Commit graph

16 commits

Author SHA1 Message Date
François Laignel
34b791ff5e webrtc: add raw payload support
This commit adds support for raw payloads such as L24 audio to `webrtcsink` &
`webrtcsrc`.

Most changes take place within the `Codec` helper structure:

* A `Codec` can now advertise a depayloader. This also ensures that a format
  not only can be decoded when necessary, but it can also be depayloaded in the
  first place.
* It is possible to declare raw `Codec`s, meaning that their caps are compatible
  with a payloader and a depayloader without the need for an encoder and decoder.
* Previous accessor `has_decoder` was renamed as `can_be_received` to account
  for codecs which can be handled by an available depayloader with or without
  the need for a decoder.
* New codecs were added for the following formats:
  * L24, L16, L8 audio.
  * RAW video.

The `webrtc-precise-sync` examples were updated to demonstrate streaming of raw
audio or video.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1501>
2024-07-16 19:32:02 +00:00
Sebastian Dröge
23d998a1db Slightly improve code making use of element factories retrieved from an element
We can use `is_some_and(...)` instead of `map_or(false, ...)`.

Also in a few places the factory was retrieved multiple times, one time
with unwrapping and another time with handling the `None` case
correctly. Instead of unwrapping, move code to handle the `None` case.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1630>
2024-06-19 13:03:37 +00:00
Taruntej Kanakamalla
83f76280f5 net/webrtc: Example for whipserver
rudimentary sample to test multiple WHIP client connections

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1339>
2024-05-29 21:03:27 +00:00
François Laignel
4259d284bd webrtc: add android webrtcsrc example
This commit adds an Android `webrtcsrc` based example with the following
features:

* A first view allows retrieving the producer list from the signaller (peer ids
  are uuids which are too long to tap, especially using an onscreen keyboard).
* Selecting a producer opens a second view. The first available video stream is
  rendered on a native Surface. All the audio streams are rendered using
  `autoaudiosink`.

Available Settings:

* Signaller URI.
* A toggle to prefer hardware decoding for OPUS, otherwise the app defaults to
  raising `opusdec`'s rank. Hardware decoding was moved aside since it was found
  to crash the app on all tested devices (2 smartphones, 1 tv).

**Warning**: in order to ease testing, this demonstration application enables
unencrypted network communication. See `AndroidManifest.xml`.

The application uses the technologies currenlty proposed by Android Studio when
creating a new project:

* Kotlin as the default language, which is fully interoperable with Java and
  uses the same SDK.
* gradle 8.6.
* kotlin dialect for gradle. The structure is mostly the same as the previously
  preferred dialect, for which examples can be found online readily.
* However, JNI code generation still uses Makefiles (instead of CMake) due to
  the need to call [`gstreamer-1.0.mk`] for `gstreamer_android` generation.
  Note: on-going work on that front:
  - https://gitlab.freedesktop.org/gstreamer/cerbero/-/merge_requests/1466
  - https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6794

Current limitations:

* x86 support is currently discarded as `gstreamer_android` libs generation
  fails (observed with `gstreamer-1.0-android-universal-1.24.3`).
* A selector could be added to let the user chose the video streams and
  possibly decide whether to render all audio streams or just select one.

Nice to have:

* Support for the synchronization features of the `webrtc-precise-sync-recv`
  example (NTP clock, RFC 7273).
* It could be nice to use Rust for the specific native code.

[`gstreamer-1.0.mk`]: https://gitlab.freedesktop.org/gstreamer/cerbero/-/blob/main/data/ndk-build/gstreamer-1.0.mk

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1578>
2024-05-24 16:14:13 +00:00
Sebastian Dröge
613ed56675 webrtcsink: Add a custom signaller example in Python
This re-implements the default webrtcsink/src signalling protocol in
Python for demonstration purposes.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1569>
2024-05-10 15:59:12 +00:00
François Laignel
83d70d3471 webrtc: add RFC 7273 support
This commit implements [RFC 7273] (NTP & PTP clock signalling & synchronization)
for `webrtcsink` by adding the "ts-refclk" & "mediaclk" SDP media attributes to
identify the clock. These attributes are handled by `rtpjitterbuffer` on the
consumer side. They MUST be part of the SDP offer.

When used with an NTP or PTP clock, "mediaclk" indicates the RTP offset at the
clock's origin. Because the payloaders are not instantiated when the offer is
sent to the consumer, the RTP offset is set to 0 and the payloader
`timstamp-offset`s are set accordingly when they are created.

The `webrtc-precise-sync` examples were updated to be able to start with an NTP
(default), a PTP or the system clock (on the receiver only). The rtp jitter
buffer will synchronize with the clock signalled in the SDP offer provided the
sender is started with `--do-clock-signalling` & the receiver with
`--expect-clock-signalling`.

[RFC 7273]: https://datatracker.ietf.org/doc/html/rfc7273

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1500>
2024-04-12 14:18:09 +02:00
François Laignel
cc43935036 webrtc: add precise synchronization example
This example demonstrates a sender / receiver setup which ensures precise
synchronization of multiple streams in a single session.

[RFC 6051]-style rapid synchronization of RTP streams is available as an option.
See the [Instantaneous RTP synchronization...] blog post for details about this
mode and an example based on RTSP instead of WebRTC.

[RFC 6051]: https://datatracker.ietf.org/doc/html/rfc6051
[Instantaneous RTP synchronization...]: https://coaxion.net/blog/2022/05/instantaneous-rtp-synchronization-retrieval-of-absolute-sender-clock-times-with-gstreamer/

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1463>
2024-04-03 19:10:40 +02:00
Sebastian Dröge
764143d971 webrtc: Remove unnecessary manual Send+Sync implementations for signallers
These are automatically implemented.

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/483

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1432>
2024-01-18 10:01:25 +02:00
Guillaume Desmottes
6dfd1c1496 use new debug and parse API
Changes from https://gitlab.freedesktop.org/gstreamer/gstreamer-rs/-/merge_requests/1355

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1403>
2023-12-04 15:58:21 +01:00
Maksym Khomenko
e4096b5157 webrtcsink: README: add documentation for custom signaller
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1340>
2023-10-06 12:58:04 +03:00
Maksym Khomenko
a9719cada2 webrtcsink: add custom signaller example
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1340>
2023-10-06 12:58:03 +03:00
Mathieu Duponchelle
a20855dfd9 webrtcsink: expose consumer-pipeline-created signal
This signal is emitted as soon as the pipeline for each consumer
is created, and can be used by applications that require a greater
level of control over webrtcsink's internals.

An example is also provided to demonstrate usage

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1220>
2023-05-25 13:15:52 +02:00
Zhao, Gang
9fa838e366 webrtc: Fix rustfmt errors
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1019>
2022-12-27 11:12:54 +02:00
Zhao, Gang
1ffeb4d44d webrtc: Move from async-std to tokio
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1019>
2022-12-26 23:10:40 +00:00
Zhao, Gang
2bc29c1fd3 webrtc: examples: Update package-lock.json
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1019>
2022-12-26 23:10:40 +00:00
Thibault Saunier
39c0dcb0d4 Plug webrtc in 2022-10-20 11:51:58 +02:00