This commit adds support for raw payloads such as L24 audio to `webrtcsink` &
`webrtcsrc`.
Most changes take place within the `Codec` helper structure:
* A `Codec` can now advertise a depayloader. This also ensures that a format
not only can be decoded when necessary, but it can also be depayloaded in the
first place.
* It is possible to declare raw `Codec`s, meaning that their caps are compatible
with a payloader and a depayloader without the need for an encoder and decoder.
* Previous accessor `has_decoder` was renamed as `can_be_received` to account
for codecs which can be handled by an available depayloader with or without
the need for a decoder.
* New codecs were added for the following formats:
* L24, L16, L8 audio.
* RAW video.
The `webrtc-precise-sync` examples were updated to demonstrate streaming of raw
audio or video.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1501>
The certificate chain was incorrectly being passed the private key instead
of certificate. With rustls 0.23.11 version, this error was being caught
and reported. As stated in the 0.23.11 release, it has a new feature
"API for determining whether a CertifiedKey's certificate and private key
matches: keys_match(). This is called from existing fallible functions
that accept a private key and certificate (for example, with_single_cert())
so these functions now detect this misconfiguration."
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1666>
This element allows wrapping an existing live "mpeg-ts source" (udpsrc,
srtsrc,...) and providing a clock based on the actual PCR of the stream.
Combined with `tsdemux ignore-pcr=True` downstream of it, this allows playing
back the content at the same rate as the (remote) provider **and** not modify
the original timestamps.
Co-authored-by: Sebastian Dröge <slomo@coaxion.net>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1640>
As suggested in the aws crate documentation, wrap SDK errors with
DisplayErrorContext so their Display implementation outputs the full
context.
Improve error display from "dispatch failure" to
"dispatch failure: io error: error trying to connect: dns error: failed
to lookup address information: Name or service not known: dns error:
failed to lookup address information: Name or service not known: failed
to lookup address information: Name or service not known
(DispatchFailure(DispatchFailure { source: ConnectorError { kind: Io,
source: hyper::Error(Connect, ConnectError(\"dns error\", Custom { kind:
Uncategorized, error: \"failed to lookup address information: Name or
service not known\" })), connection: Unknown } }))"
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1638>
We now check if the peer actually supports Datagram and refusing to
proceed if it does not. Since the datagram size can actually change
over the lifetime of a connection according to variation in path MTU
estimate, also check buffer size before trying to send.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1613>
When pad a released, then we were removing the pad from an internal
list. If the pad was not already deactivated, the deactiviation would
attempt to look for the pad in that list and panic if it was not there.
Fix by delaying removal of the pad from the list until after pad
deactivation occurs.
Also includes test.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1618>
Multiple concurrent buffers produced by the jitterbuffer will be
combined into a single buffer list which will be sent downstream.
Events or queries that interrupt the buffer flow will cause a split in
the output buffer list.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1618>
We can use `is_some_and(...)` instead of `map_or(false, ...)`.
Also in a few places the factory was retrieved multiple times, one time
with unwrapping and another time with handling the `None` case
correctly. Instead of unwrapping, move code to handle the `None` case.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1630>
We lost the environment variable checks during the addition of the
putobjectsink tests, which caused failures on MR branches.
It would be nicer to use some other mechanism to validate the tests can
run, so we don't count on only the environmnent, but for now this will
have to do.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1629>
They're not recommended by the spec to include in the RTP packets but it
is valid to include them. Pion is including them.
When parsing the size fields also make sure to only take that much of a
payload unit and to skip any trailing data (which should not exist in
the first place).
Pion is also currently storing multiple OBUs in a single payload unit,
which is not allowed by the spec but can be easily handled with this
code now.
Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/560
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1612>
It would be possible that there is no cancellable yet when unlock() is
called, then a new future is executed and it wouldn't have any
information that it is not supposed to run at all.
To solve this remember if cancellation should happen and reset this
later.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1602>
It would be possible that there is no cancellable yet when unlock() is
called, then a new future is executed and it wouldn't have any
information that it is not supposed to run at all.
To solve this remember if unlock() was called and reset this in
unlock_stop().
Also implement actual unlocking in s3hlssink.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1602>
It would be possible that there is no cancellable yet when unlock() is
called, then a new future is executed and it wouldn't have any
information that it is not supposed to run at all.
To solve this remember if unlock() was called and reset this in
unlock_stop().
Also actually implement unlock() / unlock_stop() for the sink, and don't
cancel in stop() as unlock() / unlock_stop() would've been called before
that already.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1602>
It would be possible that there is no cancellable yet when unlock() is
called, then a new future is executed and it wouldn't have any
information that it is not supposed to run at all.
To solve this remember if unlock() was called and reset this in
unlock_stop().
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1602>
- generate a new session id for every new client
use the session id in the resource url
- remove the producer-peer-id property in the WhipServer signaler as it
is redundant to have producer id in a session having only one producer
- read the 'producer-peer-id' property on the signaller conditionally
if it exists else use the session id as producer id
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1339>
- Add a new structure Session
- manage each producer using a session
- avoid send EOS when a session terminates, instead keep running
waiting for any new producer to connect
- Maintain a bin element per session
- each session bin encapsulates webrtcbin and the decoder if needed
as well as the parser and filter if requested by the application
(through request-encoded-filter)
- this will be helpful to cleanup the session's respective elements
when the corresponding producer terminates the session
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1339>
There is now two elements, rtpsend and rtprecv that represent the two
halves of a rtpsession. This avoids the potential pipeline loop if two
peers are sending/receiving data towards each other. The two halves can
be connected by setting the rtp-id property on each element to the same
value and they will behave like a combined rtpbin-like element.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1426>
Can receive and recevie one or more RTP sessions containing multiple
pt/ssrc combinations.
Demultiplexing happens internally instead of relying on separate
elements.
Co-Authored-By: François Laignel <francois@centricular.com>
Co-Authored-By: Mathieu Duponchelle <mathieu@centricular.com>
Co-Authored-By: Sebastian Dröge <sebastian@centricular.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1426>
Add to awss3sink and awss3putobjectsink elements the following
paramerters which are set on the uploaded S3 objects:
* cache-control;
* content-encoding; and
* content-language
Bugfix: Set the content-type and content-disposition values in the S3
putobject call. Previously the params were defined on the element but
unused.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1585>
Unit tests specify a 0-based offset, so printing that plus the
random initial offset on failure is just needlessly confusing,
so subtract the initial offset when printing expected/actual
values. The real values are still printed as part of the assert.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1580>
This commit adds an Android `webrtcsrc` based example with the following
features:
* A first view allows retrieving the producer list from the signaller (peer ids
are uuids which are too long to tap, especially using an onscreen keyboard).
* Selecting a producer opens a second view. The first available video stream is
rendered on a native Surface. All the audio streams are rendered using
`autoaudiosink`.
Available Settings:
* Signaller URI.
* A toggle to prefer hardware decoding for OPUS, otherwise the app defaults to
raising `opusdec`'s rank. Hardware decoding was moved aside since it was found
to crash the app on all tested devices (2 smartphones, 1 tv).
**Warning**: in order to ease testing, this demonstration application enables
unencrypted network communication. See `AndroidManifest.xml`.
The application uses the technologies currenlty proposed by Android Studio when
creating a new project:
* Kotlin as the default language, which is fully interoperable with Java and
uses the same SDK.
* gradle 8.6.
* kotlin dialect for gradle. The structure is mostly the same as the previously
preferred dialect, for which examples can be found online readily.
* However, JNI code generation still uses Makefiles (instead of CMake) due to
the need to call [`gstreamer-1.0.mk`] for `gstreamer_android` generation.
Note: on-going work on that front:
- https://gitlab.freedesktop.org/gstreamer/cerbero/-/merge_requests/1466
- https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6794
Current limitations:
* x86 support is currently discarded as `gstreamer_android` libs generation
fails (observed with `gstreamer-1.0-android-universal-1.24.3`).
* A selector could be added to let the user chose the video streams and
possibly decide whether to render all audio streams or just select one.
Nice to have:
* Support for the synchronization features of the `webrtc-precise-sync-recv`
example (NTP clock, RFC 7273).
* It could be nice to use Rust for the specific native code.
[`gstreamer-1.0.mk`]: https://gitlab.freedesktop.org/gstreamer/cerbero/-/blob/main/data/ndk-build/gstreamer-1.0.mk
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1578>
Only configure header extensions from the source pad caps if they exist
already in the source pad caps, otherwise the configuration will fail.
Extensions that are added via the signals might not exist in the source
pad caps yet and would be added later.
Also, if configuring an existing extension from the new caps fails,
remove it and try to request a new extension for it.
Additionally don't remove extensions from the caps that can't be
provided. No header extensions for them would be added to the packets,
but that's not a problem. Removing them on the other hand would cause
negotiation to fail. This only affects extensions that are already
included in the caps.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1577>
If configuring an existing extension from the new caps fails, remove it
and try to request a new extension for it.
Also remove all extensions from the list that are not provided in the
caps, instead of passing RTP packets to all of them anyway.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1577>
In our tests, the slope (found with linear regression) on a
history of the (smoothed) accumulated inter-group delays
gives a more stable congestion control. In particular,
low-end devices becomes less sensitive to spikes in
inter-group delay measurements.
This flavour of delay based bandwidth estimation with Google
Congestion Control is also what Chromium is using.
To make it easy to experiment with the new estimator, as
well as add support for new ones in the future, also add
infrastructure for making delay estimator flavour selectable
at runtime.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1566>
Rust targets without support for `AtomicU64` is still
somewhat common. Running
git grep -i 'max_atomic_width: Some(32)' | wc -l
in the Rust compiler repo currently counts to 34 targets.
Change the `RtpBasePay2::ssrc_collision` from `AtomicU64` to
`Mutex<Option<u32>>`. This way we keep support for these
targets.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1562>
While monitoring and debugging rtpgccbwe, it is very helpful
to get continuous values of what it considers the effective
bitrate. Right now such prints will stop coming once the
algorithm stabilizes. Print it in more places so it keeps
coming. Use the same format to make it simpler to extract
the values by parsing the logs.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1567>
When debugging rtpgccbwe it is helpful to know if it is
delay based or loss based band-width estimation that puts a
bound on the current target bitrate, so add logs for that.
To minimize the time we need to hold the state lock, perform
the logging after we have released the state lock.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1561>
This set of changes implements the below fixes:
- Allow certificates to be specified for client/quicsink
- Secure connection being true on server/quicsrc and false on
client/quicsink still resulted in a successful connection
instead of server rejecting the connection
- Using secure connection with ALPN was not working
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1036>
If a user constrained the supported CAPS, for instance using `video-caps`:
```shell
gst-launch-1.0 videotestsrc ! video/x-raw,format=I420 ! x264enc \
! webrtcsink video-caps=video/x-vp8
```
... a panic would occur which was internally caught without the user being
informed except for the following message which was written to stderr:
> thread 'tokio-runtime-worker' panicked at net/webrtc/src/webrtcsink/imp.rs:3533:22:
> expected audio or video raw caps: video/x-h264, [...] <br>
> note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
The pipeline kept running.
This commit converts the panic into an `Error` which bubbles up as an element
`StreamError::CodecNotFound` which can be handled by the application.
With the above `gst-launch`, this terminates the pipeline with:
> [...] ERROR webrtcsink net/webrtc/src/webrtcsink/imp.rs:3771:gstrswebrtc::
> webrtcsink:👿:BaseWebRTCSink::start_stream_discovery_if_needed::{{closure}}:<webrtcsink0>
> Error running discovery: Unsupported caps: video/x-h264, [...] <br>
> ERROR: from element /GstPipeline:pipeline0/GstWebRTCSink:webrtcsink0:
> There is no codec present that can handle the stream's type. <br>
> Additional debug info: <br>
> net/webrtc/src/webrtcsink/imp.rs(3772): gstrswebrtc::webrtcsink:👿:BaseWebRTCSink::
> start_stream_discovery_if_needed::{{closure}} (): /GstPipeline:pipeline0/GstWebRTCSink:webrtcsink0:
> Failed to look up output caps: Unsupported caps: video/x-h264, [...] <br>
> Execution ended after 0:00:00.055716661 <br>
> Setting pipeline to NULL ...
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1540>
Clippy caught the missing feature `signal` which is used by the WebRTC precise
synchronization examples. When running `cargo` `check`, `build` or `clippy`
without `no-default-dependencies`, this feature was already present due to
dependents crates.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1541>
When swapping between several development branches, compilation times can be
frustrating. This commit proposes adding features to control which signaller
to include when building the webrtc plugin. By default, all signallers are
included, just like before.
Compiling the `webrtc-precise-sync` examples with `--no-default-features`
reduces compilation to 267 crates instead of 429 when all signallers are
compiled in.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1539>
