Commit graph

71 commits

Author SHA1 Message Date
Sebastian Dröge
0331522128 webrtcsink: Configure only 4 threads for x264enc
More threads can cause more slices to be created, and Chrome simply falls
apart if there are more than a few slices and fails decoding.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1275>
2023-07-13 16:59:43 +03:00
Sebastian Dröge
ca51cf2509 webrtcsink: Translate force-keyunit events to force-IDR action signal for NVIDIA encoders
NVIDIA's v4l2 encoder elements don't handle the force-keyunit events but
instead provide a custom action signal based API for requesting a
keyframe.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1274>
2023-07-12 10:09:32 +00:00
Sebastian Dröge
bbd3d9ffe0 Remove unnecessary mut everywhere
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1273>
2023-07-11 10:09:35 +03:00
Mathieu Duponchelle
1dd13c4812 webrtcsink: fix session_id / peer_id confusion
In a few places, for instance parameter names, peer_id was still used
when session_id was actually getting passed.

Go through all instances of peer_id in webrtcsink/imp.rs and address
those mix-ups.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1269>
2023-07-07 05:33:30 +00:00
Bilal Elmoussaoui
dd2d7d9215 Use re-exported once_cell
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1268>
2023-07-06 17:50:49 +03:00
Sebastian Dröge
58adebb325 Fix a couple of typos 2023-07-06 13:50:17 +03:00
Olivier Crête
793ee66afa webrtcsink: Add LiveKit WebRTC sink and signaller
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1262>
2023-07-05 21:43:17 +00:00
Sebastian Dröge
c350f3c2af webrtcink: Use correct property types for nvvideoconvert
These are enums and not plain integers.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1256>
2023-06-26 14:48:58 +00:00
Mathieu Duponchelle
84a33ca7b9 webrtcsink: bring in signalling code from whipsink as a signaller
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1168>
2023-06-16 00:32:56 +02:00
Mathieu Duponchelle
1200ae0ee6 webrtcsink: improve debug
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1239>
2023-06-14 22:27:15 +02:00
Mathieu Duponchelle
81ae675f2d webrtcsink: don't try to use cudaconvert if not present
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1238>
2023-06-08 15:32:49 +02:00
Mathieu Duponchelle
7f78a8428e webrtcsink: dump discovery pipelines on state changes
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1238>
2023-06-08 15:32:49 +02:00
Mathieu Duponchelle
271b583876 webrtcsink: avoid panic on unprepare from an async tokio context
.. and log an error with advice on how to dispose of elements properly
from a tokio runtime.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1218>
2023-06-07 19:57:19 +00:00
Thibault Saunier
ab1ec12698 webrtcsink: Add support for pre encoded streams
This is a first step where we try to replicate encoding conditions from
the input stream into the discovery pipeline. A second patch will
implement using input buffers in the discovery pipelines.

This moves discovery to using input buffers directly. Instead of trying
to replicate buffers that `webrtcsink` is getting as input with testsrc,
directly run discovery based on the real buffers. This way we are sure
we work with the exact right stream type and we don't need encoders to
support encoding streams inputs.

We use the same logic for both encoded and raw input to avoid having
several code paths and makes it all more correct in any case.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1194>
2023-06-06 15:32:40 +02:00
Thibault Saunier
059cdecf7d webrtc: Unify the Codec structure between sink and source
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1194>
2023-06-06 15:31:45 +02:00
Thibault Saunier
cf32d9d668 webrtc: Move make_element to the utils
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1194>
2023-06-06 15:31:45 +02:00
François Laignel
4cc2498c24 webrtcsink: use spawn_blocking instead of call_async
In `webrtcsink`, we terminate a session by setting the session's pipeline to
`Null` like this:

```rust
    pipeline.call_async(|pipeline| {
        [...]
        pipeline.set_state(gst::State::Null);
        [...]
        // the following cvar is awaited in unprepare()
        cvar.notify_one();
    });
```

However, `pipeline.call_async` keeps a ref on the pipeline until it's done,
which means the `cvar` is notified before `pipeline` is actually 'disposed',
which happens in a different thread than `unprepare`'s. [`gst_rtp_bin_dispose`]
releases some resources when the pipeline is unrefed. In some cases, those
resources are actually released after the main thread has returned, leading
various issues.

This commit uses tokio runtime's `spawn_blocking` instead, which allows owning
and disposing of the pipeline before the `cvar` is notified.

[`gst_rtp_bin_dispose`]: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/blob/main/subprojects/gst-plugins-good/gst/rtpmanager/gstrtpbin.c#L3108

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1225>
2023-05-26 14:23:03 +02:00
Mathieu Duponchelle
a20855dfd9 webrtcsink: expose consumer-pipeline-created signal
This signal is emitted as soon as the pipeline for each consumer
is created, and can be used by applications that require a greater
level of control over webrtcsink's internals.

An example is also provided to demonstrate usage

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1220>
2023-05-25 13:15:52 +02:00
François Laignel
e62e9f5bd4 webrtcsink: adapt commit "abort stats collection before stopping the Signaller"
Adapt a commit [1] that was introduced as part of the forward port of the MR
'add signal "request-encoded-filter"' [2].

The deadlock said commit was fixing doesn't happen on main branch due to
changes in the element design: the Sessions are no longer aborted with the
element `State` held. However, we want to ensure the stats collection task
is terminated when the `webrtcbin` element returns from the Ready to Null
transition, meaning that the related resources are released.

[1]: gstreamer/gst-plugins-rs!1176 (0e6b9df9)
[2]: gstreamer/gst-plugins-rs!1176

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1222>
2023-05-24 21:35:39 +02:00
Mathieu Duponchelle
44a395f134 webrtcsink: further refactor connection to stats signals
- Stop passing webrtcbin around without using it

- Stop using glib::closure as clippy complains when using a unit type
  default-return

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1217>
2023-05-24 13:35:26 +02:00
Mathieu Duponchelle
e13124a426 webrtcsink: fix stats_sigid logic
First off, we just created the session, we know stats_sigid is None
at this point.

Second, don't first assign the result of connecting on-new-ssrc to the
field, then the result of connection twcc-stats, that simply doesn't
make sense.

Finally, actually check that stats_sigid *is* None before connecting
twcc-stats, as I understand it this must have been the original
intention / behavior.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1217>
2023-05-24 13:35:26 +02:00
Mathieu Duponchelle
ccf076ed1e webrtcsink: don't panic in twcc-stats callback
If webrtcbin was disposed of at this point, simply return

Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/345
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1217>
2023-05-24 13:35:26 +02:00
François Laignel
9a59763df1 webrtcsink: wait for Sessions to end
`State::finalize_session()` asynchronously sets the Session pipeline to Null.
In some cases, sessions `webrtcbin` could terminate their transition to Null
after `webrtcsink` had reached Null.

This commit adds a set of `finalizing_sessions`. When the finalization process
starts, the session is added to the set. After `webrtcbin` has reached the Null
state, the session is removed from the set and a condvar is notified.

In `unprepare`, `webrtcsink` loops until the `finalizing_sessions` set is
empty, awaiting for the condvar to be notified when it's not.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1221>
2023-05-24 10:18:47 +02:00
François Laignel
b68e2a1ed0 webrtcsink: remove unneeded mut
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1221>
2023-05-24 10:18:43 +02:00
François Laignel
7ba0073052 use Pad builders for optional name definition
Also, apply auto-naming in the following cases

* When building from a non wildcard-named template, the name of the template is
  automatically assigned to the Pad. User can override with a specific name by
  calling `name()` on the `PadBuilder`.
* When building with a target and no name was provided via the above, the
  GhostPad is named after the target.

See https://gitlab.freedesktop.org/gstreamer/gstreamer-rs/-/issues/448
Auto-naming discussion: https://gitlab.freedesktop.org/gstreamer/gstreamer-rs/-/merge_requests/1255#note_1891181

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1197>
2023-05-12 12:55:31 +02:00
François Laignel
092ae1fec8 net/webrtc: sink: add signal "request-encoded-filter"
The new "request-encoded-filter" signal is emitted when the encoder and related
elements are added to the pipeline. When defined, the element returned by the
signal is inserted between the encoder and the payloader.

The transformation can be reverted using the [insertable streams API] on the
receiver side.

[insertable streams API]: https://developer.mozilla.org/en-US/docs/Web/API/Insertable_Streams_for_MediaStreamTrack_API

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1202>
2023-05-09 11:17:32 +02:00
François Laignel
dc5ddd3022 net/webrtc: sink: abort stats collection before stopping the Signaller
In some rare cases, the webrtc-test entered a deadlock while executing
`WebRTCSink::unprepare`. Attaching gdb to a blocked instance showed:

* `gstrswebrtc::signaller:👿:Signaller::stop()` parked, waiting for a
  `Condvar` in `Signaller::stop()`. This was most likely awaiting for the
  receive task to complete while it was locked in `element.end_session()`.
  This code path is triggered from `unprepare` with the `State` `Mutex` locked.
* `webrtcsink:👿:WebRtcSink::process_stats` waiting for a contended `Mutex`,
  which is also the `State` `Mutex`. This prevented completion of the signal
  `gst_webrtc_bin_get_stats`.

This commit aborts the task in charge of periodically collecting stats and
ensures any remaining iteration completes before requesting the Signaller to
stop.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1202>
2023-05-09 10:26:11 +02:00
Mathieu Duponchelle
dbdb9bc164 webrtcsink: fix navigation data channel
At some point, presumably recently, the data channel stopped being
requested in Ready, making webrtcbin refuse to create it.

There was quite a lot of churn recently so I couldn't pinpoint the
breaking commit easily.

Fix by simply restoring the correct behavior of requesting the channel
after going to the Ready state

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/341

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1180>
2023-04-14 14:26:22 +02:00
Mathieu Duponchelle
f1fd8d84c3 webrtc: extract a BaseWebRTCSink
For documentation purposes, AwsKVSWebRTCSink should not inherit from
another element.

+ Mark base class as plugin API and update plugin cache

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1178>
2023-04-13 15:06:59 +00:00
Guillaume Desmottes
403004a85e fix typos
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1170>
2023-04-10 13:35:32 +02:00
Mathieu Duponchelle
a455819871 webrtcsink: fix tracking of signaller state
For the signaller to get stopped, we need to remember that we started it
in the first place.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1167>
2023-04-10 07:58:10 +03:00
Mathieu Duponchelle
3368f55a88 webrtcsink: don't return value from error closure
the signal doesn't expect a return value, which meant we were panicking
as soon as the signaller tried to report an error.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1167>
2023-04-10 07:58:10 +03:00
Matthew Waters
4f4e5f0d75 webrtcsink/signaller: don't call signals while having state/settings locked
It is a recipe for deadlocks if the signal callback calls back into
webrtcsink in some way.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1141>
2023-04-07 09:58:13 +10:00
Matthew Waters
1c61e46f37 webrtcsink: privatise signalling functions
The functionality is now access through the relevant signals instead.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1141>
2023-04-07 09:58:13 +10:00
Matthew Waters
2ac560975c webrtc/signaller: emit the relevant signals instead of the interface vtable
In order to support the use case of an external user providing their own
signalling mechanism, we want the signals to be used and only if nothing
is connected, fallback to the default handling.  Calling the interface
vtable directly will bypass the signal emission entirely.

Also ensure that the signals are defined properly for this case. i.e.
1. Signals the the application/external code is expected to emit are
   marked as an action signal.
2. Add accumulators to avoid calling the default class handler if
   another signal handler is connected.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1141>
2023-04-07 09:58:13 +10:00
Matthew Waters
b6e78b5f04 webrtcsink: expose signaller as a property
in the process move the signaller field to the settings struct

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1141>
2023-04-07 09:58:13 +10:00
Thibault Saunier
8236f3e5e7 webrtcsink: Port to the 'webrtcsrc' signaller object/interface
With contributions from:
Matthew Waters <matthew@centricular.com>

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1141>
2023-04-07 09:03:47 +10:00
Mathieu Duponchelle
15e1844956 webrtcsink: fix calculation of fec_ratio with multiple encoders
In this context, the bitrate variable is for all encoders, but the
max_bitrate field is per encoder. To calculate a proper FEC ratio, we
need to scale max_bitrate to the number of encoders.

+ Also clamp the fec-percentage that we set on the transceiver for extra
  safety

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1151>
2023-03-31 12:19:07 +00:00
David Revay
002a70a2a4 chore(webrtcsink): fix max-bitrate blurb and nick
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1150>
2023-03-28 16:11:05 +11:00
Vivia Nikolaidou
7a1b2d97d4 webrtcsink: Add ice-transport-policy option
Can be used to force relay ICE candidates, ensuring TURN server is used.
Proxy to the corresponding setting in webrtcbin,

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1143>
2023-03-27 16:12:13 +03:00
Mathieu Duponchelle
584392049c net/webrtc: implement AWS KVS signaller
And expose a wrapper webrtcsink variant, aws-kvs-webrtcsink.

This adds support in webrtcsink for processing a consumer offer, instead
of producing one.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1114>
2023-03-09 15:39:09 +00:00
Sebastian Dröge
fc5ed15af5 Update for gst::Element::link_many() and related API generalization
Specifically, get rid of now unneeded `&`.
2023-03-09 16:46:52 +02:00
Thibault Saunier
0ae637f531 webrtcsink: Move RUNTIME to the crate so it can be reused
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/932>
2023-02-28 17:57:14 -03:00
Matthew Waters
542c7e12b8 webrtcsink: also support nvvidconv in lieu of nvvideoconvert
nvvideoconvert may not exist and nvvidconv might on some Jetson
platforms.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1107>
2023-02-28 10:12:36 +11:00
Sebastian Dröge
3b4c48d9f5 Fix various new clippy warnings
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1062>
2023-01-25 10:31:19 +02:00
Sebastian Dröge
2c386fb792 Update for various deprecated APIs 2023-01-22 20:07:26 +02:00
Sebastian Dröge
4582ae91ab Move remaining plugins to ParamSpec builders
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1054>
2023-01-21 18:34:55 +02:00
Sebastian Dröge
812df78b75 webrtcbin: Update for StreamProducer API changes 2023-01-16 16:36:41 +02:00
Sebastian Dröge
6132788b02 Update for caps/structure-related string API changes
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1048>
2023-01-15 22:58:44 +02:00
Mathieu Duponchelle
1a8abde884 webrtcsink: fix panic on pre-bwe request error
We dispose of consumer pipelines asynchronously, potentially after the
session objects have been disposed of.

As session objects are the owner of the cc element, it is entirely
possible for the bwe-request signal to get emitted after cc has been
disposed of, as the closure only takes a weak reference to it.

Fix by simply checking if cc is None

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1044>
2023-01-11 15:09:45 +00:00