Commit graph

457 commits

Author SHA1 Message Date
Mathieu Duponchelle
f00a169081 webrtcsrc: add twcc extension to codec-preferences when present
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1245>
2023-06-15 20:41:53 +00:00
Mathieu Duponchelle
1200ae0ee6 webrtcsink: improve debug
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1239>
2023-06-14 22:27:15 +02:00
Mathieu Duponchelle
64056c5527 net/webrtc: improve documentation layout
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1239>
2023-06-14 22:27:15 +02:00
Sebastian Dröge
8a7a1f519c webrtc: Update to fastrand 2
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1240>
2023-06-09 09:36:51 +03:00
Mathieu Duponchelle
81ae675f2d webrtcsink: don't try to use cudaconvert if not present
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1238>
2023-06-08 15:32:49 +02:00
Mathieu Duponchelle
7f78a8428e webrtcsink: dump discovery pipelines on state changes
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1238>
2023-06-08 15:32:49 +02:00
Mathieu Duponchelle
7447d95f1b webrtc/signalling: fix race condition in message ordering
Spawning one task per message to send out instead of sending them out
sequentially from the one task used to poll the handler sometimes
resulted in peers receiving ICE candidates before SDP offers, triggering
hard to understand errors in the browser.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1236>
2023-06-08 13:24:45 +02:00
Mathieu Duponchelle
de0f7a08fe gstwebrtc-api: fix firefox errors about more than two stun servers
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1236>
2023-06-08 13:24:45 +02:00
Mathieu Duponchelle
cd4b90fef4 webrtcsink/utils: remove unused decoders field in DecodingInfo
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1236>
2023-06-08 01:54:13 +02:00
Mathieu Duponchelle
271b583876 webrtcsink: avoid panic on unprepare from an async tokio context
.. and log an error with advice on how to dispose of elements properly
from a tokio runtime.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1218>
2023-06-07 19:57:19 +00:00
Sebastian Dröge
c65b3429ad Use MPL as license specifier for plugins only requiring GStreamer < 1.20
And use MPL-2.0 for all that require GStreamer 1.20 or newer. The new
string is only allowed in 1.20 or newer and using it in older versions
causes failure to load the plugin.

All affected plugins are of course still MPL-2.0 licensed.

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/374

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1235>
2023-06-07 19:13:55 +03:00
Mathieu Duponchelle
fda5aed89f webrtcsink: encoded streams: address last review comments
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1194>
2023-06-06 16:05:28 +02:00
Thibault Saunier
ab1ec12698 webrtcsink: Add support for pre encoded streams
This is a first step where we try to replicate encoding conditions from
the input stream into the discovery pipeline. A second patch will
implement using input buffers in the discovery pipelines.

This moves discovery to using input buffers directly. Instead of trying
to replicate buffers that `webrtcsink` is getting as input with testsrc,
directly run discovery based on the real buffers. This way we are sure
we work with the exact right stream type and we don't need encoders to
support encoding streams inputs.

We use the same logic for both encoded and raw input to avoid having
several code paths and makes it all more correct in any case.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1194>
2023-06-06 15:32:40 +02:00
Thibault Saunier
059cdecf7d webrtc: Unify the Codec structure between sink and source
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1194>
2023-06-06 15:31:45 +02:00
Thibault Saunier
cf32d9d668 webrtc: Move make_element to the utils
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1194>
2023-06-06 15:31:45 +02:00
Thibault Saunier
ce42723ad2 webrtc: Minor documentation enhancement
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1194>
2023-06-06 15:31:45 +02:00
Mathieu Duponchelle
6346d5608e net/aws/transcriber: track discont offset in input stream
and add it up to subsequent transcripts.

This ensures synchronization is maintained even after the input stream
experiences a discontinuity and a gap in its timestamps.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1230>
2023-06-02 08:55:11 +00:00
Mathieu Duponchelle
80582923bb aws_kvs_signaller: don't force us-east-1 region
Instead use default region provider, with a fallback to us-east-1

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1228>
2023-05-30 16:04:27 +00:00
Edward Hervey
31b06e52ea rtpgccbwe: Improve packet handling
Both the delay-based *and* loss-based estimates should be computed instead of
just one. This ensures faster adaptation.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1179>
2023-05-29 08:20:36 +00:00
François Laignel
4cc2498c24 webrtcsink: use spawn_blocking instead of call_async
In `webrtcsink`, we terminate a session by setting the session's pipeline to
`Null` like this:

```rust
    pipeline.call_async(|pipeline| {
        [...]
        pipeline.set_state(gst::State::Null);
        [...]
        // the following cvar is awaited in unprepare()
        cvar.notify_one();
    });
```

However, `pipeline.call_async` keeps a ref on the pipeline until it's done,
which means the `cvar` is notified before `pipeline` is actually 'disposed',
which happens in a different thread than `unprepare`'s. [`gst_rtp_bin_dispose`]
releases some resources when the pipeline is unrefed. In some cases, those
resources are actually released after the main thread has returned, leading
various issues.

This commit uses tokio runtime's `spawn_blocking` instead, which allows owning
and disposing of the pipeline before the `cvar` is notified.

[`gst_rtp_bin_dispose`]: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/blob/main/subprojects/gst-plugins-good/gst/rtpmanager/gstrtpbin.c#L3108

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1225>
2023-05-26 14:23:03 +02:00
Mathieu Duponchelle
a20855dfd9 webrtcsink: expose consumer-pipeline-created signal
This signal is emitted as soon as the pipeline for each consumer
is created, and can be used by applications that require a greater
level of control over webrtcsink's internals.

An example is also provided to demonstrate usage

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1220>
2023-05-25 13:15:52 +02:00
Sebastian Dröge
a27be7d054 net: Update to AWS SDK 0.28
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1224>
2023-05-25 13:23:49 +03:00
François Laignel
e62e9f5bd4 webrtcsink: adapt commit "abort stats collection before stopping the Signaller"
Adapt a commit [1] that was introduced as part of the forward port of the MR
'add signal "request-encoded-filter"' [2].

The deadlock said commit was fixing doesn't happen on main branch due to
changes in the element design: the Sessions are no longer aborted with the
element `State` held. However, we want to ensure the stats collection task
is terminated when the `webrtcbin` element returns from the Ready to Null
transition, meaning that the related resources are released.

[1]: gstreamer/gst-plugins-rs!1176 (0e6b9df9)
[2]: gstreamer/gst-plugins-rs!1176

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1222>
2023-05-24 21:35:39 +02:00
Sebastian Dröge
e3c46b40a0 whipsink: Request pads with webrtcbin's pad templates and not our own
It's invalid to request pads with a pad template that is not part of the
element, and results in a critical warning.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1223>
2023-05-24 14:14:32 +00:00
Mathieu Duponchelle
44a395f134 webrtcsink: further refactor connection to stats signals
- Stop passing webrtcbin around without using it

- Stop using glib::closure as clippy complains when using a unit type
  default-return

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1217>
2023-05-24 13:35:26 +02:00
Mathieu Duponchelle
e13124a426 webrtcsink: fix stats_sigid logic
First off, we just created the session, we know stats_sigid is None
at this point.

Second, don't first assign the result of connecting on-new-ssrc to the
field, then the result of connection twcc-stats, that simply doesn't
make sense.

Finally, actually check that stats_sigid *is* None before connecting
twcc-stats, as I understand it this must have been the original
intention / behavior.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1217>
2023-05-24 13:35:26 +02:00
Mathieu Duponchelle
ccf076ed1e webrtcsink: don't panic in twcc-stats callback
If webrtcbin was disposed of at this point, simply return

Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/345
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1217>
2023-05-24 13:35:26 +02:00
François Laignel
9a59763df1 webrtcsink: wait for Sessions to end
`State::finalize_session()` asynchronously sets the Session pipeline to Null.
In some cases, sessions `webrtcbin` could terminate their transition to Null
after `webrtcsink` had reached Null.

This commit adds a set of `finalizing_sessions`. When the finalization process
starts, the session is added to the set. After `webrtcbin` has reached the Null
state, the session is removed from the set and a condvar is notified.

In `unprepare`, `webrtcsink` loops until the `finalizing_sessions` set is
empty, awaiting for the condvar to be notified when it's not.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1221>
2023-05-24 10:18:47 +02:00
François Laignel
b68e2a1ed0 webrtcsink: remove unneeded mut
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1221>
2023-05-24 10:18:43 +02:00
Thibault Saunier
04e35e86d6 webrtcsrc: Do not pass raw caps in the transceiver
That was not making sense.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1214>
2023-05-18 18:23:56 +03:00
Thibault Saunier
e73d7082a6 webrtcsrc: Fix caps used when creating transceiver
We used to pass all media keys and attributes to the caps which
incorrect. Instead we should be using only the keys from the map
and remove all information related to rtcp which is irrelevant
to create the transceiver.

This also simplifies the code.

New caps look like:

```
Caps(
    application/x-rtp(memory:SystemMemory) {
        media: (gchararray) "video",
        payload: (gint) 96,
        clock-rate: (gint) 90000,
        encoding-name: (gchararray) "VP8",
    },
    application/x-rtp(memory:SystemMemory) {
        media: (gchararray) "video",
        payload: (gint) 102,
        clock-rate: (gint) 90000,
        encoding-name: (gchararray) "H264",
        packetization-mode: (gchararray) "1",
        profile: (gchararray) "baseline",
    },
    application/x-rtp(memory:SystemMemory) {
        media: (gchararray) "video",
        payload: (gint) 104,
        clock-rate: (gint) 90000,
        encoding-name: (gchararray) "H264",
        packetization-mode: (gchararray) "0",
        profile: (gchararray) "baseline",
    },
    application/x-rtp(memory:SystemMemory) {
        media: (gchararray) "video",
        payload: (gint) 106,
        clock-rate: (gint) 90000,
        encoding-name: (gchararray) "H264",
        packetization-mode: (gchararray) "1",
        profile: (gchararray) "constrained-baseline",
    },
    application/x-rtp(memory:SystemMemory) {
        media: (gchararray) "video",
        payload: (gint) 108,
        clock-rate: (gint) 90000,
        encoding-name: (gchararray) "H264",
        packetization-mode: (gchararray) "0",
        profile: (gchararray) "constrained-baseline",
    },
    application/x-rtp(memory:SystemMemory) {
        media: (gchararray) "video",
        payload: (gint) 127,
        clock-rate: (gint) 90000,
        encoding-name: (gchararray) "H264",
        packetization-mode: (gchararray) "1",
        profile: (gchararray) "main",
    },
    application/x-rtp(memory:SystemMemory) {
        media: (gchararray) "video",
        payload: (gint) 39,
        clock-rate: (gint) 90000,
        encoding-name: (gchararray) "H264",
        packetization-mode: (gchararray) "0",
        profile: (gchararray) "main",
    },
    application/x-rtp(memory:SystemMemory) {
        media: (gchararray) "video",
        payload: (gint) 98,
        clock-rate: (gint) 90000,
        encoding-name: (gchararray) "VP9",
        profile-id: (gchararray) "0",
    },
    application/x-rtp(memory:SystemMemory) {
        media: (gchararray) "video",
        payload: (gint) 100,
        clock-rate: (gint) 90000,
        encoding-name: (gchararray) "VP9",
        profile-id: (gchararray) "2",
    },
)
```

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1214>
2023-05-18 18:23:56 +03:00
François Laignel
7ba0073052 use Pad builders for optional name definition
Also, apply auto-naming in the following cases

* When building from a non wildcard-named template, the name of the template is
  automatically assigned to the Pad. User can override with a specific name by
  calling `name()` on the `PadBuilder`.
* When building with a target and no name was provided via the above, the
  GhostPad is named after the target.

See https://gitlab.freedesktop.org/gstreamer/gstreamer-rs/-/issues/448
Auto-naming discussion: https://gitlab.freedesktop.org/gstreamer/gstreamer-rs/-/merge_requests/1255#note_1891181

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1197>
2023-05-12 12:55:31 +02:00
François Laignel
8e93d294e5 Update to argumentless {Bin,Pipeline}::new
See https://gitlab.freedesktop.org/gstreamer/gstreamer-rs/-/issues/449

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1197>
2023-05-12 12:55:31 +02:00
François Laignel
680d5221db net/webrtc: src: add signal "request-encoded-filter"
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1202>
2023-05-09 12:02:15 +02:00
François Laignel
092ae1fec8 net/webrtc: sink: add signal "request-encoded-filter"
The new "request-encoded-filter" signal is emitted when the encoder and related
elements are added to the pipeline. When defined, the element returned by the
signal is inserted between the encoder and the payloader.

The transformation can be reverted using the [insertable streams API] on the
receiver side.

[insertable streams API]: https://developer.mozilla.org/en-US/docs/Web/API/Insertable_Streams_for_MediaStreamTrack_API

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1202>
2023-05-09 11:17:32 +02:00
François Laignel
dc5ddd3022 net/webrtc: sink: abort stats collection before stopping the Signaller
In some rare cases, the webrtc-test entered a deadlock while executing
`WebRTCSink::unprepare`. Attaching gdb to a blocked instance showed:

* `gstrswebrtc::signaller:👿:Signaller::stop()` parked, waiting for a
  `Condvar` in `Signaller::stop()`. This was most likely awaiting for the
  receive task to complete while it was locked in `element.end_session()`.
  This code path is triggered from `unprepare` with the `State` `Mutex` locked.
* `webrtcsink:👿:WebRtcSink::process_stats` waiting for a contended `Mutex`,
  which is also the `State` `Mutex`. This prevented completion of the signal
  `gst_webrtc_bin_get_stats`.

This commit aborts the task in charge of periodically collecting stats and
ensures any remaining iteration completes before requesting the Signaller to
stop.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1202>
2023-05-09 10:26:11 +02:00
François Laignel
eca269cbf2 net/webrtc: src: don't set stun-server on webrtcbin when our property is None
... otherwise an error occurs about the stun-server address being an empty
string which doesn't comply with the expected address format.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1202>
2023-05-09 10:26:07 +02:00
Sebastian Dröge
cb5b527d74 Update to AWS SDK 0.27 and async-tungstenite 0.22
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1199>
2023-05-02 15:30:00 +03:00
Sebastian Dröge
5451035215 Update async-tungstenite and AWS SDK dependencies
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1187>
2023-04-21 10:48:10 +00:00
Sebastian Dröge
cc3646640e Fix a couple of new Rust 1.69 clippy warnings
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1186>
2023-04-20 16:47:45 +03:00
Edward Hervey
721d17e181 rtpgccbwe: Don't process empty lists
The structure parsing could result in an empty vector. Don't do any processing
since the loss code assumes it's non-empty for average estimates which would
result in weird/invalid results.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1181>
2023-04-15 19:35:27 +02:00
Mathieu Duponchelle
dbdb9bc164 webrtcsink: fix navigation data channel
At some point, presumably recently, the data channel stopped being
requested in Ready, making webrtcbin refuse to create it.

There was quite a lot of churn recently so I couldn't pinpoint the
breaking commit easily.

Fix by simply restoring the correct behavior of requesting the channel
after going to the Ready state

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/341

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1180>
2023-04-14 14:26:22 +02:00
Mathieu Duponchelle
f1fd8d84c3 webrtc: extract a BaseWebRTCSink
For documentation purposes, AwsKVSWebRTCSink should not inherit from
another element.

+ Mark base class as plugin API and update plugin cache

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1178>
2023-04-13 15:06:59 +00:00
Loïc Le Page
dba91bceca webrtc: fix documentation after signaller interface changes
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1175>
2023-04-12 20:19:22 +02:00
Thibault Saunier
8f2273328b webrtcsrc: Return bool en 'end-session' as required
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1172>
2023-04-12 12:17:56 +00:00
Sebastian Dröge
5dcdf645d6 net: ndi: Update to libloading 0.8
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1173>
2023-04-12 11:03:05 +03:00
Mathieu Duponchelle
f366c20869 awstranscriber: fix what we send over for translations
Prior to this commit, we were sending over words concatenated together
with no separators, for instance "Idon'twanttobeanemperor".

The translation service seems clever enough to translate the contents
anyway, but there is no reason to make its task harder than necessary,
and it didn't re-add separators when the target language was the same as
the source language, which resulted in less than ideal output.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1171>
2023-04-10 20:47:12 +00:00
Mathieu Duponchelle
408fd2030c awstranscriber: slight debug improvement
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1171>
2023-04-10 20:47:12 +00:00
Guillaume Desmottes
403004a85e fix typos
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1170>
2023-04-10 13:35:32 +02:00
Mathieu Duponchelle
a455819871 webrtcsink: fix tracking of signaller state
For the signaller to get stopped, we need to remember that we started it
in the first place.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1167>
2023-04-10 07:58:10 +03:00