Commit graph

82 commits

Author SHA1 Message Date
Tim-Philipp Müller
a84bbc66f3 Update versions to 0.9.13
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1459>
2024-02-12 19:20:50 +00:00
François Laignel
0e9d33b38b webrtc: signallers: attempt to close the ws when an error occurs
This commit discards the early error returns in the send tasks to log the error
and attempt to close the websocket.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1456>
2024-02-12 17:50:52 +01:00
François Laignel
231389a990 webrtc: only use close() to close websockets
In the signaller clients and servers, the following sequence is used to close
the websocket (in the [send task]):

```rust
    ws_sink.send(WsMessage::Close(None)).await?;
    ws_sink.close().await?;
```

tungstenite's [`WebSocket::close()` doc] states:

> Calling this function is the same as calling `write(Message::Close(..))``

So we might think they are redundant and either could be used for this purpose
(`send()` calls `write()`, then `flush()`).

The result is actually is bit different as `write()` starts by checking the
state of the connection and [returns `SendAfterClosing`] if the socket is no
longer active, which is the case when a closing request has been received from
the peer via a [call to `do_close()`]). Note that `do_close()` also enqueues a
`Close` frame.

This behaviour is visible from the server's logs:

```
1. tungstenite::protocol: Received close frame: None
2. tungstenite::protocol: Replying to close with Frame { header: FrameHeader { .., opcode: Control(Close), .. }, payload: [] }
3. gst_plugin_webrtc_signalling::server: Received message Ok(Close(None))
4. gst_plugin_webrtc_signalling::server: connection closed: None this_id=cb13892f-b4d5-4d59-95e2-b3873a7bd319
5. remove_peer{peer_id="cb13892f-b4d5-4d59-95e2-b3873a7bd319"}: gst_plugin_webrtc_signalling::server: close time.busy=285µs time.idle=55.5µs
6. async_tungstenite: websocket start_send error: WebSocket protocol error: Sending after closing is not allowed
```

1: The server's websocket receives the peer's `Close(None)`.
2: `do_close()` enqueues a `Close` frame.
3: The incoming `Close(None)` is handled by the server.
4 & 5: perform session closing.
6: `ws_sink.send(WsMessage::Close(None))` attempts to `write()` while the ws
   is no longer active. The error causes an early return, which means that
   the enqueued `Close` frame is not flushed.

Depending on the peer's shutdown sequence, this can result in the following
error, which can bubble up as a `Message` on the application's bus:

```
ERROR: from element /GstPipeline:pipeline0/GstWebRTCSrc:webrtcsrc0: GStreamer encountered a general stream error.
Additional debug info:
net/webrtc/src/webrtcsrc/imp.rs(625): gstrswebrtc::webrtcsrc:👿:BaseWebRTCSrc::connect_signaller::{{closure}}::{{closure}} (): /GstPipeline:pipeline0/GstWebRTCSrc:webrtcsrc0:
Signalling error: Error receiving: WebSocket protocol error: Connection reset without closing handshake
```

On the other hand, [`close()` ensures the ws is active] before attempting to
write a `Close` frame. If it's not, it only flushes the stream.

Thus, when we want to be able to close the websocket and/or to honor the closing
handshake in response to the peer `Close` message, the `ws_sink.close()`
variant is preferable.

This can be verified in the resulting server's logs:

```
tungstenite::protocol: Received close frame: None
tungstenite::protocol: Replying to close with Frame { header: FrameHeader { is_final: true, rsv1: false, rsv2: false, rsv3: false, opcode: Control(Close), mask: None}, payload: [] }
gst_plugin_webrtc_signalling::server: Received message Ok(Close(None))
gst_plugin_webrtc_signalling::server: connection closed: None this_id=192ed7ff-3b9d-45c5-be66-872cbe67d190
remove_peer{peer_id="192ed7ff-3b9d-45c5-be66-872cbe67d190"}: gst_plugin_webrtc_signalling::server: close time.busy=22.7µs time.idle=37.4µs
tungstenite::protocol: Sending pong/close
```

We now get the notification `Sending pong/close` (the closing handshake) instead
of `websocket start_send error` from step 6 with previous variant.

The `Connection reset without closing handshake` was not observed after this
change.

[send task]: 63b568f4a0/net/webrtc/signalling/src/server/mod.rs (L165)
[`WebSocket::close()` doc]: https://docs.rs/tungstenite/0.21.0/tungstenite/protocol/struct.WebSocket.html#method.close
[returns `SendAfterClosing`]: 85463b264e/src/protocol/mod.rs (L437)
[call to `do_close()`]: 85463b264e/src/protocol/mod.rs (L601)
[`close()` ensures the ws is active]: 85463b264e/src/protocol/mod.rs (L531)

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1456>
2024-02-12 17:50:24 +01:00
Nirbheek Chauhan
3cfb28d048 webrtc/signalling: We get the address when accepting
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1456>
2024-02-12 16:56:02 +02:00
Nirbheek Chauhan
e3190c888a webrtc/signalling: Fix potential hang and FD leak
If a peer connects via TCP and never initiates TLS, then the server
will get stuck in the accept loop. Spawn a task when accepting a TLS
connection, and timeout if it doesn't complete in 5 seconds.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1456>
2024-02-12 16:53:06 +02:00
Mathieu Duponchelle
630a1120ba webrtcsink: don't panic on failure to request pad from webrtcbin
webrtcbin will refuse pad requests for all sorts of reasons, and should
be logging an error when doing so, simply post an error message and let
the application deal with it, the reason for the refusal should
hopefully be available in the logs to the user.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1411>
2023-12-18 10:27:16 +02:00
Sebastian Dröge
f5d633d293 Update versions to 0.9.12
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1387>
2023-11-10 17:47:41 +02:00
François Laignel
3627e52673 net/webrtcsink: don't miss ice candidates
During `on_remote_description_set()` processing, current session is removed
from the sessions `HashMap`. If an ice candidate is submitted to `handle_ice()`
by that time, the session can't be found and the candidate is ignored.

This commit wraps the Session in the sessions `HashMap` so an entry is kept
while `on_remote_description_set()` is running. Incoming candidates received by
`handle_ice()` will be processed immediately or enqueued and handled when the
session is restored by `on_remote_description_set()`.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1387>
2023-11-10 17:47:41 +02:00
Maksym Khomenko
19597b3737 webrtcsrc: add turn-servers property
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1387>
2023-11-10 17:46:28 +02:00
Maksym Khomenko
8355f93f5f webrtcsrc: use @watch instead of @to-owned
@to-owned increases refcount of the element, which prevents the object from proper destruction, as the initial refcount with ElementFactory::make is larger than 1.

Instead, use @watch to create a weak reference and unbind the closure automatically if the object gets destroyed

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1387>
2023-11-10 17:32:39 +02:00
Piotr Brzeziński
ecabf02b1b webrtc: Fix paths in README
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1387>
2023-11-10 17:32:39 +02:00
Sebastian Dröge
c261dd4445 webrtcsink: Fix clippy warning
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1327>
2023-09-20 11:50:27 +03:00
Seungha Yang
4fc905c9ea webrtcsink: Propagate GstContext messages
Implement CustomBusStream so that NEED_CONTEXT and HAVE_CONTEXT
messages from session/discovery can be forwarded to parent
pipeline and also GstContext can be shared.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1327>
2023-09-20 11:47:34 +03:00
Seungha Yang
581787f651 webrtcsink: Add support for d3d11 memory and qsvh264enc
Adding d3d11 memory and qsvh264enc support

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1327>
2023-09-20 11:45:05 +03:00
Robert Ayrapetyan
2151df4d70 webrtcsink: fix TWCC extension adding
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1327>
2023-09-20 10:22:45 +03:00
Mathieu Duponchelle
1ccc93df07 webrtcsink: don't forget to setup encoders for discoveries
The "encoder-setup" signal must also be emitted for the encoders
used in discovery pipelines in order for the default settings to
be applied.

This otherwise meant that for instance the x264 encoder would
use a 60 frames latency, greatly delaying startup.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1327>
2023-09-20 09:59:17 +03:00
Sebastian Dröge
17f7b04b82 webrtcsink: NVIDIA V4L2 encoders always require NVMM memory
And if the input is not like that then a corresponding converter must be
inserted.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1327>
2023-09-20 09:40:58 +03:00
Sebastian Dröge
36cdf84655 Update version to 0.9.11 2023-07-20 15:15:07 +03:00
Mathieu Duponchelle
b4b2ca9a82 webrtcsink: fix pipeline when input caps contain max-framerate
GstVideoInfo uses max-framerate to compute its fps, but this leads
to issues in videorate when framerate is actually 0/1.

Fix this by stripping away max-framerate from input caps

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1285>
2023-07-19 09:41:43 +03:00
Sebastian Dröge
fc75502ee4 webrtcsink: Configure only 4 threads for x264enc
More threads can cause more slices to be created, and Chrome simply falls
apart if there are more than a few slices and fails decoding.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1285>
2023-07-19 09:38:44 +03:00
Sebastian Dröge
9854b299a2 webrtcsink: Translate force-keyunit events to force-IDR action signal for NVIDIA encoders
NVIDIA's v4l2 encoder elements don't handle the force-keyunit events but
instead provide a custom action signal based API for requesting a
keyframe.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1285>
2023-07-19 09:38:28 +03:00
Sebastian Dröge
bdcdbfeaaf webrtcsink: Set config-interval=-1 and aggregate-mode=zero-latency on rtph26[45]pay
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1285>
2023-07-19 09:38:18 +03:00
Sebastian Dröge
482b7e1469 webrtcsink: Set VP8/VP9 payloader based on payloader element factory name
Instead of checking the encoder's name. There are more VP8/VP9 encoders
than the ones from the vpx plugin.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1285>
2023-07-19 09:38:14 +03:00
Sebastian Dröge
fafe52475f webrtcink: Use correct property types for nvvideoconvert
These are enums and not plain integers.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1267>
2023-07-05 14:41:21 +03:00
Mathieu Duponchelle
0954af10c7 webrtc/signalling: fix race condition in message ordering
Spawning one task per message to send out instead of sending them out
sequentially from the one task used to poll the handler sometimes
resulted in peers receiving ICE candidates before SDP offers, triggering
hard to understand errors in the browser.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1254>
2023-06-20 22:30:01 +02:00
Sebastian Dröge
dfe2442c92 webrtc/signalling: Allow unknown clippy lints
tracing is adding some that require a newer Rust version than used here.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1249>
2023-06-19 20:37:53 +03:00
Mathieu Duponchelle
82f3910453 webrtcsink: don't try to use cudaconvert if not present
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1249>
2023-06-19 19:03:04 +03:00
Mathieu Duponchelle
55a6609fdb webrtcsrc: add twcc extension to codec-preferences when present
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1249>
2023-06-19 19:02:48 +03:00
Sebastian Dröge
05b2caec74 webrtc: Update to fastrand 2
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1249>
2023-06-19 19:02:16 +03:00
Mathieu Duponchelle
8248425905 webrtcsink: further refactor connection to stats signals
- Stop passing webrtcbin around without using it

- Stop using glib::closure as clippy complains when using a unit type
  default-return

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1234>
2023-06-06 22:52:43 +03:00
Mathieu Duponchelle
e9d32fb221 webrtcsink: fix stats_sigid logic
First off, we just created the session, we know stats_sigid is None
at this point.

Second, don't first assign the result of connecting on-new-ssrc to the
field, then the result of connection twcc-stats, that simply doesn't
make sense.

Finally, actually check that stats_sigid *is* None before connecting
twcc-stats, as I understand it this must have been the original
intention / behavior.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1234>
2023-06-06 22:52:34 +03:00
Mathieu Duponchelle
8ff2c6609c webrtcsink: don't panic in twcc-stats callback
If webrtcbin was disposed of at this point, simply return

Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/345
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1234>
2023-06-06 22:52:22 +03:00
Thibault Saunier
0b65a2f8af webrtcsrc: Do not pass raw caps in the transceiver
That was not making sense.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1216>
2023-05-18 18:30:08 +03:00
Thibault Saunier
482ff879a4 webrtcsrc: Fix caps used when creating transceiver
We used to pass all media keys and attributes to the caps which
incorrect. Instead we should be using only the keys from the map
and remove all information related to rtcp which is irrelevant
to create the transceiver.

This also simplifies the code.

New caps look like:

```
Caps(
    application/x-rtp(memory:SystemMemory) {
        media: (gchararray) "video",
        payload: (gint) 96,
        clock-rate: (gint) 90000,
        encoding-name: (gchararray) "VP8",
    },
    application/x-rtp(memory:SystemMemory) {
        media: (gchararray) "video",
        payload: (gint) 102,
        clock-rate: (gint) 90000,
        encoding-name: (gchararray) "H264",
        packetization-mode: (gchararray) "1",
        profile: (gchararray) "baseline",
    },
    application/x-rtp(memory:SystemMemory) {
        media: (gchararray) "video",
        payload: (gint) 104,
        clock-rate: (gint) 90000,
        encoding-name: (gchararray) "H264",
        packetization-mode: (gchararray) "0",
        profile: (gchararray) "baseline",
    },
    application/x-rtp(memory:SystemMemory) {
        media: (gchararray) "video",
        payload: (gint) 106,
        clock-rate: (gint) 90000,
        encoding-name: (gchararray) "H264",
        packetization-mode: (gchararray) "1",
        profile: (gchararray) "constrained-baseline",
    },
    application/x-rtp(memory:SystemMemory) {
        media: (gchararray) "video",
        payload: (gint) 108,
        clock-rate: (gint) 90000,
        encoding-name: (gchararray) "H264",
        packetization-mode: (gchararray) "0",
        profile: (gchararray) "constrained-baseline",
    },
    application/x-rtp(memory:SystemMemory) {
        media: (gchararray) "video",
        payload: (gint) 127,
        clock-rate: (gint) 90000,
        encoding-name: (gchararray) "H264",
        packetization-mode: (gchararray) "1",
        profile: (gchararray) "main",
    },
    application/x-rtp(memory:SystemMemory) {
        media: (gchararray) "video",
        payload: (gint) 39,
        clock-rate: (gint) 90000,
        encoding-name: (gchararray) "H264",
        packetization-mode: (gchararray) "0",
        profile: (gchararray) "main",
    },
    application/x-rtp(memory:SystemMemory) {
        media: (gchararray) "video",
        payload: (gint) 98,
        clock-rate: (gint) 90000,
        encoding-name: (gchararray) "VP9",
        profile-id: (gchararray) "0",
    },
    application/x-rtp(memory:SystemMemory) {
        media: (gchararray) "video",
        payload: (gint) 100,
        clock-rate: (gint) 90000,
        encoding-name: (gchararray) "VP9",
        profile-id: (gchararray) "2",
    },
)
```

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1216>
2023-05-18 18:30:08 +03:00
Tim-Philipp Müller
2d56989f5c git: replace LICENSE file symlinks with copies
Git will de-duplicate the contents for us anyway, and
symlinks can cause problems with some versions of git
and also on Windows.

https://github.com/mesonbuild/meson/issues/11646
https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4326

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1161>
2023-04-05 18:17:16 +03:00
Mathieu Duponchelle
c2d6273786 webrtcsink: fix calculation of fec_ratio with multiple encoders
In this context, the bitrate variable is for all encoders, but the
max_bitrate field is per encoder. To calculate a proper FEC ratio, we
need to scale max_bitrate to the number of encoders.

+ Also clamp the fec-percentage that we set on the transceiver for extra
  safety

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1161>
2023-04-05 18:16:10 +03:00
Thibault Saunier
4b867d27fe Add a webrtcsrc element
Updating the docker image to include:
https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3236

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1117>
2023-03-02 14:56:30 -03:00
Sebastian Dröge
9a779607c7 Update versions to 0.9.10 2023-03-02 13:18:00 +02:00
Thibault Saunier
e4c9ba43df webrtc: Enhance debug messages when using unknown peer ID
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1116>
2023-03-02 11:01:18 +02:00
Matthew Waters
0d3dc25414 webrtcsink: also support nvvidconv in lieu of nvvideoconvert
nvvideoconvert may not exist and nvvidconv might on some Jetson
platforms.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1116>
2023-03-02 10:53:19 +02:00
Sebastian Dröge
eb3d3b3088 Update versions to 0.9.9 2023-02-09 22:08:17 +02:00
Sebastian Dröge
5c2582d105 Update version to 0.9.8 2023-01-23 11:30:27 +02:00
Sebastian Dröge
4ba452dcc3 Update versions to 0.9.7 2023-01-19 19:06:43 +02:00
Sebastian Dröge
c818a575b4 Update versions to 0.9.6 2023-01-18 17:19:17 +02:00
Mathieu Duponchelle
53ae335d22 webrtcsink: fix panic on pre-bwe request error
We dispose of consumer pipelines asynchronously, potentially after the
session objects have been disposed of.

As session objects are the owner of the cc element, it is entirely
possible for the bwe-request signal to get emitted after cc has been
disposed of, as the closure only takes a weak reference to it.

Fix by simply checking if cc is None

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1045>
2023-01-11 18:38:13 +02:00
Sebastian Dröge
2a8a90f76f Update versions to 0.9.5 2023-01-07 16:06:17 +02:00
Sebastian Dröge
b0bd55c4d2 Update versions to 0.9.4 2022-12-27 13:14:59 +02:00
Sebastian Dröge
bae5294e8f Update versions to 0.9.3 2022-12-16 20:22:17 +02:00
Mathieu Duponchelle
fffd7dc542 webrtc/README: update command to run the signalling server
Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/277

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1017>
2022-12-16 18:51:08 +02:00
Sebastian Dröge
b4185134d1 Fix various new clippy warnings
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1017>
2022-12-16 18:51:00 +02:00