Commit graph

241 commits

Author SHA1 Message Date
François Laignel
cc43935036 webrtc: add precise synchronization example
This example demonstrates a sender / receiver setup which ensures precise
synchronization of multiple streams in a single session.

[RFC 6051]-style rapid synchronization of RTP streams is available as an option.
See the [Instantaneous RTP synchronization...] blog post for details about this
mode and an example based on RTSP instead of WebRTC.

[RFC 6051]: https://datatracker.ietf.org/doc/html/rfc6051
[Instantaneous RTP synchronization...]: https://coaxion.net/blog/2022/05/instantaneous-rtp-synchronization-retrieval-of-absolute-sender-clock-times-with-gstreamer/

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1463>
2024-04-03 19:10:40 +02:00
Guillaume Desmottes
b5cbc47cf7 web: webrtcsink: improve panic message on unexpected caps during discovery
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1524>
2024-04-02 14:25:58 +02:00
Guillaume Desmottes
35b84d219f webrtc: webrtcsink: set perfect-timestamp=true on audio encoders
Chrome audio decoder doesn't cope well with not perfect ts, generating
noises in the audio.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1524>
2024-04-02 14:25:51 +02:00
François Laignel
9f27bde36a aws: use fixed BehaviorVersion
Quoting [`BehaviorVersion` documentation]:

> Over time, new best-practice behaviors are introduced. However, these
> behaviors might not be backwards compatible. For example, a change which
> introduces new default timeouts or a new retry-mode for all operations might
> be the ideal behavior but could break existing applications.

This commit uses `BehaviorVersion::v2023_11_09()`, which is the latest
major version at the moment. When a new major version is released, the method
will be deprecated, which will warn us of the new version and let us decide
when to upgrade, after any changes if required. This is safer that using
`latest()` which would silently use a different major version, possibly
breaking existing code.

[`BehaviorVersion` documentation]: https://docs.rs/aws-config/1.1.8/aws_config/struct.BehaviorVersion.html

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1520>
2024-03-26 17:44:16 +01:00
Philippe Normand
be12c0a5f7 Fix clippy warnings after upgrade to Rust 1.77
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1512>
2024-03-21 17:33:32 +00:00
François Laignel
c5e7e76e4d webrtcsrc: add do-retransmission property
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1509>
2024-03-21 07:25:30 +00:00
François Laignel
5476e3d759 webrtcsink: prevent video-info error log for audio streams
The following error is logged when `webrtcsink` is feeded with an audio stream:

> ERROR video-info video-info.c:540:gst_video_info_from_caps:
>       wrong name 'audio/x-raw', expected video/ or image/

This commit bypasses `VideoInfo::from_caps` for audio streams.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1511>
2024-03-20 19:57:45 +01:00
Guillaume Desmottes
96337d5234 webrtc: allow resolution and framerate input changes
Some changes do not require a WebRTC renegotiation so we can allow
those.

Fix #515

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1498>
2024-03-18 14:52:01 +01:00
Guillaume Desmottes
8f997ea4e3 webrtc: janus: handle 'hangup' messages from Janus
Fix error about this message not being handled:

{
   "janus": "hangup",
   "session_id": 4758817463851315,
   "sender": 4126342934227009,
   "reason": "Close PC"
}

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1481>
2024-03-13 10:14:38 +00:00
Guillaume Desmottes
992f8d9a5d webrtc: janus: handle 'destroyed' messages from Janus
Fix this error when the room is destroyed:

ERROR   webrtc-janusvr-signaller imp.rs:413:gstrswebrtc::janusvr_signaller:👿:Signaller::handle_msg:<GstJanusVRWebRTCSignallerU64@0x55b166a3fe40> Unknown message from server: {
   "janus": "event",
   "session_id": 6667171862739941,
   "sender": 1964690595468240,
   "plugindata": {
      "plugin": "janus.plugin.videoroom",
      "data": {
         "videoroom": "destroyed",
         "room": 8320333573294267
      }
   }
}

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1481>
2024-03-13 10:14:38 +00:00
Guillaume Desmottes
9c6a39d692 webrtc: janus: handle (stopped-)talking events
Expose those events using a signal.

Fix those errors when joining a Janus room configured with
'audiolevel_event: true'.

ERROR   webrtc-janusvr-signaller imp.rs:408:gstrswebrtc::janusvr_signaller:👿:Signaller::handle_msg:<GstJanusVRWebRTCSignaller@0x560cf2a55100> Unknown message from server: {
   "janus": "event",
   "session_id": 2384862538500481,
   "sender": 1867822625190966,
   "plugindata": {
      "plugin": "janus.plugin.videoroom",
      "data": {
         "videoroom": "talking",
         "room": 7564250471742314,
         "id": 6815475717947398,
         "mindex": 0,
         "mid": "0",
         "audio-level-dBov-avg": 37.939998626708984
      }
   }
}
ERROR   webrtc-janusvr-signaller imp.rs:408:gstrswebrtc::janusvr_signaller:👿:Signaller::handle_msg:<GstJanusVRWebRTCSignaller@0x560cf2a55100> Unknown message from server: {
   "janus": "event",
   "session_id": 2384862538500481,
   "sender": 1867822625190966,
   "plugindata": {
      "plugin": "janus.plugin.videoroom",
      "data": {
         "videoroom": "stopped-talking",
         "room": 7564250471742314,
         "id": 6815475717947398,
         "mindex": 0,
         "mid": "0",
         "audio-level-dBov-avg": 40.400001525878906
      }
   }
}

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1481>
2024-03-13 10:14:38 +00:00
François Laignel
5b01e43a12 webrtc: update further to WebRTCSessionDescription sdp accessor changes
See: https://gitlab.freedesktop.org/gstreamer/gstreamer-rs/-/merge_requests/1406
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1491>
2024-03-11 13:39:19 +01:00
Guillaume Desmottes
612f863ee9 webrtc: janusvrwebrtcsink: add 'use-string-ids' property
Instead of exposing all ids properties as strings, we now have two
signaller implementations exposing those properties using their actual
type. This API is more natural and save the element and application
conversions when using numerical ids (Janus's default).

I also removed the 'joined-id' property as it's actually the same id as
'feed-id'. I think it would be better to have a 'janus-state' property or
something like that for applications willing to know when the room has
been joined.
This id is also no longer generated by the element by default, as Janus
will take care of generating one if not provided.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1486>
2024-03-07 09:34:58 +01:00
Jordan Yelloz
0414f468c6 livekit_signaller: Added missing getter for excluded-producer-peer-ids
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1484>
2024-03-04 10:08:11 -07:00
Jordan Yelloz
8b0731b5a2 webrtcsrc: Removed incorrect URIHandler from LiveKit source
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1484>
2024-03-04 09:44:01 -07:00
Jordan Yelloz
002dc36ab9 livekit_signaller: Improved shutdown behavior
Without sending a Leave request to the server before disconnecting, the
disconnected client will appear present and stuck in the room for a little
while until the server removes it due to inactivity.

After this change, the disconnecting client will immediately leave the room.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1482>
2024-02-29 08:21:13 -07:00
Jordan Yelloz
f0b408d823 webrtcsrc: Removed flag setup from WhipServerSrc
It's already done in the base class

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1461>
2024-02-28 11:25:58 -07:00
Jordan Yelloz
17b2640237 webrtcsrc: Updated readme for LiveKit source
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1461>
2024-02-28 11:25:58 -07:00
Jordan Yelloz
fa006b9fc9 webrtcsrc: Added LiveKit source element
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1461>
2024-02-28 11:25:58 -07:00
Jordan Yelloz
96037fbcc5 webrtcsink: Updated livekitwebrtcsink for new signaller constructor
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1461>
2024-02-28 11:25:58 -07:00
Jordan Yelloz
730b3459f1 livekit_signaller: Added dual-role support
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1461>
2024-02-28 11:25:49 -07:00
Guillaume Desmottes
60bb72ddc3 webrtc: janus: add joined-id property to the signaller
Fix #504

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1480>
2024-02-28 15:05:11 +01:00
Guillaume Desmottes
eabf31e6d0 webrtc: janus: rename RoomId to JanusId
Those weird ids are used in multiple places, not only for the room id,
so best to have a more generic name.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1480>
2024-02-28 15:05:11 +01:00
Guillaume Desmottes
550018c917 webrtc: janus: room id not optional in 'joined' message
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1480>
2024-02-28 14:16:46 +01:00
Guillaume Desmottes
0829898d73 webrtc: janus: remove 'audio' and 'video' from publish messages
Those are deprecated and no longer used.

See https://janus.conf.meetecho.com/docs/videoroom and
https://github.com/meetecho/janus-gateway/blob/master/src/plugins/janus_videoroom.c#L9894

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1480>
2024-02-28 13:39:04 +01:00
Guillaume Desmottes
ec17c58dee webrtc: janus: numerical room ids are u64
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1478>
2024-02-28 11:56:44 +01:00
Yorick Smilda
563eff1193 Implement GstWebRTCAPI as class instead of global instance
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1373>
2024-02-27 12:30:13 +00:00
Jordan Yelloz
594400a7f5 webrtcsrc: Made producer-peer-id optional
It may be necessary for some signalling clients but the source element
doesn't need to depend on it.

Also, the value will fall back to the pad's MSID for the first argument
to the request-encoded-filter gobject signal when it isn't available
from the signalling client.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1477>
2024-02-26 13:41:40 -07:00
Xavier Claessens
f7ffa13543 janusvr: Add string-ids property
It forces usage of strings even if it can be parsed into an integer.
This allows joining room `"133"` in a server configured with string
room ids.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1466>
2024-02-26 11:10:00 +00:00
Xavier Claessens
23955d2dbb janusvr: Room IDs can be strings
Sponsored-by: Netflix Inc.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1466>
2024-02-26 11:10:00 +00:00
Maksym Khomenko
da21dc853d webrtcsink: extensions: separate API and signal checks
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1469>
2024-02-20 19:29:46 +02:00
Maksym Khomenko
2228f882d8 webrtcsink: apply rustfmt
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1469>
2024-02-20 19:29:28 +02:00
Xavier Claessens
2572afbf15 janusvr: Add secret-key property
Every API calls have an optional "apisecret" argument.

Sponsored-by: Netflix Inc.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1465>
2024-02-16 14:04:59 +00:00
Jordan Yelloz
67b7cf9764 webrtcsink: Added sinkpad with "msid" property
This forwards to the webrtcbin sinkpad's msid when specified.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1442>
2024-02-12 15:04:44 +00:00
Sebastian Dröge
b2d5ee48cd Update to async-tungstenite 0.25
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1455>
2024-02-11 11:31:24 +02:00
Bilal Elmoussaoui
0615a16124 Use workspace features for crates metadata/deps
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1446>
2024-02-05 15:34:31 +01:00
Sebastian Dröge
91abc62ad0 webrtcsink: Fix new clippy warning
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1445>
2024-02-05 12:53:20 +02:00
Sebastian Dröge
1a55c70114 Switch git dependencies to explicitly name branch
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1445>
2024-02-05 12:51:36 +02:00
Sebastian Dröge
ffa830ae9b Update for GLib prelude re-organization
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1444>
2024-02-03 12:30:15 +02:00
Jordan Yelloz
311fda649f livekit_signaller: Added high-quality layer for video streams
This change addresses a cosmetic issue with livekit, where the
connection quality indicator seen by other users shows bad quality
unless the track is added with a high quality layer. The details of the
layer submitted aren't significant for this purpose.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1443>
2024-02-02 20:57:17 +00:00
Robert Ayrapetyan
916a8b959e doc: add http headers for webrtcsink signaller
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1419>
2024-02-01 19:31:58 +00:00
Robert Ayrapetyan
972b9e5474 doc: add docstrings for signaller object
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1419>
2024-02-01 19:31:58 +00:00
Robert Ayrapetyan
7a72b2fc25 webrtcsink-signalling: add headers support
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1419>
2024-02-01 19:31:58 +00:00
François Laignel
91bfd0f7c3 webrtc: signallers: attempt to close the ws when an error occurs
This commit discards the early error returns in the send tasks to log the error
and attempt to close the websocket.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1435>
2024-02-01 18:08:41 +01:00
François Laignel
f54d714afd webrtc: only use close() to close websockets
In the signaller clients and servers, the following sequence is used to close
the websocket (in the [send task]):

```rust
    ws_sink.send(WsMessage::Close(None)).await?;
    ws_sink.close().await?;
```

tungstenite's [`WebSocket::close()` doc] states:

> Calling this function is the same as calling `write(Message::Close(..))``

So we might think they are redundant and either could be used for this purpose
(`send()` calls `write()`, then `flush()`).

The result is actually is bit different as `write()` starts by checking the
state of the connection and [returns `SendAfterClosing`] if the socket is no
longer active, which is the case when a closing request has been received from
the peer via a [call to `do_close()`]). Note that `do_close()` also enqueues a
`Close` frame.

This behaviour is visible from the server's logs:

```
1. tungstenite::protocol: Received close frame: None
2. tungstenite::protocol: Replying to close with Frame { header: FrameHeader { .., opcode: Control(Close), .. }, payload: [] }
3. gst_plugin_webrtc_signalling::server: Received message Ok(Close(None))
4. gst_plugin_webrtc_signalling::server: connection closed: None this_id=cb13892f-b4d5-4d59-95e2-b3873a7bd319
5. remove_peer{peer_id="cb13892f-b4d5-4d59-95e2-b3873a7bd319"}: gst_plugin_webrtc_signalling::server: close time.busy=285µs time.idle=55.5µs
6. async_tungstenite: websocket start_send error: WebSocket protocol error: Sending after closing is not allowed
```

1: The server's websocket receives the peer's `Close(None)`.
2: `do_close()` enqueues a `Close` frame.
3: The incoming `Close(None)` is handled by the server.
4 & 5: perform session closing.
6: `ws_sink.send(WsMessage::Close(None))` attempts to `write()` while the ws
   is no longer active. The error causes an early return, which means that
   the enqueued `Close` frame is not flushed.

Depending on the peer's shutdown sequence, this can result in the following
error, which can bubble up as a `Message` on the application's bus:

```
ERROR: from element /GstPipeline:pipeline0/GstWebRTCSrc:webrtcsrc0: GStreamer encountered a general stream error.
Additional debug info:
net/webrtc/src/webrtcsrc/imp.rs(625): gstrswebrtc::webrtcsrc:👿:BaseWebRTCSrc::connect_signaller::{{closure}}::{{closure}} (): /GstPipeline:pipeline0/GstWebRTCSrc:webrtcsrc0:
Signalling error: Error receiving: WebSocket protocol error: Connection reset without closing handshake
```

On the other hand, [`close()` ensures the ws is active] before attempting to
write a `Close` frame. If it's not, it only flushes the stream.

Thus, when we want to be able to close the websocket and/or to honor the closing
handshake in response to the peer `Close` message, the `ws_sink.close()`
variant is preferable.

This can be verified in the resulting server's logs:

```
tungstenite::protocol: Received close frame: None
tungstenite::protocol: Replying to close with Frame { header: FrameHeader { is_final: true, rsv1: false, rsv2: false, rsv3: false, opcode: Control(Close), mask: None}, payload: [] }
gst_plugin_webrtc_signalling::server: Received message Ok(Close(None))
gst_plugin_webrtc_signalling::server: connection closed: None this_id=192ed7ff-3b9d-45c5-be66-872cbe67d190
remove_peer{peer_id="192ed7ff-3b9d-45c5-be66-872cbe67d190"}: gst_plugin_webrtc_signalling::server: close time.busy=22.7µs time.idle=37.4µs
tungstenite::protocol: Sending pong/close
```

We now get the notification `Sending pong/close` (the closing handshake) instead
of `websocket start_send error` from step 6 with previous variant.

The `Connection reset without closing handshake` was not observed after this
change.

[send task]: 63b568f4a0/net/webrtc/signalling/src/server/mod.rs (L165)
[`WebSocket::close()` doc]: https://docs.rs/tungstenite/0.21.0/tungstenite/protocol/struct.WebSocket.html#method.close
[returns `SendAfterClosing`]: 85463b264e/src/protocol/mod.rs (L437)
[call to `do_close()`]: 85463b264e/src/protocol/mod.rs (L601)
[`close()` ensures the ws is active]: 85463b264e/src/protocol/mod.rs (L531)

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1435>
2024-02-01 18:08:41 +01:00
Taruntej Kanakamalla
50e905fe4b webrtc: conditional compile for features with 1_22 dependency
Few features being used in webrtcsink like
the signal `request-aux-sender` are introduced
to webrtcbin in gstreamer release 1.22.

Rename the feature gst1_22 to v1_22 for uniformity.

Add v1_22 to default features.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1241>
2024-02-01 15:08:11 +05:30
Sebastian Dröge
4ad101b53b Use once_cell crate directly again
The glib crate does not depend on it anymore and also does not re-export
it anymore.

Also switch some usages of OnceCell to OnceLock from std.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1441>
2024-01-31 18:07:57 +02:00
Sebastian Dröge
451d928026 webrtc: Update AWS signaller to http 1
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1441>
2024-01-31 18:07:57 +02:00
Sebastian Dröge
764143d971 webrtc: Remove unnecessary manual Send+Sync implementations for signallers
These are automatically implemented.

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/483

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1432>
2024-01-18 10:01:25 +02:00
Sebastian Dröge
1af18f3028 webrtc: Require Send+Sync for signaller implementations
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1432>
2024-01-18 10:01:01 +02:00