Commit graph

851 commits

Author SHA1 Message Date
Mathieu Duponchelle
8ad882bed5 gstwebrtc-api: address issues raised by mix matrix support
1c48d7065d was mistakenly merged too
early, and there were concerns about the implementation and API design:

The fact that the frontend had to expose a text area specifically for
sending over a mix matrix, and had to manually edit in floats into the
stringified JSON was suboptimal.

Said text area was always present even when remote control was not
enabled.

The sendControlRequest API was made more complex than needed by
accepting an optional stringifier callback.

This patch addresses all those concerns:

The deserialization code in webrtcsink is now made more clever and
robust by first having it pick a numerical type to coerce to when
deserializing arrays with numbers, then making sure it doesn't allow
mixed types in arrays (or arrays of arrays as those too must share
the same inner value type).

The frontend side simply sends over strings wrapped with a request
message envelope to the backend.

The request text area is only shown when remote control is enabled.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1725>
2024-08-22 05:54:46 +00:00
Piotr Brzeziński
c4bcdea830 hlscmafsink: Add new-playlist signal
Allows you to switch output between folders without having to state change to READY to close the current playlist.
Closes the current playlist immediately and starts a new one at the currently set location.
Should be used after changing the relevant location properties.
Makes use of the send-headers signal in cmafmux.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1692>
2024-08-22 02:06:51 +00:00
Mathieu Duponchelle
5dc2d56c0e webrtcsink: store mids per-session instead of globally
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1730>
2024-08-21 21:20:40 +00:00
Mathieu Duponchelle
16ee51621e webrtcsink: fix segment format mismatch with remote offer
webrtcsink was starting the negotiation process on Ready and concurrently
moving the consumer pipeline to Playing, but when answering the remote
description was set so fast that input streams were connected (and the time
format set on appsrc) before the state change to Paused had completed.

This meant gst_base_src_start was happening after that and setting the format
back to bytes, the time segment that was next coming in then caused:

basesrc gstbasesrc.c:4255:gst_base_src_push_segment:<video_0> segment format mismatched, ignore

And the consumer pipeline errored out.

The same issue existed in theory when webrtcsink was creating the offer,
but was much harder to trigger as it required that the remote answer
came in before the state change to Paused had completed.

This commit fixes the issue by simply waiting for the state to have
changed to Paused before negotiating.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1730>
2024-08-21 21:20:40 +00:00
Piotr Brzeziński
b6406013c5 hlssink3: Fix racy test by separating events (signals) from bus messages
Was regularly failing on the CI. Bus messages are handled async here, so they need to be tracked separately.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1737>
2024-08-21 19:49:09 +00:00
Piotr Brzeziński
982a9a9aea hlssink3: Post hls-segment-added message
Posts a simple 'hls-segment-added' message with the segment location, start running time and duration.
With hlssink2, it was possible to catch 'splitmuxsink-fragment-closed', but since hlssink3 doesn't forward that message
(and hlscmafsink doesn't even use that mux), the new one was added to allow for listening for new fragments being added.

I extended the existing tests to check whether this message is posted correctly.
They theoretically only cover hlssink3, but hlscmafsink uses the same base class so it should be alright for now.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1677>
2024-08-20 18:32:59 +00:00
Sebastian Dröge
eb0a44fe67 ndisrc: Move timestamp handling from demuxer to source
This allows putting correct timestamps on buffers coming out of the
source already instead of leaving them unset until the demuxer.

And also calculate timestamps for metadata buffers.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1718>
2024-08-16 06:07:35 +00:00
Mathieu Duponchelle
1c48d7065d gstwebrtc-api example: add support for requesting mix matrix
This is one example of how a consumer might send over custom upstream
event requests to the producer.

As webrtcsink will deserialize numbers in priority as integers, we need
a custom stringifying function to ensure members of the matrix array are
indeed serialized with the floating point.

An optional stringifier parameter is thus added to the
sendControlRequest API.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1711>
2024-08-15 15:42:04 +00:00
Mathieu Duponchelle
01e28ddfe2 webrtcsink: implement generic data channel control mechanism ..
.. and deprecate data channel navigation in favor of it.

A new property, "enable-data-channel-control" is exposed, when set to
TRUE a control data channel is offered, over which can be sent typed
upstream events.

This means further upstream events will be usable, for now only
navigation and custom upstream events are handled.

In addition, send response messages to notify the consumer of whether
its requests have been handled.

In the future this can also be extended to allow the consumer to send
queries, or seek events ..

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1711>
2024-08-15 15:42:04 +00:00
Mathieu Duponchelle
0a6963f7ce gstwebrtc-api: example: use http by default
That way the webpage connects with ws:/ to the signaller.

Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/589
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1704>
2024-08-14 14:10:04 +00:00
Sebastian Dröge
102185d09d mpegtslivesrc: Handle PCR discontinuities as errors for now
More work is needed to make this work seemlessly and right now it would
simply cause invalid timestamps to be created.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1717>
2024-08-14 12:34:18 +00:00
Sebastian Dröge
ede82ca5b4 hlssink3: Don't use is-live=true
This sometimes produces imperfect timestamps that cause the fragment
duration to be slightly different than expected.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1716>
2024-08-14 13:05:40 +03:00
Sebastian Dröge
bc930122ba webrtcsrc: Make sure to always call end_session() without the state lock
This was already done in another place for the same reason: preventing a
deadlock. It's probably not correct as hinted by the FIXME comment but
better than deadlocking at least.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1701>
2024-08-13 06:04:09 +00:00
Mathieu Duponchelle
0da1c8e9c9 webrtcsink: fix assertions when finalizing
Dumping the pipeline on state changes from an async bus handler was
triggering criticals.

Instead, dump from the sync handler.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1706>
2024-08-12 09:13:06 +02:00
Sebastian Dröge
30a5987c9e rtp: mp4gpay: Don't set seqnum-base on the caps
This is supposed to be set by another layer, e.g. rtspsrc.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1693>
2024-08-10 08:06:40 +00:00
Sebastian Dröge
de42ae432c rtp: basepay: Fix off-by-one with seqnum-offset
Setting a seqnum-offset of 1 would've caused the first packet to have a
seqnum of 2 instead of 1.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1693>
2024-08-10 08:06:40 +00:00
Sebastian Dröge
c5163a73ee rtp: basepay: Don't negotiate twice in the beginning
If srcpad caps are already set as part of sinkpad caps handling, unset
the reconfigure flag so negotiation does not happen yet another time on
the first buffer.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1693>
2024-08-10 08:06:40 +00:00
Sebastian Dröge
31e836f4d6 rtp: basepay: Negotiate SSRC and PT with downstream if not set via property
This makes the new payloaders closer to the old ones, and makes usage in
webrtcbin easier.

Also properly configure default PT of subclasses. Previously any PT that
was set for these subclasses via g_object_new() would be overridden by
the default one during construction.

Additionally, do SSRC collision handling while queueing output packets.
This is the more natural place as that's where the SSRC is actually
used, it happens potentially earlier and also allows to drain any
pending packets before the SSRC change in the caps.

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/557

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1693>
2024-08-10 08:06:40 +00:00
Sebastian Dröge
914ffc8be9 rtp: basepay: Initialize class fields
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1693>
2024-08-10 08:06:40 +00:00
Sebastian Dröge
c554a5dc76 rtp: basepay: Don't unset stats on FlushStop
They are still valid and unsetting them here would cause no stats to
ever be updated again until the next state change.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1693>
2024-08-10 08:06:40 +00:00
Sebastian Dröge
035a199109 rtp: basepay: Don't use suggested SSRC on collissions if it's the current one
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1693>
2024-08-10 08:06:40 +00:00
Mathieu Duponchelle
9080c90120 net/webrtc: add support for answering to webrtcsink
Support was added to the base class when the AWS KVS signaller was
implemented, but the default signaller still only supported the case
where the producer was creating the offer.

Also extend the javascript API

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1702>
2024-08-09 14:02:48 +02:00
Mathieu Duponchelle
a9ff9615ff net/webrtc: correct signaller debug category
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1702>
2024-08-08 18:28:43 +02:00
Mathieu Duponchelle
64f0b76f71 webrtc: update README with section on embedded signalling / web services
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1671>
2024-08-08 16:40:46 +02:00
Mathieu Duponchelle
9455e09d9f webrtcsink: expose properties for running web server
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1671>
2024-08-08 16:40:46 +02:00
Mathieu Duponchelle
b709c56478 webrtcsink: expose properties for running signalling server
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1671>
2024-08-07 19:55:00 +02:00
Sebastian Dröge
6c04b59454 webrtcsrc: Don't hold the state lock while removing sessions
Removing a session can drop its bin and during release of the bin its
pads are removed, but the pad-removed handler is also taking the state
lock.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1695>
2024-08-07 09:35:15 +00:00
Sebastian Dröge
b83b6031e5 Update etherparse and async-tungstenite dependencies
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1691>
2024-08-06 09:00:32 +03:00
Dave Lucia
3a949db720 net/webrtc: Fix turn-servers nick: user -> use
Noticed this typo

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1690>
2024-08-05 12:38:51 -04:00
Sebastian Dröge
fa060b9fa0 Fix various 1.80 clippy warnings
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1688>
2024-08-05 14:14:17 +03:00
Mathieu Duponchelle
86039dd5c1 webrtc-api example: do not rely on webpack / npm proxying websocket
Instead simply use the desired address directly from the reference
example, this makes it work out of the box without placing expectations
on the web server.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1674>
2024-07-30 16:29:54 +00:00
Loïc Le Page
5a1d12419f gstwebrtc-api: always include index file in dist for convenience
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1670>
2024-07-17 08:27:31 +00:00
François Laignel
34b791ff5e webrtc: add raw payload support
This commit adds support for raw payloads such as L24 audio to `webrtcsink` &
`webrtcsrc`.

Most changes take place within the `Codec` helper structure:

* A `Codec` can now advertise a depayloader. This also ensures that a format
  not only can be decoded when necessary, but it can also be depayloaded in the
  first place.
* It is possible to declare raw `Codec`s, meaning that their caps are compatible
  with a payloader and a depayloader without the need for an encoder and decoder.
* Previous accessor `has_decoder` was renamed as `can_be_received` to account
  for codecs which can be handled by an available depayloader with or without
  the need for a decoder.
* New codecs were added for the following formats:
  * L24, L16, L8 audio.
  * RAW video.

The `webrtc-precise-sync` examples were updated to demonstrate streaming of raw
audio or video.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1501>
2024-07-16 19:32:02 +00:00
Sebastian Dröge
d4d02d70a8 rtp: Require bitstream-io < 2.4.0
Version 2.4.0 contains a breaking change that it shouldn't, and updating
to 2.4.0 requires a newer Rust version.

See https://github.com/tuffy/bitstream-io/issues/22
2024-07-16 19:13:49 +03:00
François Laignel
6e9855c36b webrtcsink: fix property types for rav1enc
Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/572
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1667>
2024-07-12 18:59:20 +02:00
Sanchayan Maity
12be9a24a6 net/quinn: Fix generation of self signed certificate
The certificate chain was incorrectly being passed the private key instead
of certificate. With rustls 0.23.11 version, this error was being caught
and reported. As stated in the 0.23.11 release, it has a new feature

"API for determining whether a CertifiedKey's certificate and private key
matches: keys_match(). This is called from existing fallible functions
that accept a private key and certificate (for example, with_single_cert())
so these functions now detect this misconfiguration."

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1666>
2024-07-12 12:26:54 +05:30
Sebastian Dröge
a8ccfe49d9 webrtc: Require livekit-protocol < 0.3.4 due to uncoordinated breaking changes
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1663>
2024-07-11 20:00:24 +03:00
Sebastian Dröge
98b28d69ce Update for new debug log macro syntax
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1658>
2024-07-08 11:25:23 +03:00
Sebastian Dröge
4123b5d1a1 mpegtslive: Update for gst::Clock::set_calibration() API changes
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1658>
2024-07-08 09:59:06 +03:00
Sanchayan Maity
2fe852166e aws/s3hlssink: Do not call abort before finishing uploads
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1653>
2024-07-06 14:44:08 +00:00
Sebastian Dröge
4ab8d92f28 mpegtslivesrc: Don't skip the first MPEG-TS packet
If every buffer contains only a single MPEG-TS packet we would otherwise
skip over everything and would never observe a PCR.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1651>
2024-07-04 17:01:43 +03:00
Philippe Normand
eee93aea52 rtp2: Fix typo on auto-header-extension property name
The rtp (de)pay elements use auto-header-extension so the new elements should do
the same.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1646>
2024-07-02 09:35:39 +01:00
Edward Hervey
95ae67752f net: New mpegtslive element
This element allows wrapping an existing live "mpeg-ts source" (udpsrc,
srtsrc,...) and providing a clock based on the actual PCR of the stream.

Combined with `tsdemux ignore-pcr=True` downstream of it, this allows playing
back the content at the same rate as the (remote) provider **and** not modify
the original timestamps.

Co-authored-by: Sebastian Dröge <slomo@coaxion.net>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1640>
2024-07-01 15:29:22 +02:00
leonardo salvatore
f303992e0c webrtcsink: initial support for vpuenc_h264 encoder for imx8mp, default values set to cover a common streaming scenario
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1639>
2024-07-01 07:34:04 +00:00
Guillaume Desmottes
a10577b42c aws: log error if sink failed to start
I find it confusing that the element was failing without reporting any
error in its logs.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1638>
2024-06-26 11:22:54 +02:00
Guillaume Desmottes
0ecbd3f953 aws: use DisplayErrorContext when displaying SDK errors
As suggested in the aws crate documentation, wrap SDK errors with
DisplayErrorContext so their Display implementation outputs the full
context.

Improve error display from "dispatch failure" to

"dispatch failure: io error: error trying to connect: dns error: failed
to lookup address information: Name or service not known: dns error:
failed to lookup address information: Name or service not known: failed
to lookup address information: Name or service not known
(DispatchFailure(DispatchFailure { source: ConnectorError { kind: Io,
source: hyper::Error(Connect, ConnectError(\"dns error\", Custom { kind:
Uncategorized, error: \"failed to lookup address information: Name or
service not known\" })), connection: Unknown } }))"

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1638>
2024-06-26 10:47:10 +02:00
Guillaume Desmottes
3b7b2cd37b aws: rely on WaitError Display implementation
The Display implementation of WaitError already displays the underlying
SDK error and the metadata, so can just use that.

Will also be used to provide more context in the next patch.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1638>
2024-06-26 10:46:46 +02:00
Sanchayan Maity
0bd98e2c34 net/quinn: Allow dropping buffers when buffer size exceeds maximum datagram size
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1613>
2024-06-25 20:15:40 +05:30
Sanchayan Maity
e00ebca63f net/quinn: Add stats property for connection statistics
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1613>
2024-06-25 20:15:40 +05:30
Sanchayan Maity
2b35f009fb net/quinn: Update quinn to 0.11.2
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1613>
2024-06-25 20:15:40 +05:30