- generate a new session id for every new client
use the session id in the resource url
- remove the producer-peer-id property in the WhipServer signaler as it
is redundant to have producer id in a session having only one producer
- read the 'producer-peer-id' property on the signaller conditionally
if it exists else use the session id as producer id
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1339>
- Add a new structure Session
- manage each producer using a session
- avoid send EOS when a session terminates, instead keep running
waiting for any new producer to connect
- Maintain a bin element per session
- each session bin encapsulates webrtcbin and the decoder if needed
as well as the parser and filter if requested by the application
(through request-encoded-filter)
- this will be helpful to cleanup the session's respective elements
when the corresponding producer terminates the session
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1339>
* Update "translation-languages" property to include G_PARAM_CONSTRUCT
so that it can be applied to initial state.
* Change default "translation-languages" value to be None instead of
cea608 specific one. Transcriberbin will be able to configure initia
state depending on selected mux method if "translation-languages" is
unspecified.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1589>
There is now two elements, rtpsend and rtprecv that represent the two
halves of a rtpsession. This avoids the potential pipeline loop if two
peers are sending/receiving data towards each other. The two halves can
be connected by setting the rtp-id property on each element to the same
value and they will behave like a combined rtpbin-like element.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1426>
Can receive and recevie one or more RTP sessions containing multiple
pt/ssrc combinations.
Demultiplexing happens internally instead of relying on separate
elements.
Co-Authored-By: François Laignel <francois@centricular.com>
Co-Authored-By: Mathieu Duponchelle <mathieu@centricular.com>
Co-Authored-By: Sebastian Dröge <sebastian@centricular.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1426>
Add to awss3sink and awss3putobjectsink elements the following
paramerters which are set on the uploaded S3 objects:
* cache-control;
* content-encoding; and
* content-language
Bugfix: Set the content-type and content-disposition values in the S3
putobject call. Previously the params were defined on the element but
unused.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1585>
Unit tests specify a 0-based offset, so printing that plus the
random initial offset on failure is just needlessly confusing,
so subtract the initial offset when printing expected/actual
values. The real values are still printed as part of the assert.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1580>
This commit adds an Android `webrtcsrc` based example with the following
features:
* A first view allows retrieving the producer list from the signaller (peer ids
are uuids which are too long to tap, especially using an onscreen keyboard).
* Selecting a producer opens a second view. The first available video stream is
rendered on a native Surface. All the audio streams are rendered using
`autoaudiosink`.
Available Settings:
* Signaller URI.
* A toggle to prefer hardware decoding for OPUS, otherwise the app defaults to
raising `opusdec`'s rank. Hardware decoding was moved aside since it was found
to crash the app on all tested devices (2 smartphones, 1 tv).
**Warning**: in order to ease testing, this demonstration application enables
unencrypted network communication. See `AndroidManifest.xml`.
The application uses the technologies currenlty proposed by Android Studio when
creating a new project:
* Kotlin as the default language, which is fully interoperable with Java and
uses the same SDK.
* gradle 8.6.
* kotlin dialect for gradle. The structure is mostly the same as the previously
preferred dialect, for which examples can be found online readily.
* However, JNI code generation still uses Makefiles (instead of CMake) due to
the need to call [`gstreamer-1.0.mk`] for `gstreamer_android` generation.
Note: on-going work on that front:
- https://gitlab.freedesktop.org/gstreamer/cerbero/-/merge_requests/1466
- https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6794
Current limitations:
* x86 support is currently discarded as `gstreamer_android` libs generation
fails (observed with `gstreamer-1.0-android-universal-1.24.3`).
* A selector could be added to let the user chose the video streams and
possibly decide whether to render all audio streams or just select one.
Nice to have:
* Support for the synchronization features of the `webrtc-precise-sync-recv`
example (NTP clock, RFC 7273).
* It could be nice to use Rust for the specific native code.
[`gstreamer-1.0.mk`]: https://gitlab.freedesktop.org/gstreamer/cerbero/-/blob/main/data/ndk-build/gstreamer-1.0.mk
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1578>
Only configure header extensions from the source pad caps if they exist
already in the source pad caps, otherwise the configuration will fail.
Extensions that are added via the signals might not exist in the source
pad caps yet and would be added later.
Also, if configuring an existing extension from the new caps fails,
remove it and try to request a new extension for it.
Additionally don't remove extensions from the caps that can't be
provided. No header extensions for them would be added to the packets,
but that's not a problem. Removing them on the other hand would cause
negotiation to fail. This only affects extensions that are already
included in the caps.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1577>
If configuring an existing extension from the new caps fails, remove it
and try to request a new extension for it.
Also remove all extensions from the list that are not provided in the
caps, instead of passing RTP packets to all of them anyway.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1577>
Otherwise when the segment durations aren't as clean cut as in the
example, multiple segments with the exact same duration in milliseconds
will get output, even though they could have been repeated.
Fix this so that people copying this code don't encounter the bug.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1574>
In our tests, the slope (found with linear regression) on a
history of the (smoothed) accumulated inter-group delays
gives a more stable congestion control. In particular,
low-end devices becomes less sensitive to spikes in
inter-group delay measurements.
This flavour of delay based bandwidth estimation with Google
Congestion Control is also what Chromium is using.
To make it easy to experiment with the new estimator, as
well as add support for new ones in the future, also add
infrastructure for making delay estimator flavour selectable
at runtime.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1566>
Rust targets without support for `AtomicU64` is still
somewhat common. Running
git grep -i 'max_atomic_width: Some(32)' | wc -l
in the Rust compiler repo currently counts to 34 targets.
Change the `RtpBasePay2::ssrc_collision` from `AtomicU64` to
`Mutex<Option<u32>>`. This way we keep support for these
targets.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1562>
While monitoring and debugging rtpgccbwe, it is very helpful
to get continuous values of what it considers the effective
bitrate. Right now such prints will stop coming once the
algorithm stabilizes. Print it in more places so it keeps
coming. Use the same format to make it simpler to extract
the values by parsing the logs.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1567>
When debugging rtpgccbwe it is helpful to know if it is
delay based or loss based band-width estimation that puts a
bound on the current target bitrate, so add logs for that.
To minimize the time we need to hold the state lock, perform
the logging after we have released the state lock.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1561>
This set of changes implements the below fixes:
- Allow certificates to be specified for client/quicsink
- Secure connection being true on server/quicsrc and false on
client/quicsink still resulted in a successful connection
instead of server rejecting the connection
- Using secure connection with ALPN was not working
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1036>
If force-aspect-ratio=false then make sure to fully fill the given
width/height with the video frame and avoid rounding errors. This makes
sure that the video is rendered in the exact position selected by the
caller and that graphics offloading is going to work more likely.
In other cases and for all overlays, make sure that the calculated
positions are staying inside (0, 0, width, height) as rendering outside
is not allowed by GTK.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1547>
In some situations, a translated alternate audio stream for a content
might be available.
Instead of going through transcription and translation of the original
audio stream, it may be preferrable for accuracy purposes to simply
transcribe the secondary audio stream.
This MR adds support for doing just that:
* Secondary audio sink pads can be requested as "sink_audio_%u"
* Sometimes audio source pads are added at that point to pass through
the audio, as "src_audio_%u"
* The main transcription bin now contains per-input stream transcription
bins. Those can be individually controlled through properties on the
sink pads, for instance translation-languages can be dynamically set
per audio stream
* Some properties that originally existed on the main element still
remain, but are now simply mapped to the always audio sink pad
* Releasing of secondary sink pads is nominally implemented, but not
tested in states other than NULL
An example launch line for this would be:
```
$ gst-launch-1.0 transcriberbin name=transcriberbin latency=8000 accumulate-time=0 \
cc-caps="closedcaption/x-cea-708, format=cc_data" sink_audio_0::language-code="es-US" \
sink_audio_0::translation-languages="languages, transcript=cc3"
uridecodebin uri=file:///home/meh/Music/chaplin.mkv name=d
d. ! videoconvert ! transcriberbin.sink_video
d. ! clocksync ! audioconvert ! transcriberbin.sink_audio
transcriberbin.src_video ! cea608overlay field=1 ! videoconvert ! autovideosink \
transcriberbin.src_audio ! audioconvert ! fakesink \
uridecodebin uri=file:///home/meh/Music/chaplin-spanish.webm name=d2 \
d2. ! audioconvert ! transcriberbin.sink_audio_0 \
transcriberbin.src_audio_0 ! fakesink
```
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1546>
If a user constrained the supported CAPS, for instance using `video-caps`:
```shell
gst-launch-1.0 videotestsrc ! video/x-raw,format=I420 ! x264enc \
! webrtcsink video-caps=video/x-vp8
```
... a panic would occur which was internally caught without the user being
informed except for the following message which was written to stderr:
> thread 'tokio-runtime-worker' panicked at net/webrtc/src/webrtcsink/imp.rs:3533:22:
> expected audio or video raw caps: video/x-h264, [...] <br>
> note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
The pipeline kept running.
This commit converts the panic into an `Error` which bubbles up as an element
`StreamError::CodecNotFound` which can be handled by the application.
With the above `gst-launch`, this terminates the pipeline with:
> [...] ERROR webrtcsink net/webrtc/src/webrtcsink/imp.rs:3771:gstrswebrtc::
> webrtcsink:👿:BaseWebRTCSink::start_stream_discovery_if_needed::{{closure}}:<webrtcsink0>
> Error running discovery: Unsupported caps: video/x-h264, [...] <br>
> ERROR: from element /GstPipeline:pipeline0/GstWebRTCSink:webrtcsink0:
> There is no codec present that can handle the stream's type. <br>
> Additional debug info: <br>
> net/webrtc/src/webrtcsink/imp.rs(3772): gstrswebrtc::webrtcsink:👿:BaseWebRTCSink::
> start_stream_discovery_if_needed::{{closure}} (): /GstPipeline:pipeline0/GstWebRTCSink:webrtcsink0:
> Failed to look up output caps: Unsupported caps: video/x-h264, [...] <br>
> Execution ended after 0:00:00.055716661 <br>
> Setting pipeline to NULL ...
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1540>
Clippy caught the missing feature `signal` which is used by the WebRTC precise
synchronization examples. When running `cargo` `check`, `build` or `clippy`
without `no-default-dependencies`, this feature was already present due to
dependents crates.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1541>
When swapping between several development branches, compilation times can be
frustrating. This commit proposes adding features to control which signaller
to include when building the webrtc plugin. By default, all signallers are
included, just like before.
Compiling the `webrtc-precise-sync` examples with `--no-default-features`
reduces compilation to 267 crates instead of 429 when all signallers are
compiled in.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1539>
This commit implements [RFC 7273] (NTP & PTP clock signalling & synchronization)
for `webrtcsink` by adding the "ts-refclk" & "mediaclk" SDP media attributes to
identify the clock. These attributes are handled by `rtpjitterbuffer` on the
consumer side. They MUST be part of the SDP offer.
When used with an NTP or PTP clock, "mediaclk" indicates the RTP offset at the
clock's origin. Because the payloaders are not instantiated when the offer is
sent to the consumer, the RTP offset is set to 0 and the payloader
`timstamp-offset`s are set accordingly when they are created.
The `webrtc-precise-sync` examples were updated to be able to start with an NTP
(default), a PTP or the system clock (on the receiver only). The rtp jitter
buffer will synchronize with the clock signalled in the SDP offer provided the
sender is started with `--do-clock-signalling` & the receiver with
`--expect-clock-signalling`.
[RFC 7273]: https://datatracker.ietf.org/doc/html/rfc7273
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1500>
When an encoder was not supported by the `VideoEncoder` `bitrate` accessors, an
`unimplemented` panic would occur which would poison `state` & `settings`
`Mutex`s resulting in other threads panicking, notably entering `end_session()`,
which lead to many failures in `BinImplExt::parent_remove_element()` until a
segmentation fault ended the process. This was observed using `vaapivp9enc`.
This commit logs a warning if an encoder isn't supported by the `bitrate`
accessors and silently by-passes `bitrate`-related operations when unsupported.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1534>
Otherwise the clock id will simply be overridden instead of unscheduling
it, and if the streaming thread of the source pad currently waits on it
then it will wait potentially for a very long time and deactivating the
pad would wait for that to happen.
Also unschedule the clock id on `Drop` of the state to be one the safe
side and not simply forget about it.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1526>
Add `self.increasing_duration` and `self.increasing_counter`
to logs to provide more details of why `overuse_filter()`
determines overuse of network.
To get access to the latest values of those fields we need
to move down the log call. But that is fine, since no other
logged data is modified between the old and new location of
`gst::log!()`.
We do not bother logging `self.last_overuse_estimate` since
that is simply the previously logged value of `estimate`. We
must put the log call before we write the latest value to it
though, in case we want to log it in the future.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1522>
Quoting [`BehaviorVersion` documentation]:
> Over time, new best-practice behaviors are introduced. However, these
> behaviors might not be backwards compatible. For example, a change which
> introduces new default timeouts or a new retry-mode for all operations might
> be the ideal behavior but could break existing applications.
This commit uses `BehaviorVersion::v2023_11_09()`, which is the latest
major version at the moment. When a new major version is released, the method
will be deprecated, which will warn us of the new version and let us decide
when to upgrade, after any changes if required. This is safer that using
`latest()` which would silently use a different major version, possibly
breaking existing code.
[`BehaviorVersion` documentation]: https://docs.rs/aws-config/1.1.8/aws_config/struct.BehaviorVersion.html
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1520>
The following error is logged when `webrtcsink` is feeded with an audio stream:
> ERROR video-info video-info.c:540:gst_video_info_from_caps:
> wrong name 'audio/x-raw', expected video/ or image/
This commit bypasses `VideoInfo::from_caps` for audio streams.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1511>
Some elements in the RTP stack assume all buffers in a `gst::BufferList`
correspond to the same timestamp. See in [`rtpsession`] for instance.
This also had the effect that `rtpsession` did not create correct RTCP as it
only saw some of the SSRCs in the stream.
`rtpgccbwe` formed a packet group by gathering buffers in a `gst::BufferList`,
regardless of whether they corresponded to the same timestamp, which broke
synchronization under certain circonstances.
This commit makes `rtpgccbwe` push the buffers as they were received: one by one.
[`rtpsession`]: bc858976db/subprojects/gst-plugins-good/gst/rtpmanager/gstrtpsession.c (L2462)
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1502>
The goal is to be able to get back the original buffer
after performing analysis on a transformed version. Then put the
various GstMeta back on the original buffer.
An example pipeline would be
.. ! originalbuffersave ! videoscale ! analysis ! originalbufferestore ! draw_overlay ! sink
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1428>
Instead of exposing all ids properties as strings, we now have two
signaller implementations exposing those properties using their actual
type. This API is more natural and save the element and application
conversions when using numerical ids (Janus's default).
I also removed the 'joined-id' property as it's actually the same id as
'feed-id'. I think it would be better to have a 'janus-state' property or
something like that for applications willing to know when the room has
been joined.
This id is also no longer generated by the element by default, as Janus
will take care of generating one if not provided.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1486>
sccparse holds last timecode in order to ignore invalid timecode
and fallback to the previous timecode. That should happen
when sccparse is handling seek event too. Otherwise single invalid
timecode before the target seek position will cause flow error.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1485>
Without sending a Leave request to the server before disconnecting, the
disconnected client will appear present and stuck in the room for a little
while until the server removes it due to inactivity.
After this change, the disconnecting client will immediately leave the room.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1482>
It may be necessary for some signalling clients but the source element
doesn't need to depend on it.
Also, the value will fall back to the pad's MSID for the first argument
to the request-encoded-filter gobject signal when it isn't available
from the signalling client.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1477>
GTK 4.14 comes with a new GL renderer that does not support GL shader
nodes anymore, so the conversion from non-premultiplied alpha to
premultiplied alpha has to happen differently.
For GTK 4.14 or newer we use the correct format directly when building the
texture, but only if a GLES3+ context is used. In that case the NGL renderer is
used by GTK, which supports non-premultiplied formats correctly and fast.
For GTK 4.10-4.12, or 4.14 and newer if a GLES2 context is used, we use a
self-mask to pre-multiply the alpha.
For GTK before 4.10, we use a GL shader and hope that it works.
Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/488
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1452>
Don't use connect(), since that is incompatible with multicast.
Instead, drop received packets that are from senders we do not want.
Also set multicast loopback = false so we don't receive RTCP RRs from
ourselves and interpret them as RTCP SRs.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1425>
If we only send a single Transport in the Transports header, then the
server is allowed to omit it in the response. This has some strange
consequences for UDP transport: specifically, we have no idea what
addr/port we will get the packets from.
In those cases, we connect() on the socket when we receive the first
packet, so we can send RTCP RRs, and also so we can ensure that we
ignore data from other addresses.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1425>
GST_PLUGIN_FEATURE_RANK=rtspsrc2:1 gst-play-1.0 [URI]
Features:
* Live streaming N audio and N video
- With RTCP-based A/V sync
* Lower transports: TCP, UDP, UDP-Multicast
* RTP, RTCP SR, RTCP RR
* OPTIONS DESCRIBE SETUP PLAY TEARDOWN
* Custom UDP socket management, does not use udpsrc/udpsink
* Supports both rtpbin and the rtpbin2 rust rewrite
- Set USE_RTPBIN2=1 to use rtpbin2 (needs other MRs)
* Properties:
- protocols selection and priority (NEW!)
- location supports rtsp[ut]://
- port-start instead of port-range
Co-Authored-by: Tim-Philipp Müller <tim@centricular.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1425>
This change addresses a cosmetic issue with livekit, where the
connection quality indicator seen by other users shows bad quality
unless the track is added with a high quality layer. The details of the
layer submitted aren't significant for this purpose.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1443>
In the signaller clients and servers, the following sequence is used to close
the websocket (in the [send task]):
```rust
ws_sink.send(WsMessage::Close(None)).await?;
ws_sink.close().await?;
```
tungstenite's [`WebSocket::close()` doc] states:
> Calling this function is the same as calling `write(Message::Close(..))``
So we might think they are redundant and either could be used for this purpose
(`send()` calls `write()`, then `flush()`).
The result is actually is bit different as `write()` starts by checking the
state of the connection and [returns `SendAfterClosing`] if the socket is no
longer active, which is the case when a closing request has been received from
the peer via a [call to `do_close()`]). Note that `do_close()` also enqueues a
`Close` frame.
This behaviour is visible from the server's logs:
```
1. tungstenite::protocol: Received close frame: None
2. tungstenite::protocol: Replying to close with Frame { header: FrameHeader { .., opcode: Control(Close), .. }, payload: [] }
3. gst_plugin_webrtc_signalling::server: Received message Ok(Close(None))
4. gst_plugin_webrtc_signalling::server: connection closed: None this_id=cb13892f-b4d5-4d59-95e2-b3873a7bd319
5. remove_peer{peer_id="cb13892f-b4d5-4d59-95e2-b3873a7bd319"}: gst_plugin_webrtc_signalling::server: close time.busy=285µs time.idle=55.5µs
6. async_tungstenite: websocket start_send error: WebSocket protocol error: Sending after closing is not allowed
```
1: The server's websocket receives the peer's `Close(None)`.
2: `do_close()` enqueues a `Close` frame.
3: The incoming `Close(None)` is handled by the server.
4 & 5: perform session closing.
6: `ws_sink.send(WsMessage::Close(None))` attempts to `write()` while the ws
is no longer active. The error causes an early return, which means that
the enqueued `Close` frame is not flushed.
Depending on the peer's shutdown sequence, this can result in the following
error, which can bubble up as a `Message` on the application's bus:
```
ERROR: from element /GstPipeline:pipeline0/GstWebRTCSrc:webrtcsrc0: GStreamer encountered a general stream error.
Additional debug info:
net/webrtc/src/webrtcsrc/imp.rs(625): gstrswebrtc::webrtcsrc:👿:BaseWebRTCSrc::connect_signaller::{{closure}}::{{closure}} (): /GstPipeline:pipeline0/GstWebRTCSrc:webrtcsrc0:
Signalling error: Error receiving: WebSocket protocol error: Connection reset without closing handshake
```
On the other hand, [`close()` ensures the ws is active] before attempting to
write a `Close` frame. If it's not, it only flushes the stream.
Thus, when we want to be able to close the websocket and/or to honor the closing
handshake in response to the peer `Close` message, the `ws_sink.close()`
variant is preferable.
This can be verified in the resulting server's logs:
```
tungstenite::protocol: Received close frame: None
tungstenite::protocol: Replying to close with Frame { header: FrameHeader { is_final: true, rsv1: false, rsv2: false, rsv3: false, opcode: Control(Close), mask: None}, payload: [] }
gst_plugin_webrtc_signalling::server: Received message Ok(Close(None))
gst_plugin_webrtc_signalling::server: connection closed: None this_id=192ed7ff-3b9d-45c5-be66-872cbe67d190
remove_peer{peer_id="192ed7ff-3b9d-45c5-be66-872cbe67d190"}: gst_plugin_webrtc_signalling::server: close time.busy=22.7µs time.idle=37.4µs
tungstenite::protocol: Sending pong/close
```
We now get the notification `Sending pong/close` (the closing handshake) instead
of `websocket start_send error` from step 6 with previous variant.
The `Connection reset without closing handshake` was not observed after this
change.
[send task]: 63b568f4a0/net/webrtc/signalling/src/server/mod.rs (L165)
[`WebSocket::close()` doc]: https://docs.rs/tungstenite/0.21.0/tungstenite/protocol/struct.WebSocket.html#method.close
[returns `SendAfterClosing`]: 85463b264e/src/protocol/mod.rs (L437)
[call to `do_close()`]: 85463b264e/src/protocol/mod.rs (L601)
[`close()` ensures the ws is active]: 85463b264e/src/protocol/mod.rs (L531)
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1435>
When accumulate-time is non-zero, we need to drain our accumulated
text once the threshold is reached.
Implement support for gaps the simplest way, by transforming it into
an empty buffer and chaining it through ourself.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1436>
Only the examples of the fallbackswitch, livesync, and togglerecord
plugins require the gtk, gio, and gst-plugin-gtk4 features. The plugins
themselves don't actually have a dependency on GTK.
Only add the features (and examples) if the examples are actually
enabled to allow building these plugins without the GTK dependency.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1438>
We were setting audio and video caps by default even when the user
might have requested only video or audio. This would then result
in a `Could not reuse transceiver` error from the webrtcbin.
Fix this by allowing the user to specify audio or video caps as
None. This allows us to maintain the earlier behaviour for backward
compatibility while allowing the user to not request audio or video
as need be.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1433>
When streaming small amounts of data, using awss3sink might not be a
good idea, as we need to accumulate at least 5 MB of data for a
multipart upload (or we flush on EOS).
The alternative, while inefficient, is to do a complete PutObject of
_all_ the data periodically so as to not lose data in case of a pipeline
failure. This element makes a start on this idea by doing a PutObject
for every buffer.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1337>
webrtcbin will refuse pad requests for all sorts of reasons, and should
be logging an error when doing so, simply post an error message and let
the application deal with it, the reason for the refusal should
hopefully be available in the logs to the user.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1399>
Implement new signaller WhipServerSignaller
- an http server using 'warp'
- handlers for the POST, OPTIONS, PATCH and DELETE
- fixed path `/whip/endpoint` as the URI
- fixed value 'whip-client' as the producer peer id
- fixed resource url `/whip/resource/whip-client`
Derive whipserversrc element from BaseWebRTCSrc
- implement constructed method for ObjectImpl to set
non-default signaller, i.e., WhipServerSignaller
- bind the properties stun-server and turn-servers to those on
the Signaller
Connect to 'webrtcbin-ready' signal in the constructor of WhipServerSignaller
- it will be emitted by the webrtcsrc when the webrtcbin element is ready
- the closure for this signal will in turn connect to webrtcbin's ice-gathering-state
and perform send with the answer sdp via the channel
- the WhipServer will hold its HTTP response in POST handler until this signal
is received or timeout which happens early
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1284>
add a new signal webrtcbin-ready in this place doing same
thing but can be used for both consumers and producers
Please note this change is only to the consumer-added
signal on the signaller interface.
The consumer-added signal on the webrtcsink is unchanged
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1284>
Also move processing from the capture thread to the streaming thread.
The NDI SDK can cause frame drops if not reading fast enough from it.
All frame processing is now handled inside the ndisrcdemux.
Also use a buffer pool for video if copying is necessary.
Additionally, make sure to use different stream ids in the stream-start
event for the audio and video pad.
This plugin now requires GStreamer 1.16 or newer.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1365>
Make it independent of the `latency`; this was inconsistent anyway,
where the default latency of zero got you a fallback duration of 100 ms
and something else got you half the latency.
Maintain a separate duration for the `in` and the `out` side so we
change the duration of repeat buffers after a caps change, not just
before.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1372>
The debug print of the event does not display details about the segment:
Unqueueing Some(Event(Event { ptr: 0x7fa3e0002580, type: "segment", seqnum: Seqnum(479), structure: Some(GstEventSegment { segment: (GstSegment) ((GstSegment*) 0x7fa3e8001d00) }) }))
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1370>
This makes the chain function almost independent of the output state. We
still do the early discard check with `buffer_is_backwards` so we don't
try to queue buffers we can't use, allowing us to fast-forward upstream
without blocking on the src task.
Don't accept `LateOverThreshold` buffers when we have `pending_caps` or
a `pending_segment`. We need to apply these first before we can sensibly
patch buffers from the new stream.
Deduplicate most of the output buffer patching code into a new
`patch_output_buffer` method.
For: https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/450
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1369>
- Separate resetting state more cleanly, introducing `set_flushing`,
`sink_reset` and `src_reset`.
- Clear the queue early when we flush, in order to unblock waits on
query responses.
- Return an error when we fail to start, pause or stop the task.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1369>
- An entirely missing duration is now only logged at debug level instead
of pretending the duration was zero and warning about it.
- Silently fix up a duration difference up to one sample.
- Error when we fail to calculate the duration; don't try to apply the
`fallback_duration` to a non-video stream.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1369>
If a packet is starting with a leading fragment but we do not expect to
receive one, then skip over it to the next OBU.
Not doing so would cause parsing of the middle of an OBU, which would
most likely fail and cause unnecessary warning messages about a
corrupted stream.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1367>
The "signaller" property used to be defined as MUTABLE_READY which meant that
the property was always set after `constructed()` was called.
Since `connect_signaller()` was called from `constructed()`, only the default
signaller was used.
This commit sets the "signaller" property as CONSTRUCT_ONLY. Using a builder,
this property will now be set before the call to `constructed()`.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1324>
During `on_remote_description_set()` processing, current session is removed
from the sessions `HashMap`. If an ice candidate is submitted to `handle_ice()`
by that time, the session can't be found and the candidate is ignored.
This commit wraps the Session in the sessions `HashMap` so an entry is kept
while `on_remote_description_set()` is running. Incoming candidates received by
`handle_ice()` will be processed immediately or enqueued and handled when the
session is restored by `on_remote_description_set()`.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1325>
The NDI closed captions specifications [1] define a variation where metadata is
attached to the video frame. This requires the AFD buffer to be v210 encoded.
This commit applies this strategy.
Another difference with previous version is that when an error occurs while
encoding or decoding a meta, next meta are also tried instead of failing
immediately.
Receiving closed captions as a standalone metadata is kept for interoperability
purposes. In this case, metadata is also expected to be v210 encoded.
[1]: http://www.sienna-tv.com/ndi/ndiclosedcaptions.html
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1356>
@to-owned increases refcount of the element, which prevents the object from proper destruction, as the initial refcount with ElementFactory::make is larger than 1.
Instead, use @watch to create a weak reference and unbind the closure automatically if the object gets destroyed
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1355>
* Simplify state/playlist management
* Fix a bug that segment is not deleted if location contains directory
and playlist-root is unset
* Split playlist update routine into two steps, adding segment
to playlist and playlist write
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1306>
This is required to take care of clock skew between
system time and pipeline time.
`track-pipeline-clock-for-pdt: true` mean utd time is
sampled for first segment and for subsequent segments
keep adding the time based on pipeline clock. difference
of segment duration and PDT time will match.
track-pipeline-clock-for-pdt: false` mean utd time is
sampled for each segment. system time may jump forward
or backward based on adjustments. If application needs
to synchronization of external events `false` is
recommended.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1145>
- connect to `format-location-full` it provide the first
sample of the fragment. preserve the running-time of the
first sample in fragment.
- on fragment-close message, find the mapping of running-time
to UTC time.
- on each subsequent fragment, calculate the offset of the
running-time with first fragment and add offset to base
utc time
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1145>
GTK expects GL textures to have premultiplied alpha. The ones we get
from GStreamer don't, leading to incorrect rendering of semitransparent
frames.
GTK 4.12 gained an API to set a different GL texture format, but it
won't help for older GTK versions. Plus, at the time of writing, it
causes a very slow download/upload path in GTK.
So, use a GTK GL shader node to premultiply the alpha without leaving
the GPU.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1312>
`quick-xml::reader::Reader::trim_text(true)` doesn't remove white spaces and
tabs from XML text. Besides, for interoperability robustness we also need to
remove carriage returns and line feeds.
Also improve the default capacities for the `SmallVec`s.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1321>
Since ab1ec126983f949804684e11e0e58c7cf3b22bc4:
webrtcsink: Add support for pre encoded streams
Discovery pipelines for remote offers were no longer fed any buffers.
While some encoders could already produce caps with no input buffers,
others, such as x264enc, simply hung forever. This resulted in no answer
getting produced if for instance video-caps were constrained to H264.
Fix this by tracking discovery pipelines at the State rather than the
InputStream level, removing the useless distinction of Initial vs.
CodecSelection discoveries, and always feeding all the current
discovery pipelines with incoming buffers.
For reference, the issue here was that codec selection discoveries were
assigned to local clones of InputStreams, not tracked anywhere, and thus
not iterated for discoveries when queuing incoming buffers from the
chain function, as it only looked at the original instance of
InputStream's in state.streams.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1319>
This fixes most, but not all, of the build errors in Windows when using
static libraries.
The ones remaining are:
- redirection of gstreamer-1.0 towards gstreamer-full-1.0
- Cairo not exporting the C++ stdlib requirement when built statically
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1307>
Previously, there was no check performed on features of plugins if these
specify GStreamer plugins. This commit adds that, and ensures that the
plugins and pkg-config targets are skipped if no outputs are to be
generated (this is already done for examples).
Closes#369
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1303>
Previously, there was no check performed on features of plugins if these
specify GStreamer plugins. This commit adds that, and ensures that the
plugins and pkg-config targets are skipped if no outputs are to be
generated (this is already done for examples).
Closes#369
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1303>
This new plugin exposes two elements, intersink and intersrc. These act
as wormholes for data in the same process and can be used to forward
data from one pipeline to another.
The implementation makes use of gstreamer-utils' StreamProducer, and
supports dynamically adding and removing consumers, before and after
producers, and changing producer names while PLAYING, both on the sink
and the src.
This initial implementation comes with a small demo, and a few tests.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1257>
This provides support GstNavigation events handling in webrtcsrc so that
a GStreamer client can be used to control remotely a GStreamer server,
similar to how the web client is capable of controlling a wpesrc.
This is part of a larger set of patches that require more work on the
sinks and sources.
server: d3d11screencapturesrc ! webrtcsink enable-data-channel-navigation=true
client: webrtcsrc enable-data-channel-navigation=true ! d3d11videosink
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1281>
When starting a webrtcsrc-signaller client in Listener mode, only the producers
started after the client connection were advertised. All currently
running producers were ignored unlike the gstwebrtc-api behavior. This
commit now lists all running producers when the client Listener connects
and advertises them through the "producer-added" signal.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1296>
Commit 08b6251a added the check to ensure only one canceller at a time for net/webrtc.
In `whipsink` and since `whipwebrtcsink` picked up the same implementation, there exists a
bug around the use of canceller. `whipsink` calls `wait_async` while passing the canceller
as an argument. The path `send_offer -> do_post -> parse_endpoint_response` results in the
canceller being replaced in each subsequent call to `wait_async`. Since `wait_async` call
does not ensure one canceller, with the async call the use of canceller/abort was subtly
broken. Similarly, for `whepsrc`.
We really don't need to use `wait_async` inside `do_post` for any `await` calls. If the
root future viz. `do_post` with `wait_async` is aborted, the child futures will be taken
care of.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1290>
The "encoder-setup" signal must also be emitted for the encoders
used in discovery pipelines in order for the default settings to
be applied.
This otherwise meant that for instance the x264 encoder would
use a 60 frames latency, greatly delaying startup.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1289>
Allows any samplerate and make it negotiable. Fixing a scenario
where transcriberbin is configured with passthrough enabled,
(and negotiated samplerate is not supported by transcriber)
and then setting passthrough=false later during playback.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1287>
We would create another chunk that ends after the fragment end, and
would from then on consider the stream always filled for the chunk
because it starts after the current fragment end (i.e. nothing would go
into this fragment).
This is obviously wrong because the actual fragment end moved further
ahead because of the additional chunk.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1277>
Change the pad health calculation to consider a pad 'healthy'
if it has received data within the last 'timeout' window. Previously,
inactive pads were constantly flip-flopping between healthy and not
healthy depending on whether they were slightly ahead of or behind
the active pad running_time.
When the health status of a pad changes, make sure to always notify
the property, so that applications that are manually controlling
the active pad can make their switching decisions.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1198>
We cannot continue with this buffer, because we cannot calculate the
time when the recording stopped or started. We also cannot safely drop
it, because that might break the stream, especially if it's encoded.
Therefore, we return an element error.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1264>
Previously livesync was waiting for the start timestamp of the current
buffer after looking at the queue and right before pushing it
downstream. This meant that it generally looked too early at the queue
and especially that upstream had to provide the next buffer already at
the start timestamp of the previous one.
Instead, now wait before looking at the queue and wait for the end
timestamp of the previous buffer. Once the previous buffer has expired,
a new buffer really needs to be available or otherwise a filler buffer
has to be pushed downstream.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1250>
I've looked at the GstQueue code again and tried making livesync behave
better with EOS. This isn't very well tested, though. My goal was to
make this look saner but I think this should be reviewed by someone who
knows the queue code.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1251>
Live input + is-live=false:
While not recording, drop input
When recording is started, offset to collapse the gap
Live input + is-live=true:
While not recording, drop input
Don't modify the offset
Non-live input + is-live=false:
While not recording, block input
Don't modify the offset
Non-live input + is-live=true:
While not recording, block input
When recording is started, offset to current running time
Co-authored-by: Jan Alexander Steffens (heftig) <jan.steffens@ltnglobal.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1206>
Fix the following use case:
- main input of fallbackswitch is finite (a media file)
- fallback input is infinite (videotestsrc)
- main input is done and send eos, which is propagated downstream
- fallbackswitch switches to fallback, sending STREAM_START which reset
EOS downstream (aggregator does that)
- fallback input keeps pushing buffers forever.
Solve it by adding a 'stop-on-eos' property so fallbackswitch stops
pushing property once the main input is eos.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1242>
Spawning one task per message to send out instead of sending them out
sequentially from the one task used to poll the handler sometimes
resulted in peers receiving ICE candidates before SDP offers, triggering
hard to understand errors in the browser.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1236>
This is a first step where we try to replicate encoding conditions from
the input stream into the discovery pipeline. A second patch will
implement using input buffers in the discovery pipelines.
This moves discovery to using input buffers directly. Instead of trying
to replicate buffers that `webrtcsink` is getting as input with testsrc,
directly run discovery based on the real buffers. This way we are sure
we work with the exact right stream type and we don't need encoders to
support encoding streams inputs.
We use the same logic for both encoded and raw input to avoid having
several code paths and makes it all more correct in any case.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1194>
In `webrtcsink`, we terminate a session by setting the session's pipeline to
`Null` like this:
```rust
pipeline.call_async(|pipeline| {
[...]
pipeline.set_state(gst::State::Null);
[...]
// the following cvar is awaited in unprepare()
cvar.notify_one();
});
```
However, `pipeline.call_async` keeps a ref on the pipeline until it's done,
which means the `cvar` is notified before `pipeline` is actually 'disposed',
which happens in a different thread than `unprepare`'s. [`gst_rtp_bin_dispose`]
releases some resources when the pipeline is unrefed. In some cases, those
resources are actually released after the main thread has returned, leading
various issues.
This commit uses tokio runtime's `spawn_blocking` instead, which allows owning
and disposing of the pipeline before the `cvar` is notified.
[`gst_rtp_bin_dispose`]: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/blob/main/subprojects/gst-plugins-good/gst/rtpmanager/gstrtpbin.c#L3108
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1225>
This signal is emitted as soon as the pipeline for each consumer
is created, and can be used by applications that require a greater
level of control over webrtcsink's internals.
An example is also provided to demonstrate usage
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1220>
Adapt a commit [1] that was introduced as part of the forward port of the MR
'add signal "request-encoded-filter"' [2].
The deadlock said commit was fixing doesn't happen on main branch due to
changes in the element design: the Sessions are no longer aborted with the
element `State` held. However, we want to ensure the stats collection task
is terminated when the `webrtcbin` element returns from the Ready to Null
transition, meaning that the related resources are released.
[1]: gstreamer/gst-plugins-rs!1176 (0e6b9df9)
[2]: gstreamer/gst-plugins-rs!1176
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1222>
First off, we just created the session, we know stats_sigid is None
at this point.
Second, don't first assign the result of connecting on-new-ssrc to the
field, then the result of connection twcc-stats, that simply doesn't
make sense.
Finally, actually check that stats_sigid *is* None before connecting
twcc-stats, as I understand it this must have been the original
intention / behavior.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1217>
`State::finalize_session()` asynchronously sets the Session pipeline to Null.
In some cases, sessions `webrtcbin` could terminate their transition to Null
after `webrtcsink` had reached Null.
This commit adds a set of `finalizing_sessions`. When the finalization process
starts, the session is added to the set. After `webrtcbin` has reached the Null
state, the session is removed from the set and a condvar is notified.
In `unprepare`, `webrtcsink` loops until the `finalizing_sessions` set is
empty, awaiting for the condvar to be notified when it's not.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1221>
In certain cases, roundedcorners would negotiate to I420 even when user
supplied a non-zero border radius.
For example, the below pipeline leads to I420 being negotiated even
though a non-zero border radius was given. Ideally, this pipeline
should have failed at the negotiation stage.
```bash
gst-launch-1.0 -v \
videotestsrc num-buffers=1000 pattern=white ! \
video/x-raw,width=320,height=180 ! \
roundedcorners border-radius-px=10 ! videobox border-alpha=0 top=-10 left=-10 right=-10 bottom=-10 fill=yellow ! \
compositor name=comp sink_0::xpos=960 sink_0::ypos=0 sink_0::width=320 sink_0::height=180 sink_0::alpha=1.0 sink_1::xpos=960 sink_1::ypos=180 sink_1::width=320 sink_1::height=180 sink_1::alpha=1.0 \
sink_2::xpos=960 sink_2::ypos=360 sink_2::width=320 sink_2::height=180 sink_2::alpha=1.0 sink_3::xpos=0 sink_3::ypos=0 sink_3::width=960 sink_3::height=720 sink_3::alpha=1.0 ! \
video/x-raw,width=1280,height=720! x264enc ! mp4mux ! filesink location=test.mp4 \
videotestsrc num-buffers=1000 pattern=red ! \
video/x-raw,width=320,height=180 ! roundedcorners border-radius-px=10 ! comp. \
videotestsrc num-buffers=1000 pattern=blue ! \
video/x-raw,width=320,height=180 ! roundedcorners border-radius-px=10 ! comp. \
videotestsrc num-buffers=1000 pattern=green ! \
video/x-raw,width=960,height=720 ! roundedcorners border-radius-px=10 ! comp.
```
If border radius is non-zero, we should not really allow negotiation
to select I420. Fix this by returning only A420 for border-radius > 0
in `transform_caps` instead of returning both like earlier.
Another example of a simpler pipeline like below which would earlier work
```bash
gst-launch-1.0 videotestsrc pattern=red ! videoconvert ! video/x-raw,width=1923,height=1087,format=I420 ! roundedcorners border-radius-px=40 ! video/x-raw,format=I420 ! videoconvert ! gtksink
```
now fails with
```bash
WARNING: erroneous pipeline: could not link roundedcorners0 to videoconvert1, roundedcorners0 can't handle caps video/x-raw, format=(string)I420
```
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1211>
After calculating the earliest pts, the fragment_start_pts and
chunk_start_pts in State are updated. However, when checking if the
stream is filled, the previous start_pts (set to None) is used instead.
This means that chunk_filled and fragment_filled will be false the first
time aggregate() is called, assuming timeout is false, all_eos is false,
and the sinkpad is not EOS. This requires aggregate() having to be
called a second time before the first fragment is sent.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1207>
In some rare cases, the webrtc-test entered a deadlock while executing
`WebRTCSink::unprepare`. Attaching gdb to a blocked instance showed:
* `gstrswebrtc::signaller:👿:Signaller::stop()` parked, waiting for a
`Condvar` in `Signaller::stop()`. This was most likely awaiting for the
receive task to complete while it was locked in `element.end_session()`.
This code path is triggered from `unprepare` with the `State` `Mutex` locked.
* `webrtcsink:👿:WebRtcSink::process_stats` waiting for a contended `Mutex`,
which is also the `State` `Mutex`. This prevented completion of the signal
`gst_webrtc_bin_get_stats`.
This commit aborts the task in charge of periodically collecting stats and
ensures any remaining iteration completes before requesting the Signaller to
stop.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1202>
Prior to this commit, we were sending over words concatenated together
with no separators, for instance "Idon'twanttobeanemperor".
The translation service seems clever enough to translate the contents
anyway, but there is no reason to make its task harder than necessary,
and it didn't re-add separators when the target language was the same as
the source language, which resulted in less than ideal output.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1171>
Session ending is bidirectional: the signaller can tell the sink that a
session was ended, and the sink can tell the signaller to end a session.
As such, two signals are needed, before this patch the second case was
not working as in essence the sink was telling itself that a session was
ended, and obviously failing to even find it when trying to end it again.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1167>
In order to support the use case of an external user providing their own
signalling mechanism, we want the signals to be used and only if nothing
is connected, fallback to the default handling. Calling the interface
vtable directly will bypass the signal emission entirely.
Also ensure that the signals are defined properly for this case. i.e.
1. Signals the the application/external code is expected to emit are
marked as an action signal.
2. Add accumulators to avoid calling the default class handler if
another signal handler is connected.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1141>
This pattern is used for subclassing and calling parent class/interface functions.
However that is not useful for the signaller object.
1. The signals are the API contract and should instead be used by
webrtcsrc/sink to ask or provide outside for/with information.
2. The default case (no signal attached)is instead handled by default class
handlers that call directly using the relevant rust trait. No parent
(GObject) vfuncs necessary.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1141>
If a basicna character is received, it will always have a channel of 0
even if it's directed at a different data channel. Fix by keeping track
of the last channel from other commands and using that when producing
text in the basicna subset.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1112>
This subproject adds a high-level web API compatible with GStreamer
webrtcsrc and webrtcsink elements and the corresponding signaling
server. It allows a perfect bidirectional communication between HTML5
WebRTC API and native GStreamer.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/946>
In this context, the bitrate variable is for all encoders, but the
max_bitrate field is per encoder. To calculate a proper FEC ratio, we
need to scale max_bitrate to the number of encoders.
+ Also clamp the fec-percentage that we set on the transceiver for extra
safety
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1151>
With this, if the transcriber element in use supports "translation_src_"
request source pads, the user can now specify what languages to
translate to and how to map them to 608 channels (only CC1 and CC3 are
supported).
For instance, translation-languages="languages, CC3=transcript, CC1=fr"
will cause the original transcript to be muxed into the CC3 channel, and
the French translation to be muxed into the CC1 channel.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1149>
To avoid special characters getting de-duplicated by the decoder, we
insert no-op control commands after those. The no-op command must be
picked according to the mode we're in however, inserting
"resume_caption_loading" commands in roll-up mode caused obvious issues.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1147>
Allowed downstream caps might hold multiple structures, simply fixating
the first structure is not enough, tttocea608 must also create caps with
a single structure from there (or remove the remaining structures, but
new caps seems cleaner)
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1146>
When passthrough=false at construction and the transcription bin
is linked after receiving video caps (and not on state change),
there could be a race where transcription-bin was linked with
tee but state change of the transcription-bin was not finished.
If upstream pushed a buffer at that point, it got a flushing flow
return and stopped streaming.
This is the same issue and the same fix as 558656deb5
for the initial passthrough=false case.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1142>
* A queue dedicated to transcript items not intended for translation.
* A queue dedicated to transcript items intended for translation. The items are
enqueued after a separator is detected or translate-lookahead was reached.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1137>
This commit adds an optional experimental translation tokenization feature.
It can be activated using the `translation_src_%u` pads property
`tokenization-method`. For the moment, the feature is deactivated by default.
The Translate ws accepts '<span></span>' tags in the input and adds matching
tags in the output. When an 'id' is also provided as an attribute of the
'span', the matching output tag also uses this 'id'.
In the context of close captions, the 'id's are of little use. However, we can
take advantage of the spans in the output to identify translation chunks, which
more or less reflect the rythm of the input transcript.
This commit adds simples spans (no 'id') to the input Transcript Items and
parses the resulting spans in the translated output, assigning the timestamps
and durations sequentially from the input Transcript Items. Edge cases such as
absence of spans, nested spans were observed and are handled here. Similarly,
mismatches between the number of input and output items are taken care of by
some sort of reconcialiation.
Note that this is still experimental and requires further testings.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1109>
This commit adds an optional transcript translation feature implemented as
request src Pads.
When requesting a src Pad, the user can specify the translation language code
using Pad properties 'language-code'.
The following properties are defined on the Element:
- 'transcribe-latency': formerly 'latency', defines the expected latency for
the Transcribe webservice.
- 'translate-latency': defines the expected latency for the Translate
webservice.
- 'transcript-lookahead': maximum transcript duration to send to translation
when a transcript is hitting its deadline and no punctuation was found.
When the input and output languages are the same, only the 'transcribe-latency'
is used for the Pad. Otherwise, the resulting latency is the addition of
'transcribe-latency' and 'translate-latency'.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1109>
We were currently returning a value based on the next chunk PTS, but the
expectation in GstAggregator is that we return a running time. This
resulted in spurious wakeups and warnings like:
0:00:01.501685123 1552995 0x55899715c1e0 WARN fmp4mux mux/fmp4/src/fmp4mux/imp.rs:1818:gstfmp4::fmp4mux:👿:FMP4Mux::drain_buffers:<fmp4mux0:sink_1> Don't have a complete GOP for the first stream on timeout in a live pipeline
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1127>
This helps gather together the details related to the `TranscriberLoop`.
One difference with previous implementation is that the ws `Client` is
build each time the loop is started instead of being reused. With the new
approach, we don't keep the connection open after EOS and we should be
more resistant in case of a connection failure.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1104>
Instead of sending transcription events to the src pad loop, this commit
enqueues the transcribed buffers immediately in the ws loop, then notifies
the src pad loop. The src pad loop is only in charge of dequeuing the buffers.
This should help with upcoming evolutions.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1104>
Create a single, global GDK GL context and the corresponding GStreamer
GL display and wrapped GStreamer GL context when initializing the first
sink and continue using that for all further sinks.
Additionally, don't create a full GStreamer GL context inside the sink
but only distribute the wrapped GL context in the pipeline so that
elements that actually need a full GL context can create one that is
sharing with that one. The sink itself does not need a full GStreamer GL
context.
Then inside the sink check that any GL memory that arrives was created
by a GL context that can share with the wrapped GDK GL context and only
then use it.
And lastly, use the correct GL contexts for a) creating a sync point and
b) actually waiting on it.
Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/318
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1099>
If creating a playlist or fragment stream fails (disk is full, the
directory is removed, ...), we will currently crash because the signal
handler expects a non-None GIOStream. The actual callback is allowed to
return None values and we handle this in the caller, so let's not have
this restriction on the signal handler.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1093>
for uploaded object default content-type is set to binary/octet-stream,
which is correct.
metadata cannot be used to set content-type and content-disposition as
setting metadata add a prefix x-amz-meta to key
e.g. setting metadate "content-type=video/mp4" actually set value as
x-amz-meta-content-type. So these has to be seaprate property.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1085>
This works around non-determinism in aggregator where depending on
timing it can happen that it consumes all buffers from both pads or
waits for another buffer on one pad while the other one already has one.
The effect in this test was that it sometimes timed out. By providing
one more buffer it is guaranteed now that at this point the muxer is
beyond the end of the first fragment.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1081>
The logic of the element requires the next buffer to be available
immediately after we are done pushing the previous, otherwise we insert
a repeat.
Making the src loop handle events and queries broke this, as upstream is
almost guaranteed not to deliver a buffer in time if we allow non-buffer
items to block upstream's push.
To fix this, replace our single-item `Option` with a `VecDeque` that we
allow to hold an unlimited number of events or queries, but only one
buffer at a time.
In addition, the code was confused about the current caps and segment.
This wasn't an issue before making the src loop handle events and
queries, as only the sinkpad cared about the current segment, using it
to buffers received, and only the srcpad cared about the current caps,
sending it just before sending the next received buffer.
Now the sinkpad cares about caps (through `update_fallback_duration`)
and the srcpad cares about the segment (when not in single-segment
mode).
Fix this by
- making `in_caps` always hold the current caps of the sinkpad,
- adding `pending_caps`, which is used by the srcpad to store
caps to be sent with the next received buffer,
- adding `in_segment`, holding the current segment of the sinkpad,
- adding `pending_segment`, which is used by the srcpad to store
the segment to be sent with the next received buffer,
- adding `out_segment`, holding the current segment of the srcpad.
Maybe a fix for
https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/298.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1082>
The first time might return `EAGAIN` if there are pending frames but
there is no decoded frame available yet. The second time it will
actually wait for frames to become available and only start returning
`EAGAIN` again once no more frames are left.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1080>
Commit ad3f1cf fixed the name of hlssink child element to be the same
for hlssink2 and hlssink3. However, we rely on element name to return
boolean in case of hlssink3 or None in case of hlssink2 as the return
value of the delete-fragment closure.
Fix this by using the factory name instead of the element name.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1076>
Simplifies state tracking and potentially reduces latency as it's not
necessary to wait until all fragments of an OBU are received.
The last OBU of a TU is marked with the marker flag to allow parsers to
detect this without first seeing the beginning of the next TU.
Also use a simple `Vec` for collecting complete OBUs instead of a
`gst_base::Adapter` as this reduces the number of allocations.
And also handle invalid packets a little bit more gracefully.
Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/244
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1072>
Allow outputting sub-fragments (chunks in CMAF terms) that are shorter
than the fragment duration and don't usually start on a keyframe. By
this the buffering requirements of the element is reduced to one chunk
duration, as is the latency.
This is used for formats like low-latency / LL-HLS and DASH.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1059>
We dispose of consumer pipelines asynchronously, potentially after the
session objects have been disposed of.
As session objects are the owner of the cc element, it is entirely
possible for the bwe-request signal to get emitted after cc has been
disposed of, as the closure only takes a weak reference to it.
Fix by simply checking if cc is None
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1044>
Without this auto-pluggers such as decodebin or parsebin will be unable to
process AV1 RTP payloads.
Tested with: `videotestsrc num-buffers=50 ! videoconvert ! av1enc ! av1parse ! rtpav1pay ! queue ! decodebin3 ! videoconvert ! queue ! autovideosink`
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1034>
Required renaming threadshare/benchmark to threadshare/ts-benchmark
because 'benchmark' as a target name is reserved for meson's
`benchmark` target.
Disabled by default because cargo decides that it has to rebuild
everything, and is really slow because of that.
Also required adding --features for setting features required by the
examples.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1028>
Required a slight rework of the build file, and how options are passed
to cargo_wrapper.py
Necessitated a bump of the required gstreamer version to 1.20, which
should be fine for the meson build since its primary function is to be
built as part of the gstreamer monorepo build to get a dev env.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1028>
If there was a previous sample in this stream then its duration needs to
be extended by the gap position, and if there was none then the start
time of the whole stream has to be shifted by the duration.
Not doing so causes timestamps to be offset wrongly by the duration of
the gap.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1015>
It attempts to produce a (nearly) gapless live stream by synchronizing
its output to the running time and forwarding the next input buffer if
its start is (nearly) flush with the end of the last output buffer.
If the input buffer is missing or too far in the future, it duplicates
the last output buffer with adjusted timestamps. If it is operating on a
raw audio stream, it will fill duplicate buffers with silence.
If an input buffer arrives too late, it is thrown away. If the last
input buffer was accepted too long ago (according to `late-threshold`),
a late input buffer is accepted anyway, but immediately considered a
duplicate. Due to the silence-filling, this has no effect on audio, but
video gets a "slideshow" effect instead of freezing completely.
The "many-repeats" property will be notified when this element has
recently duplicated a lot of buffers or recovered from such a state.
Co-authored-by: Vivia Nikolaidou <vivia@ahiru.eu>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/708>
Right now the code manually pieces together the components
in a String for efficiency. When credentials contain special
characters this can result in invalid URLs, so do it the proper
way (with Url::parse + format) to make sure components are escaped
as needed.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/949>
WHIP endpoint providers like Cloudflare do not support Trickle ICE
and need candidates to be send along with the initial offer. Instead
of sending the offer in create-offer promise, send it once the ICE
candidates have been gathered.
While at it add properties to set STUN and TURN server along with the
ICE transport policy as at least when testing the Cloudflare WHIP
endpoint seems unreachable without it. This has also been observed
with Cloudflare provided demos.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/949>
This implements WHEP specification based on
https://datatracker.ietf.org/doc/html/draft-murillo-whep-00
and has been tested with Cloudflare.
Server offers are likely to be removed from the WHEP specification
in upcoming revisions, to avoid compatibility issues. None of the
commercial services implementing WHEP support server initiated offers.
So we only support client side initiated offers.
Follows session setup and tear down as covered in Figure 1, Section 3
of the specification.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/949>
Due to how aggregator works, it depends on how buffers are pulled
whether aggregate() is called again or it is waiting for a timeout or EOS:
works:
- pad 1: 4 buffers, pad 2: 4 buffers
- aggregate ready: take all 4/4 buffers
- pad 1: 1 buffers, pad 2: 1 buffer
- aggregate ready: take all 1/1 buffers
waits:
- pad 1: 5 buffers, pad 2: 4 buffers
- aggregate ready: take all 5/4 buffers
- pad 1: 0 buffers, pad 2: 1 buffer
- aggregate not ready: waiting for timeout or EOS
Also don't manually set the clock time as that's unnecessary.
Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/274
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/998>
Especially simplify calculation of ONVIF UTC times. As a side-effect
this reduces the number of times the running times of a buffer are
calculated, and also causes streams to be interleaved correctly in ONVIF
mode if there is a non-constant UTC-to-running-time difference.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/979>
... instead of forwarding them to a Task via a channel.
This improves CPU usage by 5% according to `udpsrc-benchmark-sender`
with the `tuning` feature using default audio test buffers and
400 streams on the same ts-context.
It is expected to improve latency significantly. This is inferred
from `ts-standalone`: latency shrinks from around 5ms to 1.5µs
using the `task` sink compared to the `async-mutex` sink.
The async Mutex is mandatory here as we need to hold the lock
across await points.
This makes it easy to generate "listenable" signals and to evaluate
discontinuities.
When the `tuning` feature is activated and the `main-elem` property
is set, the element can log the parked duration in %, which is an
image of the CPU usage for the ts-context.
This commit adds a test mode to `udpsrc-benchmark-sender` which
generates default audio buffers from `ts-audiotestsrc`. The `rtp`
mode is modified so that it uses `ts-audiotestsrc`.
Contrary to the existing Task Sink, the Async and Sync Mutex Sinks
handle buffers in the `PadSinkHandler` directly. The Async Mutex
Sink uses an async Mutex for the `PadSinkHandlerInner` while the
Sync Mutex Sink uses... a sync Mutex.
All Sinks share the same settings and stats manager.
Use the `--sink` command line option to select the sink (default is
`sync-mutex` since it allows evaluating the framework with as little
overhead as possible.
Also apply various fixes:
- Only keep the segment start instead of the full `Segment`. This
helps with cache locality (`Segment` is a plain struct with many
fields) and avoids downcasting the generic `Segment` upon each
buffer handling.
- Box the `Stat`s. This should improve cache locality a bit.
- Fix EOS handling which took ages for no benefits in this
particular use case.
- Use a macro to raise log level in the main element.
- Move error handling during item processing in `handle_loop_error`.
This function was precisely designed for this and it should reduce
the `handle_item`'s Future size.
error[E0425]: cannot find value `LIBRARY_NAME` in this scope
--> net/ndi/src/ndisys.rs:336:23
|
336 | path.push(LIBRARY_NAME);
| ^^^^^^^^^^^^ not found in this scope
error[E0425]: cannot find value `LIBRARY_NAME` in this scope
--> net/ndi/src/ndisys.rs:339:33
|
339 | path::PathBuf::from(LIBRARY_NAME)
| ^^^^^^^^^^^^ not found in this scope
This can happen at the beginning of a stream if upstream is
rtpjitterbuffer and it has problems figuring out timestamps in the
beginning due to resetting / skew.
<!-- For any GStreamer usage question, please contact the community using the #gstreamer channel on IRC https://www.oftc.net/ or the mailing list on https://gstreamer.freedesktop.org/lists/ -->
#### Expected Behavior
<!-- What did you expect to happen -->
#### Observed Behavior
<!-- What actually happened -->
#### Setup
- **Operating System:**
- **Device:** Computer / Tablet / Mobile / Virtual Machine <!-- Delete as appropriate !-->
- **gst-plugins-rs Version:**
- **GStreamer Version:**
- **Command line:**
### Steps to reproduce the bug
<!-- please fill in exact steps which reproduce the bug on your system, for example: -->
1. open terminal
2. type `command`
### How reproducible is the bug?
<!-- The reproducibility of the bug is Always/Intermittent/Only once after doing a very specific set of steps-->
### Screenshots if relevant
### Solutions you have tried
### Related non-duplicate issues
### Additional Information
<!-- Any other information such as logs. Make use of <details> for long output -->
// from the global accumulator(gasig), then reads these buffers at a fixed delay time, creating the adelL, adelM and adelR buffers,
// also, It multiplies the audio samples in the right channel by 0.5 * kdel, being kdel a line of values starting at 0.5 at increments of 0.001.
// Finally, those buffers are mixed with the accumulator, and an audio envelop is applied(aseg) to them.
// The result is similar to an audio echo in which the buffered samples are read at different delay times and also modified in frecuency(right channel),
// The result is similar to an audio echo in which the buffered samples are read at different delay times and also modified in frequency(right channel),
// this creates an space effect using just one channel audio input.
//gst::element_warning!(self, gst::StreamError::Failed, ["Buffer {} is missing the GstOriginalBufferMeta, put originalbuffersave upstream in your pipeline", buffer]);
Rafael Caricio
