Commit graph

175 commits

Author SHA1 Message Date
Mathieu Duponchelle
170e769812 audio: add speechmatics transcriber
Element implemented around the Speechmatics API:

<https://docs.speechmatics.com/rt-api-ref>

The element also comes with translation support, and offers a similar
interface to the one exposed by `awstranscriber`.

The Speechmatics service has good accuracy, and can be deployed on
premises, offering an advantage over AWS transcribe.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1665>
2024-08-21 17:43:02 +00:00
Mathieu Duponchelle
1c48d7065d gstwebrtc-api example: add support for requesting mix matrix
This is one example of how a consumer might send over custom upstream
event requests to the producer.

As webrtcsink will deserialize numbers in priority as integers, we need
a custom stringifying function to ensure members of the matrix array are
indeed serialized with the floating point.

An optional stringifier parameter is thus added to the
sendControlRequest API.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1711>
2024-08-15 15:42:04 +00:00
Mathieu Duponchelle
9455e09d9f webrtcsink: expose properties for running web server
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1671>
2024-08-08 16:40:46 +02:00
Guillaume Desmottes
17910dd532 gtk4: add window-{width,height} property
Allow the application to pass the actual rendering size so overlays can
be rendered accordingly.

Fix #562

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1680>
2024-08-06 10:29:41 +00:00
Thibault Saunier
a05ab37b49 tracers: Add a tracer that dumps data flow into .pcap files
See documentation for more details

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/879>
2024-07-31 20:27:27 +00:00
François Laignel
34b791ff5e webrtc: add raw payload support
This commit adds support for raw payloads such as L24 audio to `webrtcsink` &
`webrtcsrc`.

Most changes take place within the `Codec` helper structure:

* A `Codec` can now advertise a depayloader. This also ensures that a format
  not only can be decoded when necessary, but it can also be depayloaded in the
  first place.
* It is possible to declare raw `Codec`s, meaning that their caps are compatible
  with a payloader and a depayloader without the need for an encoder and decoder.
* Previous accessor `has_decoder` was renamed as `can_be_received` to account
  for codecs which can be handled by an available depayloader with or without
  the need for a decoder.
* New codecs were added for the following formats:
  * L24, L16, L8 audio.
  * RAW video.

The `webrtc-precise-sync` examples were updated to demonstrate streaming of raw
audio or video.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1501>
2024-07-16 19:32:02 +00:00
Taruntej Kanakamalla
3a8462367e threadshare: udpsrc: add buffer-size property
Use buffer-size to set the receive buffer size
on the socket

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1636>
2024-07-15 12:13:41 +00:00
Taruntej Kanakamalla
276ec91cb2 threadshare: udpsrc: add loop property to set multicast loopback
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1636>
2024-07-15 12:13:41 +00:00
François Laignel
000c486568 rav1enc: document bitrate property unit
See:

e34e772e47/src/rate.rs (L365)

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1667>
2024-07-12 18:59:17 +02:00
Sebastian Dröge
8522c8a445 gtk4: Add support for rotations / flipping
Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/284

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1590>
2024-07-07 07:43:49 +00:00
Sebastian Dröge
6e974cf4b9 gtk4: Document paintable properties correctly
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1655>
2024-07-06 11:36:55 +00:00
Philippe Normand
eee93aea52 rtp2: Fix typo on auto-header-extension property name
The rtp (de)pay elements use auto-header-extension so the new elements should do
the same.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1646>
2024-07-02 09:35:39 +01:00
Edward Hervey
95ae67752f net: New mpegtslive element
This element allows wrapping an existing live "mpeg-ts source" (udpsrc,
srtsrc,...) and providing a clock based on the actual PCR of the stream.

Combined with `tsdemux ignore-pcr=True` downstream of it, this allows playing
back the content at the same rate as the (remote) provider **and** not modify
the original timestamps.

Co-authored-by: Sebastian Dröge <slomo@coaxion.net>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1640>
2024-07-01 15:29:22 +02:00
Sebastian Dröge
960529d90d livesync: Add sync property for allowing to output buffers as soon as they arrive
By default livesync will wait for each buffer on the clock. If sync is
set to false, it will output buffers immediately once they're available
and only waits on the clock for outputting gap filler buffers.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1635>
2024-06-26 16:21:42 +00:00
Sanchayan Maity
0bd98e2c34 net/quinn: Allow dropping buffers when buffer size exceeds maximum datagram size
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1613>
2024-06-25 20:15:40 +05:30
Sanchayan Maity
e00ebca63f net/quinn: Add stats property for connection statistics
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1613>
2024-06-25 20:15:40 +05:30
Sanchayan Maity
cf7172248c net/quinn: Allow setting some parameters from TransportConfig
As of now, we expose the below four properties from `TransportConfig`.
- Initial MTU
- Minimum MTU
- Datagram receive buffer size
- Datagram send buffer size

Maximum UDP payload size from `EndpointConfig` and upper bound from
`MtuDiscoveryConfig` are also exposed as properties.

See the below documentation for further details.
- https://docs.rs/quinn/latest/quinn/struct.TransportConfig.html
- https://docs.rs/quinn/latest/quinn/struct.MtuDiscoveryConfig.html
- https://docs.rs/quinn/latest/quinn/struct.EndpointConfig.html

While at it, also clean up passing function parameters to the functions
in utils.rs.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1613>
2024-06-25 20:15:40 +05:30
Tim-Philipp Müller
6b628485c5 rtp: Add AC-3 RTP payloader/depayloader
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1586>
2024-06-01 12:43:27 +00:00
Tamas Levai
802ff6a67c net/quinn: Make QUIC role configurable
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1575>
2024-05-31 23:20:38 +02:00
Arun Raghavan
1c54c77840 webrtcsink: Add a mechanism for SDP munging
Unfortunately, server implementations might have odd SDP-related quirks,
so let's allow clients a way to work around these oddities themselves.
For now, this means that a client can fix up the H.264 profile-level-id
as required by Twitch (whose media pipeline is more permissive than the
WHIP implementation).

Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/516
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1525>
2024-05-30 22:16:46 +03:00
Taruntej Kanakamalla
de726ca8d2 net/webrtc: multi producer support in webrtcsrc
- Add a new structure Session
  - manage each producer using a session
  - avoid send EOS when a session terminates, instead keep running
    waiting for any new producer to connect

- Maintain a bin element per session
  - each session bin encapsulates webrtcbin and the decoder if needed
    as well as the parser and filter if requested by the application
    (through request-encoded-filter)
  - this will be helpful to cleanup the session's respective elements
    when the corresponding producer terminates the session

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1339>
2024-05-29 21:03:27 +00:00
Seungha Yang
ebdcc403cf transcriberbin: Fix mux-method=cea708
* Update "translation-languages" property to include G_PARAM_CONSTRUCT
so that it can be applied to initial state.

* Change default "translation-languages" value to be None instead of
cea608 specific one. Transcriberbin will be able to configure initia
state depending on selected mux method if "translation-languages" is
unspecified.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1589>
2024-05-30 04:40:09 +09:00
Nirbheek Chauhan
9485265769 rtspsrc2: Update rtpbin2 support to use rtprecv and rtpsend
USE_RTPBIN2 is now USE_RTP2 because there is no "rtpbin2" now.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1426>
2024-05-28 19:58:09 +10:00
Matthew Waters
1600d3b055 rtpbin2: split send and receive halves into separate elements
There is now two elements, rtpsend and rtprecv that represent the two
halves of a rtpsession.  This avoids the potential pipeline loop if two
peers are sending/receiving data towards each other.  The two halves can
be connected by setting the rtp-id property on each element to the same
value and they will behave like a combined rtpbin-like element.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1426>
2024-05-28 19:58:09 +10:00
Matthew Waters
06f40e72cb rtpbin2: implement a session configuration object
Currently only contains pt-map

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1426>
2024-05-28 19:58:09 +10:00
Mathieu Duponchelle
74ec83a0ff rtpbin2: implement and use synchronization context
Co-authored-by: Sebastian Dröge <sebastian@centricular.com>
Co-Authored-By: Matthew Waters <matthew@centricular.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1426>
2024-05-28 19:58:09 +10:00
Mathieu Duponchelle
1865899621 rtpbin2: implement jitterbuffer
The jitterbuffer implements both reordering and duplicate packet
handling.

Co-Authored-By: Sebastian Dröge <sebastian@centricular.com>
Co-Authored-By: Matthew Waters <matthew@centricular.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1426>
2024-05-28 17:35:41 +10:00
Sebastian Dröge
e09ad990fa rtpbin2: Implement support for reduced size RTCP (RFC 5506)
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1426>
2024-05-28 17:35:41 +10:00
Matthew Waters
2c86f18a99 rtpbin2: add support for RFC 4585 (RTP/AVPF)
Implements the timing rules for RTP/AVPF

Co-Authored-By: Sebastian Dröge <sebastian@centricular.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1426>
2024-05-28 17:35:41 +10:00
Matthew Waters
27ad26c258 rtp: Initial rtpbin2 element
Can receive and recevie one or more RTP sessions containing multiple
pt/ssrc combinations.

Demultiplexing happens internally instead of relying on separate
elements.

Co-Authored-By: François Laignel <francois@centricular.com>
Co-Authored-By: Mathieu Duponchelle <mathieu@centricular.com>
Co-Authored-By: Sebastian Dröge <sebastian@centricular.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1426>
2024-05-28 17:35:41 +10:00
Sebastian Dröge
984a9fe5ff rtp: Don't restrict payload types for payloaders
WebRTC uses payload types 35-63 as dynamic payload types too to be able
to place more codec variants into the SDP offer.

Instead of allowing just certain payload types, completely remove any
restrictions and let the user decide. There's technically nothing wrong
with using any payload type, especially when using the encoding-name.

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/551

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1587>
2024-05-27 09:34:16 +00:00
Liam
b4fd6cf362 aws: Add system-defined metadata options to both sinks
Add to awss3sink and awss3putobjectsink elements the following
paramerters which are set on the uploaded S3 objects:

* cache-control;
* content-encoding; and
* content-language

Bugfix: Set the content-type and content-disposition values in the S3
putobject call. Previously the params were defined on the element but
unused.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1585>
2024-05-27 10:25:22 +03:00
Tim-Philipp Müller
566e6443f4 rtp: Add KLV RTP payloader/depayloader
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1580>
2024-05-25 20:21:50 +03:00
cdelguercio
f5a7de9dc3 webrtcsink: Support av1 via nvav1enc, av1enc, and rav1enc
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1572>
2024-05-23 10:16:59 +03:00
Tim-Philipp Müller
2585639054 rtp: Add Opus RTP payloader/depayloader
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1571>
2024-05-18 09:29:29 +00:00
Martin Nordholts
9a7f37e2b7 rtpgccbwe: Support linear regression based delay estimation
In our tests, the slope (found with linear regression) on a
history of the (smoothed) accumulated inter-group delays
gives a more stable congestion control. In particular,
low-end devices becomes less sensitive to spikes in
inter-group delay measurements.

This flavour of delay based bandwidth estimation with Google
Congestion Control is also what Chromium is using.

To make it easy to experiment with the new estimator, as
well as add support for new ones in the future, also add
infrastructure for making delay estimator flavour selectable
at runtime.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1566>
2024-05-14 16:25:48 +00:00
Tamas Levai
71cd80f204 net/quinn: Enable client to keep QUIC conn alive
Co-authored-by: Felician Nemeth <nemethf@tmit.bme.hu>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1568>
2024-05-11 08:51:00 +02:00
Rafael Caricio
5549dc7a15 fmp4mux: Support AV1 packaging in the fragmented mp4 plugin
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1544>
2024-05-10 20:59:49 +00:00
Sebastian Dröge
f265c3197b Update plugins cache JSON for new CI GStreamer version
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1570>
2024-05-10 14:14:51 +03:00
Sebastian Dröge
7e09481adc rtp: Add JPEG RTP payloader/depayloader
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1543>
2024-05-10 11:12:49 +03:00
Sebastian Dröge
b4576a0074 gtk4: Fix description of the plugin
A paintable is not a widget and that aspect does not belong in the short
description anyway.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1563>
2024-05-07 20:21:03 +03:00
Sanchayan Maity
80f8664564 net/quinn: Use camel case acronym
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1558>
2024-05-02 16:39:29 +00:00
Sanchayan Maity
096538989b docs: Add documentation for gst-plugin-quinn
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1036>
2024-05-01 22:30:23 +05:30
François Laignel
16b0a4d762 rtp: add mp4gpay
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1551>
2024-04-29 13:33:42 +00:00
François Laignel
b588ee59bc rtp: add mp4gdepay
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1551>
2024-04-29 13:33:42 +00:00
François Laignel
5466cafc24 rtp: add mp4apay
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1551>
2024-04-29 13:33:42 +00:00
François Laignel
812fe0a9bd rtp: add mp4adepay
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1551>
2024-04-29 13:33:42 +00:00
Sebastian Dröge
3dd800ac77 gtk4paintablesink: Implement child proxy interface
This allows setting properties on the paintable from gst-launch-1.0.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1547>
2024-04-26 12:29:10 +03:00
Sebastian Dröge
c92462b240 gtk4: Implement support for directly importing dmabufs
Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/441

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1547>
2024-04-26 12:29:10 +03:00
Mathieu Duponchelle
17d7997137 transcriberbin: add support for consuming secondary audio streams
In some situations, a translated alternate audio stream for a content
might be available.

Instead of going through transcription and translation of the original
audio stream, it may be preferrable for accuracy purposes to simply
transcribe the secondary audio stream.

This MR adds support for doing just that:

* Secondary audio sink pads can be requested as "sink_audio_%u"

* Sometimes audio source pads are added at that point to pass through
  the audio, as "src_audio_%u"

* The main transcription bin now contains per-input stream transcription
  bins. Those can be individually controlled through properties on the
  sink pads, for instance translation-languages can be dynamically set
  per audio stream

* Some properties that originally existed on the main element still
  remain, but are now simply mapped to the always audio sink pad

* Releasing of secondary sink pads is nominally implemented, but not
  tested in states other than NULL

An example launch line for this would be:

```
$ gst-launch-1.0 transcriberbin name=transcriberbin latency=8000 accumulate-time=0 \
      cc-caps="closedcaption/x-cea-708, format=cc_data" sink_audio_0::language-code="es-US" \
      sink_audio_0::translation-languages="languages, transcript=cc3"
    uridecodebin uri=file:///home/meh/Music/chaplin.mkv name=d
      d. ! videoconvert ! transcriberbin.sink_video
      d. ! clocksync ! audioconvert ! transcriberbin.sink_audio
      transcriberbin.src_video ! cea608overlay field=1 ! videoconvert ! autovideosink \
      transcriberbin.src_audio ! audioconvert ! fakesink \
    uridecodebin uri=file:///home/meh/Music/chaplin-spanish.webm name=d2 \
      d2. ! audioconvert ! transcriberbin.sink_audio_0 \
      transcriberbin.src_audio_0 ! fakesink
```

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1546>
2024-04-25 11:56:01 +02:00