Commit graph

64 commits

Author SHA1 Message Date
Guillaume Desmottes
403004a85e fix typos
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1170>
2023-04-10 13:35:32 +02:00
Mathieu Duponchelle
58c8c0edc7 webrtc: signaller iface: fix session-ended vs end-session confusion
Session ending is bidirectional: the signaller can tell the sink that a
session was ended, and the sink can tell the signaller to end a session.

As such, two signals are needed, before this patch the second case was
not working as in essence the sink was telling itself that a session was
ended, and obviously failing to even find it when trying to end it again.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1167>
2023-04-10 07:58:10 +03:00
Seungha Yang
6e36e2ddfd transcriberbin: Allow video with ANY caps features
transcriberbin does not read/write video buffers actually.
Allow ANY caps features in order to avoid unnecessary GPU
upload/download

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1165>
2023-04-08 02:40:49 +09:00
Matthew Waters
c141a82dfb webrtcsink: update docs for property and signal changes
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1141>
2023-04-07 09:58:13 +10:00
Seungha Yang
538e2e0c9e transcriberbin: Add support for runtime translation-languages update
Allows updating translation-languages at runtime

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1162>
2023-04-06 21:43:04 +09:00
Matthew Waters
a8b46f1bf4 closedcaption: add cea608tocea708 element
Implement an element that can take an input 608 caption stream and
generate a valid 708 caption stream by parsing the 608 data and
generating the equivalent DTVCCPackets and Service blocks.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1112>
2023-04-05 13:00:32 +10:00
Mathieu Duponchelle
8cb328b6f2 transcriberbin: add support for translations
With this, if the transcriber element in use supports "translation_src_"
request source pads, the user can now specify what languages to
translate to and how to map them to 608 channels (only CC1 and CC3 are
supported).

For instance, translation-languages="languages, CC3=transcript, CC1=fr"
will cause the original transcript to be muxed into the CC3 channel, and
the French translation to be muxed into the CC1 channel.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1149>
2023-03-29 01:58:37 +02:00
David Revay
002a70a2a4 chore(webrtcsink): fix max-bitrate blurb and nick
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1150>
2023-03-28 16:11:05 +11:00
Vivia Nikolaidou
7a1b2d97d4 webrtcsink: Add ice-transport-policy option
Can be used to force relay ICE candidates, ensuring TURN server is used.
Proxy to the corresponding setting in webrtcbin,

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1143>
2023-03-27 16:12:13 +03:00
François Laignel
162db2f3b9 net/aws/transcriber: fix translate lookahead
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1137>
2023-03-16 12:39:15 +01:00
François Laignel
d5d6a4daf9 net/aws/transcriber: rename prop transcript-lookahead & TranslationSrcPad
... as translate-lookahead and TranslateSrcPad.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1137>
2023-03-16 12:37:31 +01:00
François Laignel
299e25ab3c net/aws/transcriber: translate: optional experimental translation tokenization
This commit adds an optional experimental translation tokenization feature.
It can be activated using the `translation_src_%u` pads property
`tokenization-method`. For the moment, the feature is deactivated by default.

The Translate ws accepts '<span></span>' tags in the input and adds matching
tags in the output. When an 'id' is also provided as an attribute of the
'span', the matching output tag also uses this 'id'.

In the context of close captions, the 'id's are of little use. However, we can
take advantage of the spans in the output to identify translation chunks, which
more or less reflect the rythm of the input transcript.

This commit adds simples spans (no 'id') to the input Transcript Items and
parses the resulting spans in the translated output, assigning the timestamps
and durations sequentially from the input Transcript Items. Edge cases such as
absence of spans, nested spans were observed and are handled here. Similarly,
mismatches between the number of input and output items are taken care of by
some sort of reconcialiation.

Note that this is still experimental and requires further testings.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1109>
2023-03-14 13:48:32 +00:00
François Laignel
743e97738f net/aws/transcriber: add translation request src pads
This commit adds an optional transcript translation feature implemented as
request src Pads.

When requesting a src Pad, the user can specify the translation language code
using Pad properties 'language-code'.

The following properties are defined on the Element:

- 'transcribe-latency': formerly 'latency', defines the expected latency for
  the Transcribe webservice.
- 'translate-latency': defines the expected latency for the Translate
  webservice.
- 'transcript-lookahead': maximum transcript duration to send to translation
  when a transcript is hitting its deadline and no punctuation was found.

When the input and output languages are the same, only the 'transcribe-latency'
is used for the Pad. Otherwise, the resulting latency is the addition of
'transcribe-latency' and 'translate-latency'.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1109>
2023-03-14 13:48:32 +00:00
Mathieu Duponchelle
584392049c net/webrtc: implement AWS KVS signaller
And expose a wrapper webrtcsink variant, aws-kvs-webrtcsink.

This adds support in webrtcsink for processing a consumer offer, instead
of producing one.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1114>
2023-03-09 15:39:09 +00:00
François Laignel
00153754bb net/aws: use aws-sdk-transcribestreaming
Switch from manual webservice client impl to `aws-sdk-transcribestreaming`.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1104>
2023-03-01 08:47:58 +00:00
Thibault Saunier
ce3bb2f1d4 Add a webrtcsrc element
Updating the docker image to include:
https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3236

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/932>
2023-02-28 20:50:15 -03:00
Arun Raghavan
39e0acb55a hlssink3: Fix case on unspecified playlist type nick for consistency
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1089>
2023-02-10 23:07:12 +00:00
rajneeshksoni
994c79569e awss3sink: Add properties to set content-Type and content-disposition.
for uploaded object default content-type is set to binary/octet-stream,
which is correct.
metadata cannot be used to set content-type and content-disposition as
setting metadata add a prefix x-amz-meta to key
e.g. setting metadate "content-type=video/mp4" actually set value as
x-amz-meta-content-type. So these has to be seaprate property.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1085>
2023-02-09 19:04:07 +00:00
Simon Himmelbauer
3c31c98d95 spotifyaudiosrc: Support configurable bitrate
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1073>
2023-02-09 00:02:30 +02:00
rajneeshksoni
0f383a6545 hlssink3: Allow setting i-frame-only playlist.
HLS allows manifest where all segments are single ifames.
This manifest requires `EXT-X-I-FRAMES-ONLY` tag in the
manifest.
I-FRAMES-ONLY playlist segments are video only segments.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1070>
2023-02-08 14:04:46 +00:00
Sebastian Dröge
6f26e3bf79 mp4/fmp4: Update docs
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1075>
2023-02-04 16:32:17 +02:00
Sebastian Dröge
5506f8001e rtpav1pay: Add support for tu/frame aligned input
In this case every buffer can be sent out immediately and makes up a
whole frame.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1072>
2023-02-02 20:24:27 +02:00
Sebastian Dröge
d6cb9d72d8 rtpav1depay: Don't output full TUs but just OBUs as they come
Simplifies state tracking and potentially reduces latency as it's not
necessary to wait until all fragments of an OBU are received.

The last OBU of a TU is marked with the marker flag to allow parsers to
detect this without first seeing the beginning of the next TU.

Also use a simple `Vec` for collecting complete OBUs instead of a
`gst_base::Adapter` as this reduces the number of allocations.

And also handle invalid packets a little bit more gracefully.

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/244

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1072>
2023-02-02 20:24:27 +02:00
Sebastian Dröge
2a3d962dc5 fmp4mux: Add support for sub-fragments / chunking
Allow outputting sub-fragments (chunks in CMAF terms) that are shorter
than the fragment duration and don't usually start on a keyframe. By
this the buffering requirements of the element is reduced to one chunk
duration, as is the latency.

This is used for formats like low-latency / LL-HLS and DASH.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1059>
2023-01-27 19:28:27 +00:00
Guillaume Desmottes
abe4efc4a2 fmp4mux: add 'offset-to-zero' property
Add it only to 'isofmp4mux', the onvif variant already does this and
CMAF and DASH are always single-stream so you rely on inter-container
synchronization via the running-time.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1063>
2023-01-25 12:29:30 +00:00
Guillaume Desmottes
570eb7463a livesync: fix late-threshold property min value
The code is handling 0 as "always over threshold" but it was not
possible to set the property to 0.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1049>
2023-01-17 10:54:05 +01:00
rajneeshksoni
d846f527af awss3hlssink: Add stats property.
application can monitor the progress of hls segment generation
and upload progress.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1022>
2023-01-04 12:36:13 +00:00
Jan Alexander Steffens (heftig)
42385c81be Add livesync plugin
It attempts to produce a (nearly) gapless live stream by synchronizing
its output to the running time and forwarding the next input buffer if
its start is (nearly) flush with the end of the last output buffer.

If the input buffer is missing or too far in the future, it duplicates
the last output buffer with adjusted timestamps. If it is operating on a
raw audio stream, it will fill duplicate buffers with silence.

If an input buffer arrives too late, it is thrown away. If the last
input buffer was accepted too long ago (according to `late-threshold`),
a late input buffer is accepted anyway, but immediately considered a
duplicate. Due to the silence-filling, this has no effect on audio, but
video gets a "slideshow" effect instead of freezing completely.

The "many-repeats" property will be notified when this element has
recently duplicated a lot of buffers or recovered from such a state.

Co-authored-by: Vivia Nikolaidou <vivia@ahiru.eu>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/708>
2022-12-14 18:51:36 +02:00
Arun Raghavan
473e7d951b audiofx: Derive from AudioFilter where possible
Saves a little bit of code.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1013>
2022-12-14 10:35:28 -05:00
Michiel Konstapel
54741b7cc4 audiornnoise: add voice detection threshold
Add a property "voice-activity-threshold". Frames where the voice
detection score from the RNN is below the threshold will be completely
muted.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1004>
2022-12-12 11:55:38 +02:00
Guillaume Desmottes
d9de2d3d7b textahead: add settings to display previous buffers
I'll use this in Karapulse to keep displaying the few previous lyrics
rather than having them disappear right away.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1006>
2022-12-12 08:31:57 +01:00
Sebastian Dröge
99a1e30ab0 webrtchttp: Fix documentation JSON
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/949>
2022-12-05 12:47:04 +02:00
Sanchayan Maity
cc7419308b webrtchttp: whipsink: Add candidates when sending the offer
WHIP endpoint providers like Cloudflare do not support Trickle ICE
and need candidates to be send along with the initial offer. Instead
of sending the offer in create-offer promise, send it once the ICE
candidates have been gathered.

While at it add properties to set STUN and TURN server along with the
ICE transport policy as at least when testing the Cloudflare WHIP
endpoint seems unreachable without it. This has also been observed
with Cloudflare provided demos.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/949>
2022-12-05 11:04:45 +05:30
Sanchayan Maity
6be5796888 Add a WebRTC WHEP source element
This implements WHEP specification based on
https://datatracker.ietf.org/doc/html/draft-murillo-whep-00

and has been tested with Cloudflare.

Server offers are likely to be removed from the WHEP specification
in upcoming revisions, to avoid compatibility issues. None of the
commercial services implementing WHEP support server initiated offers.
So we only support client side initiated offers.

Follows session setup and tear down as covered in Figure 1, Section 3
of the specification.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/949>
2022-12-05 11:04:45 +05:30
Seungha Yang
1c145e2ba9 dav1ddec: Lower rank to primary
The rank of AOM av1dec was demoted as secondary, and thus
primary rank is sufficient.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/996>
2022-12-01 17:03:31 +00:00
Jordan Petridis
975f0141be video/gtk4: Implement support for GLTextures when possible.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/588>
2022-11-29 21:18:46 +02:00
Thibault Saunier
6b11284e8a webrtcsink: Make the turn-server prop a turn-servers list
So that we can simply specify several turn servers at once

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/973>
2022-11-16 14:48:16 +00:00
Sebastian Dröge
2b4fd40d62 mp4: Add ONVIF non-fragmented MP4 muxer
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/974>
2022-11-10 12:59:53 +02:00
François Laignel
29a490f6dc ts: introduce ts-audiotestsrc
This makes it easy to generate "listenable" signals and to evaluate
discontinuities.

When the `tuning` feature is activated and the `main-elem` property
is set, the element can log the parked duration in %, which is an
image of the CPU usage for the ts-context.

This commit adds a test mode to `udpsrc-benchmark-sender` which
generates default audio buffers from `ts-audiotestsrc`. The `rtp`
mode is modified so that it uses `ts-audiotestsrc`.
2022-11-09 07:55:04 +00:00
Sebastian Dröge
c2f403f998 gst-plugin-mp4: Add new MP4 plugin with a non-fragmented MP4 muxer 2022-11-08 19:08:47 +02:00
Sebastian Dröge
f062b7cf0d fmp4mux: Make media/trak timescales configurable
And refactor a bit of code for easier extensibility.
2022-11-07 18:06:29 +00:00
Sebastian Dröge
6706f3a4b4 fmp4mux: Add initial Opus support
Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/239
2022-11-03 16:53:01 +02:00
Sebastian Dröge
9504e4d540 docs: Remove some stale entries of renamed elements 2022-11-03 15:09:20 +02:00
Matthew Waters
8c8384c711 fmp4: add support for muxing VP9 streams in cmaf, dash and iso fmp4
As specified in https://www.webmproject.org/vp9/mp4/
2022-10-25 18:33:42 +11:00
Sebastian Dröge
fe8e0a8bf8 Update docs 2022-10-23 21:29:14 +03:00
b97a855a51 videocompare: Update README with reference 2022-10-23 17:16:22 +03:00
Nick Steel
c6578c8699 spotifyaudiosrc: convert to PushSrc
Fixes #252
2022-10-21 09:37:25 +03:00
Thibault Saunier
4942a916a8 webrtc: Uniformise GType names 2022-10-20 13:32:31 +02:00
Thibault Saunier
39c0dcb0d4 Plug webrtc in 2022-10-20 11:51:58 +02:00
9180d348bf Add video comparison element
New video/image comparison element, find images in the stream and post
metadata of comparisons of the video frames to the application.
2022-10-18 13:24:05 +00:00