Commit graph

387 commits

Author SHA1 Message Date
François Laignel
2b32d00589 net/aws/transcriber: use two queues for sending transcript items
* A queue dedicated to transcript items not intended for translation.
* A queue dedicated to transcript items intended for translation. The items are
  enqueued after a separator is detected or translate-lookahead was reached.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1137>
2023-03-16 20:29:31 +01:00
François Laignel
5a5ca76d9d net/aws/transcriber: desambiguify SrcPad output items queue
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1137>
2023-03-16 12:41:07 +01:00
François Laignel
162db2f3b9 net/aws/transcriber: fix translate lookahead
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1137>
2023-03-16 12:39:15 +01:00
François Laignel
d5d6a4daf9 net/aws/transcriber: rename prop transcript-lookahead & TranslationSrcPad
... as translate-lookahead and TranslateSrcPad.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1137>
2023-03-16 12:37:31 +01:00
François Laignel
3b3f0c1a29 net/aws/transcriber: fix transcript-lookahead prop nick
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1136>
2023-03-14 21:11:33 +01:00
François Laignel
299e25ab3c net/aws/transcriber: translate: optional experimental translation tokenization
This commit adds an optional experimental translation tokenization feature.
It can be activated using the `translation_src_%u` pads property
`tokenization-method`. For the moment, the feature is deactivated by default.

The Translate ws accepts '<span></span>' tags in the input and adds matching
tags in the output. When an 'id' is also provided as an attribute of the
'span', the matching output tag also uses this 'id'.

In the context of close captions, the 'id's are of little use. However, we can
take advantage of the spans in the output to identify translation chunks, which
more or less reflect the rythm of the input transcript.

This commit adds simples spans (no 'id') to the input Transcript Items and
parses the resulting spans in the translated output, assigning the timestamps
and durations sequentially from the input Transcript Items. Edge cases such as
absence of spans, nested spans were observed and are handled here. Similarly,
mismatches between the number of input and output items are taken care of by
some sort of reconcialiation.

Note that this is still experimental and requires further testings.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1109>
2023-03-14 13:48:32 +00:00
François Laignel
743e97738f net/aws/transcriber: add translation request src pads
This commit adds an optional transcript translation feature implemented as
request src Pads.

When requesting a src Pad, the user can specify the translation language code
using Pad properties 'language-code'.

The following properties are defined on the Element:

- 'transcribe-latency': formerly 'latency', defines the expected latency for
  the Transcribe webservice.
- 'translate-latency': defines the expected latency for the Translate
  webservice.
- 'transcript-lookahead': maximum transcript duration to send to translation
  when a transcript is hitting its deadline and no punctuation was found.

When the input and output languages are the same, only the 'transcribe-latency'
is used for the Pad. Otherwise, the resulting latency is the addition of
'transcribe-latency' and 'translate-latency'.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1109>
2023-03-14 13:48:32 +00:00
Sebastian Dröge
4eccd30ce2 Revert "aws: Temporarily enable the default features of the test-with crate"
This reverts commit 42116b5bce.
2023-03-14 13:28:28 +02:00
Sebastian Dröge
42116b5bce aws: Temporarily enable the default features of the test-with crate
Version 0.9.4 fails compiling without them enabled.

See https://github.com/yanganto/test-with/pull/57
2023-03-14 09:19:26 +02:00
Sebastian Dröge
c1bac30694 webrtc: Update to aws 0.54/0.24
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1131>
2023-03-11 09:37:14 +02:00
Mathieu Duponchelle
584392049c net/webrtc: implement AWS KVS signaller
And expose a wrapper webrtcsink variant, aws-kvs-webrtcsink.

This adds support in webrtcsink for processing a consumer offer, instead
of producing one.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1114>
2023-03-09 15:39:09 +00:00
Sebastian Dröge
fc5ed15af5 Update for gst::Element::link_many() and related API generalization
Specifically, get rid of now unneeded `&`.
2023-03-09 16:46:52 +02:00
François Laignel
b9cd71d8eb net/aws/transcriber: fix eos not being sent
For eos to be sent from the srcpad task loop, we need to go through `dequeue`.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1122>
2023-03-09 13:07:03 +01:00
François Laignel
2ea9f147ab net/aws/transcriber: fix deadlock when the pipeline is interrupted
... also makes sure to abort the taks_iter Future.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1122>
2023-03-09 13:07:03 +01:00
Sebastian Dröge
3ef8a48ded Fix a few new clippy warnings
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1120>
2023-03-07 08:47:01 +00:00
Vivia Nikolaidou
cd74d01324 ndisinkcombiner: Properly handle caps changes
We are caching one video buffer, so previously we were changing the src
caps one buffer too early.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1110>
2023-03-01 12:30:54 +00:00
François Laignel
4a988aaeb8 net/aws/transcriber: use a TranscriberLoop struct
This helps gather together the details related to the `TranscriberLoop`.
One difference with previous implementation is that the ws `Client` is
build each time the loop is started instead of being reused. With the new
approach, we don't keep the connection open after EOS and we should be
more resistant in case of a connection failure.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1104>
2023-03-01 08:47:58 +00:00
François Laignel
f1a080c94e net/aws/transcriber: own transcription items
So that we can avoid copying the content.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1104>
2023-03-01 08:47:58 +00:00
François Laignel
36ae29d746 net/aws: enqueue transcribed buffers within the ws loop
Instead of sending transcription events to the src pad loop, this commit
enqueues the transcribed buffers immediately in the ws loop, then notifies
the src pad loop. The src pad loop is only in charge of dequeuing the buffers.

This should help with upcoming evolutions.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1104>
2023-03-01 08:47:58 +00:00
François Laignel
00153754bb net/aws: use aws-sdk-transcribestreaming
Switch from manual webservice client impl to `aws-sdk-transcribestreaming`.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1104>
2023-03-01 08:47:58 +00:00
François Laignel
57f365979c net/aws: remove aws_ from the aws_transcribe* folder names
Those folders reside under `aws`, so there's shouldn't be any confusion.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1104>
2023-03-01 08:47:58 +00:00
Thibault Saunier
ce3bb2f1d4 Add a webrtcsrc element
Updating the docker image to include:
https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3236

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/932>
2023-02-28 20:50:15 -03:00
Thibault Saunier
0ae637f531 webrtcsink: Move RUNTIME to the crate so it can be reused
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/932>
2023-02-28 17:57:14 -03:00
Thibault Saunier
4ec441560b webrtc: Enhance debug messages when using unknown peer ID
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/932>
2023-02-28 19:28:51 +00:00
Matthew Waters
542c7e12b8 webrtcsink: also support nvvidconv in lieu of nvvideoconvert
nvvideoconvert may not exist and nvvidconv might on some Jetson
platforms.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1107>
2023-02-28 10:12:36 +11:00
Sebastian Dröge
9fc1404415 Update minimum supported Rust version to 1.66
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1096>
2023-02-20 11:09:01 +02:00
Arun Raghavan
487d7fb26b hlssink3: Allow GIOStream signal handlers to return None
If creating a playlist or fragment stream fails (disk is full, the
directory is removed, ...), we will currently crash because the signal
handler expects a non-None GIOStream. The actual callback is allowed to
return None values and we handle this in the caller, so let's not have
this restriction on the signal handler.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1093>
2023-02-14 11:25:44 -05:00
Sebastian Dröge
04e101c605 Optimize various error message / debug message formatting
Directly make use of format strings instead of formatting a string
beforehand and then passing it to the macros.
2023-02-13 11:50:57 +02:00
Arun Raghavan
39e0acb55a hlssink3: Fix case on unspecified playlist type nick for consistency
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1089>
2023-02-10 23:07:12 +00:00
Seungha Yang
6420fe43da rtpav1pay: Fix Leb128Bytes size parsing
There are multiple ways of encoding the value, and don't assume
that bitstream used the way used in this plugin. Instead, count
the number of used bytes.

Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/312
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1090>
2023-02-10 18:47:52 +00:00
Sebastian Dröge
ac8afc4ac0 Update to async-tungstenite 0.20
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1087>
2023-02-10 13:03:07 +02:00
Sebastian Dröge
1e13dbb99c Update versions to 0.11.0-alpha.1 2023-02-10 00:23:56 +02:00
rajneeshksoni
994c79569e awss3sink: Add properties to set content-Type and content-disposition.
for uploaded object default content-type is set to binary/octet-stream,
which is correct.
metadata cannot be used to set content-type and content-disposition as
setting metadata add a prefix x-amz-meta to key
e.g. setting metadate "content-type=video/mp4" actually set value as
x-amz-meta-content-type. So these has to be seaprate property.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1085>
2023-02-09 19:04:07 +00:00
rajneeshksoni
0f383a6545 hlssink3: Allow setting i-frame-only playlist.
HLS allows manifest where all segments are single ifames.
This manifest requires `EXT-X-I-FRAMES-ONLY` tag in the
manifest.
I-FRAMES-ONLY playlist segments are video only segments.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1070>
2023-02-08 14:04:46 +00:00
Sebastian Dröge
0ed74d0aa4 rtpgccbwe: Don't use clamp() if there's no clear min/max value
Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/305

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1078>
2023-02-06 21:56:46 +02:00
Sanchayan Maity
6006a0ba36 aws/s3hlssink: Fix deadlock on EOS
In state change to NULL, we take state lock and call stop. When stop
is called, we will try to upload queued segments in S3 request thread.
That tries to take the state lock again and deadlocks.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1076>
2023-02-03 19:09:18 +05:30
Sanchayan Maity
41aa1e51da aws/s3hlssink: Use factory name when checking name of child element
Commit ad3f1cf fixed the name of hlssink child element to be the same
for hlssink2 and hlssink3. However, we rely on element name to return
boolean in case of hlssink3 or None in case of hlssink2 as the return
value of the delete-fragment closure.

Fix this by using the factory name instead of the element name.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1076>
2023-02-03 19:08:40 +05:30
Sebastian Dröge
5506f8001e rtpav1pay: Add support for tu/frame aligned input
In this case every buffer can be sent out immediately and makes up a
whole frame.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1072>
2023-02-02 20:24:27 +02:00
Sebastian Dröge
194c4e9e9f rtpav1pay: Consider the marker flag to output packets immediately at the end of a frame
Otherwise it is necessary to wait for the beginning of the following
frame, which unnecessarily increases the latency.

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/255

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1072>
2023-02-02 20:24:27 +02:00
Sebastian Dröge
49350f738f rtpav1depay: Fix depayloading of packets starting with a leading OBU fragment followed by more OBUs
Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/288

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1072>
2023-02-02 20:24:27 +02:00
Sebastian Dröge
1756d7a516 rtpav1depay: Fix error handling
Don't error out immediately on errors anymore but try again with the
next packet.

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/289

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1072>
2023-02-02 20:24:27 +02:00
Sebastian Dröge
ed4e9a50d5 rtpav1depay: Set DISCONT flag on buffers following a corrupted packet
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1072>
2023-02-02 20:24:27 +02:00
Sebastian Dröge
d6cb9d72d8 rtpav1depay: Don't output full TUs but just OBUs as they come
Simplifies state tracking and potentially reduces latency as it's not
necessary to wait until all fragments of an OBU are received.

The last OBU of a TU is marked with the marker flag to allow parsers to
detect this without first seeing the beginning of the next TU.

Also use a simple `Vec` for collecting complete OBUs instead of a
`gst_base::Adapter` as this reduces the number of allocations.

And also handle invalid packets a little bit more gracefully.

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/244

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1072>
2023-02-02 20:24:27 +02:00
Sebastian Dröge
560bdc4cb7 Update for glib API changes 2023-01-31 12:24:07 +02:00
Sebastian Dröge
a1cce9b796 aws: Update to AWS SDK 0.54/0.24
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1066>
2023-01-27 22:10:23 +02:00
Sebastian Dröge
3b4c48d9f5 Fix various new clippy warnings
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1062>
2023-01-25 10:31:19 +02:00
Arun Raghavan
ad3f1cf534 aws: s3hlssink: Fix the name of the hlssink child element
It's easier to set child element properties if the name doesn't depend
on the factory.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1061>
2023-01-24 18:56:46 +00:00
Sebastian Dröge
2c386fb792 Update for various deprecated APIs 2023-01-22 20:07:26 +02:00
Sebastian Dröge
4582ae91ab Move remaining plugins to ParamSpec builders
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1054>
2023-01-21 18:34:55 +02:00
Sebastian Dröge
458b2386ed Update for glib API changes 2023-01-21 18:13:48 +02:00