This commit adds an optional experimental translation tokenization feature.
It can be activated using the `translation_src_%u` pads property
`tokenization-method`. For the moment, the feature is deactivated by default.
The Translate ws accepts '<span></span>' tags in the input and adds matching
tags in the output. When an 'id' is also provided as an attribute of the
'span', the matching output tag also uses this 'id'.
In the context of close captions, the 'id's are of little use. However, we can
take advantage of the spans in the output to identify translation chunks, which
more or less reflect the rythm of the input transcript.
This commit adds simples spans (no 'id') to the input Transcript Items and
parses the resulting spans in the translated output, assigning the timestamps
and durations sequentially from the input Transcript Items. Edge cases such as
absence of spans, nested spans were observed and are handled here. Similarly,
mismatches between the number of input and output items are taken care of by
some sort of reconcialiation.
Note that this is still experimental and requires further testings.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1109>
This commit adds an optional transcript translation feature implemented as
request src Pads.
When requesting a src Pad, the user can specify the translation language code
using Pad properties 'language-code'.
The following properties are defined on the Element:
- 'transcribe-latency': formerly 'latency', defines the expected latency for
the Transcribe webservice.
- 'translate-latency': defines the expected latency for the Translate
webservice.
- 'transcript-lookahead': maximum transcript duration to send to translation
when a transcript is hitting its deadline and no punctuation was found.
When the input and output languages are the same, only the 'transcribe-latency'
is used for the Pad. Otherwise, the resulting latency is the addition of
'transcribe-latency' and 'translate-latency'.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1109>
We were currently returning a value based on the next chunk PTS, but the
expectation in GstAggregator is that we return a running time. This
resulted in spurious wakeups and warnings like:
0:00:01.501685123 1552995 0x55899715c1e0 WARN fmp4mux mux/fmp4/src/fmp4mux/imp.rs:1818:gstfmp4::fmp4mux:👿:FMP4Mux::drain_buffers:<fmp4mux0:sink_1> Don't have a complete GOP for the first stream on timeout in a live pipeline
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1127>
This helps gather together the details related to the `TranscriberLoop`.
One difference with previous implementation is that the ws `Client` is
build each time the loop is started instead of being reused. With the new
approach, we don't keep the connection open after EOS and we should be
more resistant in case of a connection failure.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1104>
Instead of sending transcription events to the src pad loop, this commit
enqueues the transcribed buffers immediately in the ws loop, then notifies
the src pad loop. The src pad loop is only in charge of dequeuing the buffers.
This should help with upcoming evolutions.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1104>
Create a single, global GDK GL context and the corresponding GStreamer
GL display and wrapped GStreamer GL context when initializing the first
sink and continue using that for all further sinks.
Additionally, don't create a full GStreamer GL context inside the sink
but only distribute the wrapped GL context in the pipeline so that
elements that actually need a full GL context can create one that is
sharing with that one. The sink itself does not need a full GStreamer GL
context.
Then inside the sink check that any GL memory that arrives was created
by a GL context that can share with the wrapped GDK GL context and only
then use it.
And lastly, use the correct GL contexts for a) creating a sync point and
b) actually waiting on it.
Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/318
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1099>
