In some situations, a translated alternate audio stream for a content
might be available.
Instead of going through transcription and translation of the original
audio stream, it may be preferrable for accuracy purposes to simply
transcribe the secondary audio stream.
This MR adds support for doing just that:
* Secondary audio sink pads can be requested as "sink_audio_%u"
* Sometimes audio source pads are added at that point to pass through
the audio, as "src_audio_%u"
* The main transcription bin now contains per-input stream transcription
bins. Those can be individually controlled through properties on the
sink pads, for instance translation-languages can be dynamically set
per audio stream
* Some properties that originally existed on the main element still
remain, but are now simply mapped to the always audio sink pad
* Releasing of secondary sink pads is nominally implemented, but not
tested in states other than NULL
An example launch line for this would be:
```
$ gst-launch-1.0 transcriberbin name=transcriberbin latency=8000 accumulate-time=0 \
cc-caps="closedcaption/x-cea-708, format=cc_data" sink_audio_0::language-code="es-US" \
sink_audio_0::translation-languages="languages, transcript=cc3"
uridecodebin uri=file:///home/meh/Music/chaplin.mkv name=d
d. ! videoconvert ! transcriberbin.sink_video
d. ! clocksync ! audioconvert ! transcriberbin.sink_audio
transcriberbin.src_video ! cea608overlay field=1 ! videoconvert ! autovideosink \
transcriberbin.src_audio ! audioconvert ! fakesink \
uridecodebin uri=file:///home/meh/Music/chaplin-spanish.webm name=d2 \
d2. ! audioconvert ! transcriberbin.sink_audio_0 \
transcriberbin.src_audio_0 ! fakesink
```
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1546>
sccparse holds last timecode in order to ignore invalid timecode
and fallback to the previous timecode. That should happen
when sccparse is handling seek event too. Otherwise single invalid
timecode before the target seek position will cause flow error.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1485>
GTK 4.14 comes with a new GL renderer that does not support GL shader
nodes anymore, so the conversion from non-premultiplied alpha to
premultiplied alpha has to happen differently.
For GTK 4.14 or newer we use the correct format directly when building the
texture, but only if a GLES3+ context is used. In that case the NGL renderer is
used by GTK, which supports non-premultiplied formats correctly and fast.
For GTK 4.10-4.12, or 4.14 and newer if a GLES2 context is used, we use a
self-mask to pre-multiply the alpha.
For GTK before 4.10, we use a GL shader and hope that it works.
Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/488
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1452>
GTK expects GL textures to have premultiplied alpha. The ones we get
from GStreamer don't, leading to incorrect rendering of semitransparent
frames.
GTK 4.12 gained an API to set a different GL texture format, but it
won't help for older GTK versions. Plus, at the time of writing, it
causes a very slow download/upload path in GTK.
So, use a GTK GL shader node to premultiply the alpha without leaving
the GPU.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1312>
Allows any samplerate and make it negotiable. Fixing a scenario
where transcriberbin is configured with passthrough enabled,
(and negotiated samplerate is not supported by transcriber)
and then setting passthrough=false later during playback.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1287>
In certain cases, roundedcorners would negotiate to I420 even when user
supplied a non-zero border radius.
For example, the below pipeline leads to I420 being negotiated even
though a non-zero border radius was given. Ideally, this pipeline
should have failed at the negotiation stage.
```bash
gst-launch-1.0 -v \
videotestsrc num-buffers=1000 pattern=white ! \
video/x-raw,width=320,height=180 ! \
roundedcorners border-radius-px=10 ! videobox border-alpha=0 top=-10 left=-10 right=-10 bottom=-10 fill=yellow ! \
compositor name=comp sink_0::xpos=960 sink_0::ypos=0 sink_0::width=320 sink_0::height=180 sink_0::alpha=1.0 sink_1::xpos=960 sink_1::ypos=180 sink_1::width=320 sink_1::height=180 sink_1::alpha=1.0 \
sink_2::xpos=960 sink_2::ypos=360 sink_2::width=320 sink_2::height=180 sink_2::alpha=1.0 sink_3::xpos=0 sink_3::ypos=0 sink_3::width=960 sink_3::height=720 sink_3::alpha=1.0 ! \
video/x-raw,width=1280,height=720! x264enc ! mp4mux ! filesink location=test.mp4 \
videotestsrc num-buffers=1000 pattern=red ! \
video/x-raw,width=320,height=180 ! roundedcorners border-radius-px=10 ! comp. \
videotestsrc num-buffers=1000 pattern=blue ! \
video/x-raw,width=320,height=180 ! roundedcorners border-radius-px=10 ! comp. \
videotestsrc num-buffers=1000 pattern=green ! \
video/x-raw,width=960,height=720 ! roundedcorners border-radius-px=10 ! comp.
```
If border radius is non-zero, we should not really allow negotiation
to select I420. Fix this by returning only A420 for border-radius > 0
in `transform_caps` instead of returning both like earlier.
Another example of a simpler pipeline like below which would earlier work
```bash
gst-launch-1.0 videotestsrc pattern=red ! videoconvert ! video/x-raw,width=1923,height=1087,format=I420 ! roundedcorners border-radius-px=40 ! video/x-raw,format=I420 ! videoconvert ! gtksink
```
now fails with
```bash
WARNING: erroneous pipeline: could not link roundedcorners0 to videoconvert1, roundedcorners0 can't handle caps video/x-raw, format=(string)I420
```
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1211>
If a basicna character is received, it will always have a channel of 0
even if it's directed at a different data channel. Fix by keeping track
of the last channel from other commands and using that when producing
text in the basicna subset.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1112>
With this, if the transcriber element in use supports "translation_src_"
request source pads, the user can now specify what languages to
translate to and how to map them to 608 channels (only CC1 and CC3 are
supported).
For instance, translation-languages="languages, CC3=transcript, CC1=fr"
will cause the original transcript to be muxed into the CC3 channel, and
the French translation to be muxed into the CC1 channel.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1149>
To avoid special characters getting de-duplicated by the decoder, we
insert no-op control commands after those. The no-op command must be
picked according to the mode we're in however, inserting
"resume_caption_loading" commands in roll-up mode caused obvious issues.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1147>
Allowed downstream caps might hold multiple structures, simply fixating
the first structure is not enough, tttocea608 must also create caps with
a single structure from there (or remove the remaining structures, but
new caps seems cleaner)
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1146>
When passthrough=false at construction and the transcription bin
is linked after receiving video caps (and not on state change),
there could be a race where transcription-bin was linked with
tee but state change of the transcription-bin was not finished.
If upstream pushed a buffer at that point, it got a flushing flow
return and stopped streaming.
This is the same issue and the same fix as 558656deb5
for the initial passthrough=false case.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1142>
