If the converter configuration is set to not fill any borders, or if the
border fill color is not full opaque, then the pad has to be handled
as potentially transparent and can't be considered to obscure another
one.
This prevents pads from being wrongly skipped and doing alpha-blending
with uninitialized memory.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1172>
While so far it worked, we are about to introduce a format that break this
assuming. We have a format which consist of NV12 with alpha, and this format
does not have a direct mapping of the component against their plane indexes.
Fix this by using gst_video_format_info_component() introduced in 1.18 for
this purpose.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1151>
Improves throughput of the total convert and blend process and allows
for higher performance across slightly more threads.
Also make use of video aggregator's task pool for blending as well in
order to reduce the number of threads.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1129>
These parameters are incorrectly regarded as mutable in G-IR making them
"incompatible" with languages that are explicit about mutability like
Rust. In order to clean up the code and expected API there, update the
signatures here, right at the source (instead of overriding them in
Gir.toml and hoping for the best).
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1005>
When that property is left to its default, the width and height
property considers frames from input pads with width or height <= 0
should be left unscaled in that dimension.
Setting this property to FALSE changes that behaviour to < 0, as when
animating these properties, 0 should be a valid end value (eg. shrinking
an input stream until it disappears).
The default value of the width and height properties is set to -1, so that
the default behaviour stays consistent whether that new property is set
or not.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/923>
In the case `videoaggregator` is set as allowing format conversions,
and as we convert only on the sinkpads, we should ensure that the
chosen format is usable by the subclass. This in turns implies
that the format is usable on the srcpad.
When doing conversion *any* format can be used on the sinkpads, and this
is the only way that we can avoid race conditions during renegotiations
so we can not change that fact, we just need to ensure that the chosen
intermediary format is usable, which was not actually ensured before
that patch.
Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/issues/834
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/909>
Increases the throughput of compositing by using more CPU cycles across
multiple threads. Simple cases (the output contains one pixel from at
most one input) can have up to a 70% increase in throughput. Not so
simple cases are limited by the region with the most number of
composite operations.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/755>
When a pad has alpha != 1.0 it means that the resulting frames will
contain alpha and thus can't fully obscure with a lower zorder.
Also simplifies the other checks as blending with an OVER or on a
transparent is not a no-op as previously assumed.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/764>
The blend functions for alpha formats need to do more work than just
doing a memcpy, so we can do a memcpy when we know that a blend is not
actually needed.
1080p AYUV ! compositor background=transparent ! fakesink - 56% faster
Specifically, when we don't draw the background and the first pad we
draw completely covers the output frame, we can just copy it as-is.
The rest of the pads (if any) will get composited on top normally.
If the background is transparent and obscured by a pad that may or may
not have alpha, we can still skip drawing it entirely
AYUV 1080p ! compositor background=transparent ! fakesink - 75% faster
We don't need to waste time drawing the background when one of the
pads completely covers the output and there's no alpha on the pad or
in the video format. Speedups:
I420 1080p ! compositor ! fakesink - 72% faster
I420 1080p ! compositor background=black ! fakesink - 45% faster
This removes the crossfade-ratio property and replaces it with an
operator property. Currently this implements the following operators:
- SOURCE: Copy over the source and don't look at the destination
- OVER: Default blending of the source over the destination
- ADD: Like OVER but simply adding the alpha instead
See the example for how to implement crossfading with this.
https://bugzilla.gnome.org/show_bug.cgi?id=797169
This moves all the conversion related code to a single place, allows
less code-duplication inside compositor and makes the glmixer code less
awkward. It's also the same pattern as used by GstAudioAggregator.
The aggregated_frame is now called prepared_frame and passed to the
prepare_frame and cleanup_frame virtual methods directly. For the
currently queued buffer there is a method on the video aggregator pad
now.
Crossfading is a bit more complex than just having two pads with the
right keyframes as the blending is not exactly the same.
The difference is in the way we compute the alpha channel, in the case
of crossfading, we have to compute an additive operation between
the destination and the source (factored by the alpha property of both
the input pad alpha property and the crossfading ratio) basically so
that the crossfade result of 2 opaque frames is also fully opaque at any
time in the crossfading process, avoid bleeding through the layer
blending.
Some rationnal can be found in https://phabricator.freedesktop.org/T7773.
https://bugzilla.gnome.org/show_bug.cgi?id=784827
When caps changes while streaming, the new caps was getting processed
immediately in videoaggregator, but the next buffer in the queue that
corresponds to this new caps was not necessarily being used immediately,
which resulted sometimes in using an old buffer with new caps. Of course
there used to be a separate buffer_vinfo for mapping the buffer with its
own caps, but in compositor the GstVideoConverter was still using wrong
info and resulted in invalid reads and corrupt output.
This approach here is more safe. We delay using the new caps
until we actually select the next buffer in the queue for use.
This way we also eliminate the need for buffer_vinfo, since the
pad->info is always in sync with the format of the selected buffer.
https://bugzilla.gnome.org/show_bug.cgi?id=780682
Compositor does not support it currently and it needs special support
for handling this correctly, and is rather non-trivial to implement for
all formats.
As is done everywhere else, and avoids setting bogus values
And remove useless *<val> checks (we always provide valid values and
it's an internal function).
CID #1320700
Allows the subclass to completely override the chosen src caps.
This is needed as videoaggregator generally has no idea exactly
what operation is being performed.
- Adds a fixate_caps vfunc for fixation
- Merges gst_video_aggregator_update_converters() into
gst_videoaggregator_update_src_caps() as we need some of its info
for proper caps handling.
- Pass the downstream caps to the update_caps vfunc
https://bugzilla.gnome.org/show_bug.cgi?id=756207
