The blend functions for alpha formats need to do more work than just
doing a memcpy, so we can do a memcpy when we know that a blend is not
actually needed.
1080p AYUV ! compositor background=transparent ! fakesink - 56% faster
Specifically, when we don't draw the background and the first pad we
draw completely covers the output frame, we can just copy it as-is.
The rest of the pads (if any) will get composited on top normally.
If the background is transparent and obscured by a pad that may or may
not have alpha, we can still skip drawing it entirely
AYUV 1080p ! compositor background=transparent ! fakesink - 75% faster
We don't need to waste time drawing the background when one of the
pads completely covers the output and there's no alpha on the pad or
in the video format. Speedups:
I420 1080p ! compositor ! fakesink - 72% faster
I420 1080p ! compositor background=black ! fakesink - 45% faster
This removes the crossfade-ratio property and replaces it with an
operator property. Currently this implements the following operators:
- SOURCE: Copy over the source and don't look at the destination
- OVER: Default blending of the source over the destination
- ADD: Like OVER but simply adding the alpha instead
See the example for how to implement crossfading with this.
https://bugzilla.gnome.org/show_bug.cgi?id=797169
This moves all the conversion related code to a single place, allows
less code-duplication inside compositor and makes the glmixer code less
awkward. It's also the same pattern as used by GstAudioAggregator.
The aggregated_frame is now called prepared_frame and passed to the
prepare_frame and cleanup_frame virtual methods directly. For the
currently queued buffer there is a method on the video aggregator pad
now.
Crossfading is a bit more complex than just having two pads with the
right keyframes as the blending is not exactly the same.
The difference is in the way we compute the alpha channel, in the case
of crossfading, we have to compute an additive operation between
the destination and the source (factored by the alpha property of both
the input pad alpha property and the crossfading ratio) basically so
that the crossfade result of 2 opaque frames is also fully opaque at any
time in the crossfading process, avoid bleeding through the layer
blending.
Some rationnal can be found in https://phabricator.freedesktop.org/T7773.
https://bugzilla.gnome.org/show_bug.cgi?id=784827
When caps changes while streaming, the new caps was getting processed
immediately in videoaggregator, but the next buffer in the queue that
corresponds to this new caps was not necessarily being used immediately,
which resulted sometimes in using an old buffer with new caps. Of course
there used to be a separate buffer_vinfo for mapping the buffer with its
own caps, but in compositor the GstVideoConverter was still using wrong
info and resulted in invalid reads and corrupt output.
This approach here is more safe. We delay using the new caps
until we actually select the next buffer in the queue for use.
This way we also eliminate the need for buffer_vinfo, since the
pad->info is always in sync with the format of the selected buffer.
https://bugzilla.gnome.org/show_bug.cgi?id=780682
Compositor does not support it currently and it needs special support
for handling this correctly, and is rather non-trivial to implement for
all formats.
As is done everywhere else, and avoids setting bogus values
And remove useless *<val> checks (we always provide valid values and
it's an internal function).
CID #1320700
Allows the subclass to completely override the chosen src caps.
This is needed as videoaggregator generally has no idea exactly
what operation is being performed.
- Adds a fixate_caps vfunc for fixation
- Merges gst_video_aggregator_update_converters() into
gst_videoaggregator_update_src_caps() as we need some of its info
for proper caps handling.
- Pass the downstream caps to the update_caps vfunc
https://bugzilla.gnome.org/show_bug.cgi?id=756207
x/y/w/h are signed integers. As can be seen in GstCompositorPad.
The prototype for clamp_rectangle was wrong. This commit reverts the change
and fixes the prototype.
This reverts commit bca444ea4a.
CLAMP checks both if value is '< 0' and '> max'. Value will never be a negative
number since it is an unsigned integer. Removing that check and only checking if
it is bigger than max by using MIN().
CID 1320707
The obscured check in compositor was using the dimensions of the pad to clamp
the h/w of the pad instead of the output resolution, and was doing an incorrect
calculation to do so. Fix that by simplifying the whole calculation by using
corner coordinates. Also add a test for this bug which fell through the cracks,
and just skip all the obscured tests if the pad's alpha is 0.0.
https://bugzilla.gnome.org/show_bug.cgi?id=754107
Rather than one of the input pad video info's.
The test checking this was not constraining the output frame size
to ensure that the out of frame stream was not being displayed.
It's a waste of resources to map it if it won't be converted
or used at all. Since we moved the frame mapping down, we need
to use the GST_VIDEO_INFO accessor macros now in the code above
that instead of the GST_VIDEO_FRAME accessor macros.
https://bugzilla.gnome.org/show_bug.cgi?id=746147
For each frame, compare the frame boundaries, check if the format contains an
alpha channel, check opacity, and skip the frame if it's going to be completely
overwritten by a higher zorder frame. The check is O(n^2), but that doesn't
matter here because the number of sinkpads is small.
More can be done to avoid needless drawing, but this covers the majority of
cases. See TODOs. Ideally, a reverse painter's algorithm should be used for
optimal drawing, but memcpy during compositing is small compared to the CPU used
for frame conversion on each pad.
https://bugzilla.gnome.org/show_bug.cgi?id=746147
