Commit graph

71 commits

Author SHA1 Message Date
Mathieu Duponchelle
dd71f359be compositor: fix drawing of transparent background
When drawing the background multithreaded, y_start needs to be
scaled to obtain the correct byte offset from which to start
memsetting (yoffset).

Fixes #871

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1042>
2021-03-01 23:38:35 +00:00
Marijn Suijten
abb026ec6a gl,video: Make ptrs to VideoInfo and (GL)AllocationParams immutable
These parameters are incorrectly regarded as mutable in G-IR making them
"incompatible" with languages that are explicit about mutability like
Rust. In order to clean up the code and expected API there, update the
signatures here, right at the source (instead of overriding them in
Gir.toml and hoping for the best).

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1005>
2021-01-14 11:53:10 +00:00
Sebastian Dröge
831199a808 compositor: Don't crash in prepare_frame() if the pad was just removed
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/950>
2020-12-01 13:13:58 +02:00
Mathieu Duponchelle
fe20cac1a7 compositor: expose zero-size-is-unscaled property
When that property is left to its default, the width and height
property considers frames from input pads with width or height <= 0
should be left unscaled in that dimension.

Setting this property to FALSE changes that behaviour to < 0, as when
animating these properties, 0 should be a valid end value (eg. shrinking
an input stream until it disappears).

The default value of the width and height properties is set to -1, so that
the default behaviour stays consistent whether that new property is set
or not.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/923>
2020-11-13 14:33:54 +00:00
Thibault Saunier
d268c193ad videoaggregator: Guarantee that the output format is supported
In the case `videoaggregator` is set as allowing format conversions,
and as we convert only on the sinkpads, we should ensure that the
chosen format is usable by the subclass. This in turns implies
that the format is usable on the srcpad.

When doing conversion *any* format can be used on the sinkpads, and this
is the only way that we can avoid race conditions during renegotiations
so we can not change that fact, we just need to ensure that the chosen
intermediary format is usable, which was not actually ensured before
that patch.

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/issues/834

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/909>
2020-11-03 00:10:31 +00:00
Matthew Waters
d7abf832af compositor: blend with multiple threads
Increases the throughput of compositing by using more CPU cycles across
multiple threads.  Simple cases (the output contains one pixel from at
most one input) can have up to a 70% increase in throughput.  Not so
simple cases are limited by the region with the most number of
composite operations.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/755>
2020-09-14 14:00:58 +10:00
Thibault Saunier
5ad727f9ee compositor: Frames where we introduce alpha can't obscure anything
When a pad has alpha != 1.0 it means that the resulting frames will
contain alpha and thus can't fully obscure with a lower zorder.

Also simplifies the other checks as blending with an OVER or on a
transparent is not a no-op as previously assumed.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/764>
2020-07-31 15:52:14 -04:00
Mathieu Duponchelle
cc516695b0 plugins: uddate gst_type_mark_as_plugin_api() calls 2020-06-06 00:41:17 +02:00
Mathieu Duponchelle
e666c9ec04 plugins: Use gst_type_mark_as_plugin_api() for all non-element plugin types 2020-06-03 22:44:23 -04:00
Sebastian Dröge
18ee5e57fd compositor: Alpha inputs with the SOURCE operator can be considered opaque
We don't have to look at each pixel's alpha component because we will
directly write it over the background.
2020-01-07 20:03:59 +02:00
Seungha Yang
dc274ea9ca compositor: Add support for VUYA format
Reversed order of AYUV format. Most of core methods are prepared
already.
2019-11-04 14:50:28 +00:00
Thibault Saunier
909baa2360 Pass the code through codespell 2019-08-30 13:05:36 +00:00
Tim-Philipp Müller
ff0f659ca5 compositor: fix compiler warning due to c99-ism 2019-06-24 09:44:29 +00:00
Nirbheek Chauhan
abd80b6561 compositor: Copy frames as-is when possible
The blend functions for alpha formats need to do more work than just
doing a memcpy, so we can do a memcpy when we know that a blend is not
actually needed.

1080p AYUV ! compositor background=transparent ! fakesink - 56% faster

Specifically, when we don't draw the background and the first pad we
draw completely covers the output frame, we can just copy it as-is.
The rest of the pads (if any) will get composited on top normally.
2019-06-14 02:50:51 +05:30
Nirbheek Chauhan
bb1650cfda compositor: Sprinkle some const in prototypes
These helper functions don't edit the rectangles passed in.
2019-06-13 20:31:50 +05:30
Nirbheek Chauhan
ba4484c6a0 compositor: Skip background if transparent and obscured
If the background is transparent and obscured by a pad that may or may
not have alpha, we can still skip drawing it entirely

AYUV 1080p ! compositor background=transparent ! fakesink - 75% faster
2019-06-13 20:31:50 +05:30
Nirbheek Chauhan
ba20aec539 compositor: Skip the background when not visible
We don't need to waste time drawing the background when one of the
pads completely covers the output and there's no alpha on the pad or
in the video format. Speedups:

I420 1080p ! compositor ! fakesink - 72% faster
I420 1080p ! compositor background=black ! fakesink - 45% faster
2019-06-13 20:31:50 +05:30
Nirbheek Chauhan
ecec30c1ca compositor: Factor-out rectangle-obscuring check
We're going to use this for checking if one of the pads obscures the
background.
2019-06-13 20:31:50 +05:30
Nirbheek Chauhan
a3462a2c6d compositor: Add some comments, remove outdated ones 2019-06-13 20:31:50 +05:30
Nirbheek Chauhan
cf3287dba4 compositor: Remove unused function argument 2019-06-13 20:31:50 +05:30
Víctor Manuel Jáquez Leal
95436fa667 compositor: Fix enum type mismatch
The variable blend_mode is GstCompositorBlendMode but it is
assigned to a GstCompositorOperator enum value.
2018-10-31 19:22:35 +01:00
Sebastian Dröge
e87fa73e60 compositor: Implement different operators via per-pad property
This removes the crossfade-ratio property and replaces it with an
operator property. Currently this implements the following operators:

- SOURCE: Copy over the source and don't look at the destination
- OVER: Default blending of the source over the destination
- ADD: Like OVER but simply adding the alpha instead

See the example for how to implement crossfading with this.

https://bugzilla.gnome.org/show_bug.cgi?id=797169
2018-10-28 17:13:26 +00:00
Sebastian Dröge
c41439d16f compositor: Remove extra header for the pad declaration
There's no reason for having this separate apart from making things less
discoverable.
2018-10-27 13:59:57 +01:00
Sebastian Dröge
5485a2b9ee compositor: Define crossfade-ratio to have range [0.0,1.0]
Previously negative values had the same effect as 0.0, which was
confusing.

https://bugzilla.gnome.org/show_bug.cgi?id=796845
2018-08-16 17:08:06 +03:00
Sebastian Dröge
3e067ae333 compositor: Don't leak all buffers while crossfading and not all pads are crossfading 2018-07-26 00:20:02 +03:00
Seungha Yang
4380d57779 compositor: Update conversion info in property setter
... not in getter. Otherwise, video-converter will not be updated
with new width/height

https://bugzilla.gnome.org/show_bug.cgi?id=796828
2018-07-18 14:41:31 +03:00
Sebastian Dröge
33a8e28207 videoaggregator: Switch to a GstVideoAggregatorConvertPad subclass
This moves all the conversion related code to a single place, allows
less code-duplication inside compositor and makes the glmixer code less
awkward. It's also the same pattern as used by GstAudioAggregator.
2018-05-06 15:22:51 +02:00
Sebastian Dröge
233ee9f077 videoaggregator: Move needs_alpha pad field to the private struct
And also trigger renegotiation if the value has changed.

https://bugzilla.gnome.org/show_bug.cgi?id=795836
2018-05-05 16:32:19 +02:00
Sebastian Dröge
1c8110ab17 videoaggregator: Move aggregated_frame and the pad buffer into the private struct
The aggregated_frame is now called prepared_frame and passed to the
prepare_frame and cleanup_frame virtual methods directly. For the
currently queued buffer there is a method on the video aggregator pad
now.
2018-05-05 16:32:19 +02:00
Sebastian Dröge
a240fb7997 videoaggregator: Move GstChildProxy implementations into leaf classes
Not every subclass will want to expose the pads via the interface.

https://bugzilla.gnome.org/show_bug.cgi?id=739011
2018-05-04 16:13:16 +02:00
Mathieu Duponchelle
b59da8eced Port to latest GstAggregator segment API
The aggregator segment is now exposed on the src pad

https://bugzilla.gnome.org/show_bug.cgi?id=793946
2018-03-01 15:37:06 +01:00
Mathieu Duponchelle
14f886b7b0 aggregator: Remove klass->sinkpads_type
This posed problems for the python bindings (and possibly others).

Instead, subclasses now use add_pad_template_with_gtype.

https://bugzilla.gnome.org/show_bug.cgi?id=789986
2017-11-22 16:52:29 +01:00
Tim-Philipp Müller
eca9739bca videoaggregator: drop ABI compat in padding for new struct member
Don't really have to do that while it's in -bad and most users
are in-tree anyway.
2017-11-02 11:57:24 +00:00
Mathieu Duponchelle
663ef174d8 compositor: improve conversion debugging
https://bugzilla.gnome.org/show_bug.cgi?id=786078
2017-08-10 13:55:59 +02:00
Thibault Saunier
effb593a93 compositor: Add support for crossfade blending
Crossfading is a bit more complex than just having two pads with the
right keyframes as the blending is not exactly the same.

The difference is in the way we compute the alpha channel, in the case
of crossfading, we have to compute an additive operation between
the destination and the source (factored by the alpha property of both
the input pad alpha property and the crossfading ratio) basically so
that the crossfade result of 2 opaque frames is also fully opaque at any
time in the crossfading process, avoid bleeding through the layer
blending.

Some rationnal can be found in https://phabricator.freedesktop.org/T7773.

https://bugzilla.gnome.org/show_bug.cgi?id=784827
2017-08-03 12:59:43 -04:00
Matthew Waters
719498601f aggregator: add simple support for caps handling
Modelled off the videoaggregator caps handling as that seems the most
mature aggregtor-using implementation that has caps handling there is.

https://bugzilla.gnome.org/show_bug.cgi?id=776931
2017-05-20 16:21:17 +02:00
George Kiagiadakis
2a60a9f66f videoaggregator: delay using new caps from a sink pad until the next buffer in the queue is taken
When caps changes while streaming, the new caps was getting processed
immediately in videoaggregator, but the next buffer in the queue that
corresponds to this new caps was not necessarily being used immediately,
which resulted sometimes in using an old buffer with new caps. Of course
there used to be a separate buffer_vinfo for mapping the buffer with its
own caps, but in compositor the GstVideoConverter was still using wrong
info and resulted in invalid reads and corrupt output.

This approach here is more safe. We delay using the new caps
until we actually select the next buffer in the queue for use.
This way we also eliminate the need for buffer_vinfo, since the
pad->info is always in sync with the format of the selected buffer.

https://bugzilla.gnome.org/show_bug.cgi?id=780682
2017-05-20 16:21:16 +02:00
Thibault Saunier
2bf2696945 docs: Port all docstring to gtk-doc markdown 2017-04-12 12:57:57 -03:00
Sebastian Dröge
aee020c26f glvideomixer/compositor: Correctly error out if calculating DAR fails
CID 1320700
2017-02-28 13:06:41 +02:00
Sebastian Dröge
52fe0dbeaa compositor: Reject multiview video
Compositor does not support it currently and it needs special support
for handling this correctly, and is rather non-trivial to implement for
all formats.
2016-12-14 15:36:14 +02:00
Edward Hervey
d44a77a5e8 compositor: Check if we get a valid display ratio
As is done everywhere else, and avoids setting bogus values
And remove useless *<val> checks (we always provide valid values and
it's an internal function).

CID #1320700
2016-05-15 14:35:38 +02:00
Sebastian Dröge
905c1600b1 compositor: Set blend functions in ::negotiated_caps() instead of ::fixate_caps()
The latter should not change any state but just fixate the caps, while the
former is always called when srcpads caps are decided.

https://bugzilla.gnome.org/show_bug.cgi?id=765324
2016-05-03 16:53:25 +03:00
Reynaldo H. Verdejo Pinochet
cd4e3e0f0a Drop usage of 'overlayed' to mean 'overlaid' 2016-04-15 13:39:32 -07:00
Vineeth TM
8741806671 bad: use new gst_element_class_add_static_pad_template()
https://bugzilla.gnome.org/show_bug.cgi?id=763081
2016-03-24 14:56:51 +02:00
Matthew Waters
f1323fb6df videoaggregator: don't do caps processing that is not overridable
Allows the subclass to completely override the chosen src caps.

This is needed as videoaggregator generally has no idea exactly
what operation is being performed.

- Adds a fixate_caps vfunc for fixation
- Merges gst_video_aggregator_update_converters() into
  gst_videoaggregator_update_src_caps() as we need some of its info
  for proper caps handling.
- Pass the downstream caps to the update_caps vfunc

https://bugzilla.gnome.org/show_bug.cgi?id=756207
2016-01-27 20:36:25 +11:00
Nirbheek Chauhan
f1f8e465f2 compositor: Ensure all arguments to CLAMP are signed int
If any of the arguments to CLAMP are unsigned integers, the comparison causes
an automatic conversion of the signed int to unsigned, which causes -1 to become
UINT_MAX and get clamped to the high value of the CLAMP instead of 0.

See 716 at http://c0x.coding-guidelines.com/6.3.1.8.html

Also add a test for this.

https://bugzilla.gnome.org/show_bug.cgi?id=754576
2015-09-04 23:32:36 +02:00
Nirbheek Chauhan
8c06c85d83 compositor: variables in clamp_rectangle() should be signed
x/y/w/h are signed integers. As can be seen in GstCompositorPad.
The prototype for clamp_rectangle was wrong. This commit reverts the change
and fixes the prototype.

This reverts commit bca444ea4a.
2015-08-29 17:27:45 +01:00
Luis de Bethencourt
e4b584b3e5 compositor: remove check for below zero for unsigned value
CLAMP checks both if value is '< 0' and '> max'. Value will never be a negative
number since it is an unsigned integer. Removing that check and only checking if
it is bigger than max by using MIN().

CID 1320707
2015-08-28 15:21:13 +01:00
Nirbheek Chauhan
f12ef34410 compositor: Actually use the output resolution for clamping
The obscured check in compositor was using the dimensions of the pad to clamp
the h/w of the pad instead of the output resolution, and was doing an incorrect
calculation to do so. Fix that by simplifying the whole calculation by using
corner coordinates. Also add a test for this bug which fell through the cracks,
and just skip all the obscured tests if the pad's alpha is 0.0.

https://bugzilla.gnome.org/show_bug.cgi?id=754107
2015-08-26 15:03:05 +03:00
Mathieu Duponchelle
98c6860097 compositor: update zorder documentation.
It is not bound between 0 and 10000 anymore.
2015-06-14 23:20:38 +02:00