Commit graph

105 commits

Author SHA1 Message Date
Mathieu Duponchelle
dd71f359be compositor: fix drawing of transparent background
When drawing the background multithreaded, y_start needs to be
scaled to obtain the correct byte offset from which to start
memsetting (yoffset).

Fixes #871

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1042>
2021-03-01 23:38:35 +00:00
Marijn Suijten
abb026ec6a gl,video: Make ptrs to VideoInfo and (GL)AllocationParams immutable
These parameters are incorrectly regarded as mutable in G-IR making them
"incompatible" with languages that are explicit about mutability like
Rust. In order to clean up the code and expected API there, update the
signatures here, right at the source (instead of overriding them in
Gir.toml and hoping for the best).

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1005>
2021-01-14 11:53:10 +00:00
Mathieu Duponchelle
e0a4d3ac4e compositor/blend.c: fix MT checker pattern
When filling the checker pattern from multiple threads, y_start
needs to be taken into account to determine the shade of the
current pixel.

Example pipeline:

gst-launch-1.0 videotestsrc ! video/x-raw, width=1920, height=1080, format=I420 ! \
  queue ! compositor sink_0::xpos=200 ! video/x-raw, format=I420 ! videoconvert ! \
  xvimagesink

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/988>
2020-12-29 22:48:08 +00:00
Mathieu Duponchelle
247f821826 compositor/blend.c: fix MT fill_checker for YUY2
The initial byte offset should be calculated from the stride,
not from the dest_add variable

Example pipeline:

gst-launch-1.0 videotestsrc ! video/x-raw, width=1920, height=1080, format=YUY2 ! \
  queue ! compositor sink_0::xpos=200 ! video/x-raw, format=YUY2 ! xvimagesink

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/988>
2020-12-29 22:48:08 +00:00
Mathieu Duponchelle
8ff5079e5e compositor/blend: fix blending of subsampled components
The correct way to determine the byte offset at a certain yoffset
in a subsampled component is to shift the yoffset by the component's
hsub

This fixes out-of-bounds memory accesses and visible artefacts,
example pipeline with the samples from #802:

gst-launch-1.0 compositor name=vmixer sink_1::xpos=1910 sink_1::ypos=1080 ! \
    videoconvert ! videorate ! xvimagesink \
  filesrc location=VID_20200723_203606.mp4 ! decodebin name=demux1 ! \
    queue ! videoflip method=vertical-flip ! vmixer. \
  filesrc location=bridgeoverstubbledwater.mp4 ! decodebin name=demux2 ! \
    queue ! vmixer.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/988>
2020-12-29 22:48:08 +00:00
Sebastian Dröge
831199a808 compositor: Don't crash in prepare_frame() if the pad was just removed
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/950>
2020-12-01 13:13:58 +02:00
Mathieu Duponchelle
fe20cac1a7 compositor: expose zero-size-is-unscaled property
When that property is left to its default, the width and height
property considers frames from input pads with width or height <= 0
should be left unscaled in that dimension.

Setting this property to FALSE changes that behaviour to < 0, as when
animating these properties, 0 should be a valid end value (eg. shrinking
an input stream until it disappears).

The default value of the width and height properties is set to -1, so that
the default behaviour stays consistent whether that new property is set
or not.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/923>
2020-11-13 14:33:54 +00:00
Thibault Saunier
d268c193ad videoaggregator: Guarantee that the output format is supported
In the case `videoaggregator` is set as allowing format conversions,
and as we convert only on the sinkpads, we should ensure that the
chosen format is usable by the subclass. This in turns implies
that the format is usable on the srcpad.

When doing conversion *any* format can be used on the sinkpads, and this
is the only way that we can avoid race conditions during renegotiations
so we can not change that fact, we just need to ensure that the chosen
intermediary format is usable, which was not actually ensured before
that patch.

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/issues/834

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/909>
2020-11-03 00:10:31 +00:00
Matthew Waters
0236fa8b9e compositor: fix YUY2/UYVY/VYUY blending
Don't overead memory.

Fixup for https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/755

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/823>
2020-09-21 23:51:31 +10:00
Matthew Waters
d7abf832af compositor: blend with multiple threads
Increases the throughput of compositing by using more CPU cycles across
multiple threads.  Simple cases (the output contains one pixel from at
most one input) can have up to a 70% increase in throughput.  Not so
simple cases are limited by the region with the most number of
composite operations.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/755>
2020-09-14 14:00:58 +10:00
Jan Schmidt
8cacd54e8f compositor: fix off-by-1 error in blending ARGB
Use a more naive blend formulation that avoids off-by-one errors
at the 0x00 and 0xff alpha endpoints.

    Fixes #810

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/796>
2020-08-26 13:10:09 +00:00
Thibault Saunier
5ad727f9ee compositor: Frames where we introduce alpha can't obscure anything
When a pad has alpha != 1.0 it means that the resulting frames will
contain alpha and thus can't fully obscure with a lower zorder.

Also simplifies the other checks as blending with an OVER or on a
transparent is not a no-op as previously assumed.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/764>
2020-07-31 15:52:14 -04:00
Tim-Philipp Müller
d5a677683e Update disted generated orc backup files
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/734>
2020-07-04 14:01:56 +01:00
Tim-Philipp Müller
6bb3e01918 meson: add update-orc-dist target
Add target to update backup orc -dist.[ch] files.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/734>
2020-07-04 14:01:56 +01:00
Mathieu Duponchelle
cc516695b0 plugins: uddate gst_type_mark_as_plugin_api() calls 2020-06-06 00:41:17 +02:00
Mathieu Duponchelle
e666c9ec04 plugins: Use gst_type_mark_as_plugin_api() for all non-element plugin types 2020-06-03 22:44:23 -04:00
Niels De Graef
1b6a6cabf2 compositor: Use G_DECLARE_FINAL_TYPE 2020-03-16 15:47:58 +00:00
Sebastian Dröge
d5ee11fb36 compositor: Create a square checkerboard for UYVY/YUY2/YVYU too
Previously the "squares" were twice as wide.

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/issues/732
2020-03-09 21:51:32 +00:00
Sebastian Dröge
0227d0ca60 compositor: Define a separate checker fill function for BGRx/RGBx than for xBGR/xRGB
Otherwise we'll create a cyan or yellow checkerboard.

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/issues/736
2020-03-09 21:51:32 +00:00
Sebastian Dröge
1c147208cc compositor: memcpy() lines directly for alpha formats with SOURCE operator and alpha=1.0 2020-01-08 16:20:30 +02:00
Sebastian Dröge
18ee5e57fd compositor: Alpha inputs with the SOURCE operator can be considered opaque
We don't have to look at each pixel's alpha component because we will
directly write it over the background.
2020-01-07 20:03:59 +02:00
Seungha Yang
dc274ea9ca compositor: Add support for VUYA format
Reversed order of AYUV format. Most of core methods are prepared
already.
2019-11-04 14:50:28 +00:00
Tim-Philipp Müller
289d8e53e2 Remove autotools build system 2019-10-13 14:15:43 +01:00
Thibault Saunier
909baa2360 Pass the code through codespell 2019-08-30 13:05:36 +00:00
Tim-Philipp Müller
ff0f659ca5 compositor: fix compiler warning due to c99-ism 2019-06-24 09:44:29 +00:00
Nirbheek Chauhan
abd80b6561 compositor: Copy frames as-is when possible
The blend functions for alpha formats need to do more work than just
doing a memcpy, so we can do a memcpy when we know that a blend is not
actually needed.

1080p AYUV ! compositor background=transparent ! fakesink - 56% faster

Specifically, when we don't draw the background and the first pad we
draw completely covers the output frame, we can just copy it as-is.
The rest of the pads (if any) will get composited on top normally.
2019-06-14 02:50:51 +05:30
Nirbheek Chauhan
bb1650cfda compositor: Sprinkle some const in prototypes
These helper functions don't edit the rectangles passed in.
2019-06-13 20:31:50 +05:30
Nirbheek Chauhan
ba4484c6a0 compositor: Skip background if transparent and obscured
If the background is transparent and obscured by a pad that may or may
not have alpha, we can still skip drawing it entirely

AYUV 1080p ! compositor background=transparent ! fakesink - 75% faster
2019-06-13 20:31:50 +05:30
Nirbheek Chauhan
ba20aec539 compositor: Skip the background when not visible
We don't need to waste time drawing the background when one of the
pads completely covers the output and there's no alpha on the pad or
in the video format. Speedups:

I420 1080p ! compositor ! fakesink - 72% faster
I420 1080p ! compositor background=black ! fakesink - 45% faster
2019-06-13 20:31:50 +05:30
Nirbheek Chauhan
65615a841d compositor: Don't log per-frame under GST_INFO 2019-06-13 20:31:50 +05:30
Nirbheek Chauhan
ecec30c1ca compositor: Factor-out rectangle-obscuring check
We're going to use this for checking if one of the pads obscures the
background.
2019-06-13 20:31:50 +05:30
Nirbheek Chauhan
a3462a2c6d compositor: Add some comments, remove outdated ones 2019-06-13 20:31:50 +05:30
Nirbheek Chauhan
cf3287dba4 compositor: Remove unused function argument 2019-06-13 20:31:50 +05:30
Sebastian Dröge
779704b885 compositor: Replace shift and conv opcodes by convh in BGRA SOURCE operator
Potentially speeds up processing a bit.
2019-05-24 15:56:56 +00:00
Sebastian Dröge
2933ef6336 compositor: Remove unneeded left shift for ARGB/AYUV SOURCE operator
The alpha value is already in the lower 8 bits from the beginning in
this case.

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/issues/610
2019-05-24 15:56:56 +00:00
Thibault Saunier
27ba8d24ec doc: Port to hotdoc 2019-05-13 11:34:08 -04:00
Tim-Philipp Müller
2972b673c0 compositor: add to build after move from -bad
This replaces videomixer.

Fixes #138
2018-12-28 12:16:18 +01:00
Tim-Philipp Müller
fb8a503fd4 compositor: update disted orc backup files 2018-11-02 20:31:54 +00:00
Víctor Manuel Jáquez Leal
95436fa667 compositor: Fix enum type mismatch
The variable blend_mode is GstCompositorBlendMode but it is
assigned to a GstCompositorOperator enum value.
2018-10-31 19:22:35 +01:00
Johan Bjäreholt
4ab4db11ec compositor: fix undeclared functions 2018-10-30 13:32:33 +01:00
Sebastian Dröge
e87fa73e60 compositor: Implement different operators via per-pad property
This removes the crossfade-ratio property and replaces it with an
operator property. Currently this implements the following operators:

- SOURCE: Copy over the source and don't look at the destination
- OVER: Default blending of the source over the destination
- ADD: Like OVER but simply adding the alpha instead

See the example for how to implement crossfading with this.

https://bugzilla.gnome.org/show_bug.cgi?id=797169
2018-10-28 17:13:26 +00:00
Sebastian Dröge
c41439d16f compositor: Remove extra header for the pad declaration
There's no reason for having this separate apart from making things less
discoverable.
2018-10-27 13:59:57 +01:00
Seungha Yang
5adec0db4f meson: Replace empty configuration_data() with copy keyword
Use 'copy' keyword to avoid meson warning message.
Note that 'copy' keyword in configure_file() is available
since meson 0.47.0

https://bugzilla.gnome.org/show_bug.cgi?id=797298
2018-10-17 14:08:47 +01:00
Sebastian Dröge
5485a2b9ee compositor: Define crossfade-ratio to have range [0.0,1.0]
Previously negative values had the same effect as 0.0, which was
confusing.

https://bugzilla.gnome.org/show_bug.cgi?id=796845
2018-08-16 17:08:06 +03:00
Sebastian Dröge
3e067ae333 compositor: Don't leak all buffers while crossfading and not all pads are crossfading 2018-07-26 00:20:02 +03:00
Sebastian Dröge
aa3e64c71d compositor: Use 255 as maximum alpha instead of 256
255 will easily become 0 in the blending function as they expect
the maximum value to be 255.

Can be reproduce with

gst-launch-1.0 videotestsrc pattern=ball ! c.sink_0 \
               videotestsrc pattern=snow ! c.sink_1 \
               compositor name=c \
                 sink_0::zorder=0 sink_1::zorder=1 sink_0::crossfade-ratio=0.5 \
                 background=black ! \
               videoconvert ! xvimagesink

crossfade-ratio +/- 0.001 makes it work correctly and the same happens
at e.g. 0.25, 0.75, N*0.0625

https://bugzilla.gnome.org/show_bug.cgi?id=796846
2018-07-23 18:59:33 +03:00
Seungha Yang
4380d57779 compositor: Update conversion info in property setter
... not in getter. Otherwise, video-converter will not be updated
with new width/height

https://bugzilla.gnome.org/show_bug.cgi?id=796828
2018-07-18 14:41:31 +03:00
Sebastian Dröge
33a8e28207 videoaggregator: Switch to a GstVideoAggregatorConvertPad subclass
This moves all the conversion related code to a single place, allows
less code-duplication inside compositor and makes the glmixer code less
awkward. It's also the same pattern as used by GstAudioAggregator.
2018-05-06 15:22:51 +02:00
Sebastian Dröge
233ee9f077 videoaggregator: Move needs_alpha pad field to the private struct
And also trigger renegotiation if the value has changed.

https://bugzilla.gnome.org/show_bug.cgi?id=795836
2018-05-05 16:32:19 +02:00
Sebastian Dröge
1c8110ab17 videoaggregator: Move aggregated_frame and the pad buffer into the private struct
The aggregated_frame is now called prepared_frame and passed to the
prepare_frame and cleanup_frame virtual methods directly. For the
currently queued buffer there is a method on the video aggregator pad
now.
2018-05-05 16:32:19 +02:00