Commit graph

80 commits

Author SHA1 Message Date
Nirbheek Chauhan
abd80b6561 compositor: Copy frames as-is when possible
The blend functions for alpha formats need to do more work than just
doing a memcpy, so we can do a memcpy when we know that a blend is not
actually needed.

1080p AYUV ! compositor background=transparent ! fakesink - 56% faster

Specifically, when we don't draw the background and the first pad we
draw completely covers the output frame, we can just copy it as-is.
The rest of the pads (if any) will get composited on top normally.
2019-06-14 02:50:51 +05:30
Nirbheek Chauhan
bb1650cfda compositor: Sprinkle some const in prototypes
These helper functions don't edit the rectangles passed in.
2019-06-13 20:31:50 +05:30
Nirbheek Chauhan
ba4484c6a0 compositor: Skip background if transparent and obscured
If the background is transparent and obscured by a pad that may or may
not have alpha, we can still skip drawing it entirely

AYUV 1080p ! compositor background=transparent ! fakesink - 75% faster
2019-06-13 20:31:50 +05:30
Nirbheek Chauhan
ba20aec539 compositor: Skip the background when not visible
We don't need to waste time drawing the background when one of the
pads completely covers the output and there's no alpha on the pad or
in the video format. Speedups:

I420 1080p ! compositor ! fakesink - 72% faster
I420 1080p ! compositor background=black ! fakesink - 45% faster
2019-06-13 20:31:50 +05:30
Nirbheek Chauhan
65615a841d compositor: Don't log per-frame under GST_INFO 2019-06-13 20:31:50 +05:30
Nirbheek Chauhan
ecec30c1ca compositor: Factor-out rectangle-obscuring check
We're going to use this for checking if one of the pads obscures the
background.
2019-06-13 20:31:50 +05:30
Nirbheek Chauhan
a3462a2c6d compositor: Add some comments, remove outdated ones 2019-06-13 20:31:50 +05:30
Nirbheek Chauhan
cf3287dba4 compositor: Remove unused function argument 2019-06-13 20:31:50 +05:30
Sebastian Dröge
779704b885 compositor: Replace shift and conv opcodes by convh in BGRA SOURCE operator
Potentially speeds up processing a bit.
2019-05-24 15:56:56 +00:00
Sebastian Dröge
2933ef6336 compositor: Remove unneeded left shift for ARGB/AYUV SOURCE operator
The alpha value is already in the lower 8 bits from the beginning in
this case.

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/issues/610
2019-05-24 15:56:56 +00:00
Thibault Saunier
27ba8d24ec doc: Port to hotdoc 2019-05-13 11:34:08 -04:00
Tim-Philipp Müller
2972b673c0 compositor: add to build after move from -bad
This replaces videomixer.

Fixes #138
2018-12-28 12:16:18 +01:00
Tim-Philipp Müller
fb8a503fd4 compositor: update disted orc backup files 2018-11-02 20:31:54 +00:00
Víctor Manuel Jáquez Leal
95436fa667 compositor: Fix enum type mismatch
The variable blend_mode is GstCompositorBlendMode but it is
assigned to a GstCompositorOperator enum value.
2018-10-31 19:22:35 +01:00
Johan Bjäreholt
4ab4db11ec compositor: fix undeclared functions 2018-10-30 13:32:33 +01:00
Sebastian Dröge
e87fa73e60 compositor: Implement different operators via per-pad property
This removes the crossfade-ratio property and replaces it with an
operator property. Currently this implements the following operators:

- SOURCE: Copy over the source and don't look at the destination
- OVER: Default blending of the source over the destination
- ADD: Like OVER but simply adding the alpha instead

See the example for how to implement crossfading with this.

https://bugzilla.gnome.org/show_bug.cgi?id=797169
2018-10-28 17:13:26 +00:00
Sebastian Dröge
c41439d16f compositor: Remove extra header for the pad declaration
There's no reason for having this separate apart from making things less
discoverable.
2018-10-27 13:59:57 +01:00
Seungha Yang
5adec0db4f meson: Replace empty configuration_data() with copy keyword
Use 'copy' keyword to avoid meson warning message.
Note that 'copy' keyword in configure_file() is available
since meson 0.47.0

https://bugzilla.gnome.org/show_bug.cgi?id=797298
2018-10-17 14:08:47 +01:00
Sebastian Dröge
5485a2b9ee compositor: Define crossfade-ratio to have range [0.0,1.0]
Previously negative values had the same effect as 0.0, which was
confusing.

https://bugzilla.gnome.org/show_bug.cgi?id=796845
2018-08-16 17:08:06 +03:00
Sebastian Dröge
3e067ae333 compositor: Don't leak all buffers while crossfading and not all pads are crossfading 2018-07-26 00:20:02 +03:00
Sebastian Dröge
aa3e64c71d compositor: Use 255 as maximum alpha instead of 256
255 will easily become 0 in the blending function as they expect
the maximum value to be 255.

Can be reproduce with

gst-launch-1.0 videotestsrc pattern=ball ! c.sink_0 \
               videotestsrc pattern=snow ! c.sink_1 \
               compositor name=c \
                 sink_0::zorder=0 sink_1::zorder=1 sink_0::crossfade-ratio=0.5 \
                 background=black ! \
               videoconvert ! xvimagesink

crossfade-ratio +/- 0.001 makes it work correctly and the same happens
at e.g. 0.25, 0.75, N*0.0625

https://bugzilla.gnome.org/show_bug.cgi?id=796846
2018-07-23 18:59:33 +03:00
Seungha Yang
4380d57779 compositor: Update conversion info in property setter
... not in getter. Otherwise, video-converter will not be updated
with new width/height

https://bugzilla.gnome.org/show_bug.cgi?id=796828
2018-07-18 14:41:31 +03:00
Sebastian Dröge
33a8e28207 videoaggregator: Switch to a GstVideoAggregatorConvertPad subclass
This moves all the conversion related code to a single place, allows
less code-duplication inside compositor and makes the glmixer code less
awkward. It's also the same pattern as used by GstAudioAggregator.
2018-05-06 15:22:51 +02:00
Sebastian Dröge
233ee9f077 videoaggregator: Move needs_alpha pad field to the private struct
And also trigger renegotiation if the value has changed.

https://bugzilla.gnome.org/show_bug.cgi?id=795836
2018-05-05 16:32:19 +02:00
Sebastian Dröge
1c8110ab17 videoaggregator: Move aggregated_frame and the pad buffer into the private struct
The aggregated_frame is now called prepared_frame and passed to the
prepare_frame and cleanup_frame virtual methods directly. For the
currently queued buffer there is a method on the video aggregator pad
now.
2018-05-05 16:32:19 +02:00
Sebastian Dröge
a240fb7997 videoaggregator: Move GstChildProxy implementations into leaf classes
Not every subclass will want to expose the pads via the interface.

https://bugzilla.gnome.org/show_bug.cgi?id=739011
2018-05-04 16:13:16 +02:00
Xavier Claessens
9ae9d2686f Meson: Generate pc file for all plugins in bad
https://bugzilla.gnome.org/show_bug.cgi?id=794568
2018-04-25 11:08:09 +01:00
Mathieu Duponchelle
b59da8eced Port to latest GstAggregator segment API
The aggregator segment is now exposed on the src pad

https://bugzilla.gnome.org/show_bug.cgi?id=793946
2018-03-01 15:37:06 +01:00
Edward Hervey
3b9843d13b Update ORC fallback disted code 2018-02-14 14:36:00 +01:00
Tim-Philipp Müller
2b933a2827 Remove GstAggregator from -bad, moved to core
https://bugzilla.gnome.org/show_bug.cgi?id=739010
2017-12-02 16:14:36 +00:00
Mathieu Duponchelle
14f886b7b0 aggregator: Remove klass->sinkpads_type
This posed problems for the python bindings (and possibly others).

Instead, subclasses now use add_pad_template_with_gtype.

https://bugzilla.gnome.org/show_bug.cgi?id=789986
2017-11-22 16:52:29 +01:00
Tim-Philipp Müller
eca9739bca videoaggregator: drop ABI compat in padding for new struct member
Don't really have to do that while it's in -bad and most users
are in-tree anyway.
2017-11-02 11:57:24 +00:00
Mathieu Duponchelle
663ef174d8 compositor: improve conversion debugging
https://bugzilla.gnome.org/show_bug.cgi?id=786078
2017-08-10 13:55:59 +02:00
Tim-Philipp Müller
61b54cdb40 compositor: update disted orc fallback files 2017-08-03 20:14:41 +01:00
Thibault Saunier
effb593a93 compositor: Add support for crossfade blending
Crossfading is a bit more complex than just having two pads with the
right keyframes as the blending is not exactly the same.

The difference is in the way we compute the alpha channel, in the case
of crossfading, we have to compute an additive operation between
the destination and the source (factored by the alpha property of both
the input pad alpha property and the crossfading ratio) basically so
that the crossfade result of 2 opaque frames is also fully opaque at any
time in the crossfading process, avoid bleeding through the layer
blending.

Some rationnal can be found in https://phabricator.freedesktop.org/T7773.

https://bugzilla.gnome.org/show_bug.cgi?id=784827
2017-08-03 12:59:43 -04:00
Matthew Waters
719498601f aggregator: add simple support for caps handling
Modelled off the videoaggregator caps handling as that seems the most
mature aggregtor-using implementation that has caps handling there is.

https://bugzilla.gnome.org/show_bug.cgi?id=776931
2017-05-20 16:21:17 +02:00
George Kiagiadakis
2a60a9f66f videoaggregator: delay using new caps from a sink pad until the next buffer in the queue is taken
When caps changes while streaming, the new caps was getting processed
immediately in videoaggregator, but the next buffer in the queue that
corresponds to this new caps was not necessarily being used immediately,
which resulted sometimes in using an old buffer with new caps. Of course
there used to be a separate buffer_vinfo for mapping the buffer with its
own caps, but in compositor the GstVideoConverter was still using wrong
info and resulted in invalid reads and corrupt output.

This approach here is more safe. We delay using the new caps
until we actually select the next buffer in the queue for use.
This way we also eliminate the need for buffer_vinfo, since the
pad->info is always in sync with the format of the selected buffer.

https://bugzilla.gnome.org/show_bug.cgi?id=780682
2017-05-20 16:21:16 +02:00
Nicolas Dufresne
c483cbf904 Remove plugin specific static build option
Static and dynamic plugins now have the same interface. The standard
--enable-static/--enable-shared toggle are sufficient.
2017-05-16 14:05:52 -04:00
Thibault Saunier
2bf2696945 docs: Port all docstring to gtk-doc markdown 2017-04-12 12:57:57 -03:00
Sebastian Dröge
aee020c26f glvideomixer/compositor: Correctly error out if calculating DAR fails
CID 1320700
2017-02-28 13:06:41 +02:00
Sebastian Dröge
52fe0dbeaa compositor: Reject multiview video
Compositor does not support it currently and it needs special support
for handling this correctly, and is rather non-trivial to implement for
all formats.
2016-12-14 15:36:14 +02:00
Nirbheek Chauhan
d865d9126b Add support for Meson as alternative/parallel build system
https://github.com/mesonbuild/meson

With contributions from:

Tim-Philipp Müller <tim@centricular.com>
Matej Knopp <matej.knopp@gmail.com>
Jussi Pakkanen <jpakkane@gmail.com> (original port)

Highlights of the features provided are:
* Faster builds on Linux (~40-50% faster)
* The ability to build with MSVC on Windows
* Generate Visual Studio project files
* Generate XCode project files
* Much faster builds on Windows (on-par with Linux)
* Seriously fast configure and building on embedded

... and many more. For more details see:

http://blog.nirbheek.in/2016/05/gstreamer-and-meson-new-hope.html
http://blog.nirbheek.in/2016/07/building-and-developing-gstreamer-using.html

Building with Meson should work on both Linux and Windows, but may
need a few more tweaks on other operating systems.
2016-08-20 11:35:54 +01:00
Edward Hervey
d44a77a5e8 compositor: Check if we get a valid display ratio
As is done everywhere else, and avoids setting bogus values
And remove useless *<val> checks (we always provide valid values and
it's an internal function).

CID #1320700
2016-05-15 14:35:38 +02:00
Sebastian Dröge
905c1600b1 compositor: Set blend functions in ::negotiated_caps() instead of ::fixate_caps()
The latter should not change any state but just fixate the caps, while the
former is always called when srcpads caps are decided.

https://bugzilla.gnome.org/show_bug.cgi?id=765324
2016-05-03 16:53:25 +03:00
Reynaldo H. Verdejo Pinochet
cd4e3e0f0a Drop usage of 'overlayed' to mean 'overlaid' 2016-04-15 13:39:32 -07:00
Vineeth TM
8741806671 bad: use new gst_element_class_add_static_pad_template()
https://bugzilla.gnome.org/show_bug.cgi?id=763081
2016-03-24 14:56:51 +02:00
Matthew Waters
f1323fb6df videoaggregator: don't do caps processing that is not overridable
Allows the subclass to completely override the chosen src caps.

This is needed as videoaggregator generally has no idea exactly
what operation is being performed.

- Adds a fixate_caps vfunc for fixation
- Merges gst_video_aggregator_update_converters() into
  gst_videoaggregator_update_src_caps() as we need some of its info
  for proper caps handling.
- Pass the downstream caps to the update_caps vfunc

https://bugzilla.gnome.org/show_bug.cgi?id=756207
2016-01-27 20:36:25 +11:00
Nirbheek Chauhan
f1f8e465f2 compositor: Ensure all arguments to CLAMP are signed int
If any of the arguments to CLAMP are unsigned integers, the comparison causes
an automatic conversion of the signed int to unsigned, which causes -1 to become
UINT_MAX and get clamped to the high value of the CLAMP instead of 0.

See 716 at http://c0x.coding-guidelines.com/6.3.1.8.html

Also add a test for this.

https://bugzilla.gnome.org/show_bug.cgi?id=754576
2015-09-04 23:32:36 +02:00
Nirbheek Chauhan
8c06c85d83 compositor: variables in clamp_rectangle() should be signed
x/y/w/h are signed integers. As can be seen in GstCompositorPad.
The prototype for clamp_rectangle was wrong. This commit reverts the change
and fixes the prototype.

This reverts commit bca444ea4a.
2015-08-29 17:27:45 +01:00
Luis de Bethencourt
e4b584b3e5 compositor: remove check for below zero for unsigned value
CLAMP checks both if value is '< 0' and '> max'. Value will never be a negative
number since it is an unsigned integer. Removing that check and only checking if
it is bigger than max by using MIN().

CID 1320707
2015-08-28 15:21:13 +01:00