Commit graph

1787 commits

Author SHA1 Message Date
Stéphane Cerveau
63de6d564e videodecoder: add API to receive subframes
A video decoder can now receive subframes and start decoding
instead of waiting for the full frame to be complete.
Subframe support will reduce latency as described in the
video encoder base class.

A unit test illustrating this API is available in
tests/check/libs/videodecoder.c.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/454>
2021-07-08 13:19:21 +02:00
Seungha Yang
2551b1d976 video: Deprecate gst_video_sink_center_rect()
... and add gst_video_center_rect() method as a replacement.
The method is useful for outside of videosink subclasses as well
but the old naming might be able to mislead people.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1156>
2021-06-29 18:00:43 +09:00
Jan Schmidt
8c04f4bdae video-converter: Set up matrix tables only once.
When configuring a multi-thread converter, only allocate the
shared colour conversion matrices once for the first thread,
to avoid allocating multiple times and leaking memory.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1216>
2021-06-26 04:00:08 +00:00
Jan Alexander Steffens (heftig)
ef324fa068 video-converter: Set up gamma tables only once
When the video converter is using multiple threads, the gamma tables
were created multiple times, leaking the tables set up for the previous
thread.

Only calculate the tables once.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1140>
2021-06-25 13:55:39 +00:00
Nicolas Dufresne
be6793d0d1 videodecoder: Call drain() rather then finish() on segment-done
The finish() virtual function documentation state that "Sub-classes can refuse
to decode new data after." Though, it is very common to issue a non-flushing
seek after that event in gapless playback uses case. This fixes potential
stalls with code using segment seeks, by using drain() virtual funciton
instead.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1206>
2021-06-23 20:31:06 +00:00
Sebastian Dröge
c18dd84f79 video: Sort video formats correctly
AV12 should be right after A420 because it is the same format with just
one plane less, instead of being next to I420/NV12 which don't have an
alpha channel.

RGBP should be before GBR because it's the same format except for the
more canonical component order.

See https://gitlab.freedesktop.org/gstreamer/gstreamer-rs/-/merge_requests/790
which actually checks on the CI if the algorithm defined in
video-format.h is implemented correctly.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1189>
2021-06-03 14:00:54 +03:00
Tim-Philipp Müller
577dabf7b1 Use g_memdup2() where available and add fallback for older GLib versions
g_memdup() is deprecated since GLib 2.68 and we want to avoid
deprecation warnings with recent versions of GLib.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1171>
2021-06-02 14:21:02 +00:00
Marijn Suijten
9a502c15c6 audio,video-format: Make generate_raw_formats idempotent for assertions
When compiling without assertions `g_assert` and its contents disappear
resulting in no list being deserialized at all and the
`gst_{audio,video}_formats_raw` functions to return an empty collection.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1177>
2021-05-26 23:35:16 +02:00
Nicolas Dufresne
0df31ae9d4 video: Update video-orc-dist
This adds the new symbols needed for AV12 support.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1152>
2021-05-21 21:16:24 -04:00
Daniel Almeida
6760c7fd76 video: add support for AV12
AV12 is an internally conceived format that is actually
the combination of NV12 and an alpha plane.

This format is to add to gstreamer's webM
transparency support for vp8 and vp9. To this end, two
I420 streams are independently decoded simultaneously for
the actual content and the alpha plane respectively
and these are then combined into A420.

Since most hardware decoders output NV12, this patch adds
NV12+A to make the same workflow possible.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1152>
2021-05-21 21:16:24 -04:00
Daniel Almeida
038ba1bffe video: video-orc: remove trailing spaces
Remove unnecessary trailing spaces at the end of some orc functions.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1152>
2021-05-21 21:16:24 -04:00
Nicolas Dufresne
3897b24f69 video: Pass component index not plane index
While so far it worked, we are about to introduce a format that break this
assuming. We have a format which consist of NV12 with alpha, and this format
does not have a direct mapping of the component against their plane indexes.

Fix this by using gst_video_format_info_component() introduced in 1.18 for
this purpose.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1151>
2021-05-19 15:17:56 +00:00
Seungha Yang
87327110c2 videoaggregator: Don't try to return void
warning C4098: 'gst_video_aggregator_parallel_convert_pad_finalize':
'void' function returning a value

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1155>
2021-05-19 16:23:43 +09:00
Tim-Philipp Müller
48c0a6dd1e Revert video HDR10+ support
This reverts commit f76b731cbf.
This reverts commit 631489de23.
This reverts commit a1ed7a8f49.
This reverts commit 7d6f72e956.

Back this out again for now for the release. Needs more discussion.

https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/issues/865

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1148>
2021-05-17 15:54:15 +00:00
Matthew Waters
8a5e5ddeeb video/aggregator: add parallel convert pad class
Each required conversion will be performed concurrently

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1129>
2021-05-17 19:20:57 +10:00
Matthew Waters
c30534122e video/converter: add support for async conversion operation
Allows for users to start up multiple conversions concurrently.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1129>
2021-05-17 19:20:57 +10:00
Haihao Xiang
ca046ca73c video: add support for RGBP and BGRP formats
The two RGB planar formats are used in OpenVino [1]

gst-launch-1.0 videotestsrc ! video/x-raw,format=BGRP ! fakesink
gst-launch-1.0 videotestsrc ! video/x-raw,format=RGBP ! fakesink

[1] https://docs.openvinotoolkit.org/latest/openvino_docs_optimization_guide_dldt_optimization_guide.html

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1141>
2021-05-11 12:24:41 +08:00
Nicolas Dufresne
d3ac7bfcbf codec: Introduce GstVideoCodecAlphaMeta
This meta hold one buffer of the same codec data as the parent memory. This
extra frame luma will be used as the alpha values for the final combined
frame. This is notably used to support VP8/VP9 alpha as defined in WebM and
matroska specification.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1128>
2021-05-10 16:33:11 -04:00
Nicolas Dufresne
ac54f073d8 video: Sort includes in video.h
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1128>
2021-05-05 16:20:37 -04:00
Jordan Petridis
df88b10c7f gstvideoencoder: make sure the buffer is writable before modifying metadata
Similar to ae8d0cf3ac

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1112>
2021-04-20 16:01:15 +00:00
Stéphane Cerveau
ada8b07be2 videodecoder: use DTS if PTS unknown
The buffer should be set according to DTS if exists
when we are guessin the PTS instead of segment start.
The decoder can receive buffers which are before the segment
in case of seek for example.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1111>
2021-04-19 13:28:39 +02:00
Marijn Suijten
33167573e1 Drop @ documentation references from functions and external types
`@` references are used to reference function parameters, struct members
or enum variants _within_ the current type/function.  It cannot and
should not be used to reference to types outside that.

Since C has no notion of member functions it makes little sense to
prefix these with `@`; most of the documentation here was referencing
functions on _different_ types anyway.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1090>
2021-04-15 15:49:39 +02:00
Matthew Waters
3d9e705621 videoaggregator: allow selecting an alpha output from non-alpha inputs
e.g. if we have:

video-x/raw,format=I420 ! compositor ! video/x-raw,format=BGRA

This will currently produce a warning as the alpha-ness of the chosen
'best' format (I420) will be different from the value restricted by the
downstream caps filter.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1059>
2021-03-31 01:55:17 +00:00
Matthew Waters
98249a57db gst: don't use volatile to mean atomic
volatile is not sufficient to provide atomic guarantees and real atomics
should be used instead.  GCC 11 has started warning about using volatile
with atomic operations.

https://gitlab.gnome.org/GNOME/glib/-/merge_requests/1719

Discovered in https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/-/issues/868

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1073>
2021-03-19 04:20:19 +00:00
Vivia Nikolaidou
1517b7043d video-converter: Don't upsample/downsample/dither invalid lines
This is a fallout from the conversion to support multiple threads.
convert->upsample_p is never NULL now, it's always an allocated array of
n_threads potentially-null pointers.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1043>
2021-02-23 03:40:12 +00:00
Alicia Boya García
29aeba639a videodecoder: Fix racy critical when pool negotiation occurs during flush
I found a rather reproducible race in a WebKit LayoutTest when a player
was intantiated and a VP8/9 video was loaded, then torn down after
getting the video dimensions from the caps.

The crash occurs during the handling of the first frame by gstvpxdec.
The following actions happen sequentially leading to a crash.

(MT=Main Thread, ST=Streaming Thread)

MT: Sets pipeline state to NULL, which deactivates vpxdec's srcpad,
    which in turn sets its FLUSHING flag.

ST: gst_vpx_dec_handle_frame() -- which is still running -- calls
    gst_video_decoder_allocate_output_frame(); this in turn calls
    gst_video_decoder_negotiate_unlocked() which fails because the
    srcpad is FLUSHING. As a direct consequence of the negotiation
    failure, a pool is NOT set.

    gst_video_decoder_negotiate_unlocked() still assumes there is a
    pool, crashing in a critical in gst_buffer_pool_acquire_buffer()
    a couple statements later.

This patch fixes the bug by returning != GST_FLOW_OK when the
negotiation fails. If the srcpad is FLUSHING, GST_FLOW_FLUSHING is
returned, otherwise GST_FLOW_ERROR is used.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1031>
2021-02-16 16:57:54 +00:00
Vivia Nikolaidou
ca4240bd03 videoconvert: Support for alternate-field interlacing
Treat the data just like normal data with half the height. Also treat it
as progressive when converting from/to I420 because it requires
different handling for chroma subsampling.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1027>
2021-02-04 18:22:07 +02:00
Jakub Adam
11e6f8da92 video-hdr: Add API to check content light level equality
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/969>
2021-01-28 20:55:38 +01:00
Marijn Suijten
abb026ec6a gl,video: Make ptrs to VideoInfo and (GL)AllocationParams immutable
These parameters are incorrectly regarded as mutable in G-IR making them
"incompatible" with languages that are explicit about mutability like
Rust. In order to clean up the code and expected API there, update the
signatures here, right at the source (instead of overriding them in
Gir.toml and hoping for the best).

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1005>
2021-01-14 11:53:10 +00:00
Marijn Suijten
c70d263e48 video/video-frame: @frame in video_frame_map is out caller-allocates
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1005>
2021-01-14 11:53:10 +00:00
Marijn Suijten
a263919f06 audio,video: Add out caller-allocates to init and from_caps
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1005>
2021-01-14 11:53:10 +00:00
Sebastian Dröge
7e16eed522 videosink: Add new GstVideoSink::set_info() virtual method
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/986>
2021-01-14 11:15:40 +00:00
Sebastian Dröge
198434e71a videosink: Implement more complete BaseSink::get_times() based on the framerate
This will only make use of the framerate if the subclass is chaining up
BaseSink::set_caps(). Otherwise it will have the same behaviour as the
basesink default.

Doing so is useful if video buffers don't contain a duration to
calculate a default duration, and various video sinks already implement
a custom version of this.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/986>
2021-01-14 11:15:40 +00:00
Marijn Suijten
1f06cf60e7 video: Convert info_to_caps to take self as const ptr
This requires a slight modification to the function itself because it
was overwriting a member locally.

However, now this side-effect cannot be observed outside the function
anymore.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1008>
2021-01-14 08:14:36 +00:00
Jakub Adam
2d198ff10b video-blend: fix blending 8-bit and 16-bit frames together
Replace hardcoded 255s with the correct max value for the given color
depth. Use 64-bit integer in calculations where overflow may occur.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1000>
2021-01-08 08:04:55 +00:00
Edward Hervey
65b6994df6 videoaggregator: Pop out old buffers on timeout
This situation happens in the situation where an input stream has a framerate
exceeding the timeout latency (Ex: 1fps with a latency of 500ms) and an input
stream greater than output framerate (ex: 60fps in, 30 fps out).

The problem that would happen is that we would timeout, but then buffers from
the fast input stream would only be popped out one by one.... until a buffer
reaches the low-framerate input stream at which point they would quickly be
popped out/used. The resulting output would be "slow ... fast ... slow ... fast"
of that input fast stream.

In order to avoid this situation, whenever we detect a late buffer, check if
there's a next one and re-check with that one.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/990>
2020-12-30 16:03:13 +01:00
Stéphane Cerveau
f76b731cbf hdr: update doc
update GST_VIDEO_HDR10_PLUS_MAX_ROWS_MD_APL and
GST_VIDEO_HDR10_PLUS_MAX_COLS_MD_APL

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/979>
2020-12-15 23:50:12 +01:00
Stéphane Cerveau
9b852181d8 videodecoder: Forward hdr-format info downstream
By default the hdr-format detected by a parser should
be passed to the downstream element.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/650>
2020-12-15 16:37:46 +00:00
Stéphane Cerveau
631489de23 video-hdr: add hdr formats
Provide enum and helper method to set the hdr format
name in caps by example.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/650>
2020-12-15 16:37:46 +00:00
Stéphane Cerveau
a1ed7a8f49 video-hdr: introduce HDR10+ parser
Video can now parse a HDR10+ data structure
coming from a SEI message.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/650>
2020-12-15 16:37:46 +00:00
Stéphane Cerveau
7d6f72e956 video-hdr: add HDR10+ structure
Provides structure and GstVideoMeta

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/650>
2020-12-15 16:37:46 +00:00
Fabrice Fontaine
d86cf6314f gst-libs/gst/video/gstvideoaggregator.c: fix build with gcc 4.8
Fix the following build failure with gcc 4.8 which has been added with
d268c193ad:

../gst-libs/gst/video/gstvideoaggregator.c: In function 'gst_video_aggregator_init':
../gst-libs/gst/video/gstvideoaggregator.c:2762:3: error: 'for' loop initial declarations are only allowed in C99 mode
   for (gint i = 0; i < gst_caps_get_size (src_template); i++) {
   ^

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/974>
2020-12-14 16:42:01 +00:00
Seungha Yang
a4ba868225 video: Make use of gst_video_chroma_site_{from,to}_string() API
Replace deprecated gst_video_chroma_{from,to}_string()
to newly added gst_video_chroma_site_{from,to}_string()

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/927>
2020-12-08 07:21:28 +00:00
Seungha Yang
410efd196a video-chroma: Add support for any combination of chroma-site flags
We've been allowing only a few known chroma-site values such as
jpeg (not co-sited), mpeg2 (horizontally co-sited) and
dv (co-sited on alternate lines). That's insufficient for
representing all possible chroma-site values. By this commit,
we can represent any combination of chroma-site flags.
But, an exception here is that any combination with
GST_VIDEO_CHROMA_SITE_NONE will be considered as invalid value.

For any combination of chroma-site flags,
gst_video_chroma_to_string() method is deprecated in order to
return newly allocated string via a new gst_video_chroma_site_to_string()
method. And for consistent API naming, gst_video_chroma_from_string()
is also deprecated. Newly written code should use
gst_video_chroma_site_from_string() instead.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/927>
2020-12-08 07:21:28 +00:00
Matthew Waters
d4ff62700d video/converter: increase the number of cache lines for resampling
The exising hardcoded max default does not account for the possible
-1 offset when retrieving lines for resampling.  As a result, when
another chain has the same number of cache lines (4), the resample
operation would be attempting to generate 5 lines with a cache size
of 4 and would overwrite the first cache line.

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/issues/821

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/938>
2020-12-03 06:39:09 +00:00
Guillaume Desmottes
3ab2023ed8 videometa: gir annotate the size of plane array in new API
Fix #838

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/949>
2020-12-01 09:50:27 +01:00
Marijn Suijten
7565a0b997 video: Provide "deprecated in" version for gst_video_color_transfer fns
As requested in [1].

[1]: https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/940#note_706437

Fixes: d0f36c7e1 ("video: Rename video_color_transfer to video_transfer_function")
2020-11-25 20:19:39 +01:00
Mathieu Duponchelle
c50f4477ec video-converter: switch to using a task pool ..
.. and make use of that API in videoaggregator.

When setting certain properties, such as cropping or the scaled
size of pads, a new converter is created by videoaggregator.

Before that patch, this implied spawning new threads, potentially
at each aggregate cycle when interpolating pad properties. This
is obviously wasteful, and re-using a task pool removes that
overhead.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/896>
2020-11-12 17:38:34 +00:00
Jan Alexander Steffens (heftig)
b3fe2d3623 videoaggregator: Fix locking around vagg->info
Take `GST_OBJECT_LOCK` when writing `vagg->info`, so that reading in
subclasses is protected against races, as documented in the struct.

    /*< public >*/
    /* read-only, with OBJECT_LOCK */
    GstVideoInfo                  info;

`gst_video_aggregator_default_negotiated_src_caps` should take the
`GST_VIDEO_AGGREGATOR_LOCK` to avoid racing with
`gst_video_aggregator_reset` called by
`gst_video_aggregator_release_pad` of the last sinkpad. Otherwise it can
happen that `latency = gst_util_uint64_scale (...` gets called with a
zero framerate.

There doesn't seem to be any reason not to use the local `info` instead
of `vagg->info`, so do that.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/915>
2020-11-09 16:04:06 +00:00
Guillaume Desmottes
b005d472f7 video: fix doc warning
@mode has been renamed to
gst_video_decoder_set_interlaced_output_state() but not in the header
file, raising a doc warning.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/894>
2020-11-09 11:55:57 +00:00