Commit graph

27092 commits

Author SHA1 Message Date
Mathieu Duponchelle 08442cc792 cccombiner: implement scheduling
Prior to that, cccombiner's behaviour was essentially that of
a funnel: it strictly looked at input timestamps to associate
together video and caption buffers.

This patch instead exposes a "schedule" property, with a default
of TRUE, to control whether caption buffers should be smoothly
scheduled, in order to have exactly one per output video buffer.

This can involve rewriting input captions, for example when the
input is CDP sequence counters are rewritten, time codes are dropped
and potentially re-injected if the input video frame had a time code
meta.

Caption buffers may also get split up in order to assign captions to
the correct field when the input is interlaced.

This can also imply that the input will drift from synchronization,
when there isn't enough padding in the input stream to catch up. In
that case the element will start dropping old caption buffers once
the number of buffers in its internal queue reaches a certain limit
(configurable).

The property is exposed so that existing users of cccombiner can
revert back to the original behaviour, but should eventually be
removed, as that behaviour was simply inadequate.

This commit also disallows changing the input caption type, as
this would needlessly complicate implementation, and removes
the corresponding test.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2076>
2021-03-17 22:00:25 +00:00
Seungha Yang 80792e12d4 d3d11: Use render-target and shader-resource bind flags by default
Even if bind flags is not needed by an element, other element
might need such bind flags.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2091>
2021-03-17 22:17:22 +09:00
He Junyan 805ff6346a va: vpp: Fix features lost in transform_caps().
When we transform the caps from the sink to src, or vice versa, the
"caps" passed to us may only contain parts of the features. Which
makes our vpp lose some feature in caps and get a negotiation error.
The correct way should be:
Cleaning the format and resolution of that caps, but adding all VA,
DMA features to it, making it a full feature caps. Then, clipping it
with the pad template.

fixes: #1551

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2081>
2021-03-17 10:25:11 +01:00
Thibault Saunier 8a0224a198 wpe: Ignore 'error-cancelled' 'failures'
This happens when the user use the 'load-bytes' signal and nothing is wrong there

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2085>
2021-03-16 13:06:22 +00:00
Seungha Yang 3f08fcb4ff d3d11decoder: WARNING if ID3D11VideoDevice is unavailable, not ERROR
gst_d3d11_decoder_new() method is also used for device capability
checking during plugin init. Although we are checking hardware
flag prior to that, it doesn't guarantee ID3D11VideoDevice interface.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2088>
2021-03-16 12:39:23 +00:00
Seungha Yang 1d6cdbdc90 mediafoundation: Fix resource leak
IMFActivate would hold its internal objects unless user call ShutdownObject(),
even if we release the IMFActivate. Here internal objects may
include Direct3D objects, such as texture, device handle for example.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2087>
2021-03-16 09:47:41 +00:00
Seungha Yang 58e451325b d3d11colorconvert: Fix caps leak
GstBaseTransform::fixate_caps() takes ownership of passed
othercaps argument.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2086>
2021-03-16 08:39:46 +00:00
Víctor Manuel Jáquez Leal 2b37910631 va: example: multiple-vpp: test sharpen with dynamic controller
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2082>
2021-03-16 06:30:06 +00:00
Víctor Manuel Jáquez Leal 191b6cf5d4 va: postproc: synchronize segment with stream time
This is required to use dynamic controllable parameters.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2082>
2021-03-16 06:30:06 +00:00
Seungha Yang 309a940614 d3d11decoder: Do negotiation again per forward/reverse playback mode change
For reverse playback, we are always copying decoded
frame to downstream buffer. So the pool size can be
and need to be large enough.

In case that forward playback, however, we need to restrict
the max pool size for performance reason. Otherwise decoder
will keep copying decoded texture to downstream buffer pool
if decoding is faster than downstream throughput
performance and also there are queue element between them.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2083>
2021-03-16 05:54:41 +00:00
Seungha Yang 3a99517f7c d3d11videosink: Avoid switching conversion tool during playback
Decoder might be able to copy decoded texture to the other buffer pool
during playback depending on context. In that case, copied one
has no D3D11_BIND_DECODER bind flag.

If we used ID3D11VideoProcessor previously for decoder texture,
and incoming texture supports ID3D11VideoProcessor as well even if it has no
D3D11_BIND_DECODER flag (having D3D11_BIND_RENDER_TARGET for example),
allow zero-copying instead of using our fallback texture.

Frequent conversion tool change (between ID3D11VideoProcessor and generic shader)
might result in inconsistent image quality.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2084>
2021-03-15 21:00:22 +09:00
Víctor Manuel Jáquez Leal c74b230579 va: postproc: only drop filters if they change
Currently, at every frame the filters array is recreated. This is not
optimal, since it should be only rebuilt if the VA filter's related
properties change. This patches does that by using a flag.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2078>
2021-03-15 06:10:27 +00:00
Seungha Yang 72345875b1 d3d11decoder: Refactor device open step and negotiation
* Remove redundant method arguments
* Don't allocate staging texture if downstream supports d3d11 memory

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2079>
2021-03-14 17:34:20 +09:00
Seungha Yang 347d9ceb4e d3d11decoder: Move profile GUID handling into decoder object
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2079>
2021-03-14 16:31:31 +09:00
Seungha Yang 1c1cfc4ba7 d3d11decoder: Get rid of private struct
Completely hide member variables

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2079>
2021-03-14 16:30:19 +09:00
Seungha Yang 78e4251750 Revert "d3d11vp9dec: Add support for internal frame resizing"
This reverts commit 58a4c33a0e

We should use ID3D11VideoProcessor instead of shader
to avoid copy. We need to revisit this topic later

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2079>
2021-03-14 16:30:15 +09:00
Seungha Yang 615f52f2f7 d3d11device: Hold ID3D11VideoDevice and ID3D11VideoContext object
... instead of QueryInterface-ing per elements. Note that
ID3D11VideoDevice and ID3D11VideoContext objects might not be available
if device doesn't support video interface.
So GstD3D11Device object will create those objects only when requested.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2079>
2021-03-14 14:44:55 +09:00
Seungha Yang b08310f748 d3d11: Run gst-indent for all C++ code
Since all d3d11 plugin implementation code are C++, we need to
run gst-indent manually. This is preparation for later
"gst-indent sys/d3d11/*.cpp" run.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2077>
2021-03-14 13:40:12 +09:00
Seungha Yang 5b3e316039 d3d11: Port to C++
Direct3D11 objects are COM, and most COM C APIs are verbose
(C++ is a little better). So, by using C++ APIs, we can make code
shorter and more readable.
Moreover, "ComPtr" helper class (which is C++ only) can be
utilized, that is very helpful for avoiding error-prone COM refcounting
issue/leak.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2077>
2021-03-14 13:05:22 +09:00
Víctor Manuel Jáquez Leal 3c40ffc00f va: example: multiple-vpp: test direction change
If the driver supports it (iHD, so far) and the parameter -d is set,
the direction of the video will be changed randomly.

In the code you can select, at compilation time, if the direction
change is done by element's property or by pipeline events.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2074>
2021-03-12 12:46:41 +01:00
Víctor Manuel Jáquez Leal 0b2848fc32 va: postproc: update passthrough and reconfigure pads
Added helper function _update_passthrough() which will define and set
the pass-through mode of the filter, and it'll either reconfigure both
pads or it will just mark the src pad for renegotiation or nothing at
all.

There are cases where both pads have to be reconfigured (direction
changed, for example), other when just src pad has to (filters
updated) or none (changing to ready state).

The requirement of renegotiation depends on the need to enable/disable
its VA buffer pools.

This patch sets pass-through mode by default, so the buffer pools
aren't allocated if no filtering/direction operations are defined,
which is the correct behavior.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2074>
2021-03-12 12:34:46 +01:00
Stéphane Cerveau 451c875d40 zxing: update to support version 1.1.1
Support new API in 1.1.1
Update the supported input video format.
Update tests to use parse_launch

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2037>
2021-03-12 01:03:49 +00:00
Stéphane Cerveau b0a9ba4ccf mpegvideoparse: do not clip the frame
If the current buffer is delta unit such as P or B
frame, the buffer should not be clipped and need to
let the decoder handle the segment boundary situation.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2070>
2021-03-11 15:01:38 +01:00
Seungha Yang cdce27d975 d3d11device: Fix wrong printf formatting
Add missing '%' there

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2069>
2021-03-11 13:31:18 +09:00
Philippe Normand a6b5e3c7d5 transcoder: Add state-changed signal
Similar to GstPlayer, a new signal for state tracking is now emitted at runtime,
as a commodity for applications which then don't need to monitor the pipeline
GstBus for state changes anymore.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2028>
2021-03-10 17:13:57 +00:00
Philippe Normand 12a7bf9f7c player: Rewrite as GstPlay wrapper
For the time being the GstPlayer library remains as a wrapper for GstPlay, in
order to keep existing applications working and give them time to port to
GstPlay. GstPlayer will remain in -bad for a couple cycles and the plan for
GstPlay is to move it to -base before 1.20.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2061>
2021-03-09 18:03:48 +00:00
Philippe Normand f009802771 play: Flush API bus before exiting main loop
Otherwise the bus might attempt to dispatch queued messages after the thread
ended, causing runtime warnings.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2061>
2021-03-09 18:03:48 +00:00
Philippe Normand fae7c8dd7e play: tests: Switch user-agent test to a real HTTP server
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2061>
2021-03-09 18:03:48 +00:00
Philippe Normand 3eec2f4be8 play: tests: Refactor to use new Message bus API
Instead of relying on an extra GMainLoop, the messages are poped from the player
bus and handled synchronously. This should avoid flaky behaviors.

Fixes #608

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2061>
2021-03-09 18:03:48 +00:00
Philippe Normand 19d57dcfa5 play: Plug media_info leak
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2061>
2021-03-09 18:03:48 +00:00
Stephan Hesse e007c47c3f play: Rename internal buffering field to buffering_percent
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2061>
2021-03-09 18:03:48 +00:00
Stephan Hesse a74651ddc1 play: Introducing the new playback library
This aims to be a replacement for the GstPlayer library. In GstPlay, notifications are
sent as application messages through a dedicated GstBus. The GMainContext-based
signal dispatcher was replaced by a GObject signal adapter, now relying on the
bus to emit its signals. The signal dispatcher is now optional and fully
decoupled from the GstPlay object.

Co-authored with: Philippe Normand <philn@igalia.com>

Fixes #394

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2061>
2021-03-09 18:03:48 +00:00
Sebastian Dröge 80c1722cba avwait: Don't post messages with the mutex locked
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2063>
2021-03-09 13:01:45 +02:00
Matthew Waters 2bed220771 webrtc: don't generate duplicate rtx payloads when bundle-policy is set
It was possible to generate a SDP that had an RTX payload type
that matched one of the media payload types when providing caps via
codec_preferences without any sink pads.

Fixes

m=video 9 UDP/TLS/RTP/SAVPF 96
...
a=rtpmap:96 VP8/90000
a=rtcp-fb:96 nack pli
a=fmtp:96 apt=96

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2046>
2021-03-09 02:22:35 +00:00
Vivia Nikolaidou 4ccad5336f tests: Add negotiation tests for the interlace elements
Many complicated cases exist. Would be good to have some checks.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2062>
2021-03-08 21:02:13 +02:00
Vivia Nikolaidou cde4e74eca interlace: Discard stored_frame on EOS and PAUSED_TO_READY
Would otherwise leak it.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2062>
2021-03-08 21:02:01 +02:00
Vivia Nikolaidou cb55d30b3c interlace: Specify interlace-modes in the sink pad template
Especially specify the field-order in the interleaved mode. Otherwise it
might cause the negotiation to fail, because
GST_PAD_SET_ACCEPT_INTERSECT is not set on the sinkpad, and the
field-order is missing in the sink template but can be present in the
outside caps.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2062>
2021-03-08 21:01:50 +02:00
Seungha Yang 9da6da20e7 nvh264sldec: Reopen decoder object if larger DPB size is required
Equivalent to the d3d11h264dec fix
https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1839

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2059>
2021-03-07 16:51:59 +09:00
Seungha Yang a29c8935b7 d3d11: Fix an HLSL compiler warning
warning X3578: Output value 'main' is not completely initialized

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2052>
2021-03-04 20:34:23 +00:00
Bing Song 89f193e1eb transcoder: handle SIGINT and SIGHUP
Handle SIGINT and SIGHUP in transcoder. Or the output file maybe corrupt.

Fixes #1507

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1987>
2021-03-04 14:04:43 +00:00
Seungha Yang 3730ea3366 d3d11h264dec: Keep track of actually configured DPB size
... instead of the largest we ever seen.
Note that d3d11h264dec element holds previously configured DPB size
for later decoder object re-open decision.

This is to fix below case:
1) Initial SPS, required DPB size is 6
  - decoder object is opened with DPB size 6
  - max_dpb_size is now 6
2) SPS update with resolution change, required DPB size is 1
  - decoder object is re-opened with DPB size 1
  - max_dpb_size should be updated to 1, but it didn't happen (BUG)
3) SPS update without resolution change, only required DPB size is updated to 6
  - decoder object should be re-opened but didn't happen
    because we didn't update max_dpb_size at 2).

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2056>
2021-03-04 17:57:14 +09:00
Tim-Philipp Müller 766bd655fc interlace: add more formats, esp 10-bit, 12-bit and 16-bit ones
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2054>
2021-03-03 18:34:26 +00:00
Benjamin Gaignard 5e8daa1c5c v4l2codecs: h264: retrieve interlaced information
Lets the decoder knows if the frames are interlaced or not.
Provide this information to the driver while filling reference
pictures fields in slice params structure

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1624>
2021-03-02 22:03:34 +00:00
Nicolas Dufresne 84daea6be2 v4l2codecs: h264: Reduce controls for subsequent slices
Only the SLICE_PARAMS and PRED_WEIGHTS are needed for the second and
following slices.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1624>
2021-03-02 22:03:34 +00:00
Ezequiel Garcia 1c082b79e1 v4l2codecs: h264: Implement optional scaling matrix
The new H.264 uAPI requires that all drivers support
scaling matrix only as an option, when a non-flat
scaling matrix is provided in the bitstream headers.

Take advantage of this and avoid passing the scaling
matrix if not needed.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1624>
2021-03-02 22:03:34 +00:00
Ezequiel Garcia 78756ecba9 v4l2codecs: h264: Only set SPS control if needed
Given V4L2 controls are cached in V4L2, there is no need
to set them if they don't change. Set the SPS control
only if a new sequence was received by the parser.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1624>
2021-03-02 22:03:34 +00:00
Ezequiel Garcia ae9411e334 v4l2codecs: h264: Only slice-based need SLICE_PARAMS and PRED_WEIGHTS
Frame-based decoding mode doesn't require SLICE_PARAMS and
PRED_WEIGHTS controls.

Moreover, if the driver doesn't support these two controls, trying
to set them will fail. Fix this by only setting these on
slice-based decoding mode.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1624>
2021-03-02 22:03:34 +00:00
Ezequiel Garcia 010565eb7f v4l2codecs: h264: Add API checks
Check that the V4L2 H264 controls' sizes match
our expectation. If not, then probably there's an API
mismatch which will cause errors or decoding corruption.

Also, print a warning if the kernel version is too old.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1624>
2021-03-02 22:03:34 +00:00
Ezequiel Garcia 7d6b06ca1b v4l2codecs: h264: Update to the new uAPI
Starting from Linux v5.11, the V4L2 stateless H.264 uAPI
is updated and stable.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1624>
2021-03-02 22:03:34 +00:00
Ezequiel Garcia 56ecc19067 v4l2codecs: h264: Set the scaling matrix present flag unconditionally
We are currently always setting and passing a scaling matrix,
so need to set this flag accordingly. Passing a scaling matrix
optionally will be implemented in follow-up commit.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1624>
2021-03-02 22:03:34 +00:00