Commit graph

1434 commits

Author SHA1 Message Date
Seungha Yang
38cbc51822 nvencoder: Fix maximum QP value setting
Fixing typo

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6832>
2024-05-12 23:53:10 +01:00
Robert Mader
e869185287 v4l2codecs: decoder: Reorder caps to prefer DMA_DRM ones
Certain V4L2 fourccs don't (yet) have DRM counter parts, in which case
we can't create DMA_DRM caps for them. This is usually the case for
specific tilings, which are represented as modifiers for DMA formats.

While using these tilings is generally preferable - because of e.g.
lower memory usage - it can result in additional conversion steps when
interacting with DMA based APIs such as GL, Vulkan or KMS. In such cases
using a DMA compatible format usually ends up being the better option.

Before the addition of DMA_DRM caps, this was what playbin3 ended up
requesting in various cases - e.g. prefering NV12 over NV12_4L4 - but
the addition of DMA_DRM caps seems to confuse the selection logic.

As a simple and quite robust solution, assume that peers supporting
DMA_DRM caps always prefer these and reorder the caps accordingly.

In the future we plan to have a translation layer for cases where
there is a matching fourcc+modifier pair for a V4L2 fourcc, ensuring
optimal results.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6824>
2024-05-09 23:46:16 +01:00
Emil Pettersson
6f508684e7 vtdec: Fix deadlock when negotiating format change
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6817>
2024-05-08 09:13:50 +01:00
Seungha Yang
5e4a76fdf1 d3dshader: Fix gamma and primaries conversion pixel shader
Fixing regression introduced by the commit of f52ecb9607

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6803>
2024-05-05 18:20:03 +09:00
Seungha Yang
e2679face1 qsvh264dec,qsvh265dec: Fix nalu leaks
Fixes: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/3514
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6790>
2024-05-01 13:43:29 +00:00
Piotr Brzeziński
3d39c3ce00 vtdec: Fix PAUSED->READY deadlock when output loop is running
Makes sure the GST_PAD_STREAM_LOCK is not taken when pad is being deactivated.
The lack of this was causing deadlocks when stopping the pipeline right after producing first buffers.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6788>
2024-05-01 12:54:23 +00:00
He Junyan
ca49577cbb vah265enc: Set the correct buffer flag for output
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6764>
2024-04-29 17:27:41 +00:00
He Junyan
bda1aa3559 vah264enc: Set the correct buffer flag for output
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6764>
2024-04-29 17:27:41 +00:00
Seungha Yang
dc8d09d6d2 vabaseenc: Fix frame leak on error path
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6764>
2024-04-29 17:27:41 +00:00
He Junyan
8a76079e17 vah265enc: Do not touch the PTS of output frame
1. The PTS of all frames should not be changed.
2. Just update the DTS based on the PTS. For the frame which is not
   reordered, the DTS is equal to PTS. For frame which is reordered,
   the DTS is equal to previous DTS. For example:
   Input: F0[D0, P0] -- F1[D1, P1] -- F2[D2, P2] -- F3[D3, P3]
   Output: F0[I, D0, P0] -- F3[P, D0, P3] -- F1[B, D1, P1] -- F2[B, D2, P2]

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6764>
2024-04-29 17:27:41 +00:00
He Junyan
a9551fec9a vah264enc: Do not touch the PTS of output frame
1. The PTS of all frames should not be changed.
2. Just update the DTS based on the PTS. For the frame which is not
   reordered, the DTS is equal to PTS. For frame which is reordered,
   the DTS is equal to previous DTS. For example:
   Input: F0[D0, P0] -- F1[D1, P1] -- F2[D2, P2] -- F3[D3, P3]
   Output: F0[I, D0, P0] -- F3[P, D0, P3] -- F1[B, D1, P1] -- F2[B, D2, P2]

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6764>
2024-04-29 17:27:41 +00:00
Seungha Yang
4a7f3d0b75 d3d12decoder: Fix d3d12 resource copy
It was copying to self resource

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6755>
2024-04-29 17:44:56 +09:00
He Junyan
17ede2149b vabaseenc: No need to call _finish_subframe()
After vaav1enc is aligned to TU, there is no case that generates
multi output for one input.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6751>
2024-04-28 11:25:54 +01:00
He Junyan
c63bcb1c2a vaav1enc: Set the correct buffer flag for output
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6751>
2024-04-28 11:25:24 +01:00
He Junyan
8c4aa7067d vaav1enc: Do not change the PTS/DTS of output frames
The AV1 encoder does not reorder the frames, so there is no need
to change the timestamp related meta data of output frames, just
inheriting it from the input frames.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6751>
2024-04-28 11:25:13 +01:00
He Junyan
7ec4a7cc01 vaav1enc: Change the alignment of output to "tu"
The current output alignment is "frame", which may cause some issues
for PTS and DTS calculation. We now change the alignment to "tu",
and this is also the alignment mode for av1enc and svtav1enc.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6751>
2024-04-28 11:25:05 +01:00
Seungha Yang
06e955f94e d3d12decoder: Lock DPB while building command
Since DPB resource can be modified in output thread, protect
it when building command list.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6717>
2024-04-23 12:26:07 +01:00
Seungha Yang
d22fe06234 d3d12decoder: Hold reference pictures in fence data
Keep reference pictures alive during executing decoding commands

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6717>
2024-04-23 12:26:07 +01:00
Seungha Yang
e102560883 d3d12vp9dec: Disallow resolution change to larger size on non-keyframe
Intel GPU seems to be crashing if the case happens.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6717>
2024-04-23 12:26:06 +01:00
Seungha Yang
9daa541816 d3d12decoder: Fix potential use after free
A DPB buffer held by codec picture object may not be writable
at the moment, then gst_buffer_make_writable() will unref passed buffer.

Specifically, the use after free or double free can happen if:
* Crop meta of buffer copy is required because of non-zero
  top-left crop position
* zero-copy is possible with crop meta
* A picture was duplicated, interlaced h264 stream for example

Interlaced h264 stream with non-zero top-left crop position
is not very common but it's possible configuration in theory.

Thus gst_buffer_make_writable() should be called with
GstVideoCodecFrame.output_buffer directly.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6710>
2024-04-22 15:14:13 +01:00
Seungha Yang
72ffee6355 d3d11decoder: Fix potential use after free
A DPB buffer held by codec picture object may not be writable
at the moment, then gst_buffer_make_writable() will unref passed buffer.

Specifically, the use after free or double free can happen if:
* Crop meta of buffer copy is required because of non-zero
  top-left crop position
* zero-copy is possible with crop meta
* A picture was duplicated, interlaced h264 stream for example

Interlaced h264 stream with non-zero top-left crop position
is not very common but it's possible configuration in theory.

Thus gst_buffer_make_writable() should be called with
GstVideoCodecFrame.output_buffer directly.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6710>
2024-04-22 15:14:13 +01:00
Seungha Yang
210a225ad2 mediafoundation: Fix infinite loop in device provider
Initialize source state with GST_MF_DEVICE_NOT_FOUND to terminate
loop immediately if no available capture device is available

Fixes: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/3492
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6707>
2024-04-21 16:56:39 +01:00
Seungha Yang
180cf124a7 d3d12videosink: Disconnect window's signal on dispose
Same as the commit of 7b69d1758f
but for d3d12videosink.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6700>
2024-04-20 23:47:14 +00:00
Seungha Yang
e0143754ad d3d12videosink: Handle external HWND's mouse/keyboard events
OS will not propagate the event to child HWND if it's handled by
the parent. Thus, navigation event should be handled by parent HWND's
event handler.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6700>
2024-04-20 23:47:14 +00:00
Nicolas Dufresne
3efcfef4b5 v4l2codecs: Don't unref allocation query caps
The caps obtained from parsing the allocation query is borrowed and
should not be unreffed. This fixes criticals assertion introduced in
1.24.1.

(gst-launch-1.0:242): GStreamer-CRITICAL **: 19:48:02.667:
    gst_mini_object_unref: assertion 'GST_MINI_OBJECT_REFCOUNT_VALUE (mini_object) > 0' failed

Fixes: 5189e8b956 ("v4l2codecs: decoders: Add DMA_DRM caps support")
Closes #3462

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6682>
2024-04-18 09:56:52 +01:00
Seungha Yang
ebd27cb2d2 mediafoundation: Fix device enumeration
Do not stop device enumerate even if a device could not be opened.
Otherwise the other devices listed after the failed device will not be
reported by device provider

Fixes: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/3460
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6647>
2024-04-16 06:57:06 +00:00
Seungha Yang
4313083584 d3d12encoder: Fix buffer pool leak
Add missing buffer pool release

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6637>
2024-04-16 03:41:28 +00:00
Seungha Yang
6ddadcb830 d3d11videosink: Fix rendering on keyed mutex enabled handle
As of the commit 69b2e1565c,
keyed mutex will be handled by the memory object.

Fixes: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/3468
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6635>
2024-04-16 02:07:06 +00:00
Seungha Yang
10ffbdbb1f qsvdecoder: Release too old frames
Release too old frames manually.

Fixes: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/3163
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6586>
2024-04-09 18:56:08 +01:00
Seungha Yang
f5500906ce dwrite: Fix crash on device update
Selected blend mode should not be cleared on device update

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6521>
2024-04-08 11:38:43 +00:00
Elizabeth Figura
e1b82701df atdec: Handle channel counts greater than 2
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6546>
2024-04-05 10:14:59 +00:00
Elizabeth Figura
b7a4b9e1ab atdec: Use gst_audio_decoder_set_output_caps() directly
The code currently sets the same caps in two different ways, and neither of them correctly handle the channel mask.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6546>
2024-04-05 10:14:59 +00:00
Víctor Manuel Jáquez Leal
6f7d62eb73 msdk: sink context reference
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6527>
2024-04-03 16:34:26 +00:00
Víctor Manuel Jáquez Leal
1d7b7b04d1 va: sink reference at instantiation
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6527>
2024-04-03 16:34:26 +00:00
He Junyan
3fe9a6dc8c va: av1enc: Avoid reopen encoder or renegotiate
If parameters remain similar enough to avoid either encoder reopening
or downstream renegotiation, avoid it.

This is going to be useful for dynamic parameters setting.

To check if the stream parameters changed, so the internal encoder has
to be closed and opened again, are required two steps:

1. If input caps, format, profile, chroma or rate control mode have changed.
2. If any of the calculated variables and element properties have changed.

Later on, only if the output caps also changed, the pipeline
is renegotiated.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6519>
2024-04-02 21:45:33 +00:00
He Junyan
ac23e04236 va: av1enc: Improve the LAST reference assignment
The last frame which has the smallest diff should be consider as
the first choice rather than the golden frame. Especially when only
one reference available, this way can improve the BD rate about 5
percentage.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6507>
2024-04-02 09:57:51 +01:00
He Junyan
c4bb6d301e va: av1enc: Fix the reference number setting bug
The current way will let the total reference number surplus the
reference number set by the "ref-frames" property.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6507>
2024-04-02 09:57:51 +01:00
Alexander Slobodeniuk
f1d28fdcf7 d3d11videosink: disconnect signals before releasing the window
It might happen that the key event arrives when the d3d11videosink
is stopping. In case of GstD3D11WindowWin32 it can raise a
navigation event even when the sink is already freed, because the
window object's refcount may reach 0 in the window thread. In
other words sometimes the GstD3D11WindowWin32 lives few ms more
then the GstD3D11VideoSink, because it's freed asynchronously.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6492>
2024-03-30 19:06:31 +00:00
Nicolas Dufresne
fa44d14d2b v4l2codecs: alphadecoder: Explicitly pass 64 bit integers as such through varargs
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6469>
2024-03-28 11:16:28 +00:00
Piotr Brzeziński
4020144a65 vtdec: Fix caps criticals during negotiation
Calling gst_pad_peer_query_caps() without a filter can give us EMPTY caps, whereas all the code below
assumes that's not the case. Replacing query+intersect with a filtered query ensures we always get a subset
of the template caps back.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6454>
2024-03-27 16:24:07 +00:00
Piotr Brzeziński
11c6432cca vtdec: Ignore output loop errors in drain() if we're flushing
In an early non-linked scenario, this was causing a ton of criticals about the queue array,
because the output callback would still fire for leftover frames that were still being processed by VT
at the time the output loop stopped. This makes sure they're flushed correctly as well.

Also renames gst_vtdec_loop to gst_vtdec_output_loop for consistency with related functions.

wip

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6411>
2024-03-20 14:21:24 +01:00
Piotr Brzeziński
bf88fb152d vtdec: Fix a deadlock during ProRes playback
Sometimes a call to negotiate (and thus drain) can happen from the output loop
(via finish_frame()), which will tell VT to output all internal frames, but that won't succeed
if we happen to decide to wait for the queue to empty (because the loop is waiting for draining to finish and
will not make space in the queue!). This commit adds an override for the queue size limit if we're draining/flushing.

This bug could happen for any formats, but was especially obvious for ProRes, which has dpb_size of 0.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6411>
2024-03-20 14:21:24 +01:00
Ruijing Dong
b547c8eebb va: enc : checking surface alignment attribute
Apply surface alignment attribute when availalbe,
also fix frame cropping issue for va h265 encoder.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6399>
2024-03-18 20:55:38 +01:00
Seungha Yang
9061e464a8 d3d12: Fix SDK debug layer activation
Debug layer must be enabled before creating device. Otherwise
already opened devices before the activation will be removed

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6383>
2024-03-16 15:58:13 +01:00
Robert Mader
aef872944e v4l2codecs: decoders: Add DMA_DRM caps support
In order to simplify caps negotiations for clients and, notably, be more
compatible with va* decoders.
Crucially this allows clients to know ahead of time whether buffers will
actually be DMABufs.

Similar to GstVaBaseDec we only announce system memory caps if the peer
has ANY caps. Further more, and again like va decoders, we fail in
`decide_allocation()` if DMA_DRM caps are used without VideoMeta.
Apart from buggy peers this can happen e.g. when a peer with ANY caps
is used in combination with caps filters.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6376>
2024-03-14 23:32:00 +00:00
Robert Mader
383545d856 v4l2codecs: decoders: Introduce and use set_output_state helper class
Allowing us to avoid some code duplication. This will become more
important with upcoming changes to caps generation.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6376>
2024-03-14 23:32:00 +00:00
Robert Mader
84d015cabb v4l2codecs: decoder: Clean up select_src_format()
Most importantly rely on video info helpers instead of manual parsing
of caps, which will allow us to use additional helpers in the future.

While on it, tighen the check for supported formats - failing that
indicates a bug in caps negotiation - and make some style changes.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6376>
2024-03-14 23:32:00 +00:00
Robert Mader
b5fe2319f3 v4l2codecs: decoder: Generalize size enumeration caps
By reducing the generated caps to the minimal number of fields and
using intersections instead of merges. This will allow us to reuse the
result in the future.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6376>
2024-03-14 23:32:00 +00:00
Robert Mader
7b6d6fe080 v4l2codecs: decoders: Use src template for negotiation filter
This ensures we don't create filter caps that are not supported by the
individual codec implementations, as well as that the resulting caps
have the required fields so they can be turned into a GstVideoFormat.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6376>
2024-03-14 23:32:00 +00:00
Seungha Yang
615b7ca7ca asio: Fix {input,output}-channels property handling
Fixing regression introduced by the commit 06dc931b52

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6372>
2024-03-14 13:46:28 +00:00