Commit graph

1617 commits

Author SHA1 Message Date
Seungha Yang
4ede9fef07 dwrite: Fix crash on device update
Selected blend mode should not be cleared on device update

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6508>
2024-04-02 17:36:28 +00:00
Seungha Yang
106187cc59 nvencoder: Add support for RGB formats
Adding RGBA, RGBx, BGRA, BGRx, VUYA and RGB10A2_LE format support for performance.
However, these formats are not still recommended if upstream can support
native YUV formats (e.g., NV12, P010) since NVENC does not expose
conversion related optiones. Note that VUYA format is 4:4:4 YUV format
already but NVENC runtime will convert it to 4:2:0 format internally

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6417>
2024-04-02 13:07:29 +00:00
Seungha Yang
e6f496a240 cuda: Add support for VUYA format
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6417>
2024-04-02 13:07:29 +00:00
Seungha Yang
707ac69f5f qsv: Add support for d3d12 interop in encoder
Since QSV API does not support D3D12, try to import d3d12 resource
into d3d11 texture. Note that resource sharing requires
D3D12_SHARED_RESOURCE_COMPATIBILITY_TIER_2 for NV12 texure sharing.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6501>
2024-04-02 11:54:39 +00:00
He Junyan
78f1299ce4 va: av1enc: Avoid reopen encoder or renegotiate
If parameters remain similar enough to avoid either encoder reopening
or downstream renegotiation, avoid it.

This is going to be useful for dynamic parameters setting.

To check if the stream parameters changed, so the internal encoder has
to be closed and opened again, are required two steps:

1. If input caps, format, profile, chroma or rate control mode have changed.
2. If any of the calculated variables and element properties have changed.

Later on, only if the output caps also changed, the pipeline
is renegotiated.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6441>
2024-04-02 09:06:07 +00:00
He Junyan
075937dc19 va: vp9enc: Avoid reopen encoder or renegotiate
If parameters remain similar enough to avoid either encoder reopening
or downstream renegotiation, avoid it.

This is going to be useful for dynamic parameters setting.

To check if the stream parameters changed, so the internal encoder has
to be closed and opened again, are required two steps:

1. If input caps, format, profile, chroma or rate control mode have changed.
2. If any of the calculated variables and element properties have changed.

Later on, only if the output caps also changed, the pipeline
is renegotiated.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6441>
2024-04-02 09:06:07 +00:00
He Junyan
40629b84ee va: av1enc: Improve the LAST reference assignment
The last frame which has the smallest diff should be consider as
the first choice rather than the golden frame. Especially when only
one reference available, this way can improve the BD rate about 5
percentage.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6379>
2024-04-02 03:54:52 +00:00
He Junyan
1745a82516 va: av1enc: Fix the reference number setting bug
The current way will let the total reference number surplus the
reference number set by the "ref-frames" property.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6379>
2024-04-02 03:54:52 +00:00
Seungha Yang
bb268eafa1 webview2: Add support for d3d12 interop
Enable shared copy to D3D12 resource

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6499>
2024-04-02 03:43:59 +09:00
Seungha Yang
d00c26cdc8 d3d12screencapturesrc: Use gst_d3d12_memory_get_d3d11_texture()
... and use fence to wait for GPU sync

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6499>
2024-04-02 03:43:59 +09:00
He Junyan
e023546162 va: encoder: Fix the unit of bitrate in debug log message
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6483>
2024-04-01 15:39:13 +00:00
He Junyan
9c3bd3950e va: vp9enc: Adjust the coded buffer size to avoid failure
Some extreme case such as "videotestsrc pattern=1" can generate pure
white noise videoes, for which encoder may generate too big output
for current coded buffer size. We now consider the qindex and bitrate
to avoid that.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6483>
2024-04-01 15:39:13 +00:00
He Junyan
2560f4c581 va: vp9enc: Fix the frame size not enough issue for super frame
The current code forgets to add the current last frame size into
the total super frame size.

Fixes: #3427
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6483>
2024-04-01 15:39:13 +00:00
Cheah, Vincent Beng Keat
249a086612 msdk: Fix mjpeg BGRx encode
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6401>
2024-04-01 11:23:24 +00:00
Seungha Yang
7d0e90b7bc d3d12: Move core part to gst-libs
Move buffer pool, converter, and device abstraction layer to
public library

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6494>
2024-04-01 10:37:50 +00:00
Seungha Yang
7829315305 d3d12memory: Define new D3D12 map flags
Define GST_MAP_READ_D3D12 and GST_MAP_READ_D3D12 flags

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6494>
2024-04-01 10:37:50 +00:00
Seungha Yang
abeccdd6bc d3d12: Make resource getter methods consistent
Returns COM pointer directly everywhere

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6494>
2024-04-01 10:37:49 +00:00
Seungha Yang
706d5402fa d3d12: Remove device11on12 wrapping layer
It was added to avoid symbol conflict between DirectX-header project
and Windows SDK, but symbol conflict does not happen

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6494>
2024-04-01 10:37:49 +00:00
Seungha Yang
927e249557 d3d12bufferpool: Use d3dx12.h format table
The format table in SDK header defines all required information
already.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6494>
2024-04-01 10:37:49 +00:00
Seungha Yang
0527f27204 d3d12: Add a helper method for device equality check
GstD3D12Device object itself is not singltons anymore but
underlying private struct is singltons.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6494>
2024-04-01 10:37:49 +00:00
Mengkejiergeli Ba
faddaa8ba5 msdkvpp: Set colorimetry for src caps
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6316>
2024-04-01 02:20:50 +00:00
Mengkejiergeli Ba
72bb59b37c msdkenc: Set VideoFullRange according to input colorimetry range
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6316>
2024-04-01 02:20:50 +00:00
Seungha Yang
f7bdf91ad7 webview2: Add support for javascript injection
Allow javascript injection for various custom use cases.
For example, scrollbars and scrolling can be disabled via

gst-launch-1.0 webview2src location=https://gstreamer.freedesktop.org \
    javascript="document.querySelector('body').style.overflow='hidden'" ! ...

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6487>
2024-03-30 16:44:28 +00:00
Seungha Yang
85d422f7c6 webview2: Use IContainerVisual for offscreen rendering
Capturing from hidden HWND fails sometimes for some reason.
Instead of rendering to hidden HWND, render webpage to container
visual and create WGC item from the container visual object.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6487>
2024-03-30 16:44:28 +00:00
Alexander Slobodeniuk
7b69d1758f d3d11videosink: disconnect signals before releasing the window
It might happen that the key event arrives when the d3d11videosink
is stopping. In case of GstD3D11WindowWin32 it can raise a
navigation event even when the sink is already freed, because the
window object's refcount may reach 0 in the window thread. In
other words sometimes the GstD3D11WindowWin32 lives few ms more
then the GstD3D11VideoSink, because it's freed asynchronously.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6476>
2024-03-30 16:02:12 +00:00
Mengkejiergeli Ba
41c52f5739 msdk: Fix session close failure
In the case of multi-channels transcoding, a context with child
sesseion can be parent for others, so we need to check if the
msdkcontext has any child session in the list to avoid session
leaks. Otherwise, we will see the failure of closing a parent
session because one of its child's child session not released.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6259>
2024-03-29 04:20:47 +00:00
Seungha Yang
4888a25bac webview2: Add Microsoft WebView2 based web browser source
Adding webview2src element

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4631>
2024-03-28 10:49:39 +00:00
He Junyan
a170fa4f46 va: {av1, vp9}enc: Use g_free() to free frames
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6466>
2024-03-28 16:29:50 +08:00
Nicolas Dufresne
cee05343f0 v4l2codecs: alphadecoder: Explicitly pass 64 bit integers as such through varargs
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6452>
2024-03-27 14:01:45 -04:00
Piotr Brzeziński
82294fd665 vtdec: Fix caps criticals during negotiation
Calling gst_pad_peer_query_caps() without a filter can give us EMPTY caps, whereas all the code below
assumes that's not the case. Replacing query+intersect with a filtered query ensures we always get a subset
of the template caps back.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6429>
2024-03-27 09:04:28 +00:00
He Junyan
7ac9d26533 va: vp9enc: Correct the flags for registering properties
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6437>
2024-03-27 10:24:12 +08:00
He Junyan
fa50571d3b va: av1enc: Correct the flags for registering properties
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6437>
2024-03-27 10:21:08 +08:00
He Junyan
73c1c1fe7f va: {vp9, av1}enc: Do not use g_slice_new() to create frames
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6437>
2024-03-27 10:18:04 +08:00
He Junyan
71eff7ce97 va: encoder: update the bitrate change correctly
We should update and notify the bitrate change at a common place,
no matter whether the bitrate is calculated or not.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6433>
2024-03-26 18:15:20 +00:00
He Junyan
56703704a5 va: av1enc: enable ICQ and QVBR modes
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6433>
2024-03-26 18:15:20 +00:00
He Junyan
5c51abfa71 va: vp9enc: enable ICQ and QVBR modes
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6433>
2024-03-26 18:15:20 +00:00
He Junyan
d90f718b1a va: h265enc: enable ICQ and QVBR modes
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6433>
2024-03-26 18:15:20 +00:00
He Junyan
f45f52a786 va: h264enc: enable ICQ and QVBR modes
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6433>
2024-03-26 18:15:20 +00:00
He Junyan
85bcea1776 va: encoder: Enable ICQ and QVBR mode in rate control map
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6433>
2024-03-26 18:15:20 +00:00
He Junyan
3f1ce4460a va: encoder: Set the quality_factor parameter in rate control
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6433>
2024-03-26 18:15:20 +00:00
Seungha Yang
bbdfa00b7b d3d12: Add support for cross-compile
... and fix bunch of GCC reported warnings

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6435>
2024-03-25 11:41:25 +00:00
Seungha Yang
6c80d2f5f8 d3d12: Allow building without WGC support
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6435>
2024-03-25 11:41:25 +00:00
Seungha Yang
2649d2ae62 d3d12: Port to d3dshader library
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6434>
2024-03-25 00:02:12 +09:00
Seungha Yang
cab1f3e547 d3d11: Port to d3dshader library
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6434>
2024-03-24 22:28:14 +09:00
Seungha Yang
43da9b4249 d3d12: Update shader to be Shader Model 5.0 compatible
And use fxc HLSL compiler

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6434>
2024-03-24 17:15:22 +09:00
Seungha Yang
0c148a96db d3d12: Add IPC elements
Adding d3d12ipcsink and d3d12ipcsrc elements, equivalent to D3D11 ones.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6428>
2024-03-22 23:09:12 +09:00
Seungha Yang
0006ad1f86 d3d12: Add buffer copy helper method
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6428>
2024-03-22 23:09:05 +09:00
Seungha Yang
c612af6d42 d3d12commandqueue: Always invoke notify asynchronously
Otherwise the callback thread is unpredictable

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6428>
2024-03-22 20:45:01 +09:00
Seungha Yang
18091f0115 d3d12memory: Update for API interop
Add support for destroy notify in case of wrapped memory, and
allow setting external fence for interop

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6428>
2024-03-22 20:01:16 +09:00
Seungha Yang
0d559bead9 d3d12: Make primary fence sharable
Create primary fence with D3D12_FENCE_FLAG_SHARED flag so that
the fence can be shared with other APIs or processes

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6428>
2024-03-22 18:57:26 +09:00
Piotr Brzeziński
982c73dfdd vtdec: Ignore output loop errors in drain() if we're flushing
In an early non-linked scenario, this was causing a ton of criticals about the queue array,
because the output callback would still fire for leftover frames that were still being processed by VT
at the time the output loop stopped. This makes sure they're flushed correctly as well.

Also renames gst_vtdec_loop to gst_vtdec_output_loop for consistency with related functions.

wip

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6397>
2024-03-20 13:07:25 +00:00
Piotr Brzeziński
112951ce55 vtdec: Fix a deadlock during ProRes playback
Sometimes a call to negotiate (and thus drain) can happen from the output loop
(via finish_frame()), which will tell VT to output all internal frames, but that won't succeed
if we happen to decide to wait for the queue to empty (because the loop is waiting for draining to finish and
will not make space in the queue!). This commit adds an override for the queue size limit if we're draining/flushing.

This bug could happen for any formats, but was especially obvious for ProRes, which has dpb_size of 0.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6397>
2024-03-20 13:07:25 +00:00
Seungha Yang
17f92ab400 asio: Add support for MinGW build
Drop MSVC specific bits and remove unused dependency

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6404>
2024-03-20 10:53:16 +00:00
Seungha Yang
f0761a7358 asio: Drop external SDK header dependency
Build ASIO plugin using our tiny SDK header

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6404>
2024-03-20 10:53:16 +00:00
He Junyan
3b57825ca6 va: Implement the vavp9enc plugin
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3293>
2024-03-20 08:53:11 +00:00
Seungha Yang
b417c62940 d3d12device: Set debugging friendly object name
Build object name with DXGI adapter index

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6395>
2024-03-19 12:03:55 +00:00
Seungha Yang
6de5eab1e4 d3d12: Suppress expected leak reports
Such leaks are expected and intended ones

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6395>
2024-03-19 12:03:55 +00:00
Seungha Yang
a1c8fc6163 d3d12device: Keep device object permanently
Because ID3D12Device objects are singletons per adapter,
GstD3D12Device was following the API design, that is, keep track
of global GstD3D12Device objects and reuses it.
That means ID3D12Device object can be released at the time
when GstD3D12Device is destroyed.

But exetrnal APIs such as NVENC does not seem to be happy
with the released ID3D12Device, that could be a driver bug though.

Let's hold already opened ID3D12Device permanently without releasing
it for now.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6395>
2024-03-19 12:03:55 +00:00
Seungha Yang
17d6c7350c d3d12commandqueue: Add drain method
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6395>
2024-03-19 12:03:55 +00:00
Seungha Yang
393fb4733c d3d12: Use native device handle if possible
Various abstraction objects such as command queue/list/allocator
can be constructed without GstD3D12Device

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6395>
2024-03-19 12:03:55 +00:00
Seungha Yang
331522210f d3d12commandqueue: Allow empty command list
Just increase fence value and signal the queue in that case

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6395>
2024-03-19 12:03:54 +00:00
Ruijing Dong
bc858976db va: enc : checking surface alignment attribute
Apply surface alignment attribute when availalbe,
also fix frame cropping issue for va h265 encoder.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6282>
2024-03-18 16:34:42 +00:00
Seungha Yang
8acf844fed d3d12: Fix SDK debug layer activation
Debug layer must be enabled before creating device. Otherwise
already opened devices before the activation will be removed

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6382>
2024-03-16 21:25:38 +09:00
Robert Mader
5189e8b956 v4l2codecs: decoders: Add DMA_DRM caps support
In order to simplify caps negotiations for clients and, notably, be more
compatible with va* decoders.
Crucially this allows clients to know ahead of time whether buffers will
actually be DMABufs.

Similar to GstVaBaseDec we only announce system memory caps if the peer
has ANY caps. Further more, and again like va decoders, we fail in
`decide_allocation()` if DMA_DRM caps are used without VideoMeta.
Apart from buggy peers this can happen e.g. when a peer with ANY caps
is used in combination with caps filters.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5890>
2024-03-14 17:32:13 +00:00
Robert Mader
513d0d8cbb v4l2codecs: decoders: Introduce and use set_output_state helper class
Allowing us to avoid some code duplication. This will become more
important with upcoming changes to caps generation.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5890>
2024-03-14 17:32:13 +00:00
Robert Mader
a95acbcc11 v4l2codecs: decoder: Clean up select_src_format()
Most importantly rely on video info helpers instead of manual parsing
of caps, which will allow us to use additional helpers in the future.

While on it, tighen the check for supported formats - failing that
indicates a bug in caps negotiation - and make some style changes.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5890>
2024-03-14 17:32:13 +00:00
Robert Mader
73b69d8ca0 v4l2codecs: decoder: Generalize size enumeration caps
By reducing the generated caps to the minimal number of fields and
using intersections instead of merges. This will allow us to reuse the
result in the future.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5890>
2024-03-14 17:32:13 +00:00
Robert Mader
65896dab75 v4l2codecs: decoders: Use src template for negotiation filter
This ensures we don't create filter caps that are not supported by the
individual codec implementations, as well as that the resulting caps
have the required fields so they can be turned into a GstVideoFormat.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5890>
2024-03-14 17:32:13 +00:00
Seungha Yang
7aff9c8600 asio: Fix {input,output}-channels property handling
Fixing regression introduced by the commit 06dc931b52

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6370>
2024-03-14 21:01:40 +09:00
Michiel Westerbeek
a4aa9e197e gstcudaconvertscale, gstvavpp, videoconvertscale: downgrade 'Can't keep DAR' to debug
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5730>
2024-03-13 16:06:56 +00:00
Seungha Yang
94dfef68e1 d3d12device: Fix IDXGIFactory2 leak
factory passed to gst_d3d12_device_find_adapter() method is valid
handle already

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6340>
2024-03-12 22:06:01 +00:00
L. E. Segovia
71510860af meson: Require tinyalsa >= 1.1.0 when building its plugin
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6311>
2024-03-12 20:30:11 +00:00
L. E. Segovia
9c8549c31c tinyalsasink: Fix missing const and deprecations with tinyalsa v2
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6311>
2024-03-12 20:30:11 +00:00
Seungha Yang
c9aaf39279 cuda,d3d11,d3d12bufferpool: Disable preallocation
Do not chain up to parent's GstBufferPool::start() which will do
preallocation. We don't want it to be preallocated
since there are various cases where negotiated downstream buffer pool is
not used at all (e.g., zero-copy decoding, IPC elements).

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6326>
2024-03-12 18:07:29 +00:00
Antonio Larrosa
7b8fa42f8a va{h264,h265,av1}enc: fix potential crash on devices without rate control
This fixes a crash in `gst_va_h264_enc_class_init` and `gst_va_h265_enc_class_init`
(and probably also in gst_va_av1_enc_class_init) when calling
`g_object_class_install_properties (object_class, n_props, properties);`

When rate_control_type is 0, the following code is executed in :

```
  } else {
    n_props--;
    properties[PROP_RATE_CONTROL] = NULL;
  }
```

n_props has initially a value of N_PROPERTIES but PROP_RATE_CONTROL
is not the last element in the array, so it's making
g_object_class_install_properties fail to iterate over the
properties array.

This applies the same fix to gstvah264enc.c, gstvah265enc.c and
gstvaav1enc.c.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6319>
2024-03-12 16:58:07 +00:00
Piotr Brzeziński
d3fba31da0 macos: Move atdec from applemedia (-bad) to osxaudio (-good)
osxaudio has a few helper methods potentially useful in atdec (or future atenc), like GStreamer -> CoreAudio
channel mapping. Doesn't make sense to duplicate them in applemedia, and atdec is the only audio-oriented
element there anyway.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6223>
2024-03-12 09:55:10 +00:00
He Junyan
861c1a44be va: av1enc: Init the output_frame_num when resetting gf group
Fixes: #3359
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6308>
2024-03-11 12:38:57 +00:00
Seungha Yang
4db7eb0290 d3d12screencapturesrc: Add support for WGC API
Adding support for window and monitor capturing by using
Windows Graphics Capture API.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6256>
2024-03-08 01:05:24 +09:00
Seungha Yang
63ef405131 d3d12memory: Implement NT handle caching and custom user data support
Same as the d3d11 memory implementation.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6256>
2024-03-08 01:03:28 +09:00
Sebastian Dröge
df00962cb8 ajasink: Make logging between ajasrc and ajasink more consistent
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6208>
2024-03-06 11:09:58 +00:00
Sebastian Dröge
396aa55958 ajasrc: Improve clock handling
Provide a clock from the source that is a monotonic system clock with
the rate corrected based on the measured and ideal capture rate of the
frames.

If this clock is selected as pipeline clock, then provide perfect
timestamps to downstream.

Otherwise, if the pipeline clock is the monotonic system clock, use the
internal clock for converting back to the monotonic system clock.

Otherwise, use the monotonic system clock time calculated in the above
case and convert that to the pipeline clock.

In all cases this will give a smoother time than the previous code,
which simply took the difference between the driver provided capture
time and the current real-time clock time, and applied that to the
current pipeline clock time.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6208>
2024-03-06 11:09:58 +00:00
Sebastian Dröge
86e3009448 ajasrc: Move frame drop detection after the frame transfer
Otherwise there's a small window between querying the state and doing
the transfer in which a frame could be dropped, and we would then output
the frame right after the dropped one as if it was the dropped frame.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6208>
2024-03-06 11:09:58 +00:00
Sebastian Dröge
170bf0cc8e ajasrc: Improve debug output related to frame transfers
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6208>
2024-03-06 11:09:58 +00:00
He Junyan
a5a1944db4 MSDK: Set the job type when create context from external handle
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6221>
2024-03-06 05:22:23 +00:00
He Junyan
1a78d61b9f vah265enc: Set backward_num to 1 in low delay mode
In low delay B mode, the P frame is converted as B frame with forward
references. For example, One P frame may refers to P-1, P-2 and P-3 in
list0 and refers to P-3, P-2 and P-1 in list1.
So the num in list0 and list1 does not reflect the forward_num and
backward_num. The vaapi does not provide ref num for forward or backward
so far. In this case, we just consider the backward_num to be 1 conservatively.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6249>
2024-03-05 23:39:43 +00:00
He Junyan
b9c28920e0 vah265enc: Improve B pyramid mode in HEVC
If the reference frame number is bigger than 2, we can enable the
pyramid B mode. We do not need to assign a reference frame to each
pyramid level.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6249>
2024-03-05 23:39:43 +00:00
He Junyan
196de61035 vah265enc: Expand log2_max_pic_order_cnt if needed
In b_pyramid mode, B frames can be ref and prevPicOrderCntLsb can
be the B frame POC which is smaller than the P frame. This can cause
POC diff bigger than MaxPicOrderCntLsb/2 and generate wrong POC value.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6249>
2024-03-05 23:39:43 +00:00
Seungha Yang
d0713e029c d3d11memory, d3d12memory: Fix outstanding memory count tracing
Gets being released memory back to queue even if allocator is flushing
in order to count the number of outstanding memory objects.
Also, clear queue if there's no outstanding memory object and
allocator is flushing

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6240>
2024-02-29 11:57:50 +00:00
Alexander Slobodeniuk
f92c27a49e d3d11window_win32: fix crash on RC unprepare() vs window_proc()
Unprepare method posts WM_GST_D3D11_DESTROY_INTERNAL_WINDOW
command to the window queue, and from that moment considers
internal_hwnd to be released, and so it sets it to null.
The problem is that it's possible that right at that moment
the window thread might be already processing some other
command, or just another command might be already in the queue.
On practice we met a crash when WM_PAINT got processed in between
(unprepare already finished and WM_GST_D3D11_DESTROY_INTERNAL_WINDOW
was not handled yet)

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6187>
2024-02-26 23:17:05 +00:00
He Junyan
a734f95790 vabasedec: Fix a possible NULL pointer dereference
The format in _get_preferred_format_and_caps_features() may be NULL.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6206>
2024-02-23 16:55:50 +00:00
Víctor Manuel Jáquez Leal
95436d1eb4 vabasedec: disable derived images for i965 driver
Since it has a very poor performance at reading derived images, which is the
most common use case for decoders.

Partially fixes: #3325

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6202>
2024-02-23 14:20:14 +00:00
Víctor Manuel Jáquez Leal
9567f8f444 vapostproc: optimization for va memory to system memory only
When the conversion is only caps feature from memory:VAMemory to system memory,
it's possible to optimize by doing a pseudo pass-through since the va-backed
buffers are the same for system memory buffers.

This change will also mitigates #2940

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6174>
2024-02-23 12:40:38 +00:00
Seungha Yang
06dc931b52 asiosink: Fix channel selection
Fixing copy paste mistake

Fixes: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/3321
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6170>
2024-02-23 11:21:08 +00:00
Víctor Manuel Jáquez Leal
cc7726ea39 vabase: fail decide allocation if dmabuf without videometa
If the allocation query received from downstream doesn't handle GstVideoMeta but
it requests memory:DMABuf caps feature, it's incomplete, so we rather reject the
negotiation.

Both in base decoder, base transform and compositor.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6155>
2024-02-23 09:52:50 +00:00
Seungha Yang
5d62f408f2 nvcodec: Add plugin status message
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6167>
2024-02-22 23:30:29 +00:00
Seungha Yang
fa477a4a25 amfcodec: Add plugin status message
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6167>
2024-02-22 23:30:29 +00:00
Víctor Manuel Jáquez Leal
301e281777 vabasedec: refactor format and capsfeature selection
This is a simplification of the venerable
gst_va_base_dec_get_preferred_format_and_caps_features() function, which
predates since gstreamer-vaapi. It's used to select the format and the
capsfeature to use when setting the output state. It was complex and hard to
follow. This refactor simplifies a lot the algorithm.

The first thing to remove _downstream_has_video_meta() since, most of the time
it will be called before the caps negotiation, and allocation queries make sense
only after caps negotiation. It might work during renegotiation but, in that
case, caps feature change is uncommon. Better a simple and common approach.

Also, for performance, instead of dealing with caps features as strings, GQuarks
are used.

The refactor works like this:

1. If peer pad returns any caps, the returned caps feature is system memory and
   looks for a proper format in the allowed caps.

2. The allowed caps are traversed at most 3 times: one per each valid caps
   feature. First VAMemory, later DMABuf, and last system memory. The first to
   match in allowed caps is picked, and the first format matching with the
   chroma is picked too.

Notice that, right now, using playbin videoconvert never return any.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6154>
2024-02-22 20:59:13 +00:00
Mengkejiergeli Ba
fc38459e50 msdk: Fix possible memory leaks
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6161>
2024-02-22 11:53:06 +00:00
Seungha Yang
911d840288 wasapi: Fix alloc/free function mismatch
... and fix leak in wasapi device provider

Fixes: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/3326
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6169>
2024-02-21 21:03:32 +00:00
Seungha Yang
06b9864dae wasapi2: Fix task memory leak
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6169>
2024-02-21 21:03:32 +00:00
Tim-Philipp Müller
9b84d6398a qsv: use new plugin status message API
Minimal example.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3832>
2024-02-21 00:58:19 +00:00
Seungha Yang
354af30d9f wasapi2: Respect ringbuffer buffer/latency time
Decide buffer size based on configured buffer/latency time
if low-latency is disabled, so that ringbuffer can buffer more
than minimum required size.

Fixes: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/2870
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6141>
2024-02-20 11:37:09 +00:00
Seungha Yang
93f0135798 wasapi2: Fix choppy rendering
This reverts questionable commit 009bc15f33
which looks completely wrong.

The GstWasapi2RingBuffer:buffer_size variable is used to
calculate available buffer size we can write
(i.e., available size = buffer_size - padding_size).
But the commit makes the size to be exactly same as buffer period.
Then, it can confuse this element as if the endpoint buffer is full on
I/O event callback (if padding size is equal to buffer period)
but it's not true.

Fixes: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/2870
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6132>
2024-02-19 04:39:25 +00:00
Robert Mader
6b024b50ab v4l2codecs: h264: Fix a memory leak on renegotiation
We only use this anchor when streaming, in which case output_state is
set and needs to get unreffed.
This is in line with how it is handled for all other codecs.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6127>
2024-02-15 22:37:11 +01:00
Xavier Claessens
d86a6715e1 aja: Replace global semaphore with per-device flock()
The global semaphore was never closed/unlinked, causing permission
denied issue if the device is later used by another user. Properly
removing the semaphore when stopping the pipeline would still leave it
open in case of a crash.

With a GStreamer specific name, it was also not preventing other apps to access
the device concurrently.

Finally, if the system has multiple cards, the lock should be per card
and not global (to be confirmed).

Fixes: #3283.

Sponsored-by: Netflix Inc.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6117>
2024-02-15 09:30:20 +00:00
Seungha Yang
cfa8386dd0 nvdecoder: Enable zero-copy only if explicitly enabled
Keep pre-1.24 behavior unless user specifies the number of
output surface size. We are calculating output surface size
conservatively, and it can result in over allocation.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6102>
2024-02-13 12:37:07 +00:00
Seungha Yang
59358e439d nvh265dec: Don't convert unknown video format to string
gst_video_format_to_string() method does not allow unknown format

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6102>
2024-02-13 12:37:06 +00:00
Seungha Yang
693dbc9667 d3d12screencapturesrc: Fix choppy display
According to recommendation from MS, IDXGIOutputDuplication::ReleaseFrame()
needs to be called just before IDXGIOutputDuplication::AcquireNextFrame()
for performance reasons, so that driver can accumulate dirty rects
and update texture at once. But it seems to cause choppy output.
Do release acquired frame immediately once processing done,
like d3d11 implementation does.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6092>
2024-02-11 13:13:18 +00:00
Seungha Yang
36c4ef361b d3d12: Add VP8 decoder
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6088>
2024-02-10 21:23:51 +09:00
Seungha Yang
e2eff3e4a3 d3d12decoder: Fix bitstream buffer usage
Resource state of a buffer in upload heap should stay in
generic-read although it's effectively in common state. Some drivers
complains about the wrong state.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6088>
2024-02-10 21:23:28 +09:00
Tim-Philipp Müller
9fe9e60012 aja: suppress compiler warnings for aja ntv2 subproject
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6087>
2024-02-10 10:10:06 +00:00
Seungha Yang
c5e241932c amfcodec: Update plugin cache
Adding AV1 encoder documentation (added in 1.22) with various
new properties

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6084>
2024-02-10 01:18:27 +00:00
Seungha Yang
b4ee86d1e7 ksdeviceprovider: Fix crash while probing device caps
Ignore unexpected media type reported by driver, instead of abort

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6081>
2024-02-09 14:09:07 +00:00
Guillaume Desmottes
550a21ccbe avdtpsink: post error message when failing to start
Subclasses are supposed to report errors messages if their start
implementation failed.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1678>
2024-02-09 10:09:13 +00:00
Sebastian Dröge
3ca5a2554f ajasink: Add HANC/VANC ancillary data from GstAncillaryMeta
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5488>
2024-02-08 15:28:39 +00:00
Sebastian Dröge
efa7e70d16 ajasrc: Add GstAncillaryMeta for any HANC/VANC ancillary data that is captured
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5488>
2024-02-08 15:28:39 +00:00
Seungha Yang
d444fe3fdc nvcodec: Update plugin cache
Updating plugin cache for IPC elements, JPEG encoder, and newly added
properties. Also removing Gst*SLDec which does not exist anymore
but renamed to Gst*Dec.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6070>
2024-02-07 22:06:39 +00:00
Sebastian Dröge
ddd9dcc559 ajasink: Remove workaround for NTV SDK bug that is fixed since a while
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6056>
2024-02-05 16:24:13 +00:00
Seungha Yang
35e5178f0e d3d12: Fix potential self thread join
Fence data could hold GstD3D12Device directly or indirectly.
Then if it's holding last refcount, the device object will
be released from the device object's internal thread,
and will try join self thread.
Delegates it to other global background thread to avoid
self thread joining.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6042>
2024-02-05 01:37:32 +09:00
Seungha Yang
1c0f224f05 d3d12memory: Remove unused method
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6042>
2024-02-03 23:22:58 +09:00
Seungha Yang
6dc902a962 nvcodec: Add JPEG encoder
Adding nvJPEG library based JPEG encoder element

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6021>
2024-01-31 12:51:28 +00:00
Seungha Yang
fc4b136ac8 nvcodec: Move runtime compiler check to plugin init function
Required information for other elements

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6021>
2024-01-31 12:51:28 +00:00
Mengkejiergeli Ba
bf7f4eef20 msdk: Use gst_video_info_dma_drm_to_video_info to extract video info
Note that we need mappings for all drm_fourcc, otherwise we will get
GST_VIDEO_FORMAT_UNKNOWN for some formats...

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5991>
2024-01-31 06:24:13 +00:00
He Junyan
637d253e9f va: baseenc: Fix the wrong parameter order to call va_map_buffer()
The current gst_va_base_enc_copy_output_data() uses wrong parameter
order to call va_map_buffer().

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6020>
2024-01-30 15:50:23 +00:00
Seungha Yang
51162acc31 cuda: Report device open error
Call gst_cuda_result() with CUDA_ERROR_NO_DEVICE error code if
we could not open device, so that application can catch the error

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6006>
2024-01-30 14:30:41 +00:00
Seungha Yang
07ba225183 cudaipc: Add support for custom meta forwarding
Forward custom meta to peer ipcsrc elements

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6006>
2024-01-30 14:30:41 +00:00
He Junyan
b4eb97101f va: Implement the vaav1enc plugin
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3015>
2024-01-30 08:24:26 +00:00
He Junyan
f4779f1fab va: baseenc: Extend the create_output_buffer() to accept prefix data
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3015>
2024-01-30 08:24:25 +00:00
He Junyan
726cc895d8 va: encoder: Add copy_output_data() helper function
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3015>
2024-01-30 08:24:25 +00:00
He Junyan
39144f612e va: encoder: extend prepare_output() virtual function
The output of VP9 and AV1 encoder is a little different from the H264
and H265 encoder, it may contain repeat frames and so the output frame
number may be more than the input. We need to call finish_subframe()
when some frame will be repeated later. So we need to extend the
current prepare_output() virtual function.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3015>
2024-01-30 08:24:25 +00:00
Seungha Yang
1a7c85a261 qsvh264enc: Always specify field order
qsv runtime will report error when encoding interlaced stream
if session was initialized with unknown field order
but tff/bff is specified in per frame encoding parameter.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5835>
2024-01-25 14:25:16 +00:00
Seungha Yang
3855646f1d d3d12: Add MPEG-2 decoder
Most of code additions are just gluing already implemented
GstDxva and GstD3D12Decoder interfaces

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5979>
2024-01-25 11:45:23 +00:00
Ruben Gonzalez
5e3fafd025 SA: Remove extra check since libva min version is 1.12
The libva version was updated to 1.12 in PR 4781 [1]
[1] https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4781

NOTE: I also sort va_sources
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5978>
2024-01-25 00:45:55 +00:00
Seungha Yang
3991801d4a d3d12av1dec: Fix object type name on multi-gpu system
Fixing typo

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5964>
2024-01-24 22:35:17 +00:00
Seungha Yang
4fb284346e d3d12h264enc: Fix profile support check
Add missing CheckFeatureSupport() call, and remove unused field

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5952>
2024-01-22 20:10:24 +09:00
Seungha Yang
f44d5d18f3 d3d12: Pass target rectangle to ClearRenderTargetView()
Some drivers seem to be crashing if ClearRenderTargetView() is called
for P010/P016 texture's second subresource (UV plane) without
specified target rectangle.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5952>
2024-01-22 19:40:08 +09:00
Seungha Yang
4e1bf149d0 d3d12: Add H.264 video encoder
Adding video encoder element

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5808>
2024-01-19 12:47:35 +00:00
Mengkejiergeli Ba
b8cb0d1d7f msdkdec: Fix leaks
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5930>
2024-01-18 08:43:48 +00:00
Piotr Brzeziński
3f67e70f39 vtenc: Fix PAUSED->READY deadlock when output loop is running
Explicitly calls gst_vtenc_pause_output_loop when going PAUSED->READY to make sure GST_PAD_STREAM_LOCK is not taken.

Before this change, a deadlock would occur if pipeline got stopped right after one output buffer was generated by vtenc.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5933>
2024-01-18 07:37:58 +00:00
Mengkejiergeli Ba
a6ab4fa98b msdkdec: Let msdk base decoder maintain the output_state
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5929>
2024-01-17 10:04:03 +08:00
Mengkejiergeli Ba
2c61fd92e2 msdkvpp: Fix dma caps negotiation
Modify the fix_output_format in vpp to directly generate caps with
negotiated src caps, and we have the correct dma caps negotiation in
fix_output_format function. And thus, we can remove the redundant
negotiation of using function pad_accept_memory in vpp.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5845>
2024-01-16 14:04:05 +08:00
Seungha Yang
0cceb6f68f d3d12memory: Don't use persistent staging buffer map
Persistent map is not recommended in case of readback

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5919>
2024-01-14 13:10:56 +00:00
Seungha Yang
077470913d d3d12decoder: Use D3D12_HEAP_FLAG_CREATE_NOT_ZEROED flag
Since the first access to a texture is always write, zero initialization
is unnecessary

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5919>
2024-01-14 13:10:56 +00:00
Chris Spencer
865227b750 vkimagebufferpool: allow specifying initial image layout
The pool currently defaults to performing a layout transition to
VK_IMAGE_LAYOUT_TRANSFER_DST_OPTIMAL, with some special exceptions for
video usages. This may not be a legal transition depending on the usage.
Provide an API to explicitly control the initial image layout.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5881>
2024-01-13 05:54:00 +00:00
Ratchanan Srirattanamet
facb000afe androidmedia: add NDK implementation of Android MediaCodec
This reduces the amount of Java <-> native calls, which should reduce
overhead a bit. It also paves a way to share the code between Android
and a libhybris-based backend (where GNU/Linux system uses Android
driver) in the future.

Bug: https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/issues/1242
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4115>
2024-01-11 19:00:13 +00:00
Ratchanan Srirattanamet
a50ce9c6b0 androidmedia: allow multiple implementations of codec and format
When implementing NDK media support, it would be useful to also have JNI
implementation in the same binary as NDK media compatibility is lower.
As such, implement a rudimentary vtable system for gstamc-codec and
gstamc-format, and allow choosing the implementation at static_init()
time.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4115>
2024-01-11 19:00:13 +00:00
Ratchanan Srirattanamet
d0ffcb46df androidmedia: call all static_init() functions from single entry point
This allows the implementations to do custom logic behind the hood. For
example, when NDK implementation is added, the entrypoint can chooses to
statically initialize the NDK implementations or the JNI one.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4115>
2024-01-11 19:00:13 +00:00
Mengkejiergeli Ba
0d04660c5d msdk: Fix memory leaks
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5819>
2024-01-11 14:52:45 +08:00
Víctor Manuel Jáquez Leal
7f14bfe8b8 va: basedec: fix return type
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5903>
2024-01-10 12:48:16 +01:00
Mengkejiergeli Ba
14c5d3ee91 msdk: Modify the caps order when register plugins
With this patch, the caps is registered in the order of memory features
as: VAMemory, DMABuf then raw caps in linux path, and D3D11Memory then
raw caps in windows path. It helps to prioritize the video memory for all
msdk elements when doing negotiation.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5898>
2024-01-10 01:14:21 +00:00
Mengkejiergeli Ba
7d62166098 msdk: Remove useless function for vp8dec and vc1dec in windows path
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5898>
2024-01-10 01:14:21 +00:00
Robert Mader
a64f2bf628 v4l2codecs: Always chain up to parent decide_allocation function for all codecs
Apply the changes to the codecs previously left out, fixing playback
issues seen with VP9.

See: 70ff80a873 ("v4l2codecs: Always chain up to parent decide_allocation function")
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5896>
2024-01-09 17:29:52 +00:00
Robert Mader
75b7e5fcb3 v4l2codecs/av1decoder: Allow output caps to be updated
To bring AV1 in line - needed for the next commit.

See: d3c5fc815e ("v4l2codecs: Allow output caps to be updated")
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5896>
2024-01-09 17:29:52 +00:00
Seungha Yang
a242b93f33 d3d12decoder: Release decoder and heap on stop
Sometimes driver crash happens if ID3D12VideoDevice got closed first,
and then ID3D12VideoDecoderHeap/ID3D12VideoDecoder are released.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5895>
2024-01-09 13:52:44 +00:00
Seungha Yang
6ffed9b3dc d3d12videosink: Fix crash on set_buffer()
set_buffer() can be called with null buffer

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5895>
2024-01-09 13:52:44 +00:00
Seungha Yang
7f2b4b6b09 d3d12: Fix warnings reported by debug layer
Fixing below debug layer report
ID3D12Device::CreateCommittedResource: Ignoring InitialState D3D12_RESOURCE_STATE_COPY_DEST.
Buffers are effectively created in state D3D12_RESOURCE_STATE_COMMON.

Buffer resource will be automatically promoted to D3D12_RESOURCE_STATE_COPY_DEST
at the very first COPY operation time.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5895>
2024-01-09 13:52:44 +00:00
Seungha Yang
e1ecd1de2f d3d12: Simplify fence data setup
Adding COM and GstMiniObject specific methods to skip passing
free function pointer

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5895>
2024-01-09 13:52:44 +00:00
Seungha Yang
2c3f3c0cb6 d3d12device: Print live objects on finalize()
The report from debug layer can help tracing resource leak

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5895>
2024-01-09 13:52:44 +00:00
Seungha Yang
0077c4beb0 d3d12decoder: Set buffer flag on output buffer
... and remove unnecessary condition check

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5891>
2024-01-07 13:38:44 +00:00
Seungha Yang
02563605e8 d3d12videosink: Add gamma, primaries and sampling filter properties
Add properties to control conversion methods

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5891>
2024-01-07 13:38:44 +00:00
Seungha Yang
368d8b9252 d3d12videosink: Add support for 3D transformation
Add x, y, and z axis rotation with scaling support

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5891>
2024-01-07 13:38:44 +00:00
Seungha Yang
72237d2563 d3d12videosink: Add support for MSAA
Adding "msaa" property to support MSAA rendering

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5891>
2024-01-07 13:38:44 +00:00
Seungha Yang
de331217aa d3d12videosink: Add support for fullscreen mode
Adding "fullscreen-on-alt-enter" and "fullscreen" properties so that
use can control fullscreen mode switch

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5891>
2024-01-07 13:38:44 +00:00
Seungha Yang
24d515f57d d3d11screencapturesrc: Hide symbols
Fix potential linking error in case of static build

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5883>
2024-01-05 21:27:35 +09:00
Seungha Yang
ae3ed20f41 d3d12: Add screen capture element
Since DXGI desktop duplication API does not work with Direct3D12 device,
this element will use Direct3D11 device to acquire frame.
Then other rendering operations (e.g., texture copy, render pipeline) will
happen using Direct3D12 API

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5883>
2024-01-05 21:27:35 +09:00
Mengkejiergeli Ba
5d488c444e msdkvpp: Remove passthrough condition in propose_allocation
According to basetransform func: gst_base_transform_do_bufferpool,
it is the upstream to decide if it needs a bufferpool from vpp, so
we don't need passthrough condition in subclass propose_allocation.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5879>
2024-01-04 04:35:40 +00:00
Seungha Yang
1b86c9d370 d3d12decoder: Try zero-copy in case of reverse playback too
In case of tier 1 decoder, always use reference-only picture to avoid
fixed-size pool limitation and output decoded picture without
copy even for negative rate. Also do not use copy queue for GPU to GPU
copy. Copy queue is specialized for upload/download and may occupy
PCIE bandwidth. Use direct queue as recommended by vendors.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5877>
2024-01-03 02:08:53 +09:00
Seungha Yang
1fb44b8160 d3d12decoder: Fix crash on flush
On flush event, baseclass will discard all pictures from DPB
but there can be still in-flight commands not finished yet.
Use our command queue, allocator and fence data helper objects
to keep resource available during command execution.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5877>
2024-01-03 01:36:42 +09:00
Seungha Yang
d3b3a1c00a d3d12: Add {set,get}_user_data() methods to command allocator
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5877>
2024-01-03 01:19:39 +09:00
Seungha Yang
9d5277e70e d3d12decoder: Fix barrier usage
Common state promotion and decay does not seem to be applied to
decoder commands. Use barriers explicitly

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5877>
2024-01-03 01:17:32 +09:00
Seungha Yang
087e39564d d3d12videosink: Fix buffer leak
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5875>
2024-01-02 13:02:47 +00:00
Seungha Yang
b0ef890726 d3d12device: Store adapter index
... and remove unused fence object

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5875>
2024-01-02 13:02:47 +00:00
Seungha Yang
0577d1c9de d3d12: Reduce shader visible descriptor size
Shader visible descriptors occupy GPU resource and there are hardware
limits. Thus, in order to minimize the amount of shader visible heaps,
only non shader visible descriptor heap (staging) will be held by d3d12memory.
Then converter will copy the staging descriptor to shader visible
descriptor heap per draw.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5875>
2024-01-02 13:02:47 +00:00
Seungha Yang
0ce6e752e4 d3d12: Use CREATE_NOT_ZEROED heap flag if possible
Zero initialization would have overhead and it's not required
most cases except for textures. Use CREATE_NOT_ZEROED flag
in case of buffer resource or if a texture will be rendered without any
prior read operation.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5875>
2024-01-02 13:02:47 +00:00
Seungha Yang
a1ba38cd64 d3d12: Add compositor element
Adding d3d12compositor element. d3d12compositor will build GPU commands
asynchronously and each command is serialized at final render stage.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5875>
2024-01-02 13:02:47 +00:00
Seungha Yang
996346f8d3 d3d12converter: Don't map output buffer with write flag
Conversion will happen when constructed command list is executed,
not by converter element. Thus this object should not map output buffer
with write flag which will result in error if multiple threads
are building commands for the same output target frame.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5875>
2024-01-02 13:02:47 +00:00
Seungha Yang
bffbf0d951 d3d12converter: Add support for texture upload
If buffer is not a d3d12 memory or allocated by other device,
upload to internal d3d12 memory

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5875>
2024-01-02 13:02:47 +00:00
Seungha Yang
f4fe17d8d2 d3d12converter: Add support for blending
Create new PSO if blend state update is required

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5875>
2024-01-02 13:02:47 +00:00
Seungha Yang
7c701058ed d3d12: Add testsrc element
Adding testsrc element with d2d interop support via d3d11on12

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5875>
2024-01-02 13:02:46 +00:00
Seungha Yang
abe1f5044d cuda: Prefer CUBIN over PTX
System installed NVRTC library might be newer version than
driver, then generate PTX can be incompatible with the driver.
Instead of the intermediate code PTX, use actual assembly code
directly.

Fixes: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/3108
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5639>
2024-01-02 10:10:09 +00:00
Seungha Yang
2f091e7118 d3d12: Add video sink element
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5870>
2023-12-29 14:39:00 +00:00
Seungha Yang
660f2d7d27 d3d12: Add convert element
Implement converter object with convert element

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5870>
2023-12-29 14:39:00 +00:00
Seungha Yang
4198bd6932 d3d12: Add helper object for fence operation
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5870>
2023-12-29 14:39:00 +00:00
Seungha Yang
eeb57061d2 d3d12: Enable plugin only for Windows8 or newer
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5870>
2023-12-29 14:39:00 +00:00
Seungha Yang
3e49d6a75f d3d12: Define more formats
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5870>
2023-12-29 14:39:00 +00:00
Seungha Yang
48cfca413d d3d12: Add header containing core features
... and include the single header instead of listing many ones

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5870>
2023-12-29 14:39:00 +00:00
Seungha Yang
41d8a12649 d3d12: Remove unused methods
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5870>
2023-12-29 14:39:00 +00:00
Seungha Yang
5ce2a7f64f d3d12bufferpool: Don't pre-allocate memory for size calculation
Unlike d3d11, we can know CPU accessible memory layout without
allocation

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5870>
2023-12-29 14:39:00 +00:00
Seungha Yang
d316356bf2 d3d12memory: Add alloc_wrapped() method
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5870>
2023-12-29 14:39:00 +00:00
Seungha Yang
3308a976bd d3d12: Add upload element
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5870>
2023-12-29 14:39:00 +00:00
Seungha Yang
e4f794cbdd d3d12bufferpool: Wait fence before reusing buffer
Buffer can be released without waiting fence for previous commands

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5870>
2023-12-29 14:39:00 +00:00
Seungha Yang
1c5bba4b6b d3d12decoder: Remove ID3D12Device4 interface requirement
Old OS may not support the interface. And allow 11_0 feature level
hardware.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5870>
2023-12-29 14:38:59 +00:00
Seungha Yang
4e5d4a45a3 d3d12decoder: Reduce the number of resource barriers
Single barrier per texture is sufficient in case of array-of-textures.
Avoid unnecessary decay barriers

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5870>
2023-12-29 14:38:59 +00:00
Seungha Yang
e6bdb0458c d3d12decoder: Use flexible task queue
Instead of using fixed size command allocator array, make it
resizable.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5870>
2023-12-29 14:38:59 +00:00
Seungha Yang
efc023e76e d3d12: Rework command scheduling
* Use single fence object per queue and remove GstD3D12Fence
  implementation
* Add a helper method for texture copy
* Run background thread and release unused resource from the thread

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5870>
2023-12-29 14:38:59 +00:00
Seungha Yang
6d7d9291c3 d3d12: Add resource pool objects
Adding pool objects for command list, command allocator, and descriptor
heap

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5870>
2023-12-29 14:38:59 +00:00
Seungha Yang
93458c0155 d3d12memory: Add more SRV/RTV getter methods
Adding a method so that memory object can create SRV/RTV
on external descriptor heap. And remove unused methods

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5870>
2023-12-29 14:38:59 +00:00
Seungha Yang
2578f234dd d3d12: Remove d3d11 dependency
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5870>
2023-12-29 14:38:59 +00:00
Víctor Manuel Jáquez Leal
fdfd51397b vajpegdec: only support progressive mjpeg streams
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5838>
2023-12-22 04:38:06 +00:00
Marek Vasut
5f3d4215a0 v4l2codecs: Switch gst_codec_picture_ts_ns() to gst_util_uint64_scale_int()
Instead of plain multiplication, use gst_util_uint64_scale_int()
to achieve the same effect with additional checks.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5791>
2023-12-20 18:47:39 +00:00