Commit graph

1505 commits

Author SHA1 Message Date
Seungha Yang
4ac46ce82b d3d12screencapturesrc: Performance improvement
Process captured frame using d3d11 instead of d3d12, and use shared
fence when copying processed d3d11 texture to d3d12 resource.
In this way, capture CPU thread does not need to wait for fence signal.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6733>
2024-04-25 22:51:01 +00:00
Seungha Yang
cecb0f2148 d3d12decoder: Lock DPB while building command
Since DPB resource can be modified in output thread, protect
it when building command list.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6709>
2024-04-23 10:08:19 +00:00
Seungha Yang
27c02a0b80 d3d12decoder: Hold reference pictures in fence data
Keep reference pictures alive during executing decoding commands

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6709>
2024-04-23 10:08:19 +00:00
Seungha Yang
0f5f170a40 d3d12vp9dec: Disallow resolution change to larger size on non-keyframe
Intel GPU seems to be crashing if the case happens.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6709>
2024-04-23 10:08:18 +00:00
Seungha Yang
700c00eda3 d3d12decoder: Fix potential use after free
A DPB buffer held by codec picture object may not be writable
at the moment, then gst_buffer_make_writable() will unref passed buffer.

Specifically, the use after free or double free can happen if:
* Crop meta of buffer copy is required because of non-zero
  top-left crop position
* zero-copy is possible with crop meta
* A picture was duplicated, interlaced h264 stream for example

Interlaced h264 stream with non-zero top-left crop position
is not very common but it's possible configuration in theory.

Thus gst_buffer_make_writable() should be called with
GstVideoCodecFrame.output_buffer directly.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6706>
2024-04-22 13:28:06 +00:00
Seungha Yang
b9e51facdd d3d11decoder: Fix potential use after free
A DPB buffer held by codec picture object may not be writable
at the moment, then gst_buffer_make_writable() will unref passed buffer.

Specifically, the use after free or double free can happen if:
* Crop meta of buffer copy is required because of non-zero
  top-left crop position
* zero-copy is possible with crop meta
* A picture was duplicated, interlaced h264 stream for example

Interlaced h264 stream with non-zero top-left crop position
is not very common but it's possible configuration in theory.

Thus gst_buffer_make_writable() should be called with
GstVideoCodecFrame.output_buffer directly.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6706>
2024-04-22 13:28:06 +00:00
Seungha Yang
b9feb47de5 mediafoundation: Fix infinite loop in device provider
Initialize source state with GST_MF_DEVICE_NOT_FOUND to terminate
loop immediately if no available capture device is available

Fixes: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/3492
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6704>
2024-04-21 19:21:54 +09:00
Seungha Yang
5179cbccfa d3d12testsrc: Use shared 11on12 device
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6697>
2024-04-20 04:16:48 +09:00
Seungha Yang
045c9ac6b9 dwrite: Use shared 11on12 device
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6697>
2024-04-20 04:16:48 +09:00
Seungha Yang
cd97275a19 d3d12device: Hold d3d11on12 device to be shared
d3d11on12 device seems to be occupying a bit of GPU memory
Hold the instance in GstD3D12Device so that it can be shared

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6697>
2024-04-20 04:16:41 +09:00
Seungha Yang
8e4fe98361 d3d12videosink: Handle mouse double click and modifier
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6693>
2024-04-19 16:11:30 +00:00
Seungha Yang
dabce9635d d3d12videosink: Disconnect window's signal on dispose
Same as the commit of 7b69d1758f
but for d3d12videosink.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6692>
2024-04-19 21:25:06 +09:00
Seungha Yang
6659bd6362 d3d12videosink: Handle external HWND's mouse/keyboard events
OS will not propagate the event to child HWND if it's handled by
the parent. Thus, navigation event should be handled by parent HWND's
event handler.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6692>
2024-04-19 21:24:52 +09:00
Sebastian Dröge
d42ee6e48c vavp9enc: Preserve PTS and other frame metadata
See also https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4150

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6680>
2024-04-19 07:18:35 +00:00
Seungha Yang
ba443e7a64 webview2: Handle double click and modifier
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6491>
2024-04-18 20:19:38 +00:00
Seungha Yang
e0def6b355 d3d11videosink: Handle double click and modifier
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6491>
2024-04-18 20:19:38 +00:00
Nicolas Dufresne
e34f765a7e v4l2codecs: Don't unref allocation query caps
The caps obtained from parsing the allocation query is borrowed and
should not be unreffed. This fixes criticals assertion introduced in
1.24.1.

(gst-launch-1.0:242): GStreamer-CRITICAL **: 19:48:02.667:
    gst_mini_object_unref: assertion 'GST_MINI_OBJECT_REFCOUNT_VALUE (mini_object) > 0' failed

Fixes: 5189e8b956 ("v4l2codecs: decoders: Add DMA_DRM caps support")
Closes #3462

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6679>
2024-04-18 06:13:17 +00:00
Mengkejiergeli Ba
49dee23e65 msdk: Add main-422-12 profile to hevc
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6591>
2024-04-17 19:31:09 +00:00
Seungha Yang
a7fe79c4de mediafoundation: Fix device enumeration
Do not stop device enumerate even if a device could not be opened.
Otherwise the other devices listed after the failed device will not be
reported by device provider

Fixes: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/3460
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6598>
2024-04-15 14:52:32 +00:00
Seungha Yang
15434ce51d dwrite: D3D12 integration
Adding d3d12 backend text renderer/blender by using d3d11on12 interop.
And subclassing renderer object per backend (i.e., d3d11, d3d12, and bitmap)

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6620>
2024-04-14 09:55:41 +00:00
Seungha Yang
f6ec4e6256 d3d12overlaycompositor: Add support for d3d12 memory
Don't allocate d3d12 texture if overlay is d3d12 memory already

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6620>
2024-04-14 09:55:41 +00:00
Seungha Yang
13d892d336 d3d12: Move gst_d3d12_buffer_copy_into method to library
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6620>
2024-04-14 09:55:41 +00:00
Seungha Yang
e597dcdf0d d3d12encoder: Fix buffer pool leak
Add missing buffer pool release

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6628>
2024-04-13 14:55:09 +00:00
Seungha Yang
d002acdd0c d3d11videosink: Fix rendering on keyed mutex enabled handle
As of the commit 69b2e1565c,
keyed mutex will be handled by the memory object.

Fixes: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/3468
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6600>
2024-04-12 14:46:20 +00:00
Seungha Yang
0cc206e939 qsvdecoder: Release too old frames
Release too old frames manually.

Fixes: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/3163
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6583>
2024-04-09 16:49:47 +00:00
Mathieu Duponchelle
4b93581c15 ajasrc: always post details about detected format
.. instead of only when there is a mismatch.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6438>
2024-04-04 18:23:31 +00:00
Víctor Manuel Jáquez Leal
4b5e930bc8 va: encoders: don't assert at target percentage when QVBR
Instead of asserting, just get the max value between the current value and 10,
which is the minimum required by QVBR.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6465>
2024-04-03 13:51:31 +02:00
Víctor Manuel Jáquez Leal
3b4b979ec4 msdk: sink context reference
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6398>
2024-04-03 08:48:25 +00:00
Víctor Manuel Jáquez Leal
0d48ede4a9 va: sink reference at instantiation
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6398>
2024-04-03 08:48:25 +00:00
Seungha Yang
b9ebe979ee qsvencoder: Handle d3d12 context
GstD3D12Device objetct's internal resources are singletons per adapter
already though, the object itself is not a singleton.
Due to the singleton design (unlike other APIs such as d3d11),
d3d12 device context sharing is not a strict requirement
for zero-copy, but handles context ones to make things less noisy.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6513>
2024-04-03 02:05:25 +00:00
Seungha Yang
ad6670ba81 d3d12decoder: Always output sharable texture
Because shared heap's additional costs is not significant,
use D3D12_HEAP_FLAG_SHARED for resource can be shared over process
boundary. And enables render target for d3d11 interop in the process.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6513>
2024-04-03 02:05:25 +00:00
He Junyan
cbdc22d697 va: av1enc: Change the set_property to make it atomic
The inside encoder may be set in other threads, so we should make
its accessing atomic.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6471>
2024-04-02 19:01:22 +00:00
He Junyan
db74da3f6f va: vp9enc: Change the set_property to make it atomic
The inside encoder may be set in other threads, so we should make
its accessing atomic.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6471>
2024-04-02 19:01:22 +00:00
Víctor Manuel Jáquez Leal
8bede7937c va{vp9,av1}enc: reconfigure when properties change
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6471>
2024-04-02 19:01:22 +00:00
He Junyan
d179fca13f va: h265enc: Change the set_property to make it atomic
The inside encoder may be set in other threads, so we should make
its accessing atomic.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6471>
2024-04-02 19:01:22 +00:00
He Junyan
33bc8d75ad va: h265enc: set the reconf flag when cpb_size updated
This feature can be changed dynamically in playing state, so we
need to set reconf flag to trigger reconfig.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6471>
2024-04-02 19:01:22 +00:00
He Junyan
045449e190 va: h264enc: Change the set_property to make it atomic
The inside encoder may be set in other threads, so we should make
its accessing atomic.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6471>
2024-04-02 19:01:22 +00:00
He Junyan
812e7efb90 va: h264enc: set the reconf flag when cpb_size updated
This feature can be changed dynamically in playing state, so we
need to set reconf flag to trigger reconfig.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6471>
2024-04-02 19:01:22 +00:00
Seungha Yang
4ede9fef07 dwrite: Fix crash on device update
Selected blend mode should not be cleared on device update

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6508>
2024-04-02 17:36:28 +00:00
Seungha Yang
106187cc59 nvencoder: Add support for RGB formats
Adding RGBA, RGBx, BGRA, BGRx, VUYA and RGB10A2_LE format support for performance.
However, these formats are not still recommended if upstream can support
native YUV formats (e.g., NV12, P010) since NVENC does not expose
conversion related optiones. Note that VUYA format is 4:4:4 YUV format
already but NVENC runtime will convert it to 4:2:0 format internally

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6417>
2024-04-02 13:07:29 +00:00
Seungha Yang
e6f496a240 cuda: Add support for VUYA format
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6417>
2024-04-02 13:07:29 +00:00
Seungha Yang
707ac69f5f qsv: Add support for d3d12 interop in encoder
Since QSV API does not support D3D12, try to import d3d12 resource
into d3d11 texture. Note that resource sharing requires
D3D12_SHARED_RESOURCE_COMPATIBILITY_TIER_2 for NV12 texure sharing.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6501>
2024-04-02 11:54:39 +00:00
He Junyan
78f1299ce4 va: av1enc: Avoid reopen encoder or renegotiate
If parameters remain similar enough to avoid either encoder reopening
or downstream renegotiation, avoid it.

This is going to be useful for dynamic parameters setting.

To check if the stream parameters changed, so the internal encoder has
to be closed and opened again, are required two steps:

1. If input caps, format, profile, chroma or rate control mode have changed.
2. If any of the calculated variables and element properties have changed.

Later on, only if the output caps also changed, the pipeline
is renegotiated.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6441>
2024-04-02 09:06:07 +00:00
He Junyan
075937dc19 va: vp9enc: Avoid reopen encoder or renegotiate
If parameters remain similar enough to avoid either encoder reopening
or downstream renegotiation, avoid it.

This is going to be useful for dynamic parameters setting.

To check if the stream parameters changed, so the internal encoder has
to be closed and opened again, are required two steps:

1. If input caps, format, profile, chroma or rate control mode have changed.
2. If any of the calculated variables and element properties have changed.

Later on, only if the output caps also changed, the pipeline
is renegotiated.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6441>
2024-04-02 09:06:07 +00:00
He Junyan
40629b84ee va: av1enc: Improve the LAST reference assignment
The last frame which has the smallest diff should be consider as
the first choice rather than the golden frame. Especially when only
one reference available, this way can improve the BD rate about 5
percentage.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6379>
2024-04-02 03:54:52 +00:00
He Junyan
1745a82516 va: av1enc: Fix the reference number setting bug
The current way will let the total reference number surplus the
reference number set by the "ref-frames" property.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6379>
2024-04-02 03:54:52 +00:00
Seungha Yang
bb268eafa1 webview2: Add support for d3d12 interop
Enable shared copy to D3D12 resource

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6499>
2024-04-02 03:43:59 +09:00
Seungha Yang
d00c26cdc8 d3d12screencapturesrc: Use gst_d3d12_memory_get_d3d11_texture()
... and use fence to wait for GPU sync

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6499>
2024-04-02 03:43:59 +09:00
He Junyan
e023546162 va: encoder: Fix the unit of bitrate in debug log message
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6483>
2024-04-01 15:39:13 +00:00
He Junyan
9c3bd3950e va: vp9enc: Adjust the coded buffer size to avoid failure
Some extreme case such as "videotestsrc pattern=1" can generate pure
white noise videoes, for which encoder may generate too big output
for current coded buffer size. We now consider the qindex and bitrate
to avoid that.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6483>
2024-04-01 15:39:13 +00:00