Commit graph

3472 commits

Author SHA1 Message Date
Seungha Yang
25514c8fc1 d3d12: Add v216, v210, r210, v308, IYU2, RGB, BGR format support
Reuse the compute shader implemented for d3d11

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7092>
2024-06-23 22:30:40 +09:00
Seungha Yang
c1498851b3 d3d12: Add support for UYVY, VYUY, and YVYU
Use already implemented compute shaders

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7092>
2024-06-23 22:30:40 +09:00
Seungha Yang
6fcb5f6ae7 d3d12: Add RGB{16,15} and BGR{16,15} format support
d3d12 device can support B5G6R5_UNORM and B5G5R5A1_UNORM formats
in pixel shader. If the format is not supported by device,
U16_UINT format with compute shader will be used, like d3d11converter

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7092>
2024-06-23 22:30:40 +09:00
Seungha Yang
94bb9fec58 d3d12: Add BGRA64 and BGR10A2 format support
Map BGRA64 and BGR10A2 to Y416 and Y410, respectively,
since it's possible RGB space decoder output

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7092>
2024-06-23 22:30:30 +09:00
Seungha Yang
66cc3ff84e d3d12: Add support for DXGI native packed YUV formats
Adding YUY2, Y210, Y216, and Y416 format support

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7092>
2024-06-23 22:11:32 +09:00
Seungha Yang
dd4d85272e d3d12converter: Fix Y410 conversion
Adding format conversion helper and use compute shader in case that
output format does not support RTV.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7088>
2024-06-23 10:53:18 +00:00
Seungha Yang
f432e61bdc d3d12memory: Add support for UAV descriptor cache
Cache shader invisible UAV descriptor in memory

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7088>
2024-06-23 10:53:18 +00:00
Seungha Yang
ec06b0445e d3d12: Format table refactoring
Hide format table from header. This is a preparation for compute
shader based format support

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7088>
2024-06-23 10:53:18 +00:00
Seungha Yang
5d18d8a109 d3d12converter: Upload shader buffer resources earlier
Schedule (semi-)static resource upload at converter creation time.
And use single resource for all vertex, index, and constant
buffers, since separate resources will waste GPU memory.
Note that size and address of a committed resource are 64K aligned
even if requested buffer size is small.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7081>
2024-06-21 20:16:19 +09:00
Seungha Yang
c5785fbeec d3d12converter: Make gamma remap work as intended
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7073>
2024-06-20 17:54:12 +00:00
Seungha Yang
e90880e988 d3d12device: Don't warn for out of range device index
It can happen during enumeration as well, and it's expected error

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7070>
2024-06-20 14:23:13 +00:00
Seungha Yang
2983c50a9f d3d12device: Dump device feature support
... and use CD3DX12FeatureSupport helper class in d3dx12.h

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7070>
2024-06-20 14:23:13 +00:00
Seungha Yang
bdb12db5c6 d3d12device: Prevent too many in-flight GPU commands
Even if each element is checking its own in-flight commands,
total number of commands can get larger in case of complex pipeline.
Limits total number of in-flight commands at command queue level

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7066>
2024-06-20 00:58:45 +09:00
Seungha Yang
143efa2f86 d3d12commandqueue: Detect device removed event
Early return if device removed event is detected

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7066>
2024-06-20 00:07:41 +09:00
Seungha Yang
327fb9924c d3d12basefilter: Add adapter property
Allows initial GPU adapter selection

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7066>
2024-06-20 00:03:28 +09:00
Seungha Yang
2ec7d82bd7 d3d12: Move fence setter helper method to gst-libs
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7057>
2024-06-19 13:19:48 +00:00
Seungha Yang
fa7c4a2e39 d3d12converter: Update API signature
Always use device's main direct queue, and control gpu waiting
behavior by using boolean value

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7057>
2024-06-19 13:19:48 +00:00
Seungha Yang
f6eb3a01c0 d3d12memory: Hide fence value from header
Instead of exposing fence value to wait in header, user setter/getter
methods.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7057>
2024-06-19 13:19:47 +00:00
Seungha Yang
49c4247cb3 d3d12device: Add helper method for getting fence handle
Add get_fence_handle() method so that caller can get command queue's
dedicated fence handle from device

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7057>
2024-06-19 13:19:47 +00:00
Seungha Yang
289bc1d440 d3d12: Remove notify_com and notify_mini_object helper methods
Use private macros instead of exposing multiple APIs for the same thing

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7057>
2024-06-19 13:19:47 +00:00
Seungha Yang
6942868bda d3d12commandqueue: Update API name and arguments
Accepts multiple fences since single command list may have
multiple dependent resources which are associated with
different GPU engines

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7057>
2024-06-19 13:19:47 +00:00
Seungha Yang
713ea313f8 d3d12device: Use HRESULT return code if possible
Make function signature consistent with that of command queue

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7057>
2024-06-19 13:19:47 +00:00
Stéphane Cerveau
a4c976dd20 vkdecoder: support layered and non dedicated DPB
As NVIDIA Amperium. In this case the each output buffer is also a DPB,
but using a different view layer.

Still pending a validation layer issue:

VUID-VkVideoBeginCodingInfoKHR-flags-07244

Co-authored-by: Victor Jaquez <vjaquez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6954>
2024-06-19 10:37:32 +00:00
Daniel Morin
55b12a80d6 analytics: Add validation on classification analytics-meta
- Add valiation on parameters passed to gst_analytics_cls_add_cls_mtd.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7046>
2024-06-18 19:13:10 +00:00
Edward Hervey
a4a21feb87 tsdemux: Fix maximum PCR/DTS values
* PTS/DTS are stored as 33 bit
* PCR is 33bit multiplied by 300

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7050>
2024-06-18 15:56:09 +00:00
Seungha Yang
339c81bac8 d3d12: Promote decoder and videosink rank to primary
It's proven that d3d12 performs better than d3d11 while
consumes less resources in various cases.
Assign primary+ rank to decoder and videosink in case of Windows10/11,
so that it can be tested widely

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7045>
2024-06-18 14:40:37 +00:00
Stéphane Cerveau
f40d947c4a vkoperation: support for query_result_status
query_result_status can be optional so we should not create
the query pool if the queue does not support it,
ie, AMD does not support VK_QUERY_TYPE_RESULT_STATUS_ONLY_KHR

In other use case such as VK_QUERY_TYPE_VIDEO_ENCODE_FEEDBACK_KHR, the
query pool must be created.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7043>
2024-06-18 12:13:41 +00:00
Stéphane Cerveau
71ebb20267 vkphysicaldevice: rename query to query_result_status
As only queryResultStatusSupport can be optional,
the variable name should be more specific.

queryResultStatusSupport reports VK_TRUE if query type
VK_QUERY_TYPE_RESULT_STATUS_ONLY_KHR and use of
VK_QUERY_RESULT_WITH_STATUS_BIT_KHR are supported.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7043>
2024-06-18 12:13:41 +00:00
He Junyan
7589647c13 av1parse: Do not return error when expectedFrameId mismatch
According to the SPEC:
  The frame id numbers (represented in display_frame_id, current_frame_id,
  and RefFrameId[ i ]) are not needed by the decoding process, but allow
  decoders to spot when frames have been missed and take an appropriate action.

So we should just print out warning and should not return error in parser when
mismatching. The decoder itself is already robust to handle the reference missing.

Fixes #3622

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7047>
2024-06-18 09:03:41 +00:00
Seungha Yang
e5d2dd8e8d d3d12videosink: Add direct-swapchain property
Because DXGI flip mode swapchain will disallow GDI operation
to a HWND once swapchain is configured, videosink has been creating
child window of application's window. However, since window creation
would take a few milliseconds, it can cause performance issue such as
UI freezing. Adding a property so that videosink can attach
DXGI swapchain diretly to application's window in order to improve
performance.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7013>
2024-06-17 16:05:00 +00:00
Seungha Yang
dd1d2f5ff7 d3d12videosink: Add external-window-only property
Adding a new property in order to avoid unintended interanl window
creation.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7013>
2024-06-17 16:05:00 +00:00
Seungha Yang
37e1847464 d3d12videosink: Add support for window handle update
A large refactoring commit for adding features and improve performance

* Reuse internal converter and overlay compositor:
  Converter can be reused as long as input and display formats are not
  changed. Also overlay compositor reconstruction is required only if
  display format is changed

* Don't wait for full GPU flush on resize or close:
  D3D12 swapchain requires GPU idle in order to resize backbuffer.
  Thus CPU side waiting is required for swapchain related commands
  to be finished. However, don't need to wait for full GPU flushing.

* Support multiple sink on a single external window
  Keep installed subclass window procedure even if there's no associated
  our internal HWND. This will make window procedure hooking less racy.
  Then parent HWND's message will be transferred to our internal HWNDs
  if needed.

* Adding support for window handle update
  Application can change target HWND even when videosink is playing or
  paused state. So, users can call gst_video_overlay_set_window_handle()
  against d3d12videosink anytime. The videosink will be able to update
  internal state and setup resource upon requested.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7013>
2024-06-17 16:05:00 +00:00
Seungha Yang
c00c36e33b d3d12overlaycompositor: Remove unused parameter
Don't need to check fence value of overlay buffer since
window uses global direct command queue

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7013>
2024-06-17 16:05:00 +00:00
Seungha Yang
f91be03550 d3d12videosink: Calculate display resolution only per caps change
Don't need to calculate it per window property update

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7013>
2024-06-17 16:05:00 +00:00
Seungha Yang
db47f46ad7 d3d12commandqueue: Fix deadlock on drain()
Don't take lock if the drain() is called from the GC thread

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7013>
2024-06-17 16:05:00 +00:00
Seungha Yang
a2df44da7d d3d12: Workaround for Intel iGPU decoder crash
Observed Intel GPU driver crash when multiple decoders are
configured in a process. It might be because of frequent
command queue alloc/free or too many in-flight decoding commands.
In order to make command queue persistent and limit the number of
in-flight command lists, holds global decoding command queue.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7019>
2024-06-17 15:15:07 +00:00
Piotr Brzeziński
e5b5d223b4 vtdec: Use GST_VIDEO_DECODER_ERROR instead of aborting when frame has an ERROR flag
This was already being used in handle_frame() for errors that happen when queueing a frame for decoding,
let's do the same when a frame is flagged with an error in the output callback.
From quick testing, this makes seeking more reliable (previously, it would sometimes cause a decoding error
and shut the whole decoder down due to GST_FLOW_ERROR).

Also manually sets the max error count to actually stop processing if too many errors occur.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6446>
2024-06-17 12:42:53 +00:00
Piotr Brzeziński
a5c437c643 vtdec: Handle some errors without stopping the decoder
ReferenceMissingErr is not critical and the simplest solution is to just ignore it. The frame has
the FrameDropped flag set when it occurs, so we can just drop it as usual.
BadDataErr is also not immediately critical, but in its case let's set the ERROR flag,
so the output loop can use GST_VIDEO_DECODER_ERROR to count and error out if it happens too many times.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6446>
2024-06-17 12:42:53 +00:00
Sebastian Dröge
522704d308 av1dec: Don't treat decoding errors as fatal and print more error details
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7040>
2024-06-17 08:18:55 +00:00
Seungha Yang
6449bc3171 d3d12: Add support for DXGI debug layer
Will be enabled if GST_ENABLE_D3D12_DXGI_DEBUG env is set
and dxgidebug.dll is available.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7016>
2024-06-15 13:24:24 +00:00
Zach van Rijn
4729d24bb6 pcapparse: Avoid unaligned memory access
Fixes: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/3602
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7030>
2024-06-14 10:32:54 -05:00
Seungha Yang
0ed9c39835 nvcomp: Add nvCOMP library based GPU lossless compression plugin
Adding NVIDIA nvCOMP library based plugin for lossless raw video
compression/decompression. To build this plugin, user should
install nvCOMP SDK first and specify the SDK path via
"nvcomp-sdk-path" build option or NVCOMP_SDK_PATH env.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6912>
2024-06-13 18:19:08 +00:00
Seungha Yang
cee01d7fbd cuda: Load 1D memcpy method symbols
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6912>
2024-06-13 18:19:08 +00:00
Mathieu Duponchelle
deed29732b codectimestamper: never set DTS to NONE
If we want to avoid the DTS going backward, then we can set DTS to
last_dts as a last resort.

Log a warning in this case

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6977>
2024-06-13 16:25:57 +00:00
Khem Raj
9a3c707fd4 uvcgadget: Use g_path_get_basename instead of libc basename
Musl does not implement GNU basename and have fixed a bug where the
prototype was leaked into string.h [1], which resullts in compile errors
with GCC-14 and Clang-17+

| sys/uvcgadget/configfs.c:262:21: error: call to undeclared function 'basename'
ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
|   262 |     const char *v = basename (globbuf.gl_pathv[i]);
|       |                     ^

Use glib function instead makes it portable across musl and glibc on
linux

[1] https://git.musl-libc.org/cgit/musl/commit/?id=725e17ed6dff4d0cd22487bb64470881e86a92e7a

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7006>
2024-06-12 12:20:45 -07:00
Matthew Waters
050c622b7f vulkan/swapper: expose choose_queue() in docs
It was missing a doc trigraph.

Also mark input queue argument as nullable.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7023>
2024-06-12 23:37:58 +10:00
Sebastian Dröge
53f8621e00 av1enc: Handle force-keyunit events properly by requesting keyframes
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7008>
2024-06-12 10:41:55 +00:00
Seungha Yang
00536ea232 wasapi2: Adjust log level in device enumeration path
Audio device at requested index might not be available, but that's
expected case when enumerating devices.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6996>
2024-06-11 11:11:30 +00:00
Mengkejiergeli Ba
248556d823 msdkvpp: Add a huge value to inbuf pts and set mfx surface timestamp
It can be seen as a WA in the case of multi-channel transcoding (like
decoder output to two channels, one for encoder and one for vpp).
Normally, encoder sets min pts of a huge value to avoid negative dts,
while vpp set pts without this addtional huge value, which are likely to
cause input surface pts does not fit with encoder (since both encoder
and vpp accept the same buffer from decoder, means they modify the timestamp
of one mfx surface). So we add this huge value to vpp to ensure enc and
vpp set the same value to input mfx surface meanwhile does not break
encoder's setting min pts for dts protection.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6971>
2024-06-11 05:21:42 +00:00
Seungha Yang
20852ba028 d3d12videosink: Disconnect window signal handler on dispose as intended
Fixing typo

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7012>
2024-06-10 23:33:26 +09:00