Commit graph

120212 commits

Author SHA1 Message Date
Seungha Yang
c1498851b3 d3d12: Add support for UYVY, VYUY, and YVYU
Use already implemented compute shaders

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7092>
2024-06-23 22:30:40 +09:00
Seungha Yang
6fcb5f6ae7 d3d12: Add RGB{16,15} and BGR{16,15} format support
d3d12 device can support B5G6R5_UNORM and B5G5R5A1_UNORM formats
in pixel shader. If the format is not supported by device,
U16_UINT format with compute shader will be used, like d3d11converter

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7092>
2024-06-23 22:30:40 +09:00
Seungha Yang
94bb9fec58 d3d12: Add BGRA64 and BGR10A2 format support
Map BGRA64 and BGR10A2 to Y416 and Y410, respectively,
since it's possible RGB space decoder output

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7092>
2024-06-23 22:30:30 +09:00
Seungha Yang
66cc3ff84e d3d12: Add support for DXGI native packed YUV formats
Adding YUY2, Y210, Y216, and Y416 format support

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7092>
2024-06-23 22:11:32 +09:00
Seungha Yang
dd4d85272e d3d12converter: Fix Y410 conversion
Adding format conversion helper and use compute shader in case that
output format does not support RTV.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7088>
2024-06-23 10:53:18 +00:00
Seungha Yang
f432e61bdc d3d12memory: Add support for UAV descriptor cache
Cache shader invisible UAV descriptor in memory

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7088>
2024-06-23 10:53:18 +00:00
Seungha Yang
ec06b0445e d3d12: Format table refactoring
Hide format table from header. This is a preparation for compute
shader based format support

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7088>
2024-06-23 10:53:18 +00:00
George Hazan
9e1abc0797 parsebin: fix compilation problem with Visual C++ 2017
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7089>
2024-06-22 20:56:03 +03:00
Tim-Philipp Müller
03a60c9950 subprojects: bump opus wrap to 1.5.2
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7085>
2024-06-22 12:26:22 +01:00
Seungha Yang
5d18d8a109 d3d12converter: Upload shader buffer resources earlier
Schedule (semi-)static resource upload at converter creation time.
And use single resource for all vertex, index, and constant
buffers, since separate resources will waste GPU memory.
Note that size and address of a committed resource are 64K aligned
even if requested buffer size is small.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7081>
2024-06-21 20:16:19 +09:00
Edward Hervey
a20569b1ba basesink: Remove field not used
It was never actually used since it was introduced as part of the commit
introducing support for instant rate

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7077>
2024-06-21 09:29:48 +02:00
Seungha Yang
c5785fbeec d3d12converter: Make gamma remap work as intended
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7073>
2024-06-20 17:54:12 +00:00
Sebastian Dröge
8117ee4fa3 queue, queue2, multiqueue: Timestamps of gap events must be valid
This is checked in gst_event_new_gap() so doesn't have to be checked again here,
but simply can be asserted with a g_return_if_fail().

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7071>
2024-06-20 20:02:43 +03:00
Sebastian Dröge
909cd6d865 queue: queue2: multiqueue: Don't work with segment.position if buffers have no timestamps
If the first buffers have no timestamp then the sink position would be
initialized to 0. The source pad might output this buffer, which would then
initialize the source position to 0 too.

Afterwards two buffers with a valid but huge timestamp might arrive before any
of them are output on the source pad. The first one would set the sink position
to a huge value, the second one would notice that the difference between the
huge value and 0 is certainly larger than max-size-time and consider the queue
as full.

Instead, simply don't update the times from buffers without timestamps and
assume whatever was set before is still valid, i.e. the buffer has the same
timestamp as the previous one.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7071>
2024-06-20 20:02:43 +03:00
Seungha Yang
e90880e988 d3d12device: Don't warn for out of range device index
It can happen during enumeration as well, and it's expected error

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7070>
2024-06-20 14:23:13 +00:00
Seungha Yang
2983c50a9f d3d12device: Dump device feature support
... and use CD3DX12FeatureSupport helper class in d3dx12.h

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7070>
2024-06-20 14:23:13 +00:00
Edward Hervey
11ff2b2835 decodebin3: Fix keyframe drop probe handling
We were storing the probe id in a different structure (DecodebinOutputStream)
than the pad it is targetting (which is in MultiQueueSlot).

The problem is that when re-targetting outputs (to a different slot)... we would
end up having an invalid probe id, or not have a reference to an existing one.

Instead, store the probe id in the same structure as the pad it's targetting

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7069>
2024-06-20 12:56:46 +00:00
Edward Hervey
6aa0c8442d decodebin3: Fix detection of selection done
We should not assert if there are still some old streams that are waiting to be
deactivated.

Instead wait for them to be gone before posting the selection done message

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7069>
2024-06-20 12:56:46 +00:00
Seungha Yang
bdb12db5c6 d3d12device: Prevent too many in-flight GPU commands
Even if each element is checking its own in-flight commands,
total number of commands can get larger in case of complex pipeline.
Limits total number of in-flight commands at command queue level

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7066>
2024-06-20 00:58:45 +09:00
Seungha Yang
143efa2f86 d3d12commandqueue: Detect device removed event
Early return if device removed event is detected

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7066>
2024-06-20 00:07:41 +09:00
Seungha Yang
327fb9924c d3d12basefilter: Add adapter property
Allows initial GPU adapter selection

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7066>
2024-06-20 00:03:28 +09:00
Seungha Yang
2ec7d82bd7 d3d12: Move fence setter helper method to gst-libs
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7057>
2024-06-19 13:19:48 +00:00
Seungha Yang
fa7c4a2e39 d3d12converter: Update API signature
Always use device's main direct queue, and control gpu waiting
behavior by using boolean value

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7057>
2024-06-19 13:19:48 +00:00
Seungha Yang
f6eb3a01c0 d3d12memory: Hide fence value from header
Instead of exposing fence value to wait in header, user setter/getter
methods.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7057>
2024-06-19 13:19:47 +00:00
Seungha Yang
49c4247cb3 d3d12device: Add helper method for getting fence handle
Add get_fence_handle() method so that caller can get command queue's
dedicated fence handle from device

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7057>
2024-06-19 13:19:47 +00:00
Seungha Yang
289bc1d440 d3d12: Remove notify_com and notify_mini_object helper methods
Use private macros instead of exposing multiple APIs for the same thing

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7057>
2024-06-19 13:19:47 +00:00
Seungha Yang
6942868bda d3d12commandqueue: Update API name and arguments
Accepts multiple fences since single command list may have
multiple dependent resources which are associated with
different GPU engines

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7057>
2024-06-19 13:19:47 +00:00
Seungha Yang
713ea313f8 d3d12device: Use HRESULT return code if possible
Make function signature consistent with that of command queue

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7057>
2024-06-19 13:19:47 +00:00
Stéphane Cerveau
a4c976dd20 vkdecoder: support layered and non dedicated DPB
As NVIDIA Amperium. In this case the each output buffer is also a DPB,
but using a different view layer.

Still pending a validation layer issue:

VUID-VkVideoBeginCodingInfoKHR-flags-07244

Co-authored-by: Victor Jaquez <vjaquez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6954>
2024-06-19 10:37:32 +00:00
Tim-Philipp Müller
8d845d4a02 rtpdtmfsrc: minor logging clean-up
Only serialise event structure for debug logging purposes
if logging is actually enabled.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7060>
2024-06-19 07:32:49 +00:00
Tim-Philipp Müller
62047a9f8d rtpdtmfsrc: fix leak when shutting down mid-event
.. and update rtpdtmfdepay unit test to trigger
the potential leak more reliably (without the fix).

Fixes https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/3633

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7060>
2024-06-19 07:32:49 +00:00
Thibault Saunier
c8dfb7c8f0 validate: tool: Use gst_macos_main
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7056>
2024-06-18 22:31:40 +00:00
Tim-Philipp Müller
e35c08c067 testsuites: remove prores reverse playback test from known issues list
Looks like it may have been fixed since (either in ffmpeg and/or gstreamer).

See https://gitlab.freedesktop.org/gstreamer/gst-libav/-/issues/45

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7042>
2024-06-18 20:27:15 +00:00
Daniel Morin
55b12a80d6 analytics: Add validation on classification analytics-meta
- Add valiation on parameters passed to gst_analytics_cls_add_cls_mtd.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7046>
2024-06-18 19:13:10 +00:00
Sebastian Dröge
9da1268656 video-info: Don't crash in gst_video_info_is_equal() if one videoinfo is zero-initialized
Instead handle it like gst_audio_info_is_equal() and consider both different.
And also add a shortcut for the pointers to both infos being equal.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7055>
2024-06-18 17:47:14 +00:00
Edward Hervey
a4a21feb87 tsdemux: Fix maximum PCR/DTS values
* PTS/DTS are stored as 33 bit
* PCR is 33bit multiplied by 300

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7050>
2024-06-18 15:56:09 +00:00
Seungha Yang
339c81bac8 d3d12: Promote decoder and videosink rank to primary
It's proven that d3d12 performs better than d3d11 while
consumes less resources in various cases.
Assign primary+ rank to decoder and videosink in case of Windows10/11,
so that it can be tested widely

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7045>
2024-06-18 14:40:37 +00:00
Stéphane Cerveau
f40d947c4a vkoperation: support for query_result_status
query_result_status can be optional so we should not create
the query pool if the queue does not support it,
ie, AMD does not support VK_QUERY_TYPE_RESULT_STATUS_ONLY_KHR

In other use case such as VK_QUERY_TYPE_VIDEO_ENCODE_FEEDBACK_KHR, the
query pool must be created.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7043>
2024-06-18 12:13:41 +00:00
Stéphane Cerveau
71ebb20267 vkphysicaldevice: rename query to query_result_status
As only queryResultStatusSupport can be optional,
the variable name should be more specific.

queryResultStatusSupport reports VK_TRUE if query type
VK_QUERY_TYPE_RESULT_STATUS_ONLY_KHR and use of
VK_QUERY_RESULT_WITH_STATUS_BIT_KHR are supported.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7043>
2024-06-18 12:13:41 +00:00
He Junyan
7589647c13 av1parse: Do not return error when expectedFrameId mismatch
According to the SPEC:
  The frame id numbers (represented in display_frame_id, current_frame_id,
  and RefFrameId[ i ]) are not needed by the decoding process, but allow
  decoders to spot when frames have been missed and take an appropriate action.

So we should just print out warning and should not return error in parser when
mismatching. The decoder itself is already robust to handle the reference missing.

Fixes #3622

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7047>
2024-06-18 09:03:41 +00:00
Tim-Philipp Müller
7d05af9680 rtpdtmfdepay: add unit test for caps fixation issue with downstream audioconvert
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7036>
2024-06-18 00:11:28 +01:00
Tim-Philipp Müller
ab61233f30 rtpdtmfdepay: fix caps negotiation with audioconvert
Specify "layout" field in src template to make sure it's
set and gets fixated properly if the downstream element
supports both interleaved and non-interleaved caps.

Fixes

  gst_pad_set_caps: assertion 'caps != NULL && gst_caps_is_fixed (caps)' failed

critical with e.g.

  gst-launch-1.0 rtpdtmfsrc ! rtpdtmfdepay ! audioconvert ! fakesink

Not that the layout really matters in our case since we always
output mono anyway, but non-interleaved requires adding AudioMeta,
so this is the easiest fix.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7036>
2024-06-18 00:11:28 +01:00
Seungha Yang
e5d2dd8e8d d3d12videosink: Add direct-swapchain property
Because DXGI flip mode swapchain will disallow GDI operation
to a HWND once swapchain is configured, videosink has been creating
child window of application's window. However, since window creation
would take a few milliseconds, it can cause performance issue such as
UI freezing. Adding a property so that videosink can attach
DXGI swapchain diretly to application's window in order to improve
performance.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7013>
2024-06-17 16:05:00 +00:00
Seungha Yang
dd1d2f5ff7 d3d12videosink: Add external-window-only property
Adding a new property in order to avoid unintended interanl window
creation.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7013>
2024-06-17 16:05:00 +00:00
Seungha Yang
37e1847464 d3d12videosink: Add support for window handle update
A large refactoring commit for adding features and improve performance

* Reuse internal converter and overlay compositor:
  Converter can be reused as long as input and display formats are not
  changed. Also overlay compositor reconstruction is required only if
  display format is changed

* Don't wait for full GPU flush on resize or close:
  D3D12 swapchain requires GPU idle in order to resize backbuffer.
  Thus CPU side waiting is required for swapchain related commands
  to be finished. However, don't need to wait for full GPU flushing.

* Support multiple sink on a single external window
  Keep installed subclass window procedure even if there's no associated
  our internal HWND. This will make window procedure hooking less racy.
  Then parent HWND's message will be transferred to our internal HWNDs
  if needed.

* Adding support for window handle update
  Application can change target HWND even when videosink is playing or
  paused state. So, users can call gst_video_overlay_set_window_handle()
  against d3d12videosink anytime. The videosink will be able to update
  internal state and setup resource upon requested.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7013>
2024-06-17 16:05:00 +00:00
Seungha Yang
c00c36e33b d3d12overlaycompositor: Remove unused parameter
Don't need to check fence value of overlay buffer since
window uses global direct command queue

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7013>
2024-06-17 16:05:00 +00:00
Seungha Yang
f91be03550 d3d12videosink: Calculate display resolution only per caps change
Don't need to calculate it per window property update

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7013>
2024-06-17 16:05:00 +00:00
Seungha Yang
db47f46ad7 d3d12commandqueue: Fix deadlock on drain()
Don't take lock if the drain() is called from the GC thread

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7013>
2024-06-17 16:05:00 +00:00
Seungha Yang
a2df44da7d d3d12: Workaround for Intel iGPU decoder crash
Observed Intel GPU driver crash when multiple decoders are
configured in a process. It might be because of frequent
command queue alloc/free or too many in-flight decoding commands.
In order to make command queue persistent and limit the number of
in-flight command lists, holds global decoding command queue.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7019>
2024-06-17 15:15:07 +00:00
Piotr Brzeziński
e5b5d223b4 vtdec: Use GST_VIDEO_DECODER_ERROR instead of aborting when frame has an ERROR flag
This was already being used in handle_frame() for errors that happen when queueing a frame for decoding,
let's do the same when a frame is flagged with an error in the output callback.
From quick testing, this makes seeking more reliable (previously, it would sometimes cause a decoding error
and shut the whole decoder down due to GST_FLOW_ERROR).

Also manually sets the max error count to actually stop processing if too many errors occur.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6446>
2024-06-17 12:42:53 +00:00