Commit graph

189 commits

Author SHA1 Message Date
Seungha Yang
d33f5a1de9 d3d12videosink: Add support for mouse scroll events
Handle WM_MOUSEHWHEEL and WM_MOUSEWHEEL events

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7774>
2024-10-31 15:57:43 +00:00
Seungha Yang
8c3d516a14 d3d12: Additional fixes for MinGW build
Various fixes for GCC build, including actual bug fixes

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7722>
2024-10-24 06:54:30 +09:00
Seungha Yang
703cc1783e d3d12: Fix MinGW build with installed DirectX-Headers
Required for cerbero MinGW build

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7154>
2024-10-22 21:25:11 +00:00
Seungha Yang
fb6a5c8a0c d3d12mipmapping: Fix debug category
Fixing copying-and-pasting mistake

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7701>
2024-10-20 16:01:10 +00:00
L. E. Segovia
4af2121189 meson: d3d12: fix build with MinGW GCC 14
Also apply the d3d11 fix since both use the same header.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7658>
2024-10-12 19:01:46 -03:00
Seungha Yang
7c06001705 d3d12: Shorten various names
Update names of various objects and method to be shorter, for instance
GstD3D12CommandAllocator is changed to GstD3D12CmdAlloc.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7642>
2024-10-10 10:35:36 -04:00
Sebastian Dröge
b7b24573ce common: Use more efficient versions of GstCapsFeatures API where possible
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7432>
2024-09-26 19:26:18 +03:00
Seungha Yang
2d91521dfc d3d12: Fix resource allocation on old Windows version
D3D12_HEAP_FLAG_CREATE_NOT_ZEROED flag was introduced as of
Windows 10 May 2020 Update, and older versions don't understand
the heap flag. Checks the feature support and enables the
D3D12_HEAP_FLAG_CREATE_NOT_ZEROED only if it's supported by OS

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7573>
2024-09-26 12:48:55 +00:00
Seungha Yang
86af5a5f20 d3d12: Add colorconvert and scale elements
In addition to existing d3d12convert element which supports
color conversion and rescale at once, adding
separate color-conversion-only and scale-only elements

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7560>
2024-09-24 20:58:47 +09:00
Seungha Yang
cef201734c d3d12: Add d3d12mipmapping element
Adding a new element for texture conversion from single mip level
texture to mipmapping enabled RGBA texture

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7555>
2024-09-23 13:52:37 +00:00
Seungha Yang
51e1834e81 d3d12: Use D3D12_FILTER_MIN_MAG_MIP_LINEAR filter by default
... instead of D3D12_FILTER_MIN_MAG_LINEAR_MIP_POINT, since we supports
mipmap texture now.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7555>
2024-09-23 13:52:37 +00:00
Seungha Yang
71d26ee4f8 d3d12swapchainsink: Add auto-resize mode
Automatically resize swapchain backbuffer to be identical to
stream resolution if user calls resize() signal with zero resolution

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7550>
2024-09-20 19:19:15 +00:00
Seungha Yang
9dee102867 d3d12swapchainsink: Add support for MSAA
Adding "msaa" property and enable MSAA if supported by device

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7550>
2024-09-20 19:19:15 +00:00
Seungha Yang
ce5321be68 d3d12videosink: Use converter config for initial MSAA setup
Avoid redundant pso creation

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7550>
2024-09-20 19:19:15 +00:00
Seungha Yang
965d898deb d3d12swapchainsink: Add sampling-method property
Allow setting sampler filter method to use

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7550>
2024-09-20 19:19:15 +00:00
Seungha Yang
62caa76155 d3d12convert: Use new sampler filter update method
... instead of creating new converter

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7550>
2024-09-20 19:19:15 +00:00
Seungha Yang
5182cac9be d3d12decoder: Disable sub-allocated bitstream buffer
This sub-allocation causes decoding artifacts for some reason
on Intel platform

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7535>
2024-09-18 10:23:26 +00:00
Seungha Yang
d0505fba55 d3d12: Add async-depth property
Adding a property to control the number of in-flight GPU commands
(default is unlimited). Note that actual maximum number is defined
in d3d12device's direct command queue object which is 32 now,
thus total number of scheduled GPU commands cannot exceed 32.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7444>
2024-09-04 12:56:43 +00:00
Seungha Yang
4bb3854772 d3d12: Add d3d12swapchainsink element
Adding a new videosink element for Windows composition API based
applications. Unlike d3d12videosink, this element will create only
DXGI swapchain by using IDXGIFactory2::CreateSwapChainForComposition()
without actual window handle, so that video scene can be composed
via Windows native composition API, such as DirectComposition.
Note that this videosink does not support GstVideoOverlay interface
because of the design.

The swapchain created by this element can be used with
* DirectComposition's IDCompositionVisual in Win32 app
* WinRT and WinUI3's UI.Composition in Win32/UWP app
* UWP and WinUI3 XAML's SwapChainPanel

See also examples in this commit which show usage of the videosink

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7287>
2024-08-19 11:07:17 +09:00
Seungha Yang
c95873bbd5 d3d12screencapturesrc: Always release acquired frame
AcquireNextFrame() call should be paired with ReleaseFrame().

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7293>
2024-08-02 15:36:14 +00:00
Seungha Yang
9b6a3170ae d3d12screencapturesrc: Do not recreate d3d11 device on capture error
Already opened d3d11 device including shader pipeline can be reused

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7293>
2024-08-02 15:36:14 +00:00
Seungha Yang
f5cd00fbd2 d3d12screencapturesrc: Fix deadlock on error
Don't try to wait for non-signalled fence

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7293>
2024-08-02 15:36:14 +00:00
Seungha Yang
88efaa35a8 d3d12download: Do not overwrite fence of non-writable memory
A fence configured in GstD3D12Memory should be used only for
write access to be completed. And because d3d12 -> d3d11 copy path
is read access to d3d12 resource, we should not set fence to
memory. Otherwise another read access to the d3d12 resource
will wait for d3d11 device context's copy operation although
simultaneous read access is allowed.

Use background thread to keep d3d12 resource and wait for d3d11 device's
copy operation instead.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7243>
2024-07-27 03:23:22 +09:00
Seungha Yang
800961cf28 d3d12decoder: Add support for d3d11 output again
Although d3d12download supports d3d12 to d3d11 texture copy,
this feature might be useful if an application is not ready to d3d12
support and it expects output type of decodebin(3) is d3d11.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7208>
2024-07-22 23:17:25 +09:00
Seungha Yang
8c2bbd8760 meson: d3d12: Use configuration file
Move defines to config header

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7208>
2024-07-22 23:02:48 +09:00
Seungha Yang
72895ed0fa d3d12: Always allocate output texture using shared heap
... if downstream preference is unknown (e.g., no proposed
buffer pool by downstream), so that produced textures can be
shareable with other APIs such as d3d11 or vulkan, or other processes

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7193>
2024-07-18 14:17:43 +00:00
Seungha Yang
b37bfc02f5 d3d12: Remove unnecessary event handles
null event NT handle to ID3D12Fence::SetEventOnCompletion()
will block the calling CPU thread already, thus it has no point that
creating an event NT handle in order to immediate wait for fence at CPU-side.
Note that passing a valid event NT handle to the fence API might be useful
when we need to wait for the fence value later (or timeout is required),
or want to wait for multiple fences at once via WaitForMultipleObjects().
But it's not a considered use case for now.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7176>
2024-07-16 19:17:15 +00:00
Seungha Yang
783e1ea327 d3d12videosink: Fix mouse event handling
GstD3D12Window.priv.input_info is referenced by mouse event handler
in order to calculate corresponding original position
if scene is rotated/flipped by the videosink.
Fixing regression introduced by recent d3d12videosink refactoring

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7177>
2024-07-16 17:02:24 +00:00
Seungha Yang
dbc8a1397d d3d12compositor: Fix transparent background mode with YUV output
In case of YUV format without alpha channel, zero clear value
for each channle will result in green color. Use calculated black
background color with alpha=0 for transparent background mode instead.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7181>
2024-07-16 14:18:12 +00:00
Seungha Yang
2a6967a8cd d3d12videosink: Clear cached buffer on format change
Otherwise converter will try to read memory of which layout/format
might be different from configured converter pipeline

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7167>
2024-07-15 10:20:46 +00:00
Seungha Yang
e692bf0fb9 d3d12memorycopy: Enhance d3d12 to d3d11 copy
If a d3d12 memory holds non-direct-queue fence but the fence was
created with D3D12_FENCE_FLAG_SHARED flag, use the fence instead of
waiting for fence at CPU side. Note that d3d12ipcsrc or
d3d12screencapture elements will hold such sharable fence.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7139>
2024-07-08 09:57:04 +00:00
Seungha Yang
c0a02affa4 d3d12: Add support for resource copy between d3d11 and d3d12
If driver can support cross-api resource sharing, use device-to-device
resource copy in d3d12upload/download elements.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7119>
2024-07-01 20:18:42 +00:00
Seungha Yang
c013e88295 d3d12decoder: Use sub-allocated bitstream buffer
Since a buffer resource will occupy at least 64KB,
allocating upload resource per decoding command might not be
an optimal approach. Instead, use sub-region of a upload resource
for multiple decoding command if sub-regions are not overlapped
each other.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7108>
2024-06-29 20:47:39 +09:00
Seungha Yang
69ea20b739 d3d12videosink: Present on GstVideoOverlay::expose()
... so that updated backbuffer can be swapped and presented

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7079>
2024-06-24 08:46:24 +00:00
Seungha Yang
dd4d85272e d3d12converter: Fix Y410 conversion
Adding format conversion helper and use compute shader in case that
output format does not support RTV.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7088>
2024-06-23 10:53:18 +00:00
Seungha Yang
327fb9924c d3d12basefilter: Add adapter property
Allows initial GPU adapter selection

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7066>
2024-06-20 00:03:28 +09:00
Seungha Yang
2ec7d82bd7 d3d12: Move fence setter helper method to gst-libs
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7057>
2024-06-19 13:19:48 +00:00
Seungha Yang
fa7c4a2e39 d3d12converter: Update API signature
Always use device's main direct queue, and control gpu waiting
behavior by using boolean value

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7057>
2024-06-19 13:19:48 +00:00
Seungha Yang
f6eb3a01c0 d3d12memory: Hide fence value from header
Instead of exposing fence value to wait in header, user setter/getter
methods.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7057>
2024-06-19 13:19:47 +00:00
Seungha Yang
49c4247cb3 d3d12device: Add helper method for getting fence handle
Add get_fence_handle() method so that caller can get command queue's
dedicated fence handle from device

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7057>
2024-06-19 13:19:47 +00:00
Seungha Yang
289bc1d440 d3d12: Remove notify_com and notify_mini_object helper methods
Use private macros instead of exposing multiple APIs for the same thing

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7057>
2024-06-19 13:19:47 +00:00
Seungha Yang
6942868bda d3d12commandqueue: Update API name and arguments
Accepts multiple fences since single command list may have
multiple dependent resources which are associated with
different GPU engines

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7057>
2024-06-19 13:19:47 +00:00
Seungha Yang
713ea313f8 d3d12device: Use HRESULT return code if possible
Make function signature consistent with that of command queue

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7057>
2024-06-19 13:19:47 +00:00
Seungha Yang
339c81bac8 d3d12: Promote decoder and videosink rank to primary
It's proven that d3d12 performs better than d3d11 while
consumes less resources in various cases.
Assign primary+ rank to decoder and videosink in case of Windows10/11,
so that it can be tested widely

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7045>
2024-06-18 14:40:37 +00:00
Seungha Yang
e5d2dd8e8d d3d12videosink: Add direct-swapchain property
Because DXGI flip mode swapchain will disallow GDI operation
to a HWND once swapchain is configured, videosink has been creating
child window of application's window. However, since window creation
would take a few milliseconds, it can cause performance issue such as
UI freezing. Adding a property so that videosink can attach
DXGI swapchain diretly to application's window in order to improve
performance.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7013>
2024-06-17 16:05:00 +00:00
Seungha Yang
dd1d2f5ff7 d3d12videosink: Add external-window-only property
Adding a new property in order to avoid unintended interanl window
creation.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7013>
2024-06-17 16:05:00 +00:00
Seungha Yang
37e1847464 d3d12videosink: Add support for window handle update
A large refactoring commit for adding features and improve performance

* Reuse internal converter and overlay compositor:
  Converter can be reused as long as input and display formats are not
  changed. Also overlay compositor reconstruction is required only if
  display format is changed

* Don't wait for full GPU flush on resize or close:
  D3D12 swapchain requires GPU idle in order to resize backbuffer.
  Thus CPU side waiting is required for swapchain related commands
  to be finished. However, don't need to wait for full GPU flushing.

* Support multiple sink on a single external window
  Keep installed subclass window procedure even if there's no associated
  our internal HWND. This will make window procedure hooking less racy.
  Then parent HWND's message will be transferred to our internal HWNDs
  if needed.

* Adding support for window handle update
  Application can change target HWND even when videosink is playing or
  paused state. So, users can call gst_video_overlay_set_window_handle()
  against d3d12videosink anytime. The videosink will be able to update
  internal state and setup resource upon requested.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7013>
2024-06-17 16:05:00 +00:00
Seungha Yang
c00c36e33b d3d12overlaycompositor: Remove unused parameter
Don't need to check fence value of overlay buffer since
window uses global direct command queue

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7013>
2024-06-17 16:05:00 +00:00
Seungha Yang
f91be03550 d3d12videosink: Calculate display resolution only per caps change
Don't need to calculate it per window property update

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7013>
2024-06-17 16:05:00 +00:00
Seungha Yang
a2df44da7d d3d12: Workaround for Intel iGPU decoder crash
Observed Intel GPU driver crash when multiple decoders are
configured in a process. It might be because of frequent
command queue alloc/free or too many in-flight decoding commands.
In order to make command queue persistent and limit the number of
in-flight command lists, holds global decoding command queue.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7019>
2024-06-17 15:15:07 +00:00