Commit graph

1631 commits

Author SHA1 Message Date
Matthew Waters
58db0e6ed8 decklinkvideosink: schedule frames before they need to be displayed
This removes most occurances of 'late' frame notifications produced by
decklink.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7107>
2024-07-19 01:28:16 +00:00
Seungha Yang
72895ed0fa d3d12: Always allocate output texture using shared heap
... if downstream preference is unknown (e.g., no proposed
buffer pool by downstream), so that produced textures can be
shareable with other APIs such as d3d11 or vulkan, or other processes

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7193>
2024-07-18 14:17:43 +00:00
Robert Mader
f392e9b369 vabase: Stop aligning VideoInfo during DMABUF import
Doing so resets the stride from the VideoMeta and it wasn't done before
the commit below. While on it, drop the plane size check as we can't
reliably predict the correct size when using DRM modifiers.

Fixes: 89b0a6fa23 ("va: refactor buffer import")
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7187>
2024-07-18 09:10:23 +00:00
Robert Mader
b30a83ed5a vabase: Use correct VideoInfo during DMABUF import
The changes to the VideoInfo, notably the stride from the VideoMeta,
were lost. Avoid such mistakes by explicitly using the VideoInfo from
drm_info.

Fixes: 9f5b2c4e25 ("va: use GstVideoInfoDmaDrm when importing buffers")
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7187>
2024-07-18 09:10:23 +00:00
Nirbheek Chauhan
65ff618bb6 avfdeviceprovider: Fix debug category initialization
The device monitor calls into avfvideosrc functions without
initializing the debug category, which causes multiple criticals.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7192>
2024-07-18 07:20:06 +00:00
Seungha Yang
b37bfc02f5 d3d12: Remove unnecessary event handles
null event NT handle to ID3D12Fence::SetEventOnCompletion()
will block the calling CPU thread already, thus it has no point that
creating an event NT handle in order to immediate wait for fence at CPU-side.
Note that passing a valid event NT handle to the fence API might be useful
when we need to wait for the fence value later (or timeout is required),
or want to wait for multiple fences at once via WaitForMultipleObjects().
But it's not a considered use case for now.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7176>
2024-07-16 19:17:15 +00:00
Seungha Yang
783e1ea327 d3d12videosink: Fix mouse event handling
GstD3D12Window.priv.input_info is referenced by mouse event handler
in order to calculate corresponding original position
if scene is rotated/flipped by the videosink.
Fixing regression introduced by recent d3d12videosink refactoring

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7177>
2024-07-16 17:02:24 +00:00
Robert Mader
556e47580a va: Blocklist i965 driver for encoding
The driver - AKA intel-vaapi-driver - has been unmaintained for four years
now and encoding appears to be broken in various cases. As it's unlikely
that the situation will improve, blocklist the driver for encoding.
Decoding appears to be stable enough to keep it enabled.

The driver can still be used by setting the `GST_VA_ALL_DRIVERS` env
variable.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7170>
2024-07-16 15:53:37 +00:00
Seungha Yang
dbc8a1397d d3d12compositor: Fix transparent background mode with YUV output
In case of YUV format without alpha channel, zero clear value
for each channle will result in green color. Use calculated black
background color with alpha=0 for transparent background mode instead.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7181>
2024-07-16 14:18:12 +00:00
Seungha Yang
3922ae5d92 d3d11compositor: Fix transparent background mode with YUV output
In case of YUV format without alpha channel, zero clear value
for each channle will result in green color. Use calculated black
background color with alpha=0 for transparent background mode instead.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7181>
2024-07-16 14:18:12 +00:00
Sebastian Dröge
3eec4703c6 ajasrc: Fix handling of timestamps and don't rely on driver frame counters
The driver frame counters (processed, dropped, buffer level) are not
always correct apparently, and don't allow reliably assigning a frame
number to captured frames.

Instead of relying on them, count the number of frames directly here and
detect dropped frames based on the capture times of the frames: if more
than 1.75 frame durations are between two frames, then there must've
been a dropped frame.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7163>
2024-07-16 09:59:36 +00:00
Seungha Yang
2a6967a8cd d3d12videosink: Clear cached buffer on format change
Otherwise converter will try to read memory of which layout/format
might be different from configured converter pipeline

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7167>
2024-07-15 10:20:46 +00:00
Seungha Yang
e692bf0fb9 d3d12memorycopy: Enhance d3d12 to d3d11 copy
If a d3d12 memory holds non-direct-queue fence but the fence was
created with D3D12_FENCE_FLAG_SHARED flag, use the fence instead of
waiting for fence at CPU side. Note that d3d12ipcsrc or
d3d12screencapture elements will hold such sharable fence.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7139>
2024-07-08 09:57:04 +00:00
Nirbheek Chauhan
0b9ec1b657 applemedia: Disable some deprecation errors
This needs significant work to use the new Metal→Vulkan integration
extension `VK_EXT_metal_objects`

```
MoltenVK/mvk_deprecated_api.h:132:1: note: 'vkGetMTLDeviceMVK' has been explicitly marked deprecated here
MVK_DEPRECATED_USE_MTL_OBJS
^
MoltenVK/mvk_deprecated_api.h:74:52: note: expanded from macro 'MVK_DEPRECATED_USE_MTL_OBJS'
 #define MVK_DEPRECATED_USE_MTL_OBJS   VKAPI_ATTR [[deprecated("Use the VK_EXT_metal_objects extension instead.")]]
                                                    ^
../sys/applemedia/videotexturecache-vulkan.mm:303:20: error: 'vkSetMTLTextureMVK' is deprecated:
Use the VK_EXT_metal_objects extension instead.
    VkResult err = vkSetMTLTextureMVK (memory->vulkan_mem.image, texture);
                   ^
MoltenVK/mvk_deprecated_api.h:151:1: note: 'vkSetMTLTextureMVK' has been explicitly marked deprecated here
MVK_DEPRECATED_USE_MTL_OBJS
^
MoltenVK/mvk_deprecated_api.h:74:52: note: expanded from macro 'MVK_DEPRECATED_USE_MTL_OBJS'
 #define MVK_DEPRECATED_USE_MTL_OBJS   VKAPI_ATTR [[deprecated("Use the VK_EXT_metal_objects extension instead.")]]
                                                    ^
2 errors generated.
```

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7091>
2024-07-07 13:20:12 +00:00
Nirbheek Chauhan
156a016313 meson: Find MoltenVK with the objc++ compiler everywhere
When building for iOS in Cerbero, as of MoltenVK SDK 1.3.283, we have
to statically link to libMoltenVK since it no longer ships a dylib.
This requires linking to libc++, so we find the dep with the objc++
compiler to ensure that meson uses the right linker.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7091>
2024-07-07 13:20:12 +00:00
Nirbheek Chauhan
adedac6bd9 meson: Fix some confusing code in applemedia's build file
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7091>
2024-07-07 13:20:12 +00:00
Nirbheek Chauhan
4293719206 meson: Fix vulkan automagic in applemedia plugin
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7091>
2024-07-07 13:20:12 +00:00
Víctor Manuel Jáquez Leal
426616cca7 vavpp: simplify gst_va_vpp_transform_caps()
The code is simplified by using GQuarks for looking for caps features, and
removing inner loops.

Also, it's used the pad template caps to compare with the incoming caps because
is cheaper at the beginning of negotiation, where the pad template caps is used.
And, since the ANY caps where removed, there's no need to check for an initial
intersection.

Finally, the completion of caps features is done through a loop.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6698>
2024-07-04 08:54:16 +00:00
He Junyan
f235e0308d vadeinterlace: Do not append ANY caps into pad template
Just like the vapostproc, we delete the ANY caps in pad template to
avoid unexpected negotiation.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6698>
2024-07-04 08:54:15 +00:00
He Junyan
38a4224c8c vapostproc: Do not append ANY caps into pad template
The ANY caps in pad template caps seems to mess up the DMA negotiation.
The command of:

GST_GL_API=opengl gst-launch-1.0 -vf videotestsrc ! video/x-raw,format=NV12 !
  vapostproc ! "video/x-raw(memory:DMABuf)" ! glimagesink

fails to negotiate, but in fact, the vapostproc can convert the input NV12
formant into the RGBA format to render.

The ANY may help the passthough mode, but we should make the negotiate correct
first.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6698>
2024-07-04 08:54:15 +00:00
Lim, Siew Hoon
e82eb628b1 vacompositor: Initialize the allocation related variables in decide_allocation()
Prevent garbage value has been pass thru and causing
pipeline fail to run later on.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7097>
2024-07-04 06:48:46 +00:00
Lim, Siew Hoon
f498ba25da vabasedec: Initialize the allocation related variables in decide_allocation()
Prevent garbage value has been pass thru and causing
pipeline fail to run.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7097>
2024-07-04 06:48:45 +00:00
Lim, Siew Hoon
c270c5643d vabasetranform: Initialize the allocation related variables in decide_allocation()
Prevent garbage value has been pass thru and causing
pipeline fail to run.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7097>
2024-07-04 06:48:45 +00:00
He Junyan
e9801cb0a5 vah265enc: Set the trellis only when HW supports it
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6663>
2024-07-04 05:59:07 +00:00
He Junyan
2049c7e050 vah264enc: Init missing fields in reset_state()
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6663>
2024-07-04 05:59:06 +00:00
He Junyan
5706e3566c vah264enc: Set the trellis only when HW supports it
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6663>
2024-07-04 05:59:06 +00:00
He Junyan
5b27aff33b va: baseenc: Set the trellis parameter anyway
The driver may enable trellis by default. So we should also set the
trellis info to driver even when the trellis option is turned off.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6663>
2024-07-04 05:59:06 +00:00
Seungha Yang
c0a02affa4 d3d12: Add support for resource copy between d3d11 and d3d12
If driver can support cross-api resource sharing, use device-to-device
resource copy in d3d12upload/download elements.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7119>
2024-07-01 20:18:42 +00:00
Sebastian Dröge
f8fb72b0d7 ajasrc: Drop some frames after signal recovery
After signal recovery the capture times for the next frames are simply
wrong. Experimentally this affected 2-3 frames and seemed to be related
to the buffer fill level after signal recovery, so drop at least 5
frames and up to fill level + 1 frames in this situation.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7106>
2024-07-01 11:37:08 +00:00
Sebastian Dröge
63dec7c704 ajasrc: Reset clock after signal loss or signal change
Otherwise timestamps would continue as if there was no gap, and the next
frames until the clock has compensated would be all too late.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7106>
2024-07-01 11:37:08 +00:00
Seungha Yang
c013e88295 d3d12decoder: Use sub-allocated bitstream buffer
Since a buffer resource will occupy at least 64KB,
allocating upload resource per decoding command might not be
an optimal approach. Instead, use sub-region of a upload resource
for multiple decoding command if sub-regions are not overlapped
each other.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7108>
2024-06-29 20:47:39 +09:00
He Junyan
77f779e86a vaav1enc: Move repeat header data to a dedicated buffer
When enable parallel encoding, it is possible that the unshown frame
is not output but it is already be marked as a repeated frame header.
So we need to use a dedicated buffer to hold the repeat frame header,
don't mix it with the orignal frame data.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6867>
2024-06-27 12:48:37 +00:00
Seungha Yang
76fd732d42 qsv: Check d3d shared resource tier using D3D11 API
We can check the tier using d3d11 API. Thus, don't need to
create d3d12 device

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7099>
2024-06-26 16:25:06 +00:00
Piotr Brzeziński
32d4e5f73c vtenc: Fix redistribute latency spam
Just a quick fix to only report the maximum noticed delay (measured by frames inside the encoder) instead of changing
the reported latency every time the number there changes, which is way too often.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7094>
2024-06-24 16:08:37 +02:00
Seungha Yang
69ea20b739 d3d12videosink: Present on GstVideoOverlay::expose()
... so that updated backbuffer can be swapped and presented

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7079>
2024-06-24 08:46:24 +00:00
Seungha Yang
dd4d85272e d3d12converter: Fix Y410 conversion
Adding format conversion helper and use compute shader in case that
output format does not support RTV.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7088>
2024-06-23 10:53:18 +00:00
Seungha Yang
327fb9924c d3d12basefilter: Add adapter property
Allows initial GPU adapter selection

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7066>
2024-06-20 00:03:28 +09:00
Seungha Yang
2ec7d82bd7 d3d12: Move fence setter helper method to gst-libs
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7057>
2024-06-19 13:19:48 +00:00
Seungha Yang
fa7c4a2e39 d3d12converter: Update API signature
Always use device's main direct queue, and control gpu waiting
behavior by using boolean value

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7057>
2024-06-19 13:19:48 +00:00
Seungha Yang
f6eb3a01c0 d3d12memory: Hide fence value from header
Instead of exposing fence value to wait in header, user setter/getter
methods.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7057>
2024-06-19 13:19:47 +00:00
Seungha Yang
49c4247cb3 d3d12device: Add helper method for getting fence handle
Add get_fence_handle() method so that caller can get command queue's
dedicated fence handle from device

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7057>
2024-06-19 13:19:47 +00:00
Seungha Yang
289bc1d440 d3d12: Remove notify_com and notify_mini_object helper methods
Use private macros instead of exposing multiple APIs for the same thing

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7057>
2024-06-19 13:19:47 +00:00
Seungha Yang
6942868bda d3d12commandqueue: Update API name and arguments
Accepts multiple fences since single command list may have
multiple dependent resources which are associated with
different GPU engines

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7057>
2024-06-19 13:19:47 +00:00
Seungha Yang
713ea313f8 d3d12device: Use HRESULT return code if possible
Make function signature consistent with that of command queue

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7057>
2024-06-19 13:19:47 +00:00
Seungha Yang
339c81bac8 d3d12: Promote decoder and videosink rank to primary
It's proven that d3d12 performs better than d3d11 while
consumes less resources in various cases.
Assign primary+ rank to decoder and videosink in case of Windows10/11,
so that it can be tested widely

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7045>
2024-06-18 14:40:37 +00:00
Seungha Yang
e5d2dd8e8d d3d12videosink: Add direct-swapchain property
Because DXGI flip mode swapchain will disallow GDI operation
to a HWND once swapchain is configured, videosink has been creating
child window of application's window. However, since window creation
would take a few milliseconds, it can cause performance issue such as
UI freezing. Adding a property so that videosink can attach
DXGI swapchain diretly to application's window in order to improve
performance.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7013>
2024-06-17 16:05:00 +00:00
Seungha Yang
dd1d2f5ff7 d3d12videosink: Add external-window-only property
Adding a new property in order to avoid unintended interanl window
creation.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7013>
2024-06-17 16:05:00 +00:00
Seungha Yang
37e1847464 d3d12videosink: Add support for window handle update
A large refactoring commit for adding features and improve performance

* Reuse internal converter and overlay compositor:
  Converter can be reused as long as input and display formats are not
  changed. Also overlay compositor reconstruction is required only if
  display format is changed

* Don't wait for full GPU flush on resize or close:
  D3D12 swapchain requires GPU idle in order to resize backbuffer.
  Thus CPU side waiting is required for swapchain related commands
  to be finished. However, don't need to wait for full GPU flushing.

* Support multiple sink on a single external window
  Keep installed subclass window procedure even if there's no associated
  our internal HWND. This will make window procedure hooking less racy.
  Then parent HWND's message will be transferred to our internal HWNDs
  if needed.

* Adding support for window handle update
  Application can change target HWND even when videosink is playing or
  paused state. So, users can call gst_video_overlay_set_window_handle()
  against d3d12videosink anytime. The videosink will be able to update
  internal state and setup resource upon requested.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7013>
2024-06-17 16:05:00 +00:00
Seungha Yang
c00c36e33b d3d12overlaycompositor: Remove unused parameter
Don't need to check fence value of overlay buffer since
window uses global direct command queue

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7013>
2024-06-17 16:05:00 +00:00
Seungha Yang
f91be03550 d3d12videosink: Calculate display resolution only per caps change
Don't need to calculate it per window property update

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7013>
2024-06-17 16:05:00 +00:00