Adding a property to control the number of in-flight GPU commands
(default is unlimited). Note that actual maximum number is defined
in d3d12device's direct command queue object which is 32 now,
thus total number of scheduled GPU commands cannot exceed 32.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7444>
Sometimes under certain loads, VT can error out with kVTVideoEncoderMalfunctionErr or kVTVideoEncoderNotAvailableNowErr.
These have been reported to happen more often than usual if CopyProperty/SetProperty() is used close to the encode call.
Both can be worked around by restarting the encoding session.
These errors can be returned either directly from VTCompressionSessionEncodeFrame() or later in the encoding callback.
This patch handles both scenarios the same way - a session restart is be attempted on the next encode_frame() call.
If the error is returned immediately by the encode call, it's possible that some correct frames will still be given to
the output callback, but for simplicity (+ because I wasn't able to verify this scenario) let's just discard those.
In addition, this commit also simplifies the beach/drop logic in enqueue_buffer.
Related bug reports in other projects:
http://www.openradar.me/45889262https://github.com/aws/amazon-chime-sdk-ios/issues/170#issuecomment-741908622
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7173>
Since bins can set the context of their children elements, the set_context()
vmethod shouldn't call bus messages post methods, since it locks the parent
object, the bin, which might be already locked, leading to a deadlock.
Fixes: #3706
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7378>
Adding a new videosink element for Windows composition API based
applications. Unlike d3d12videosink, this element will create only
DXGI swapchain by using IDXGIFactory2::CreateSwapChainForComposition()
without actual window handle, so that video scene can be composed
via Windows native composition API, such as DirectComposition.
Note that this videosink does not support GstVideoOverlay interface
because of the design.
The swapchain created by this element can be used with
* DirectComposition's IDCompositionVisual in Win32 app
* WinRT and WinUI3's UI.Composition in Win32/UWP app
* UWP and WinUI3 XAML's SwapChainPanel
See also examples in this commit which show usage of the videosink
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7287>
num_backward_references > 0 means we need to cache several frames
after the current frame. But the basetransform class does not
provide any _drain() kind function, so we do not have the chance
to push out our cached frames when EOS or set caps event comes.
Rather than losing the last several frames, we should just give up
the backward reference here.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7348>
The current code forgets to push the first several frames if the forward
reference > 0. They are just cached in history array and will never be
deinterlaced and pushed.
For the first several frames, even the forward reference frames are not
enough, we still need to deinterlace them as normal and push them after that.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7348>
Fixing warnings
GStreamer-CRITICAL **: 01:21:25.862: gst_value_set_int_range_step:
assertion 'start < end' failed
Although when QSV runtime reports a codec is supported, resolution query
fails sometimes, espeically VP9 encoder case on Windows.
Don't try to register an element if resolution query returned an error
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7250>
A fence configured in GstD3D12Memory should be used only for
write access to be completed. And because d3d12 -> d3d11 copy path
is read access to d3d12 resource, we should not set fence to
memory. Otherwise another read access to the d3d12 resource
will wait for d3d11 device context's copy operation although
simultaneous read access is allowed.
Use background thread to keep d3d12 resource and wait for d3d11 device's
copy operation instead.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7243>
Doing so resets the stride from the VideoMeta and it wasn't done before
the commit below. While on it, drop the plane size check as we can't
reliably predict the correct size when using DRM modifiers.
Fixes: 89b0a6fa23 ("va: refactor buffer import")
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7187>
null event NT handle to ID3D12Fence::SetEventOnCompletion()
will block the calling CPU thread already, thus it has no point that
creating an event NT handle in order to immediate wait for fence at CPU-side.
Note that passing a valid event NT handle to the fence API might be useful
when we need to wait for the fence value later (or timeout is required),
or want to wait for multiple fences at once via WaitForMultipleObjects().
But it's not a considered use case for now.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7176>
GstD3D12Window.priv.input_info is referenced by mouse event handler
in order to calculate corresponding original position
if scene is rotated/flipped by the videosink.
Fixing regression introduced by recent d3d12videosink refactoring
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7177>
The driver - AKA intel-vaapi-driver - has been unmaintained for four years
now and encoding appears to be broken in various cases. As it's unlikely
that the situation will improve, blocklist the driver for encoding.
Decoding appears to be stable enough to keep it enabled.
The driver can still be used by setting the `GST_VA_ALL_DRIVERS` env
variable.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7170>
The driver frame counters (processed, dropped, buffer level) are not
always correct apparently, and don't allow reliably assigning a frame
number to captured frames.
Instead of relying on them, count the number of frames directly here and
detect dropped frames based on the capture times of the frames: if more
than 1.75 frame durations are between two frames, then there must've
been a dropped frame.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7163>
If a d3d12 memory holds non-direct-queue fence but the fence was
created with D3D12_FENCE_FLAG_SHARED flag, use the fence instead of
waiting for fence at CPU side. Note that d3d12ipcsrc or
d3d12screencapture elements will hold such sharable fence.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7139>
This needs significant work to use the new Metal→Vulkan integration
extension `VK_EXT_metal_objects`
```
MoltenVK/mvk_deprecated_api.h:132:1: note: 'vkGetMTLDeviceMVK' has been explicitly marked deprecated here
MVK_DEPRECATED_USE_MTL_OBJS
^
MoltenVK/mvk_deprecated_api.h:74:52: note: expanded from macro 'MVK_DEPRECATED_USE_MTL_OBJS'
#define MVK_DEPRECATED_USE_MTL_OBJS VKAPI_ATTR [[deprecated("Use the VK_EXT_metal_objects extension instead.")]]
^
../sys/applemedia/videotexturecache-vulkan.mm:303:20: error: 'vkSetMTLTextureMVK' is deprecated:
Use the VK_EXT_metal_objects extension instead.
VkResult err = vkSetMTLTextureMVK (memory->vulkan_mem.image, texture);
^
MoltenVK/mvk_deprecated_api.h:151:1: note: 'vkSetMTLTextureMVK' has been explicitly marked deprecated here
MVK_DEPRECATED_USE_MTL_OBJS
^
MoltenVK/mvk_deprecated_api.h:74:52: note: expanded from macro 'MVK_DEPRECATED_USE_MTL_OBJS'
#define MVK_DEPRECATED_USE_MTL_OBJS VKAPI_ATTR [[deprecated("Use the VK_EXT_metal_objects extension instead.")]]
^
2 errors generated.
```
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7091>
When building for iOS in Cerbero, as of MoltenVK SDK 1.3.283, we have
to statically link to libMoltenVK since it no longer ships a dylib.
This requires linking to libc++, so we find the dep with the objc++
compiler to ensure that meson uses the right linker.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7091>
The code is simplified by using GQuarks for looking for caps features, and
removing inner loops.
Also, it's used the pad template caps to compare with the incoming caps because
is cheaper at the beginning of negotiation, where the pad template caps is used.
And, since the ANY caps where removed, there's no need to check for an initial
intersection.
Finally, the completion of caps features is done through a loop.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6698>
The ANY caps in pad template caps seems to mess up the DMA negotiation.
The command of:
GST_GL_API=opengl gst-launch-1.0 -vf videotestsrc ! video/x-raw,format=NV12 !
vapostproc ! "video/x-raw(memory:DMABuf)" ! glimagesink
fails to negotiate, but in fact, the vapostproc can convert the input NV12
formant into the RGBA format to render.
The ANY may help the passthough mode, but we should make the negotiate correct
first.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6698>
After signal recovery the capture times for the next frames are simply
wrong. Experimentally this affected 2-3 frames and seemed to be related
to the buffer fill level after signal recovery, so drop at least 5
frames and up to fill level + 1 frames in this situation.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7106>
Since a buffer resource will occupy at least 64KB,
allocating upload resource per decoding command might not be
an optimal approach. Instead, use sub-region of a upload resource
for multiple decoding command if sub-regions are not overlapped
each other.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7108>
When enable parallel encoding, it is possible that the unshown frame
is not output but it is already be marked as a repeated frame header.
So we need to use a dedicated buffer to hold the repeat frame header,
don't mix it with the orignal frame data.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6867>
Because DXGI flip mode swapchain will disallow GDI operation
to a HWND once swapchain is configured, videosink has been creating
child window of application's window. However, since window creation
would take a few milliseconds, it can cause performance issue such as
UI freezing. Adding a property so that videosink can attach
DXGI swapchain diretly to application's window in order to improve
performance.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7013>
A large refactoring commit for adding features and improve performance
* Reuse internal converter and overlay compositor:
Converter can be reused as long as input and display formats are not
changed. Also overlay compositor reconstruction is required only if
display format is changed
* Don't wait for full GPU flush on resize or close:
D3D12 swapchain requires GPU idle in order to resize backbuffer.
Thus CPU side waiting is required for swapchain related commands
to be finished. However, don't need to wait for full GPU flushing.
* Support multiple sink on a single external window
Keep installed subclass window procedure even if there's no associated
our internal HWND. This will make window procedure hooking less racy.
Then parent HWND's message will be transferred to our internal HWNDs
if needed.
* Adding support for window handle update
Application can change target HWND even when videosink is playing or
paused state. So, users can call gst_video_overlay_set_window_handle()
against d3d12videosink anytime. The videosink will be able to update
internal state and setup resource upon requested.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7013>
Observed Intel GPU driver crash when multiple decoders are
configured in a process. It might be because of frequent
command queue alloc/free or too many in-flight decoding commands.
In order to make command queue persistent and limit the number of
in-flight command lists, holds global decoding command queue.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7019>
This was already being used in handle_frame() for errors that happen when queueing a frame for decoding,
let's do the same when a frame is flagged with an error in the output callback.
From quick testing, this makes seeking more reliable (previously, it would sometimes cause a decoding error
and shut the whole decoder down due to GST_FLOW_ERROR).
Also manually sets the max error count to actually stop processing if too many errors occur.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6446>
ReferenceMissingErr is not critical and the simplest solution is to just ignore it. The frame has
the FrameDropped flag set when it occurs, so we can just drop it as usual.
BadDataErr is also not immediately critical, but in its case let's set the ERROR flag,
so the output loop can use GST_VIDEO_DECODER_ERROR to count and error out if it happens too many times.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6446>
Musl does not implement GNU basename and have fixed a bug where the
prototype was leaked into string.h [1], which resullts in compile errors
with GCC-14 and Clang-17+
| sys/uvcgadget/configfs.c:262:21: error: call to undeclared function 'basename'
ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
| 262 | const char *v = basename (globbuf.gl_pathv[i]);
| | ^
Use glib function instead makes it portable across musl and glibc on
linux
[1] https://git.musl-libc.org/cgit/musl/commit/?id=725e17ed6dff4d0cd22487bb64470881e86a92e7a
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7006>
It can be seen as a WA in the case of multi-channel transcoding (like
decoder output to two channels, one for encoder and one for vpp).
Normally, encoder sets min pts of a huge value to avoid negative dts,
while vpp set pts without this addtional huge value, which are likely to
cause input surface pts does not fit with encoder (since both encoder
and vpp accept the same buffer from decoder, means they modify the timestamp
of one mfx surface). So we add this huge value to vpp to ensure enc and
vpp set the same value to input mfx surface meanwhile does not break
encoder's setting min pts for dts protection.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6971>
