Schedule (semi-)static resource upload at converter creation time.
And use single resource for all vertex, index, and constant
buffers, since separate resources will waste GPU memory.
Note that size and address of a committed resource are 64K aligned
even if requested buffer size is small.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7081>
As NVIDIA Amperium. In this case the each output buffer is also a DPB,
but using a different view layer.
Still pending a validation layer issue:
VUID-VkVideoBeginCodingInfoKHR-flags-07244
Co-authored-by: Victor Jaquez <vjaquez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6954>
query_result_status can be optional so we should not create
the query pool if the queue does not support it,
ie, AMD does not support VK_QUERY_TYPE_RESULT_STATUS_ONLY_KHR
In other use case such as VK_QUERY_TYPE_VIDEO_ENCODE_FEEDBACK_KHR, the
query pool must be created.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7043>
As only queryResultStatusSupport can be optional,
the variable name should be more specific.
queryResultStatusSupport reports VK_TRUE if query type
VK_QUERY_TYPE_RESULT_STATUS_ONLY_KHR and use of
VK_QUERY_RESULT_WITH_STATUS_BIT_KHR are supported.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7043>
According to the SPEC:
The frame id numbers (represented in display_frame_id, current_frame_id,
and RefFrameId[ i ]) are not needed by the decoding process, but allow
decoders to spot when frames have been missed and take an appropriate action.
So we should just print out warning and should not return error in parser when
mismatching. The decoder itself is already robust to handle the reference missing.
Fixes#3622
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7047>
Because DXGI flip mode swapchain will disallow GDI operation
to a HWND once swapchain is configured, videosink has been creating
child window of application's window. However, since window creation
would take a few milliseconds, it can cause performance issue such as
UI freezing. Adding a property so that videosink can attach
DXGI swapchain diretly to application's window in order to improve
performance.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7013>
A large refactoring commit for adding features and improve performance
* Reuse internal converter and overlay compositor:
Converter can be reused as long as input and display formats are not
changed. Also overlay compositor reconstruction is required only if
display format is changed
* Don't wait for full GPU flush on resize or close:
D3D12 swapchain requires GPU idle in order to resize backbuffer.
Thus CPU side waiting is required for swapchain related commands
to be finished. However, don't need to wait for full GPU flushing.
* Support multiple sink on a single external window
Keep installed subclass window procedure even if there's no associated
our internal HWND. This will make window procedure hooking less racy.
Then parent HWND's message will be transferred to our internal HWNDs
if needed.
* Adding support for window handle update
Application can change target HWND even when videosink is playing or
paused state. So, users can call gst_video_overlay_set_window_handle()
against d3d12videosink anytime. The videosink will be able to update
internal state and setup resource upon requested.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7013>
Observed Intel GPU driver crash when multiple decoders are
configured in a process. It might be because of frequent
command queue alloc/free or too many in-flight decoding commands.
In order to make command queue persistent and limit the number of
in-flight command lists, holds global decoding command queue.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7019>
This was already being used in handle_frame() for errors that happen when queueing a frame for decoding,
let's do the same when a frame is flagged with an error in the output callback.
From quick testing, this makes seeking more reliable (previously, it would sometimes cause a decoding error
and shut the whole decoder down due to GST_FLOW_ERROR).
Also manually sets the max error count to actually stop processing if too many errors occur.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6446>
ReferenceMissingErr is not critical and the simplest solution is to just ignore it. The frame has
the FrameDropped flag set when it occurs, so we can just drop it as usual.
BadDataErr is also not immediately critical, but in its case let's set the ERROR flag,
so the output loop can use GST_VIDEO_DECODER_ERROR to count and error out if it happens too many times.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6446>
Adding NVIDIA nvCOMP library based plugin for lossless raw video
compression/decompression. To build this plugin, user should
install nvCOMP SDK first and specify the SDK path via
"nvcomp-sdk-path" build option or NVCOMP_SDK_PATH env.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6912>
Musl does not implement GNU basename and have fixed a bug where the
prototype was leaked into string.h [1], which resullts in compile errors
with GCC-14 and Clang-17+
| sys/uvcgadget/configfs.c:262:21: error: call to undeclared function 'basename'
ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
| 262 | const char *v = basename (globbuf.gl_pathv[i]);
| | ^
Use glib function instead makes it portable across musl and glibc on
linux
[1] https://git.musl-libc.org/cgit/musl/commit/?id=725e17ed6dff4d0cd22487bb64470881e86a92e7a
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7006>
It can be seen as a WA in the case of multi-channel transcoding (like
decoder output to two channels, one for encoder and one for vpp).
Normally, encoder sets min pts of a huge value to avoid negative dts,
while vpp set pts without this addtional huge value, which are likely to
cause input surface pts does not fit with encoder (since both encoder
and vpp accept the same buffer from decoder, means they modify the timestamp
of one mfx surface). So we add this huge value to vpp to ensure enc and
vpp set the same value to input mfx surface meanwhile does not break
encoder's setting min pts for dts protection.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6971>
Adding a property to control error reporting behavior when output
window is closed in playing or paused state. This can be useful
for apps where an app wants to close window even if it's playing
a stream, and the closed window is expected.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6939>
Not all GPUs can support arbitrary offset of
D3D12_PLACED_SUBRESOURCE_FOOTPRINT when copying GPU memory between
texture and buffer. Instead of calculating size/offset per plane,
calculate the entire size and offsets at once.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6967>
drm_fourcc.h should be picked up via the pkgconfig include, not the
system includedir directly.
Also consolidate the libdrm usage in va and msdk.
All this allows it to be picked up consistently (via the subproject,
for example).
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6932>
This patch fixes this critical warning when registering MSDK:
_dma_fmt_to_dma_drm_fmts: assertion 'fmt != GST_VIDEO_FORMAT_UNKNOWN' failed
It was because the HEVC string with possible output formats has an extra space
that could not be parsed correctly.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6853>
When transforming from unknown alignment to frame or obu, the TU timestamp
was not properly transferred. Fix this by saving the TU DTS as the first
DTS seen within the the TU data, and the PTS as the last PTS seen in that
TU data. Finally, reset the TU timestamp after each TU have completed.
Fixes#1496
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6895>
Adds a separate vtenc_h265a element (with a _hw variant as usual) for the HEVCWithAlpha codec type.
Decided to go with a separate element to not break existing uses of the normal HEVC encoder.
The preserve_alpha property is still only used for ProRes, no need for it here because we explicitly say we want alpha
when using the new element.
For now, the HEVCWithAlpha has an issue where it does not throttle the amount of input frames queued internally.
I added a quick workaround where encode_frame() will block until enqueue_frame() callback notifies it that some space
has been freed up in the internal queue. The limit was set to 5, which should be enough I guess? Hopefully this is not
too prone to race conditions.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6664>
`StdVideoDecodeH265PictureInfo.flags.IsReference` refers to section 3.132 ITU-T
H.265 specification:
reference picture: A picture that is a short-term reference picture or a
long-term reference picture.
`GstH265Picture.ref` doesn't reflect this, but we need to query the NAL type of
the processed slice.
This patch fixes the validation layer error
`VUID-vkCmdBeginVideoCodingKHR-slotIndex-07239` while using the NVIDIA driver.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6901>
Conceptually identical to the present signal of d3d11videosink.
This signal will be emitted with current render target
(i.e., swapchain backbuffer) and command queue. Signal handler
can record GPU commands for an overlay image or to blend
an image to the render target.
In addition to d3d12 resources, videosink will send
d3d11 and d2d resources depending on "overlay-mode"
property, so that signal handler can render by using
preferred/required DirectX API.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6838>
The idea of using separate command queue per videosink was that
swapchain is bound to a command queue and we need to flush the
command queue when window size is changed. But the separate
queue does not seem to improve performance a lot.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6838>
The prime_fds for multi planes may be the same. For example, on Intel's
platform, the NV12 surface may have the same FD for the plane0 and the
plane1. Then, the DRM_IOCTL_GEM_CLOSE will close the same handle twice
and get an "Invalid argument 22" error the second time.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6914>
From the spec (chapter 34, v1.3.283):
````
UNORM: the components are unsigned normalized values in the range [0, 1]
SRGB: the R, G and B components are unsigned normalized value that represent
values using sRGB nonlinear encoding, while the A component (if one
exists) is a regular unsigned normalized value
```
The difference is the storage encoding, the first one is aimed for image
transfers, while the second is for shaders, mostly in the swapchain stage in the
pipeline, and it's done automatically if needed [1].
As far as I have checked, other frameworks (FFmpeg, GTK+), when import or export
images from/to Vulkan, use exclusively UNORM formats, while SRGB formats are
ignored.
My conclusion is that Vulkan formats are related on how bits are stored in
memory rather their transfer functions (colorimetry).
This patch does two interrelated changes:
1. It swaps certain color format maps to try first, in both
gst_vulkan_format_from_video_info() and gst_vulkan_format_from_video_info_2(),
the UNORM formats, when comparing its usage, and later check for SRGB.
2. It removes the code that check for colorimetry in
gst_vulkan_format_from_video_info_2(), since it not storage related.
1. https://community.khronos.org/t/noob-difference-between-unorm-and-srgb/106132/7
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6797>
This mirrors the behaviour in vp8enc / vp9enc and is generally more
useful than using any framerate from the caps as it provides some degree
of accuracy if the stream doesn't have timestamps perfectly according to
the framerate.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6891>
The encoder can specify the a preferred_output_delay value to get better throughput
performance. The higher delay may get better HW performance, but it may increases
the encoder and pipeline latency.
When the output queue length is smaller than preferred_output_delay, the encoder
will not block to waiting for the encoding output. It will continue to prepare and
send more commands to GPU, which may improve the encoder throughput performance.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4359>
If the muxer times out because of the latency deadline it can happen
that some pads have no caps yet. In that case skip creation of streams
for these pads and create updated section tables once the first buffer
arrives later.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6823>
This makes sure that for sparse streams (KLV, DVB subtitles, ...) the
muxer does not wait until the next buffer is available for them but
times out on the latency deadline and outputs data.
For non-live pipelines it will still be necessary for upstream to
correctly produce gap events for sparse streams.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6823>
The FORCE_KEYFRAME frame which has GST_VIDEO_CODEC_FRAME_FLAG_FORCE_KEYFRAME
bit set should be the sync point. So we should let it be an IDR frame to begin
a new GOP, rather than just promote it to an I frame.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6619>
The FORCE_KEYFRAME frame which has GST_VIDEO_CODEC_FRAME_FLAG_FORCE_KEYFRAME
bit set should be the sync point. So we should let it be an IDR frame to begin
a new GOP, rather than just promote it to an I frame.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6619>
Set the P and B frame qp to I frame value to avoid generating delta
QP between different frame types. For ICQ and QVBR modes, we can
only set the qpi value, so the qpp and qpb values should be set to
the same value as the qpi.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6841>
Address below message reported by SDK debug layer.
ID3D12Device::CheckFeatureSupport: Unsupported Decode Profile Specified.
Use ID3D12VideoDevice::CheckFeatureSupport with D3D12_FEATURE_VIDEO_DECODE_PROFILES
to retrieve a list of supported profiles
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6839>
Certain V4L2 fourccs don't (yet) have DRM counter parts, in which case
we can't create DMA_DRM caps for them. This is usually the case for
specific tilings, which are represented as modifiers for DMA formats.
While using these tilings is generally preferable - because of e.g.
lower memory usage - it can result in additional conversion steps when
interacting with DMA based APIs such as GL, Vulkan or KMS. In such cases
using a DMA compatible format usually ends up being the better option.
Before the addition of DMA_DRM caps, this was what playbin3 ended up
requesting in various cases - e.g. prefering NV12 over NV12_4L4 - but
the addition of DMA_DRM caps seems to confuse the selection logic.
As a simple and quite robust solution, assume that peers supporting
DMA_DRM caps always prefer these and reorder the caps accordingly.
In the future we plan to have a translation layer for cases where
there is a matching fourcc+modifier pair for a V4L2 fourcc, ensuring
optimal results.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6645>
Doing it in gst_play_new() means that bindings that directly call
g_object_new() with the GType wouldn't end up initializing both.
This affects at least the Python and GJS bindings.
gst_init() is nonetheless only called from gst_play_new() once because
calling it from class_init would likely lead to problems as that's
called from somewhere in the middle of GObject.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6801>
Don't add an extra ref if non-floating as that ref will never be
unreffed.
gst_bin_add() is transfer floating (alias to transfer none).
Fixes a leak when a non-floating ref was provided as a return value in
the request-aux-sender signal.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6807>
Use of VK_PIPELINE_STAGE_2_ALL_COMMANDS_BIT instead of
specific VK_PIPELINE_STAGE_2_VIDEO_DECODE_BIT_KHR
Fix for VUID-vkCmdPipelineBarrier2-srcStageMask-03849
pDependencyInfo->pImageMemoryBarriers[0].srcStageMask
(VK_PIPELINE_STAGE_2_VIDEO_DECODE_BIT_KHR) is not compatible with
the queue family properties
(VK_QUEUE_GRAPHICS_BIT|VK_QUEUE_COMPUTE_BIT|VK_QUEUE_TRANSFER_BIT|
VK_QUEUE_SPARSE_BINDING_BIT|VK_QUEUE_PROTECTED_BIT) of this
command buffer. The Vulkan spec states: The srcStageMask member
of any element of the pMemoryBarriers, pBufferMemoryBarriers, or
pImageMemoryBarriers members of pDependencyInfo must only
include pipeline stages valid for the queue family that was
used to create the command pool that commandBuffer was allocated
from (
https://www.khronos.org/registry/vulkan/specs/1.3-extensions/
html/vkspec.html#VUID-vkCmdPipelineBarrier2-srcStageMask-03849)
The frame barrier should use a compatible srcStageMask for all
the queues.
Remove reset_pipeline_stage_mask as it is redundant
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6780>
Add pitch tests with different forward and backward playback rates.
Those tests depend on the libSoundTouch version to validate the buffers
checksums. The actual version uses libSoundTouch 2.3.2, use the
`--force-fallback-for=soundtouch` meson option to build using the same
version.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6247>
- fully protect accesses to the libsoundtouch API that is not
thread-safe.
- fully protect accesses to GstPitch members that could be read by a
downstream query thread while written by an upstream streaming thread
or a user thread.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6247>
- use the `GST_PITCH_GET_PRIVATE` accessor when needed
- rename `out_seg_rate` to `output_rate` to use the same name as the parameter
- rename `seg_arate` to `segment_applied_rate` to improve readability
- apply gst-indent to gstpitch.hh/cc
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6247>
When changing playing rate, the output segment was not correctly
calculated because the stream time ratio was computed using the previous
input segment rate instead of using the actual rate. This was producing
wrong results for the output segment start and end values.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6247>
The h26xdecoder 'stop' method was not called
as the vulkan h26x class rewires the video decoder
'stop' base method to its own one.
It was causing some memory leaks such as dangling parser
and dpb in h26xdecoder base class.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6782>
1. The PTS of all frames should not be changed.
2. Just update the DTS based on the PTS. For the frame which is not
reordered, the DTS is equal to PTS. For frame which is reordered,
the DTS is equal to previous DTS. For example:
Input: F0[D0, P0] -- F1[D1, P1] -- F2[D2, P2] -- F3[D3, P3]
Output: F0[I, D0, P0] -- F3[P, D0, P3] -- F1[B, D1, P1] -- F2[B, D2, P2]
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6703>
1. The PTS of all frames should not be changed.
2. Just update the DTS based on the PTS. For the frame which is not
reordered, the DTS is equal to PTS. For frame which is reordered,
the DTS is equal to previous DTS. For example:
Input: F0[D0, P0] -- F1[D1, P1] -- F2[D2, P2] -- F3[D3, P3]
Output: F0[I, D0, P0] -- F3[P, D0, P3] -- F1[B, D1, P1] -- F2[B, D2, P2]
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6703>
Fixing deadlock in below case
* GC lock is taken by background thread, and the background thread calls
gst_d3d12_ipc_client_release_imported_data() which takes ipc lock
* ipc lock is already taken in ipc thread and trying to pushing GC data
via gst_d3d12_command_queue_set_notify()
* gst_d3d12_command_queue_set_notify() is trying to take GC lock
but it's already taken by background thread
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6749>
Ideally, GPU waiting should be scheduled just before executing command list.
But handling the case outside of converter is a bit complicated.
Under an assumption that constructed command list will be executed
immediately, schedules GPU-side waiting inside of conversion method
to simplify the flow.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6749>
Since we don't want to expose video decoding API outside of GStreamer, the
header is removed from installation and both source files are renamed as
-private.
The header must remain in gst-libs because is referred by GstVulkanQueue,
which's the decoder factory.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6723>
First it derived mapping was disabled for P010 formats, but also there's an
issue with interlaced frames.
It would be possible to disable derived mapping only for interlaced (H.264
decoder and vadeinterlace) but it would spread the hacks along the code. It's
simpler and contained to disable derived completely for Mesa <23.3
Fixes: #3450
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6729>
Instead of duplicating the GStreamer format to DRM fourcc mapping, this patch
uses the GstVideo library helpers. This duplicates the big O of looking for,
since the two lists are traversed, but it's less error prone.
Partially reverts commit 547f3e8622.
Fixes: #3354
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6731>
If waylandsink received buffer rate is high which causes frame
drop, the cached staged buffer will be replaced when next buffer
needs to be rendered and be freed after redraw. But there is
chance to get memory leak if ended without redraw. So need to
free staged buffer when do gst_wl_window_finalize().
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6670>
