This patch fixes this critical warning when registering MSDK:
_dma_fmt_to_dma_drm_fmts: assertion 'fmt != GST_VIDEO_FORMAT_UNKNOWN' failed
It was because the HEVC string with possible output formats has an extra space
that could not be parsed correctly.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6853>
When transforming from unknown alignment to frame or obu, the TU timestamp
was not properly transferred. Fix this by saving the TU DTS as the first
DTS seen within the the TU data, and the PTS as the last PTS seen in that
TU data. Finally, reset the TU timestamp after each TU have completed.
Fixes#1496
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6895>
Adds a separate vtenc_h265a element (with a _hw variant as usual) for the HEVCWithAlpha codec type.
Decided to go with a separate element to not break existing uses of the normal HEVC encoder.
The preserve_alpha property is still only used for ProRes, no need for it here because we explicitly say we want alpha
when using the new element.
For now, the HEVCWithAlpha has an issue where it does not throttle the amount of input frames queued internally.
I added a quick workaround where encode_frame() will block until enqueue_frame() callback notifies it that some space
has been freed up in the internal queue. The limit was set to 5, which should be enough I guess? Hopefully this is not
too prone to race conditions.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6664>
`StdVideoDecodeH265PictureInfo.flags.IsReference` refers to section 3.132 ITU-T
H.265 specification:
reference picture: A picture that is a short-term reference picture or a
long-term reference picture.
`GstH265Picture.ref` doesn't reflect this, but we need to query the NAL type of
the processed slice.
This patch fixes the validation layer error
`VUID-vkCmdBeginVideoCodingKHR-slotIndex-07239` while using the NVIDIA driver.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6901>
Conceptually identical to the present signal of d3d11videosink.
This signal will be emitted with current render target
(i.e., swapchain backbuffer) and command queue. Signal handler
can record GPU commands for an overlay image or to blend
an image to the render target.
In addition to d3d12 resources, videosink will send
d3d11 and d2d resources depending on "overlay-mode"
property, so that signal handler can render by using
preferred/required DirectX API.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6838>
The idea of using separate command queue per videosink was that
swapchain is bound to a command queue and we need to flush the
command queue when window size is changed. But the separate
queue does not seem to improve performance a lot.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6838>
The prime_fds for multi planes may be the same. For example, on Intel's
platform, the NV12 surface may have the same FD for the plane0 and the
plane1. Then, the DRM_IOCTL_GEM_CLOSE will close the same handle twice
and get an "Invalid argument 22" error the second time.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6914>
From the spec (chapter 34, v1.3.283):
````
UNORM: the components are unsigned normalized values in the range [0, 1]
SRGB: the R, G and B components are unsigned normalized value that represent
values using sRGB nonlinear encoding, while the A component (if one
exists) is a regular unsigned normalized value
```
The difference is the storage encoding, the first one is aimed for image
transfers, while the second is for shaders, mostly in the swapchain stage in the
pipeline, and it's done automatically if needed [1].
As far as I have checked, other frameworks (FFmpeg, GTK+), when import or export
images from/to Vulkan, use exclusively UNORM formats, while SRGB formats are
ignored.
My conclusion is that Vulkan formats are related on how bits are stored in
memory rather their transfer functions (colorimetry).
This patch does two interrelated changes:
1. It swaps certain color format maps to try first, in both
gst_vulkan_format_from_video_info() and gst_vulkan_format_from_video_info_2(),
the UNORM formats, when comparing its usage, and later check for SRGB.
2. It removes the code that check for colorimetry in
gst_vulkan_format_from_video_info_2(), since it not storage related.
1. https://community.khronos.org/t/noob-difference-between-unorm-and-srgb/106132/7
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6797>
This mirrors the behaviour in vp8enc / vp9enc and is generally more
useful than using any framerate from the caps as it provides some degree
of accuracy if the stream doesn't have timestamps perfectly according to
the framerate.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6891>
The encoder can specify the a preferred_output_delay value to get better throughput
performance. The higher delay may get better HW performance, but it may increases
the encoder and pipeline latency.
When the output queue length is smaller than preferred_output_delay, the encoder
will not block to waiting for the encoding output. It will continue to prepare and
send more commands to GPU, which may improve the encoder throughput performance.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4359>
If the muxer times out because of the latency deadline it can happen
that some pads have no caps yet. In that case skip creation of streams
for these pads and create updated section tables once the first buffer
arrives later.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6823>
This makes sure that for sparse streams (KLV, DVB subtitles, ...) the
muxer does not wait until the next buffer is available for them but
times out on the latency deadline and outputs data.
For non-live pipelines it will still be necessary for upstream to
correctly produce gap events for sparse streams.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6823>
The FORCE_KEYFRAME frame which has GST_VIDEO_CODEC_FRAME_FLAG_FORCE_KEYFRAME
bit set should be the sync point. So we should let it be an IDR frame to begin
a new GOP, rather than just promote it to an I frame.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6619>
The FORCE_KEYFRAME frame which has GST_VIDEO_CODEC_FRAME_FLAG_FORCE_KEYFRAME
bit set should be the sync point. So we should let it be an IDR frame to begin
a new GOP, rather than just promote it to an I frame.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6619>
Set the P and B frame qp to I frame value to avoid generating delta
QP between different frame types. For ICQ and QVBR modes, we can
only set the qpi value, so the qpp and qpb values should be set to
the same value as the qpi.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6841>
Address below message reported by SDK debug layer.
ID3D12Device::CheckFeatureSupport: Unsupported Decode Profile Specified.
Use ID3D12VideoDevice::CheckFeatureSupport with D3D12_FEATURE_VIDEO_DECODE_PROFILES
to retrieve a list of supported profiles
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6839>
Certain V4L2 fourccs don't (yet) have DRM counter parts, in which case
we can't create DMA_DRM caps for them. This is usually the case for
specific tilings, which are represented as modifiers for DMA formats.
While using these tilings is generally preferable - because of e.g.
lower memory usage - it can result in additional conversion steps when
interacting with DMA based APIs such as GL, Vulkan or KMS. In such cases
using a DMA compatible format usually ends up being the better option.
Before the addition of DMA_DRM caps, this was what playbin3 ended up
requesting in various cases - e.g. prefering NV12 over NV12_4L4 - but
the addition of DMA_DRM caps seems to confuse the selection logic.
As a simple and quite robust solution, assume that peers supporting
DMA_DRM caps always prefer these and reorder the caps accordingly.
In the future we plan to have a translation layer for cases where
there is a matching fourcc+modifier pair for a V4L2 fourcc, ensuring
optimal results.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6645>
Doing it in gst_play_new() means that bindings that directly call
g_object_new() with the GType wouldn't end up initializing both.
This affects at least the Python and GJS bindings.
gst_init() is nonetheless only called from gst_play_new() once because
calling it from class_init would likely lead to problems as that's
called from somewhere in the middle of GObject.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6801>
Don't add an extra ref if non-floating as that ref will never be
unreffed.
gst_bin_add() is transfer floating (alias to transfer none).
Fixes a leak when a non-floating ref was provided as a return value in
the request-aux-sender signal.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6807>
Use of VK_PIPELINE_STAGE_2_ALL_COMMANDS_BIT instead of
specific VK_PIPELINE_STAGE_2_VIDEO_DECODE_BIT_KHR
Fix for VUID-vkCmdPipelineBarrier2-srcStageMask-03849
pDependencyInfo->pImageMemoryBarriers[0].srcStageMask
(VK_PIPELINE_STAGE_2_VIDEO_DECODE_BIT_KHR) is not compatible with
the queue family properties
(VK_QUEUE_GRAPHICS_BIT|VK_QUEUE_COMPUTE_BIT|VK_QUEUE_TRANSFER_BIT|
VK_QUEUE_SPARSE_BINDING_BIT|VK_QUEUE_PROTECTED_BIT) of this
command buffer. The Vulkan spec states: The srcStageMask member
of any element of the pMemoryBarriers, pBufferMemoryBarriers, or
pImageMemoryBarriers members of pDependencyInfo must only
include pipeline stages valid for the queue family that was
used to create the command pool that commandBuffer was allocated
from (
https://www.khronos.org/registry/vulkan/specs/1.3-extensions/
html/vkspec.html#VUID-vkCmdPipelineBarrier2-srcStageMask-03849)
The frame barrier should use a compatible srcStageMask for all
the queues.
Remove reset_pipeline_stage_mask as it is redundant
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6780>
Add pitch tests with different forward and backward playback rates.
Those tests depend on the libSoundTouch version to validate the buffers
checksums. The actual version uses libSoundTouch 2.3.2, use the
`--force-fallback-for=soundtouch` meson option to build using the same
version.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6247>
- fully protect accesses to the libsoundtouch API that is not
thread-safe.
- fully protect accesses to GstPitch members that could be read by a
downstream query thread while written by an upstream streaming thread
or a user thread.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6247>
- use the `GST_PITCH_GET_PRIVATE` accessor when needed
- rename `out_seg_rate` to `output_rate` to use the same name as the parameter
- rename `seg_arate` to `segment_applied_rate` to improve readability
- apply gst-indent to gstpitch.hh/cc
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6247>
When changing playing rate, the output segment was not correctly
calculated because the stream time ratio was computed using the previous
input segment rate instead of using the actual rate. This was producing
wrong results for the output segment start and end values.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6247>
The h26xdecoder 'stop' method was not called
as the vulkan h26x class rewires the video decoder
'stop' base method to its own one.
It was causing some memory leaks such as dangling parser
and dpb in h26xdecoder base class.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6782>
1. The PTS of all frames should not be changed.
2. Just update the DTS based on the PTS. For the frame which is not
reordered, the DTS is equal to PTS. For frame which is reordered,
the DTS is equal to previous DTS. For example:
Input: F0[D0, P0] -- F1[D1, P1] -- F2[D2, P2] -- F3[D3, P3]
Output: F0[I, D0, P0] -- F3[P, D0, P3] -- F1[B, D1, P1] -- F2[B, D2, P2]
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6703>
1. The PTS of all frames should not be changed.
2. Just update the DTS based on the PTS. For the frame which is not
reordered, the DTS is equal to PTS. For frame which is reordered,
the DTS is equal to previous DTS. For example:
Input: F0[D0, P0] -- F1[D1, P1] -- F2[D2, P2] -- F3[D3, P3]
Output: F0[I, D0, P0] -- F3[P, D0, P3] -- F1[B, D1, P1] -- F2[B, D2, P2]
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6703>
Fixing deadlock in below case
* GC lock is taken by background thread, and the background thread calls
gst_d3d12_ipc_client_release_imported_data() which takes ipc lock
* ipc lock is already taken in ipc thread and trying to pushing GC data
via gst_d3d12_command_queue_set_notify()
* gst_d3d12_command_queue_set_notify() is trying to take GC lock
but it's already taken by background thread
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6749>
Ideally, GPU waiting should be scheduled just before executing command list.
But handling the case outside of converter is a bit complicated.
Under an assumption that constructed command list will be executed
immediately, schedules GPU-side waiting inside of conversion method
to simplify the flow.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6749>
Since we don't want to expose video decoding API outside of GStreamer, the
header is removed from installation and both source files are renamed as
-private.
The header must remain in gst-libs because is referred by GstVulkanQueue,
which's the decoder factory.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6723>
First it derived mapping was disabled for P010 formats, but also there's an
issue with interlaced frames.
It would be possible to disable derived mapping only for interlaced (H.264
decoder and vadeinterlace) but it would spread the hacks along the code. It's
simpler and contained to disable derived completely for Mesa <23.3
Fixes: #3450
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6729>
Instead of duplicating the GStreamer format to DRM fourcc mapping, this patch
uses the GstVideo library helpers. This duplicates the big O of looking for,
since the two lists are traversed, but it's less error prone.
Partially reverts commit 547f3e8622.
Fixes: #3354
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6731>
If waylandsink received buffer rate is high which causes frame
drop, the cached staged buffer will be replaced when next buffer
needs to be rendered and be freed after redraw. But there is
chance to get memory leak if ended without redraw. So need to
free staged buffer when do gst_wl_window_finalize().
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6670>
`rsvg_handle_get_dimensions()` and `rsvg_handle_render_cairo()` are
deprecated, and the replacement librsvg functions as specified in the
migration guide are `rsvg_handle_get_intrinsic_size_in_pixels()` and
`rsvg_handle_render_document()`.
However, those are not drop-in replacements, and actually have
breaking semantics for our use-case:
1. `intrinsic_size_in_pixels()` requires SVGs to have width+height or
the viewBox attribute, but `get_dimensions()` does not. It will
calculate the geometry based on element extents recursively.
2. `render_cairo()` simply renders the SVG at its intrinsic size on
the specified surface starting at the top-left, maintaining
whatever transformations have been applied to the cairo surface,
including distorted aspect ratio.
However, `render_document()` does not do that, it is specifically
for rendering at the specified aspect ratio inside the specified
viewport, and if you specify a viewPort that does not match the
aspect ratio of the SVG, librsvg will center it.
Matching the old behaviour with the new APIs is a lot of work for no
benefit. We'd be duplicating code that is already there in librsvg in
one case and undoing work that librsvg is doing in the other case.
The aspect ratio handling in this element is also kinda atrocious.
There is no option to scale the SVG while maintaining the aspect
ratio. Overall, element needs a rewrite.
Let's just disable deprecations. The API is not going anywhere.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6726>
There was an issue with this equality check, which was to figure out what to do
with PCR pids (whether they were part of the streams present or not) and whether
we ignore PCR or not.
Turns out ... we already took care of that further up in the function.
The length check can be simplified by just checking whether the length of
the *original* PMT and the new PMT are identical. Since we don't store "magic"
PCR streams in those, we can just use them as-is.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6713>
A DPB buffer held by codec picture object may not be writable
at the moment, then gst_buffer_make_writable() will unref passed buffer.
Specifically, the use after free or double free can happen if:
* Crop meta of buffer copy is required because of non-zero
top-left crop position
* zero-copy is possible with crop meta
* A picture was duplicated, interlaced h264 stream for example
Interlaced h264 stream with non-zero top-left crop position
is not very common but it's possible configuration in theory.
Thus gst_buffer_make_writable() should be called with
GstVideoCodecFrame.output_buffer directly.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6706>
A DPB buffer held by codec picture object may not be writable
at the moment, then gst_buffer_make_writable() will unref passed buffer.
Specifically, the use after free or double free can happen if:
* Crop meta of buffer copy is required because of non-zero
top-left crop position
* zero-copy is possible with crop meta
* A picture was duplicated, interlaced h264 stream for example
Interlaced h264 stream with non-zero top-left crop position
is not very common but it's possible configuration in theory.
Thus gst_buffer_make_writable() should be called with
GstVideoCodecFrame.output_buffer directly.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6706>
The goal of this code was, for programs which were updates (i.e. adding/removing
streams but not completely changing) to allow dynamic addition/removal of
streams without completely removing everything.
But this wasn't 100% tested and there are a bunch of issues which make it fail
in plenty of ways.
For now disable that feature and force the legacy "add all pads again and then
remove old ones" behaviour to make it switch.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6651>
In addition to device removed status monitoring in gst_d3d11_result()
method, if ID3D11Device4 interface is available,
an event handle will be used for device removed status update.
And "device-removed" signal is removed since applications can monitor
the device removed status via gobject notify
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6699>
Adding new property in order to notify users of device removed status.
Once device removed status is detected, application should release
all ID3D12Device objects corresponding to the adapter, including
GstD3D12Device object. Otherwise D3D12CreateDevice() call for the
adapter will fail.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6699>
It seems that when D3D11CreateDevice collides in time
with other D3D11 calls, in particular the proccess of
creating a shader, it can corrupt the memory in the driver.
D3D11 spec doesn't seem to require any thread safety from
D3D11CreateDevice. Following MSDN, it is supposed to be called
in the beginning of the proccess, while GStreamer calls it with each
new pipeline.
Such crashes in the driver were frequently reproducing on the
Intel UHD 630 machine.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6686>
