... based on MediaFoundation work queue API.
By this commit, wasapi2 plugin will make use of pull mode scheduling
with audioringbuffer subclass.
There are several drawbacks of audiosrc/audiosink subclassing
(not audiobasesrc/audiobasesink) for WASAPI API, which are:
* audiosrc/audiosink classes try to set high priority to
read/write thread via MMCSS (Multimedia Class Scheduler Service)
but it's not allowed in case of UWP application.
In order to use MMCSS in UWP, application should use MediaFoundation
work queue indirectly.
Since audiosrc/audiosink scheduling model is not compatible with
MediaFoundation's work queue model, audioringbuffer subclassing
is required.
* WASAPI capture device might report larger packet size than expected
(i.e., larger frames we can read than expected frame size per period).
Meanwhile, in any case, application should drain all packets at that moment.
In order to handle the case, wasapi/wasapi2 plugins were making use of
GstAdapter which is obviously sub-optimal because it requires additional
memory allocation and copy.
By implementing audioringbuffer subclassing, we can avoid such inefficiency.
In this commit, all the device read/write operations will be moved
to newly implemented wasapi2ringbuffer class and
existing wasapi2client class will take care of device enumeration
and activation parts only.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2306>
This add src format validation, this avoid registering element for
drivers we don't support any of their src formats. This also special
case the AlphaDecodeBin wrapper, as we know that alphacombine element
only support I420 and NV12 for now.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2272>
codecalpha is a new plugin introduced to support VP8/VP9 alpha as
defined in the WebM and Matroska specifications. It splits the stream
into two streams, one for the alpha and one for the actual content,
then it decodes them separately with vpxdec and finally combine the
results as A420 or AV12 (i.e. YUV + an extra alpha plane).
The workflow above is setup by means of a bin, gstcodecalphabin.
This patch simulates the same workflow into the v4l2codecs namespace,
thus using the new v4l2 stateless decoders for hardware acceleration.
This is so we can register the new alpha decode elements only if the
hardware produces formats we support, i.e. I420 or NV12 for now.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2272>
We already declare the support of HEVC screen content extension profiles
in the profile mapping list, but we fail to generate the correct VA picture
parameters buffers. This may cause the GPU hang.
We need to fill the buffer of VAPictureParameterBufferHEVCExtension correctly.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2255>
The function of gst_h265_get_profile_from_sps() is better than the
function gst_h265_profile_tier_level_get_profile() when we recognize
the profile of the stream, becaue it considers the compatibility.
It is also used by h265parse to recognize the profile. So it is
better to keep the same behaviour with the parse and other decoders.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2255>
We already declare the support of HEVC range extension profiles in
the profile mapping list, but we fail to generate the correct VA
picture and slice parameters buffers. This may cause the GPU hang.
We need to fill the buffer of VAPictureParameterBufferHEVCExtension
and VASliceParameterBufferHEVCExtension correctly.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2255>
VA-API HEVC decoding needs to known which is the last slice of a
picture, but slices are processed sequencially, so we know the
last slice until all the slices are already pushed into the
VABuffer array.
In order to mark the last slice, they are pushed into the
VABuffer array with a delay of one slice: the first slice is
hold, and when the second slice come, the first one is pushed
while holding the second, and so on. Finally, at end_picture(),
the last slice is marked and pushed into the array.
Co-author: Victor Jaquez <vjaquez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2246>
The VA acceleration now has more usages in linux-like platforms,
such as the MSDK. The different plugins based on the VA acceleration
need to share some common logic and types. We now move the display
related functions and types into a common va lib.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2196>
The VP9 related definitions in mfxvp9.h are available under the
condition of 'MFX_VERSION >= MFX_VERSION_NEXT', which implies that these
definitions are never used in a public release.
This is in preparation for oneVPL support because mfxvp9.h was
removed from oneVPL
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1503>
We have only one copy of gst_va_base_dec_parent_class inside the
vabasedec, so it can not handle the case when there are multi va
decoders inside one pipeline. The pipeline:
gst-launch-1.0 filesrc location=xxx.h264 ! h264parse \
! vah264dec ! msdkh265enc ! vah265dec ! fakesink
generates a assertion of
"invalid cast from 'GstVaH264Dec' to 'GstH265Decoder"
and gets a crash.
We should keep the parent_class for each decoder type.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2231>
VADecPictureParameterBufferAV1.mode_control_fields.bits were filled
twice, overwriting to zeros the first assignation. This patch unifies
both assignations.
Also it makes explicit an enum casting between libva and gstreamer; it
removes the assignation to zero a deprecated parameter; and use an
appropriate assertion.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2223>
Before creating output duplication interface, call SetThreadDesktop()
with HDESK of the current input desktop in case a desktop switch has
occurred.
This allows d3d11desktopdupsrc to capture Windows User Account Control
(UAC) prompts, which appear on a separate secure desktop. Otherwise
IDXGIOutput1::DuplicateOutput() will return E_ACCESSDENIED and the
element won't produce any frames as long as the UAC screen is active.
Note that in order to access secure desktop the application still has to
run at LOCAL_SYSTEM privileges. For GStreamer applications running with
regular user privileges this change has no effect.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2209>
IDXGIOutputDuplication can become invalid for example when there's
desktop switch, resolution change or Windows User Account Control prompt
appears on screen.
When that happens, try to re-create the duplication interface for the
changed output. Note that in the case of UAC prompt this operation will
fail if the GStreamer process doesn't run at LOCAL_SYSTEM privileges. In
such situation the source element won't create any frames as long as the
output is occupied by UAC screen.
In order to enable UAC access to sufficiently privileged GStreamer
processes, call SetThreadDesktop() with the desktop handle that
currently receives user input before creating our output duplication.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2204>
The problem is for uploading YUV frames using derived images, is that
derived images imply tiling, so frames are wrongly uploaded.
Though derived for reading might work we cannot know the Intel graphics
generation to validate the caching. Overall, it's safer to disable derived
images for i965.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2127>
iHD driver sets a tiled DRM modifier if surface's usage hint is set to
VPP_WRITE. This result in a garbled rendering when using glimagesink.
This patch changes the usage hint to generic if the caps feature is
DMABuf. Either way only iHD driver, so far, uses the usage hint flag.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2127>
Some capture devices might not provide channel mask value which will
result in capturing failure because of unknown channel mask in case
that device generates more than 2 channels. Although it might not
be correct, we can assume channel mask with the given number of channels.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2177>
DMABuf allocator already implements DMABuf Sync, meaning that doing
mmap/munmap (unless the mode have changed) is not required. In fact, on
systems with IOMMU it makes the kernel redo the mmu table which is visible
in the CPU usage.
This change reduces CPU usage when decoding
bbb_sunflower_2160p_60fps_normal.mp4 on RK3399 SoC from over 30% to
around 15%.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2152>
Implementation of mem_copy() virtual method for GstVaAllocator.
It's a deep copy where a new VA memory is popped out from the pool or,
if pool is empty, a new memory is allocated. The original memory is
mapped to read, and if its VAImage is not derived and size to copy is
the whole surface, the mapped VAImage of the original memory is put in
the new memory. Otherwise a slow memcpy is done between both memories.
Fixes: #1568
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2136>
The picture buffer (V4L2 CAPTURE buffer) was being released immediatly
when the request was done. This was problematic since even after the
request is done, the picture buffer might still be used as a reference
and should not be reused for further decoding yet.
This change effectively bind the picture buffer lifetime to the request.
So that if the picture is never showned (decode only frame) or the request
queue is full before the buffer is displayed, the picture buffer will
remain alive.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2142>
Different VA drives might have different definitions for RGB32 color
formats because different bit interpretation. Sadly the specification
doesn't clarify these interpretations. So VA users have to figure out
what's the correct mapping with it's rendering color format
definition.
This patch aims to fix the static map structure after the
VAImageFormats are queried. There is another static map with the
different interpretations of the RGB32 formats, and compare them with
the given VAImageFormat, then with the GStreamer color format, update
the mapping table.
Finally, some RGB32 color formats were added.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2129>
Add missing include for sys/ioctl.h so that these warnings dissapear
when compiling:
../subprojects/gst-plugins-bad/sys/v4l2codecs/gstv4l2decoder.c:179:9:
warning: implicit declaration of function ‘ioctl’
[-Wimplicit-function-declaration]
Signed-off-by: Daniel Almeida <daniel.almeida@collabora.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2140>
If the application did not define one yet, define our own
VK_DEFINE_NON_DISPATCHABLE_HANDLE that is independent of the
architecture.
Vulkan, by default, provides a define that depends on the architecture,
which causes the symbol type to be different. This causes an
architecture dependent .gir file, which then causes multilib
installation problems because the .gir files can't be shared.
Make it possible to override the format specifier and provide
a default one that is compatible with the default non dispatchable
handle.
Return VK_NULL_HANDLE from functions that return a non-dispatchable
handle.
Fixes#1566
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2130>
vapostproc tries to be in passthrough mode as much as possible. But
they might be situations where the user might force to process the
frames. For example, when upstream sets the crop meta and the user
wants VA do that cropping, rather than downstream.
For those situations this property will disable the passthrough mode,
if it's enabled.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2058>
If incoming buffers have crop meta it's done by vapostproc, iif
vapostproc is not in passthrough mode and downstream doesn't handle
it.
This patch announces the crop meta API in proposed bufferpool, while
it stops filtering meta APIs, since it was only filter crop api.
Also if downstream supports crop and video metas, vapostporoc
announces both meta APIs in upstream bufferpool.
Finally, the meta is removed from the buffer if the crop is enabled.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2058>
This new struct describes the input and output GstBuffers to
post-process, including VA flags. It also contains the VASurfaceID and
VARectangle, but those are private, completed inside GstVaFilter.
It is used for pass arguments to gst_va_filter_convert_surface() function.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2058>
When an input buffer needs to be copied into a VA memory, it's
required to create a buffer pool. This patch uses the
propose_allocation() caps to instantiate the allocator and pool,
instead of the negotiated caps, which rather represents the resolution
to display.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2058>
Derived images are direct maps to surfaces bits, but in Intel Gen7 to
Gen9, that memory is not cachable, thus reading can be very slow (it
might produce timeout is tests such as fluster).
This patch tries first to define if derived images are possible, and
later use them only if mapping is not for reading.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2128>
This plugin, for decoders more concretely, assumes that a VA config
can do certain color conversions when mapping frames onto CPU's
memory.
This assumption was valid for i965 and Gallium drivers which generates
valid outputs in bitstreams testers (v.gr. fluster). Nonetheless, iHD,
even when it generates acceptable rendered frames, output's MD5 of
tests weren't valid.
This patch append the image formats, for color conversion when mapping
to memory, for non-iHD drivers.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2128>
Enable zero-copy if downstream proposed pool and therefore decoder
can know the amount of buffer required by downstream.
Otherwise decoder will copy when our DPB pool has no sufficient
buffers for later decoding operation.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2097>
