In some occasions, a buffer pool is created for pre-initialization
purposes regardless of whether a valid image size is available or
not. However, during actual decode stage, the vaapidecode element
is expected to update the srcpad caps with the new dimensions, thus
also triggering a reset of the underlying bufferpool.
The ensure_surface() and ensure_image() functions shall only relate
to the underlying backing store. The actual current flags are to be
updated only through ensure_{surface,image}_is_current() or very other
particular cases in GstMemory hooks.
If the surface proxy is updated into the GstVaapiVideoMemory, then
it is assumed it is the most current representation of the current
video frame. Likewise, make a few more arrangements to have the
"current " flags set more consistently.
When interacting with SW elements, the buffers and underlying video
memory could be mapped as read/write. However, we need to use those
buffers again as plain VA surfaces, we have to make sure the VA image
is thus committed back to VA surface memory.
This fixes pipelines involving avdec_* and vaapi{postproc,sink}.
Hence vaapisink can display buffers decoded by gst-libav, or HW decoded
buffers can be further processed in-place, e.g. with a textoverlay.
https://bugzilla.gnome.org/show_bug.cgi?id=704078
[ported to current git master branch, amended commit message]
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Add GST_VAAPI_VIDEO_BUFFER_POOL_ACQUIRE_FLAG_NO_ALLOC params flag that
can be used to disable early allocations of vaapi video metas on buffers,
thus delagating that to the bufferpool user.
Make sure to always prefer native internal formats for the VA surfaces
that get allocated. Also disable "direct-rendering" mode in this case.
This is needed so that to make sure that anything that gets out of the
decoder, or anything that gets into the encoder, is in native format
for the hardware, and thus the driver doesn't need to perform implicit
conversions in there. Interop with SW elements is still available with
fast implementations of VA imaging APIs.
Forbid shares of GstMemory instances, and rather make copy of it. This
effectively copies the GstMemory structure and enclosed metadata, but
this does not copy the VA surface contents itself. It should though.
This fixes preroll and makes sure to not download garbage for the first
frame when a SW rendering sink is used.
Use an image pool to hold VA images to be used for downloads/uploads
of contents for the associated surface.
This is an optmization for size. So, instead of creating as many VA
images as there are buffers (then VA surfaces) allocated, we only
maintain a minimal set of live VA images, thus preserving memory
resources.
Disable read-write mappings if "direct-rendering" is not supported.
Since the ordering of read and write operations is not specified,
this would require to always download the VA surface on _map(), then
commit the temporary VA image back to the VA surface on _unmap().
Some SW decoding plug-in elements still use R/W mappings though.
https://bugzilla.gnome.org/show_bug.cgi?id=733242
Fix error messages introduced in the previous commit for the _map()
imaplementation. Also use the new get_image_data() helper function
to determine the base pixels data buffer from a GstVaapiImage when
updating the video info structure from it.
Allow raw pixels of the whole frame to be mapped read-only. i.e. in
cases where the buffer pool is allocated without VideoMeta API, thus
individual planes cannot be mapped.
This is initial support for Firefox >= 30.
https://bugzilla.gnome.org/show_bug.cgi?id=731886
While creating the vaapi video allocator, make sure the associated
surface pool has correct format instead of defaulting to NV12 video
format even though there is no direct rendering support.
https://bugzilla.gnome.org/show_bug.cgi?id=732691
Make sure to always update the VA surface pointer whenever the proxy
changes. This used to only work when the VA surface is written to, in
interop with SW element ("upload" feature), and this now fixes cases
when the VA surface is needed for reading, in interop with SW element
("download" feature).
Always expose I420 format by default when the VA surface could be
mapped for interoperability with non harware accelerated elements.
However, the default behaviour remains the auto-plugging of vaapi
elements, down to the sink.
Side effect: "direct-rendering" mode is also disabled most of the
times as plain memcpy() from uncached speculative write combining
memory is not going to be efficient enough.
Fix support for VA surface download capability in vaapidecode element
for GStreamer >= 1.2. This is a fix to supporting libva-vdpau-driver,
but also the libva-intel-driver while performing hardware accelerated
conversions from the native VA surface format (NV12) to the desired
output VA image format.
For instance, this fixes pipelines involving vaapidecode ! xvimagesink.
https://bugzilla.gnome.org/show_bug.cgi?id=733243
When GstVideoMeta maps were used, the supporting functions incorrectly
used gst_buffer_get_memory() instead of gst_buffer_peek_memory(), thus
always increasing the associated GstMemory reference count and giving
zero chance to actually release that, and subsequently the VA display.
Reset the buffer pool allocator only if the config caps changed in a
sensible way: format or resolution change. i.e. don't bother with
other caps like colorimetry et al. as this doesn't affect the way to
allocate VA surfaces or images.
Enable read and write mappings only if direct-rendering is supported.
Otherwise, this means that we may need to download data from the VA
surface first for correctness, even if the VA surface doesn't need to
be read at all. i.e. sometimes, READWRITE mappings are meant for
surfaces that are written to first, and read afterwards for further
processing.
https://bugzilla.gnome.org/show_bug.cgi?id=704078
Fix check for direct-rendering if the creation of VA surfaces with
an explicit pixel format is not support, e.g. VA-API < 0.34.0, and
that we tried to allocate a VA surface based on the corresponding
chroma type. i.e. in that particular case, we have to make sure that
the derived image has actually the expected format.
Fix GstVaapiVideoBufferPool::reset_buffer() to reset the underlying
memory resources, and more particularly the VA surface proxy. Most
importantly, the GstVaapiVideoMeta is retained. Cached surface in
memory are released, thus triggering a new allocation the next time
we need to map the buffer.
Make sure GstVaapiVideoMemory allocates VA surface proxies from a
pool stored in the parent VA memory allocator.
This fixes the following scenario:
- VA video buffer 1 is allocated from a buffer pool
- Another video buffer is created, and inherits info from buffer 1
- Buffer 1 is released, thus pushing it back to the buffer pool
- New buffer alloc request comes it, this yields buffer 1 back
- At this stage, buffers 1 and 2 still share the same underlying VA
surface, but buffer 2 was already submitted downstream for further
processing, thus conflicting with additional processing we were
about to perform on buffer 1.
Maybe the core GstBufferPool implementation should have been fixed
instead to actually make sure that the returned GstBuffer memory we
found from the pool is writable?
Fix reference counting issue whereby gst_memory_init() does not hold
an extra reference to the GstAllocator. So, there could be situations
where the last instance of GstVaapiVideoAllocator gets released before
a dangling GstVaapiVideoMemory object, thus possibly leading to a crash.
Adapt GstVaapiVideoMemory allocator to support creation of VA surfaces
with an explicit pixel format. This allows for direct rendering to
VA surface memory from a software decoder.
Allow plain gst_buffer_map() interface to work with gstreamer-vaapi
video buffers, i.e. expose the underlying GstVaapiSurfaceProxy to the
caller. This is the only sensible enough thing to do in this mode as
the underlying surface pixels need to be extracted through an explicit
call to the gst_video_frame_map() function instead.
A possible use-case of this is to implement a "handoff" signal handler
to fakesink or identity element for further processing.
Fix gst_vaapi_video_allocator_new() to silently check for direct-rendering
mode support, and not trigger fatal-criticals if either test surface or
image could not be created. Typical case: pixel format mismatch, e.g. NV12
supported by most hardware vs. I420 supported by most software decoders.
On map, ensure we have GST_MAP_WRITE flags since this is only what we
support for now. Likewise, on unmap, make sure that the VA image is
unmapped for either read or write, while still committing it to the
VA surface if write was requested.
Implement GstVideoMeta::{,un}map() to support raw YUV buffer upload when
the last component is unmapped. Downloads are not supported yet. The aim
was to first support SW decoding + HW accelerated rendering (vaapisink).
e.g. for Wayland.
Add initial support for GstVaapiVideoMemory backed buffer pool. The memory
object currently holds a reference to GstVaapiVideoMeta.
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
