There were duplicated code in gst_video_meta_unmap_vaapi_memory() and
gst_vaapi_video_memory_unmap() when unmapping.
This patch refactors both methods adding the common function
unmap_vaapi_memory(). This also ensures, if direct rendering is enabled, it
is correctly reset.
Additionally, only when mapping flag has the WRITE bit, it set the image as
current, which was done in gst_video_meta_map_vaapi_memory() but no in
gst_vaapi_video_memory_map().
In order to make this, the mapping flags were required, so instead of
overloading mem_unmap() virtual function, mem_unmap_full() is overloaded.
https://bugzilla.gnome.org/show_bug.cgi?id=774213
There were duplicated code in gst_video_meta_map_vaapi_memory() and
gst_vaapi_video_memory_map() when doing the READ and WRITE mapping.
This patch refactors both methods adding the common function
map_vaapi_memory().
Additionally, only when flag has the READ bit it calls
ensure_images_is_current(), which was done in
gst_video_meta_map_vaapi_memory() but no in
gst_vaapi_video_memory_map().
https://bugzilla.gnome.org/show_bug.cgi?id=772151
The variable member allowed_sinkpad_caps is constructed querying the
current VA display. Bearing that in mind, the variable shall be freed
when the VA display changes or is removed.
This patch moves the freeing of allowed_sinkpad_caps to close(), when
the VA display is freed.
The thread that handles window's events should be finished during
pipeline's shutdown, otherwise it will remain alive during pipeline
re-activation, leading to unexpected problems.
This patch fixes failures of intensive_state_change scenario of
gst-validate
https://bugzilla.gnome.org/show_bug.cgi?id=774241
Returning GST_FLOW_ERROR always when gst_pad_push fails might lead to
deadlock during seek.
This patch returns the same error of gst_pad_push() and log out the
return value.
https://bugzilla.gnome.org/show_bug.cgi?id=774030
In commit 6d11a00 were introduced a regression when gstreamer-vaapi is
compiled with out EGL/GLX support: it shall not support
GST_VAAPI_MAKE_GLTEXUPLOAD_CAPS.
This patch guards the inclusion of GST_VAAPI_MAKE_GLTEXUPLOAD_CAPS in the
allowed src caps for vaapedecode if EGL/GLX.
Previously the frame map counter increased independently if the map succeeded
or not. This leaded to critical messages and crashes if the frame was unable
to be mapped, but the counter increased.
This patch increases the map counter only if the map operation occurred.
https://bugzilla.gnome.org/show_bug.cgi?id=773939
When the allocator is created, it stores the allocation caps. But sometimes
the "allocation caps" may be different from the "negotiation caps".
In this case, the allocator should store the negotiation caps since they
are the ones used for frame mapping with GstVideoMeta.
When vaapispostproc is used, this is not a problem since the element is assume
to resize. But when using a vaapi decoder only, with a software renderer, it
fails in this case.
https://bugzilla.gnome.org/show_bug.cgi?id=773323
Just as vaapipostproc, VA decoder's context can be queried to get the possible
raw formats, so, the src caps can negotiate the exact caps that the context
supports.
Since when the sink pad allocator is created, it is decided if the required
one is vaapi allocator or dmabuf allocator, there is no need to force its set
again.
When running gst-discoverer-1.0, in certain media, vaapipostroc is stopped
meanwhile it is transforming caps. The problem is that stop() calls
gst_vaapi_plugin_base_close(), which nullifies the element's va display, but
the va display is used in tranform_caps() when it is extracting the possible
format conversions. This display disappearing generates warning messages.
This patch holds a local reference of va display at ensure_allowed_raw_caps()
hence it doesn't go away meanwhile it is used, even if the
gst_vaapi_plugin_base_close() is called in other thread.
https://bugzilla.gnome.org/show_bug.cgi?id=773593
Enable the direct rendering with linear surfaces if the negotiated src caps
are video/x-raw without features.
Pass also the caps, since they are needed to know the requested caps features.
Just as we did in ensure_sinkpad_allocator(), let's move the error message
into the ensure_srcpad_allocator() from the caller,
gst_vaapi_plugin_base_decide_allocation()
Instead of receiving the GstVideoInfo structure as parameter, get the original
GstCaps from ensure_sinkpad_buffer_pool(), in this way we could decide better
which allocator instantiate.
If the allocator was configured to use direct upload or rendering, the
generated derived image created at mapping needs to be destroyed after
unmapping, because, in order to process the surface, it should not be marked
as "busy" by the driver.
The way to experiment with the direct rendering is through and internal
compiler pre-processor flag.
The current change set enables a way to specified at run-time, as a flag
passed to the allocator at instanciation time.
This patch is to change the inheritance of GstVaapiDisplay to GstObject,
instead of GstVaapiMiniObject. In this way we can use all the available
infrastructure for GObject/GstObject such as GstTracer, GIR, etc.
In addition, a new debug category for GstVaapiDisplay is created to make it
easier to trace debug messages. It is named "vaapidisplay" and it transverse
all the VA display backends (DRM, GLX, EGL, Wayland, ...)
This patch is a step forward to expose GstVaapiDisplay for users in a future
library.
https://bugzilla.gnome.org/show_bug.cgi?id=768266
Signed-off-by: Víctor Manuel Jáquez Leal <victorx.jaquez@intel.com>
Since vaapipostproc is only registered if the driver supports it, all the
support for dynamic loading were removed. Though some leftovers remained.
https://bugzilla.gnome.org/show_bug.cgi?id=773589
In the spirit of the codec split, this patch removes the documentation of
vaapidecode and adds a page per each possible decoder.
Nonetheless, only those available in the compilation system are going to be
instrospected, because the rest are not registered.
vaapidecode has a member named allowed_caps, but this name is not enough
explicit. This patch renames allowed_caps to allowed_sinkpad_caps.
No functional changes were included.
https://bugzilla.gnome.org/show_bug.cgi?id=773497
In order to clarify the use of flag as input parameter, it is renamed to
surface_alloc_flag, since it is used when creating a VA surface with certain
properties.
https://bugzilla.gnome.org/show_bug.cgi?id=773497
When caps reconfiguration is called, the new downstream frame size might be
different. Thus, if the downstream caps change,the display's texture map is
reset.
In addition, during pipeline shutdown, textures in texture map have to be
released, since each one have a reference to the GstVaapiDisplay object, which
is a dangerous circular reference.
https://bugzilla.gnome.org/show_bug.cgi?id=769293
Signed-off-by: Víctor Manuel Jáquez Leal <victorx.jaquez@intel.com>
instances when created and reuse
This patch improves performance when glimagesink uploads a GL texture.
It caches the GStVaapiTexture instances in GstVaapiDisplay{GLX,EGL}, using an
instance of GstVaapiTextureMap, so our internal texture structure can be found
by matching the GL texture id for each frame upload process, avoiding the
internal texture structure creation and its following destruction.
https://bugzilla.gnome.org/show_bug.cgi?id=769293
Signed-off-by: Víctor Manuel Jáquez Leal <victorx.jaquez@intel.com>
set_format() is called by upstream when the stream capabilites has changed.
Before, if the new stream is compatible with the old one the VA decoder was
not destroyed. Nonetheless, with this behavoir, the VA decoder ignores
when the upstreamer parsers gets more details of the stream, such as the
framerate. Hence, when the src caps are negotiates, the further sink caps
updates are ignored.
This patch forces the VA decoder destroying and recreation when set_format()
is called.
https://bugzilla.gnome.org/show_bug.cgi?id=770921
When calling gst_video_frame_map() with GST_MAP_WRITE flag, it doesn't call
ensure_image_is_current(), which means it doesn't guarentee VAImage is valid
in this case.
https://bugzilla.gnome.org/show_bug.cgi?id=766978
In a multiple video cards system, a X11 environment may have different VA
capabilities. This patch tracks the DISPLAY environment variable to
invalidates the GStreamer features cache. Also tracks WAYLAND_DISPLAY.
https://bugzilla.gnome.org/show_bug.cgi?id=770357
Encoders claim to support a whole bunch of input formats but then
just error out if the format is not actually supported, even if
there's a converter in front. This means they're not fit for
autoplugging in encodebin or camerabin yet and therefore should
not have a rank. People can still use them in custom pipelines.
https://bugzilla.gnome.org/show_bug.cgi?id=769266
Check earlier if upstream video source has activated the dmabuf-import
io-mode (hack to disappear soon), thus we can avoid the re-assignation of a
new allocator.
Add the method gst_allocator_get_vaapi_image_size() for the
GstVaapiVideoAllocator, which gets the size of the allocated images with the
current video info.
This method replaces the direct call to the allocator's image info when the
pool is configured.
Depends on media, video size is sometimes updated with new allocator.
It leads to dismatch between bufferpool's set size and real allocated buffer size.
In this case, it causes every buffer is freed during release in bufferpool,
which should be reused. This affects performance.
https://bugzilla.gnome.org/show_bug.cgi?id=769248
In order to register only the available decoders, this patch queries the
created test VA display, which uses the currently used back-end (X11, Wayland,
DRM, …) on the used display device.
https://bugzilla.gnome.org/show_bug.cgi?id=724352
In order to register only the available encoders, this patch queries the
created test VA display, which uses the currently used back-end (X11,
Wayland, DRM, …) on the used display device.
https://bugzilla.gnome.org/show_bug.cgi?id=724352
Split the vaapidecode to all the supported codecs with the format
vaapi{codec}dec.
vaapidecode is stil registered as a GObject type, but not as a
GStreamer feature, so it can be used internally by vaapidecodebin without
changing its code too much.
https://bugzilla.gnome.org/show_bug.cgi?id=734093
Since the elements dependant of the VA video processor are now only registered
if it is available, vaapidecodebin code can be simplified a lot, removing all
the code required to check if the VA video processor was available.
https://bugzilla.gnome.org/show_bug.cgi?id=768899
Delay the GstVaapiDisplay instantiating until when changing the state from
READY to PAUSE. In this way the element has more chances to find an already
created GstVaapiDisplay, or a GL context, in the pipeline.
https://bugzilla.gnome.org/show_bug.cgi?id=766206
The function gst_vaapi_create_display_from_gl_context() cretes a
GstVaapiDisplay given a GstGLContext. But it didn't created a GLX VA display
when the GL platform was GLX, but a plain X11 VA display.
This patch fixes that, by querying the GL platform earlier.
https://bugzilla.gnome.org/show_bug.cgi?id=766206
Using the GstContext mechanism, it is possible to find if the pipeline
shares a GstGLContext, even if we are not to negotiating GLTextureUpload
meta. This is interesting because we could negotiate system memory caps
feature, but enable DMABuf if the GstGLContext is EGL with some extensions.
https://bugzilla.gnome.org/show_bug.cgi?id=766206
Since the driver checkup is done at registering, there is no need to do it
when changing the element state from NULL to READY. This patch remove this
vmethod from vaapidecode.
Query the GstVaapiDisplay to know if the driver supports video
postprocessing. If does, then register vaapipostproc and vaapidecodebin
elements.
This patch will simplify the design of vaapidecodebin.
https://bugzilla.gnome.org/show_bug.cgi?id=724352
Move some of the logic in gst_vaapi_plugin_base_driver_is_whitelisted() to a
new function gst_vaapi_driver_is_whitelisted(), in this way, it can be used
when registering the plugin's feature set with the test VA display.
https://bugzilla.gnome.org/show_bug.cgi?id=724352
This patch tries to instantiate a GstVaapiDisplay when registering the plugin
features, if it fails, no gstreamer-vaapi element is registering.
The purpose of this patch is to avoid a situation where the user has
gstreamer-vaapi installed but their VA-API setup is not functional, which may
lead to unexpected behavior.
https://bugzilla.gnome.org/show_bug.cgi?id=724352
There are two main external dependencies that define the feature set of this
plugin: a) the kernel and b) the VA driver
This patch tracks both dependencies, if any of them change, GStreamer will
re-inspect the plugin.
The kernel is tracked through the device files /dev/dri/card*
The VA driver is tracked through the files VA_DRIVERS_PATH/*_drv_video.so,
where VA_DRIVERS_PATH is the one defined in libva package configuration. Also,
the environment variables LIBVA_DRIVERS_PATH and LIBVA_DRIVER_NAME are tracked
since they modify the driver lookup.
Additionally, the environment variable GST_VAAPI_ALL_DRIVERS is tracked too.
https://bugzilla.gnome.org/show_bug.cgi?id=724352
This is a fix for a regression of previous commit, which updates the filters
only when the property is set, because it is also required to update the
filter when the color balance interface change its values.
In case that sink caps and src caps are same, and no filtering parameter set,
pass-through mode is enabled.
If new filtering parameter is set during playback, it makes it reconfiguring,
so that pass-through mode is changed
In addition, updating filter is performed during reconfiguration, if needed.
https://bugzilla.gnome.org/show_bug.cgi?id=751876
In order to handle correctly seek and other operations, vaapiencode should
flush all the remaining data from the encoder without pushing it downstream.
This patch implements the flush() vmethod, only after of pausing the
source pad task, and restarting it again after the flush stop.
https://bugzilla.gnome.org/show_bug.cgi?id=767176
Signed-off-by: Víctor Manuel Jáquez Leal <victorx.jaquez@intel.com>
To avoid surface-exhausted situation during reverse playback,
drop frames except for key frame.
Also, to avoid the corruption of the parser state, flush() vmethod
doesn't destroy the VA decoder when playing in reverse.
https://bugzilla.gnome.org/show_bug.cgi?id=742922
Signed-off-by: Víctor Manuel Jáquez Leal <victorx.jaquez@intel.com>
The queue in GstVaapiDecode adds an extra reference to the frames. This patch
unref that extra reference earlier making the code simpler to follow.
https://bugzilla.gnome.org/show_bug.cgi?id=768652
Calling drain() vmethod means "decode any data it can at this point, but that
more data may arrive after". Hence, vaapidecode should check if there is data
in the output adapter and process them, without destroying the decoded picture
buffer (dpb).
Since this operation is done by gst_vaapidecode_internal_flush(), the operation
was refactored into a new function gst_vaapidecode_flush_output_adapter().
https://bugzilla.gnome.org/show_bug.cgi?id=768652
Calling flush() vmethod means "to flush all remaining data from the decoder
without pushing it downstream".
Nonetheless flush() is calling gst_vaapidecode_internal_flush(), which calls
gst_video_decoder_have_frame() if there is still something in the input
adapter, which may push buffers to downstream by calling handle_frame().
This patch changes this behavior by calling gst_vaapidecode_purge() rather
than gst_vaapidecode_internal_flush(), which does what we want: flushes the VA
decoder and releases all the rest of decoded frames.
https://bugzilla.gnome.org/show_bug.cgi?id=768652
This patch is a fix for my bad review of commit 6d73ca8d. The element should
be able to return the available raw caps handled by the VA display, but that
only should happen when there a VA display. If there's none, the element
should use the caps template.
https://bugzilla.gnome.org/show_bug.cgi?id=768161
Though USE_{JPEG,VP8,VP9,H265}_ENCODER macros definition depend on USE_ENCODER
macro, it is clearer to nest them, showing explicitly the dependency relation.
Under certain conditions the element might receive a positive context query
but without a context instance. This situation will lead to a segmentation
fault when traversing the context list in the pipeline.
https://bugzilla.gnome.org/show_bug.cgi?id=767946
If vaapisink received a caps query before getting a VA display, it returned
only the surfaces related caps. This behavior broke the autovideosink
negotiation.
This patch returns the pad's template caps if no VA display, otherwise the
caps are crafted as before.
https://bugzilla.gnome.org/show_bug.cgi?id=767699
In cases where we know the video meta must be present, add it to
the pool configuration.
Signed-off-by: Scott D Phillips <scott.d.phillips@intel.com>
https://bugzilla.gnome.org/show_bug.cgi?id=766184
if gst_buffer_pool_set_config returns FALSE, check the modified
config and retry set_config if the config is still acceptable.
Signed-off-by: Scott D Phillips <scott.d.phillips@intel.com>
https://bugzilla.gnome.org/show_bug.cgi?id=766184
There's no need to check for the display in the plugin object when
decide_allocation() vmethod is called, because the display will created or
re-created along the method execution.
Get the pool config just before use it, to avoid a memory leak if the
allocator cannot be instantiated. Similarly, return FALSE if the configuration
cannot be set, avoid keep a not used allocator in the pool.
Instead of instantiating an allocator per vaapivideobufferpool, only one
allocator is instantiated per element's pad and shared among future pools.
If the pad's caps changes, the allocator is reset.
https://bugzilla.gnome.org/show_bug.cgi?id=765435
Originally, vaapivideobufferpool has a set of boolean variables for the
buffer configuration options.
This pach changes these boolean variables for a single bitwise, just as
it is used in pluginbase. Hence, the internal enum was moved to
vaapivideobufferpool header.
https://bugzilla.gnome.org/show_bug.cgi?id=765435
This patch refactors the code in pluginbase in order to centralize the buffer
pool instantiation. As the buffer pool config may have different options, these
are gathered using a bitwise flag.
https://bugzilla.gnome.org/show_bug.cgi?id=765435
Originally vaapivideobufferpool instantiates its own allocator regardless the
received configuration, and it relies in custom configuration options to
choose which kind of allocator instantiate.
This patch transfers the responsibility of the allocator instantiate to
vaapipluginbase and pass it to the vaapivideobufferpool through its
configuration.
* gst/vaapi/gstvaapipluginbase.c
+ set_dmabuf_allocator(): inserts a dmabuf allocator in the bufferpool
+ ensure_sinkpad_buffer_pool(): set a normal vaapi video allocator in
bufferpool configuration
+ gst_vaapi_plugin_base_propose_allocation(): call set_dmabuf_allocator() if
needed.
+ gst_vaapi_plugin_base_decide_allocation(): set a normal vaapi video
allocator in bufferpool configuration
* gst/vaapi/gstvaapivideobufferpool.c
+ gst_vaapi_video_buffer_pool_set_config(): instead of instantiate the
allocator, process the received one through its configuration.
* gst/vaapi/gstvaapivideobufferpool.h: removed
GST_BUFFER_POOL_OPTION_DMABUF_MEMORY since it is not used anymore.
* gst/vaapi/gstvaapivideomemory.c
+ gst_vaapi_is_dmabuf_allocator(): new helper function to identify a dmabuf
allocator with the vaapi qdata.
https://bugzilla.gnome.org/show_bug.cgi?id=765435
Add a mutex to postproc to protect concurrent access to data members.
Previously set_caps() could release the allowed_srcpad_caps while
transform_caps was in the middle of using it.
Signed-off-by: Scott D Phillips <scott.d.phillips@intel.com>
https://bugzilla.gnome.org/show_bug.cgi?id=766940
Since gstreamer-vaapi is coupled with gstreamer releases, there is no need to
keep compatibility definition.
This patch removes the definition of
GST_BUFFER_POOL_OPTION_VIDEO_GL_TEXTURE_UPLOAD_META since it is in
gst-plugins-base version 1.2.2
This is a helper function to improve the readability of
ensure_sinkpad_buffer_pool(). It makes clearer when the buffer pool needs to be
re-instantiated.
This buffer pool may still be processing buffers when a caps renegotiation is
done. This one-liner patch deactivates the pool to drain it before it
de-allocation.
Right now, the decoders create a buffer pool for their sink pad which is not
used at all, because the decoders have never proposed it to upstream.
This patch avoids the buffer pool instantiating when the element inherits from
the GstVideoDecoder class.
This patch is a bit of optimization, since the bufferpool configuration is get
when the pool is created. Hence, we only need to request it when the pool from
the allocation query is reused.
If the offered pool in decide_allocation() vmethod doesn't have the
VAAPI_VIDEO_META option, it is destroyed immediatly and the pointer cleared,
so it could be created later.
When validating the caps from bufferpool config, this patch distinguishes the
error from no caps received (NULL) from the invalid caps (cannot be converted
into GstVideoInfo structure).
Use GST_{ERROR,WARNING}_OBJECT instead of GST_{ERROR,WARNING}, thus the logs
will show the name of the vaapipluginbase instance that failed.
Also, the code-style is fixed, where some error labels need to be surrounded
by braces.
gst_video_info_set_format() and gst_video_info_from_caps() call, internally,
gst_video_info_init(), hence it is not required to call it before them. This
patch removes these spurious calls.
Instead of a dummy alloc() vmethod, the allocator instance set the flag
GST_ALLOCATOR_FLAG_CUSTOM_ALLOC, which is used by the framework to avoid call
gst_allocator_alloc() on the allocator.
This function helper make sense for GStreamer 1.2, but it is not helpful for
greater version since the validation is already done in the API implementation.
Thus, it is removed.
Instead of using gst_video_info_to_caps () to generated the fixed src caps,
this patch enables the first step for caps negotiation with a possible
following caps filter.
_get_preferred_caps() will traverse the possible src caps looking for the one
wit the preferred feature and the preferred color format. Then the color
format, the frame size and the frame rate are fixated.
https://bugzilla.gnome.org/show_bug.cgi?id=758548
Instead of the allowed_srcpad_caps variable, this patch uses the othercaps
from fixate_caps() vmethod to find the preferred caps feature and color
format.
https://bugzilla.gnome.org/show_bug.cgi?id=758548
When running transform_caps() vmethod, returning the srcpad caps, the caps are
early restricted to the element properties set: width, height, format and
force keep aspect.
A new file was added gstvaapipostprocutil.{c,h} where the utilities functions
are stored.
https://bugzilla.gnome.org/show_bug.cgi?id=758548
