There are situations where gst_video_decoder_flush() is called, and
this subsequently produces a gst_video_decoder_reset() that kills the
currently active GstVideoCodecFrame. This means that it no longer
exists by the time we reach GstVideoDecoder::finish() callback, thus
possibly resulting in a crash if we assumed spare data was still
available for decode (current_frame_size > 0).
Try to honour GstVideoDecoder::reset() behaviour from GStreamer 1.0
that means a flush, thus performing the actual operations there like
calling gst_video_decoder_have_frame() if pending data is available.
Review all interactions between the main video decoder stream thread
and the decode task to derive a correct sequence of operations for
decoding. Also avoid extra atomic operations that become implicit under
the GstVideoDecoder stream lock.
Fix hard reset for seek cases by flushing the GstVaapiDecoder queue
and completely purge any decoded output frame that may come out from
it. At this stage, the GstVaapiDecoder shall be in a complete clean
state to start decoding over new buffers.
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
vaapidecode used to wait up to one second past the expected time of
presentation for the last decoded frame. This is not realistic in
practice when it comes to video pause/resume. Changed behaviour to
unconditionnally wait for a free VA surface prior to continuing the
decoding. The decode task will continue pushing the output frames to
the downstream element while also reporting errors at the same time
to the main thread.
https://bugzilla.gnome.org/show_bug.cgi?id=707108
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
The srcpad caps exposed for GStreamer 1.2 were missing any useful info
like framerate, pixel-aspect-ratio, interlace-mode et al. Not to mention
that it relied on possibly un-initialized data. Fix srcpad caps to be
initialized from a sanitized copy of GstVideoDecoder output state caps.
Note: the correct way to expose the srcpad caps triggers an additional
issue in core GStreamer auto-plugging capabilities as the correct caps
to be exposed should be format=ENCODED with memory:VASurface caps feature
at the minimum. In some situations, we could determine the underlying
VA surface format, but this is not always possible. e.g. cases where it
is not allowed to expose the underlying VA surface data, or when the
VA driver implementation cannot actually provide such information.
Currently, the decoder only supports YUV 4:2:0 output. So, expose the
output formats for GStreamer 1.2 in caps to a realistic subset. This
means NV12, I420 or YV12 but also ENCODED if we cannot determine the
underlying VA surface format, or if it is actually not allowed to get
access to the surface contents.
Fix vaapidecode srcpad caps to only expose RGBA video format for the
meta:GstVideoGLTextureUploadMeta feature. That's only what is supported
so far. Besides, drop this meta from the vaapisink sinkpad caps since
we really don't support that for rendering.
https://bugzilla.gnome.org/show_bug.cgi?id=711828
Add gst_caps_set_interlaced() helper function that would reset the
interlace-mode field to "progressive" for GStreamer >= 1.0, or the
interlaced field to "false" for GStreamer 0.10.
If the allocation meta GST_VIDEO_GL_TEXTURE_UPLOAD_META_API_TYPE is
requested, and more specifically under a GLX configuration, then add
the GstVideoGLTextureUploadMeta to the output buffer.
https://bugzilla.gnome.org/show_bug.cgi?id=703236
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Move VA video buffer memory from "video/x-surface,type=vaapi" format,
as expressed in caps, to the more standard use of caps features. i.e.
add "memory:VASurface" feature attribute to the associated caps.
https://bugzilla.gnome.org/show_bug.cgi?id=703271
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Fix gst_vaapidecode_query() to correctly display the query type name,
instead of randomly displaying that we shared the underlying display.
Also add debug info for the GstVaapiSink::query() handler, i.e. the
supplied query type name actually.
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Add support for the new GstContext API from GStreamer 1.2.x.
- implement the GstElement::set_context() hook ;
- reply to the `context' query from downstream elements.
https://bugzilla.gnome.org/show_bug.cgi?id=703235
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Port vaapidecode and vaapisink plugins to GStreamer API >= 1.2. This
is rather minimalistic so that to test the basic functionality.
Disable vaapipostproc plugin for now as further polishing is needed.
Also disable GstVideoContext interface support since this API is now
gone in 1.2.x. This is preparatory work for GstContext support.
https://bugzilla.gnome.org/show_bug.cgi?id=703235
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Fix _getcaps() implementation to not report codecs with size information
filled in the returned caps. That's totally useless nowadays. Ideally,
this is a hint to insert a video parser element, thus allowing future
optimizations, but this is not a strict requirement for gstreamer-vaapi,
which is able to parse the elementary bitstreams itself.
https://bugzilla.gnome.org/show_bug.cgi?id=704734
If there is no frame delimiter at the end of the stream, e.g. no
end-of-stream or end-of-sequence marker, and that the current frame
was fully parsed correctly, then assume that last frame is complete
and submit it to the decoder.
https://bugzilla.gnome.org/show_bug.cgi?id=705123
Signed-off-by: Guangxin.Xu <Guangxin.Xu@intel.com>
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Add support for GstVideoCropMeta in GStreamer >= 1.0.x builds and gst-vaapi
specific meta information to hold video cropping details. Make the sink
support video cropping in X11 and GLX modes.
Add gst_vaapi_decoder_get_frame_with_timeout() helper function that will
wait for a frame to be decoded, until the specified timeout in microseconds,
prior to returning to the caller.
This is a fix to performance regression from 851cc0, whereby the vaapidecode
loop executed on the srcpad task was called to often, thus starving all CPU
resources.
Rework heuristics to detect when downstream element ran into errors,
and thus failing to release any VA surface in due time for the current
frame to get decoded. In particular, recalibrate the render time base
when the first frame gets submitted downstream, or when there is no
timestamp that could be inferred.
Rework GstVideoDecoder::handle_frame() to decode the current frame,
while possibly waiting for a free surface, and separately submit all
decoded frames from a task. This makes it possible to pop and render
decoded frames as soon as possible.
Add support for interlaced streams with GStreamer 1.0 too. Basically,
this enables vaapipostproc, though it is not auto-plugged yet. We also
make sure to reply to CAPS queries, and happily handle CAPS events.
Make gst_vaapi_decoder_get_codec_state() return the original codec state,
i.e. make the GstVaapiDecoder object own the return state so that callers
that want an extra reference to it would just gst_video_codec_state_ref()
it before usage. This aligns the behaviour with what we had before with
gst_vaapi_decoder_get_caps().
This is an ABI incompatible change, library major version was bumped from
previous release (0.5.2).
Handle GST_QUERY_CAPS, which is the GStreamer 1.0 mechanism to retrieve
the set of allowed caps, i.e. it works similar to GstPad::get_caps().
This fixes fallback to SW decoding if no HW decoder is available.
Introduce a new configure option --with-gstreamer-api that determines
the desired GStreamer API to use. By default, GStreamer 1.0 is selected.
Also integrate more compatibility glue into gstcompat.h and plugins.
Use new GstVaapiVideoBufferPool to maintain video buffers. Implement
GstBaseSink::propose_allocation() to expose that pool to upstream
elements; and also implement GstVideoDecoder::decide_allocation() to
actually use that pool (from downstream), if any, or create one.
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Port vaapidecode and vaapisink plugins to GStreamer API >= 1.0. This
is rather minimalistic so that to test the basic functionality.
Disable vaapiupload, vaapidownload and vaapipostproc plugins. The latter
needs polishing wrt. to GStreamer 1.x functionality and the former are
totally phased out in favor of GstVaapiVideoMemory map/unmap facilities,
which are yet to be implemented.
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Don't return static caps that don't mean anything for the underlying codecs
that are actually supported for decoding. i.e. always allocate a VA display
and retrieve the exact set of HW decoders available. That VA display may be
re-used later on during negotiation through GstVideoContext "prepare-context".
This fixes fallback to SW decoding if no HW decoder is available.
Move GstVaapiVideoMeta from core libgstvaapi decoding library to the
actual plugin elements. That's only useful there. Also inline reference
counting code from GstVaapiMiniObject.
Use gst_element_class_set_static_metadata() from GStreamer 1.0, which
basically is the same as gst_element_class_set_details_simple() in
GStreamer 0.10 context.
Move GstImplementsInterface and GstVideoContext support functions up
so that to keep a clear separation between the plugin element and its
interface hooks.
Decode-only frames may not have a valid surface proxy. So, simply discard
them gracefully, i.e. don't create meta data information. GstVideoDecoder
base class will properly handle this case and won't try to push any buffer
to downstream elements.
Implement GstVideoDecoder::reset() as a destruction of the VA decoder
and the creation of a new VA decoder.
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Split GstVideoDecoder::handle_frame() implementation into two functions:
(i) one for decoding the provided GstVideoCodecFrame and (ii) another one
for purging all decoded frames and submit them downstream.
Update plugin elements with the new GstVaapiVideoMeta API.
This also fixes support for subpictures/overlay because GstVideoDecoder
generates a sub-buffer from the GstVaapiVideoBuffer. So, that sub-buffer
is marked as read-only. However, when comes in the textoverlay element
for example, it checks whether the input buffer is writable. Since that
buffer read-only, then a new GstBuffer is created. Since gst_buffer_copy()
does not preserve the parent field, the generated buffer in textoverlay
is not exploitable because we lost all VA specific information.
Now, with GstVaapiVideoMeta information attached to a standard GstBuffer,
all information are preserved through gst_buffer_copy() since the latter
does copy metadata (qdata in this case).
Fix calculation of the time-out value for cases where no VA surface is
available for decoding. In this case, we need to wait until downstream
sink consumed at least one surface. The time-out was miscalculated as
it was always set to <current-time> + one second, which is not suitable
for streams with larger gaps.
Don't call gst_video_decoder_drop_frame() if gst_video_decoder_finish_frame()
was already called before and it returned an error. In that case, we were
releasing the frame again, thus leading to a "double-free" condition.
Maintain decoded surfaces as GstVideoCodecFrame objects instead of
GstVaapiSurfaceProxy objects. The latter will tend to be reduced to
the strict minimum: a context and a surface.
Make sure to push all decoded frames downstream as soon as possible.
This makes sure we don't need to wait for a new frame to be ready to
be decoded before receiving new decoded frames.
This also separates the decode process and the output process. The latter
could be moved to a specific GstTask later on.
Directly use the GstVideoCodecState associated with the VA decoder
instead of parsing caps again.
Signed-off-by: Sreerenj Balachandran <sreerenj.balachandran@intel.com>
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
