If a multiview stream is decoded, multiple view components are submitted
as is downstream. It is the responsibility of the sink element to display
the required view components. By default, always select the frame buffer
that matches the view-id of the very first frame to be displayed.
However, introduce a "view-id" property to allow the selection of a
specific view component of interest to display.
Always record the VA surface that is currently being rendered, no matter
the fact we are using texturedblit or overlay. That's because in some
occasions, we need to refresh or resize the displayed contents based on
new events. e.g. user-resized window.
Besides, it's simpler to track the last video buffer in GstVaapiSink than
through the base sink "last-sample".
Add a "display-name" property to vaapisink so that the end user could
select the desired output. Keep "display-name" in-line with the existing
"display" (GstVaapiDisplayXXX type).
So, for X11 or GLX, the "display-name" is the usual display name as we
know for XOpenDisplay(); for Wayland, the "display-name" is the name used
for wl_display_connect(); and for DRM, the "display-name" is actually the
DRI device name.
https://bugzilla.gnome.org/show_bug.cgi?id=722247
Force early initializatin of the GstVaapiDisplay so that to make sure
that the sink element display object is presented first to upstream
elements, as it will be correctly featuring the requested display type
by the user.
Otherwise, we might end up in situations where a VA/X11 display is
initialized in vaapidecode, then we try VA/DRM display in vaapisink
(as requested by the "display" property), but this would cause a failure
because we cannot acquire a DRM display that was previously acquired
through another backend (e.g. VA/X11).
This maps GstVideoColorimetry information in vaapisink's sinkpad caps
to GST_VAAPI_COLOR_STANDARD_* flags, if per-buffer information was not
available.
https://bugzilla.gnome.org/show_bug.cgi?id=722255
[factored out code, added SMPTE240M, handle per-buffer flags]
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
When gst_vaapisink_video_overlay_set_window_handle() is called early,
before the pipeline has been set to PLAYING, the display has not yet
been initialized and _PLUGIN_BASE_DISPLAY_TYPE() is not yet
up-to-date. For this reason the foreign XID is not attached.
Now _ensure_display() is called earlier.
https://bugzilla.gnome.org/show_bug.cgi?id=722244
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Factor out propose_allocation() hooks, creation of video buffer pool
for the sink pad, conversion from raw YUV buffers to VA surface backed
buffers. Update vaapidecode, vaapiencode and vaapipostproc to cope
with the new GstVaapiPluginBase abilities.
Move common VA display creation code to GstVaapiPluginBase, with the
default display type remaining "any". Also add a "display-changed"
hook so that subclasses could perform additional tasks when/if the
VA display changed, due to a new display type request for instance.
All plug-ins are updated to cope with the new internal APIs.
Introduce a new GstVaapiPluginBase object that will contain all common
data structures and perform all common tasks. First step is to have a
single place to hold VA displays.
While we are at it, also make sure to store and subsequently release
the appropriate debug category for the subclasses.
Fix GstElement::set_context() implementation for all plug-in elements
to avoid leaking an extra reference to the VA display, thus preventing
correct cleanup of VA resources in GStreamer 1.2 builds.
Currently, the decoder only supports YUV 4:2:0 output. So, expose the
output formats for GStreamer 1.2 in caps to a realistic subset. This
means NV12, I420 or YV12 but also ENCODED if we cannot determine the
underlying VA surface format, or if it is actually not allowed to get
access to the surface contents.
Fix vaapidecode srcpad caps to only expose RGBA video format for the
meta:GstVideoGLTextureUploadMeta feature. That's only what is supported
so far. Besides, drop this meta from the vaapisink sinkpad caps since
we really don't support that for rendering.
https://bugzilla.gnome.org/show_bug.cgi?id=711828
If the allocation meta GST_VIDEO_GL_TEXTURE_UPLOAD_META_API_TYPE is
requested, and more specifically under a GLX configuration, then add
the GstVideoGLTextureUploadMeta to the output buffer.
https://bugzilla.gnome.org/show_bug.cgi?id=703236
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Move VA video buffer memory from "video/x-surface,type=vaapi" format,
as expressed in caps, to the more standard use of caps features. i.e.
add "memory:VASurface" feature attribute to the associated caps.
https://bugzilla.gnome.org/show_bug.cgi?id=703271
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Fix gst_vaapidecode_query() to correctly display the query type name,
instead of randomly displaying that we shared the underlying display.
Also add debug info for the GstVaapiSink::query() handler, i.e. the
supplied query type name actually.
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Add support for the new GstContext API from GStreamer 1.2.x.
- implement the GstElement::set_context() hook ;
- reply to the `context' query from downstream elements.
https://bugzilla.gnome.org/show_bug.cgi?id=703235
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Port vaapidecode and vaapisink plugins to GStreamer API >= 1.2. This
is rather minimalistic so that to test the basic functionality.
Disable vaapipostproc plugin for now as further polishing is needed.
Also disable GstVideoContext interface support since this API is now
gone in 1.2.x. This is preparatory work for GstContext support.
https://bugzilla.gnome.org/show_bug.cgi?id=703235
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
In GStreamer 0.10 builds, make sure that the GstVaapiUploader helper
is setup in case upstream elements allocate buffers themselves without
honouring our GstVaapiSink::bufer_alloc() hook.
In particular, this fixes support for OGG video streams with WebKit.
https://bugzilla.gnome.org/show_bug.cgi?id=703934
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Handle raw video buffers that were not created from a VA video buffer
pool. Use the generic GstVideo API to copy buffers in GStreamer 1.0.x
builds instead of the GstVaapiUploader.
https://bugs.freedesktop.org/show_bug.cgi?id=55818
Other GStreamer sinks, like xvimagesink, have a force-aspect-ratio property,
which allows you to say that you don't want the sink to respect aspect
ratio. Add the same property to vaapisink.
http://lists.freedesktop.org/archives/libva/2012-September/001298.html
Signed-off-by: Simon Farnsworth <simon.farnsworth at onelan.co.uk>
Fix GstBaseSink::get_caps() implementation for GStreamer 1.0.X builds
by honouring the filter caps argument. More precisely, this fixes the
following pipeline: gst-launch-1.0 videotestsrc ! vaapisink
https://bugzilla.gnome.org/show_bug.cgi?id=705192
Signed-off-by: Guangxin.Xu <Guangxin.Xu@intel.com>
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Fix creation of GLX texture, to not depend on the GstCaps video size that
could be wrong, especially in presence of frame cropping. So, use the size
from the source VA surfaces.
An optimization could be to reduce the texture size to the actual visible
size on screen. i.e. scale down the texture size to match the screen dimensions,
while preserving the VA surface aspect ratio. However, some VA drivers don't
honour that.
Add support for GstVideoCropMeta in GStreamer >= 1.0.x builds and gst-vaapi
specific meta information to hold video cropping details. Make the sink
support video cropping in X11 and GLX modes.
Expose all raw video formats in the static caps template since the
vaapisink is supporting raw data. We will get the exact set of formats
supported by the driver dynamically through the _get_caps() routine.
This also fixes an inconsistency wrt. GStreamer 0.10 builds.
https://bugzilla.gnome.org/show_bug.cgi?id=702178
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Now that VA/GLX capable buffers are generated by default on X11, thus
depending on a VA/GLX display, we stil want to use vaPutSurface() for
rendering since it is faster.
Anyway, OpenGL rendering in vaapisink was only meant for testing and
enabling "fancy" effects to play with. This has no real value. So,
disable OpenGL rendering by default.
When render-mode is "overlay", then it is not really useful to peek into
the GstBaseSink::last_buffer, since we have our own video_buffer already
recorded and maintained into GstVaapiSink.
Fix memory leak of GstSample objects in GstVideoOverlayInterface::expose().
This also fixes extra unreferencing of the underlying GstBuffer in the common
path afterwards (for both 0.10 or 1.0).
Introduce a new configure option --with-gstreamer-api that determines
the desired GStreamer API to use. By default, GStreamer 1.0 is selected.
Also integrate more compatibility glue into gstcompat.h and plugins.
Use new GstVaapiVideoBufferPool to maintain video buffers. Implement
GstBaseSink::propose_allocation() to expose that pool to upstream
elements; and also implement GstVideoDecoder::decide_allocation() to
actually use that pool (from downstream), if any, or create one.
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Port vaapidecode and vaapisink plugins to GStreamer API >= 1.0. This
is rather minimalistic so that to test the basic functionality.
Disable vaapiupload, vaapidownload and vaapipostproc plugins. The latter
needs polishing wrt. to GStreamer 1.x functionality and the former are
totally phased out in favor of GstVaapiVideoMemory map/unmap facilities,
which are yet to be implemented.
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Improve check for raw YUV format modes by avoiding checks against strings
("video/x-raw-yuv") for each new GstBuffer allocation. In the usual case,
GstBaseSink::set_caps() is called first and if VA surface format mode is
used, then GstBaseSink::buffer_alloc() is not called. If the latter is
called before set_caps(), then we just make a full check. This one is
pretty rare though, e.g. it usually happens once for custom pipelines.
Make gst_vaapi_reply_to_query() first check whether the query argument
is actually a video-context query, i.e. with type GST_QUERY_TYPE_CUSTOM.
Then, make sure vaapisink propagates the query to the parent class if
it is not a video-context query.
Move GstVaapiVideoMeta from core libgstvaapi decoding library to the
actual plugin elements. That's only useful there. Also inline reference
counting code from GstVaapiMiniObject.
If the raw YUV buffer was created from vaapisink, through the buffer_alloc()
hook, then it will have a valid GstVaapiVideoMeta object attached to it.
However, we previously assumed in that case that it was a "native" VA buffer,
thus not calling into GstVaapiUploader::process().
Use gst_element_class_set_static_metadata() from GStreamer 1.0, which
basically is the same as gst_element_class_set_details_simple() in
GStreamer 0.10 context.
Move GstImplementsInterface and GstVideoContext support functions up
so that to keep a clear separation between the plugin element and its
interface hooks.
Use GstVideoInfo and gst_video_info_from_caps() helper wherever possible.
Also use the newly added gst_vaapi_image_format_from_structure() helper
in GstVaapiUploader::ensure_allowed_caps().
Update plugin elements with the new GstVaapiVideoMeta API.
This also fixes support for subpictures/overlay because GstVideoDecoder
generates a sub-buffer from the GstVaapiVideoBuffer. So, that sub-buffer
is marked as read-only. However, when comes in the textoverlay element
for example, it checks whether the input buffer is writable. Since that
buffer read-only, then a new GstBuffer is created. Since gst_buffer_copy()
does not preserve the parent field, the generated buffer in textoverlay
is not exploitable because we lost all VA specific information.
Now, with GstVaapiVideoMeta information attached to a standard GstBuffer,
all information are preserved through gst_buffer_copy() since the latter
does copy metadata (qdata in this case).
Intermediate elements may produce a sub-buffer from a valid GstVaapiVideoBuffer
for non raw YUV cases. Make sure vaapisink now understands those buffers.
Make vaapisink expose only the set of supported caps for raw YUV buffers.
Add gst_vaapi_uploader_get_caps() helper function to determine the set
of supported YUV caps as source (for images). This function actually
tries to zero and upload each image to a 64x64 test surface. Of course,
this relies on VA drivers to not claim success if vaPutImage() is not
correctly supported.
Add new GstVaapiUploader helper to upload raw YUV buffers to VA surfaces.
It is up to the caller to negotiate source caps (for images) and output
caps (for surfaces). gst_vaapi_uploader_has_direct_rendering() is available
to help decide between the creation of a GstVaapiVideoBuffer or a regular
GstBuffer on sink pads.
Signed-off-by: Zhao Halley <halley.zhao@intel.com>
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
If either dimension is out-of-bounds, then scale window to fit the
display size, even if the output is to be rotated. Use the standard
gst_video_sink_center_rect() function to center and scale the window
wrt. the outer (display) bounds.
Keep VA surface proxy associated with the surface that is currently
being displayed. This makes sure that surface is not released back
to the pool of surfaces free to use for decoding. This is necessary
with VA driver implementations that support rendering to an overlay
pipe. Otherwise, there could be cases where we are decoding into a
surface that is being displayed, hence some flickering.
This is not useful in practice but for raw performance evaluation when
the sink is invoked with display=drm sync=false. fakesink could also be
used though.
If vaapisink is in the GStreamer pipeline, then we shall allocate a
unique GstVaapiDisplay and propagate it upstream. i.e. subsequent
queries from vaapidecode shall get a valid answer from vaapisink.
Move display types from gstvaapipluginutil.* to gstvaapidisplay.* so that
we could simplify characterization of a GstVaapiDisplay. Also rename "auto"
type to "any", and add a "display-type" attribute.
vaapisink is now built with support for multiple display types, whenever
they are enabled. The new "display" attribute is used to select a particular
renderer.
GStreamer -base plugins >= 0.10.31 are now required, so the checks for
new APIs like GstXOverlay::set_window_handle() and ::set_render_rectangle()
are no longer necessary.
Bump GStreamer plugins -base required version to 0.10.31, needed for
gst_x_overlay_got_window_handle().
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Bump GStreamer required version to 0.10.14, needed for
gst_element_class_set_details_simple().
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Don't forcibly resize foreign X windows. The user is responsible for
their size and vaapisink shall not change this.
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Fix gst_vaapisink_xoverlay_set_window_handle() when it is called before
caps got negotiated. Besides, when a foreign window is provided by the
user, so should the render rect.
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Allow rendering flags, as a combination of GstVaapiSurfaceRenderFlags,
to be set to the video buffer. In particular, this is mostly useful for
basic deinterlacing.
With the new video/x-surface abstraction, we can't rely on having a VA
specific sink downstream. Also, there was no particular reason to do that.
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
This new interface allows for upstream and downstream display sharing
that works in both static and dynamic pipelines.
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
