Currently the src caps are set immediately after the sink caps are set, but in
that moment the pipeline might not fully constructed and the video sink has
not negotiated its supported caps and features. As a consequence, in many cases
of playback, the least optimized caps feature is forced. This is partially the
responsible of bug #744039.
Also, vaapidecode doesn't attend the reconfigure events from downstream,
which is a problem too, since the video sink can be changed with different
caps features.
This patch delays the src caps, setting them until the first frame arrives to
the decoder, assuming until that very moment the whole pipeline is already
negotiated. Particularly, it checks if the src pad needs to be reconfigured,
as a consequence of a reconfiguration event from downstream.
A key part of this patch is the new GstVaapiCapsFeature
GST_VAAPI_CAPS_FEATURE_NOT_NEGOTIATED, which is returned when the src pad
doesn't have a peer yet. Also, for a better report of the caps allowed
through the src pad and its peer, this patch uses gst_pad_get_allowed_caps()
instead of gst_pad_peer_query_caps() when looking for the preferred feature.
v3: move the input_state unref to close(), since videodecoder resets at
some events such as navigation.
v4: a) the state_changed() callback replaces the input_state if the media
changed, so this case is also handled.
b) since the parameter ref_state in gst_vaapidecode_update_src_caps() is
always the input_state, the parameter were removed.
c) there were a lot of repeated code handling the input_state, so I
refactored it with the function gst_vaapi_decode_input_state_replace().
https://bugzilla.gnome.org/show_bug.cgi?id=744618
Signed-off-by: Víctor Manuel Jáquez Leal <victorx.jaquez@intel.com>
Signed-off-by: Sreerenj Balachandran <sreerenj.balachandran@intel.com>
Working on bug #743687, I realized that vaapidecode always adds to its buffer
pool the config option GST_BUFFER_POOL_OPTION_VIDEO_GL_TEXTURE_UPLOAD_META if
the decide_allocation()'s query has GST_VIDEO_GL_TEXTURE_UPLOAD_META_API_TYPE.
Nevertheless, there are occasions where the query has the API type, but the
last negotiated caps don't have the feature meta:GstVideoGLTextureUploadMeta.
Under this contradiction, vaapidecode adds the GLTextureUploadMeta API to its
buffer pool configuration, and adds its buffer's meta to each output buffer,
even if the negotiated caps feature is memory:SystemMemory with I420 color
format.
This kind of output buffers chokes ClutterAutoVideosSink, since it uses a map
that relates caps <-> GL upload method. If it receives a buffer with color
format I420, it assumes that it doesn't have a texture upload meta, because
only those with RGB color format has it. Our buffers, with I420 format, say
that they have the upload meta too. In that case the mapped method is a dummy
one which does nothing. I reported this issue in bug #744039 (the patch,
obviously, was rejected).
This patch workarounds the problem: the buffer pool's configuration option
GST_BUFFER_POOL_OPTION_VIDEO_GL_TEXTURE_UPLOAD_META is set if and only if the
query has the GST_VIDEO_GL_TEXTURE_UPLOAD_META_API_TYPE *and* the negotiated
caps feature is meta:GstVideoGLTextureUploadMeta.
I have tested these patches with gst-master (1.5), gst-1.4 and gst-1.2 and
in all they seem to work correctly.
https://bugzilla.gnome.org/show_bug.cgi?id=744618
[adapted to fit current EGL changes]
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Add support for GstVideoGLTextureOrientation modes. In particular,
add orientation flags to the GstVaapiTexture wrapper and the GLX
implementations. Default mode is that texture memory is laid out
with top lines first, left row first. Flags indicate whether the
X or Y axis need to be inverted.
Some frameworks (EFL) expect BGRA textures for storage. However,
adding support for that broadly into GStreamer framework implies
two kinds of hacks: (i) libgstgl helpers currently do not support
BGRA textures correctly, (ii) we need to better parse downstream
suggested caps and intersect them with what the VA plugin elements
can offer to them for GL texturing.
When multiple display servers are available, the glimagesink element
(from GStreamer 1.4) may not be able to derive a global display in
Wayland. Rather, a "window"-specific display is created. In this case,
the GstGLDisplay handle available through GstGLContext is invalid.
So, try to improve heuristics for display server characterisation in
those particular situations.
Add initial support for EGL through GstVideoGLTextureUploadMeta.
Fix gst_vaapi_ensure_display() to allocate a GstVaapiDisplay off the
downstream supplied GstGLContext configuration, i.e. use its native
display handle to create a GstVaapiDisplay of type X11 or Wayland ;
and use the desired OpenGL API to allocate the GstVaapiDisplayEGL
wrapper.
https://bugzilla.gnome.org/show_bug.cgi?id=741079
Sync video texture sizes to GstVideoGLTextureUploadMeta private date,
i.e. GstVaapiVideoMetaTexture, on a regular basis. In particular, we
now update the texture size from the GstVideoMeta, if any, or reset
to some defaults otherwise.
If a GstGLContext is supplied by the downstream element, then make
sure that the VA plugin element gets a compatible display to what
is requested by the GL context. e.g. re-allocate a VA/GLX display
when a GLX context is provided by the downstream element.
Record GL context supplied by downstream elements. This can be useful,
and further needed, to enforce run-time check that the GL context is
compatible for use by libgstvaapi. e.g. check that we don't create a
VA/GLX display for EGL/X11 contexts.
https://bugzilla.gnome.org/show_bug.cgi?id=725643
Original-path-by: Matthew Waters <ystreet00@gmail.com>
Reset the VA decoder after updating the base plugin caps, and most
importantly, after GstVideoDecoder negotiation. The reason behind
this is that the negotiation could trigger a last decide_allocation()
where we could actually derive a new GstVaapiDisplay to use from the
downstream element. e.g. GLX backend.
Query caps filtering should be always done on top of allowed caps instead
of existing fixed caps on a particular pad.
This fixes the mvc stream decoding when there is a base view(high profile)
and non-base view(stereo-high profile).
According to documentation[1] when receiving a GST_QUERY_CAPS
the return value should be all formats that this elements supports,
taking into account limitations of peer elements further downstream
or upstream, sorted by order of preference, highest preference first.
This patch add those limitations intersecting with the received
filter in the query. Also takes into account the already negotiated
caps. Also adds the processing of the query on the SRC pad.
1. http://gstreamer.freedesktop.org/data/doc/gstreamer/head/pwg/html/section-nego-getcaps.htmlhttps://bugzilla.gnome.org/show_bug.cgi?id=744406
Otherwise the condition could become true before the lock
is taken and the g_cond_signal() could be called
before the g_cond_wait(), so the g_cond_wait() is never
awoken.
https://bugzilla.gnome.org/show_bug.cgi?id=740645
GstVaapiVideoMemory quark is not needed any more, and the actual
implementation was already removed bfore the merge. i.e. this is
an oversight for a hunk that was not meant to be pushed.
Allow v4l2src element to connected to vaapipostproc or vaapisink when
"io-mode" is set to "dmabuf-import". In practice, this is a more likely
operational mode with uvcvideo. Supporting v4lsrc with "io-mode" set
to "dmabuf" could work, but with more demanding driver or kernel reqs.
Note: with GStreamer 1.4, v4l2src (gst-plugins-good) needs to be built
with --without-libv4l2.
https://bugzilla.gnome.org/show_bug.cgi?id=743635
Allow imports of v4l2 buffers into VA surfaces for further operation
with vaapi plugins, e.g. vaapipostproc or vaapiencode_* elements.
https://bugzilla.gnome.org/show_bug.cgi?id=735362
[fixed memory leaks, ported to new dma_buf infrastructure, cleanups]
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Reword surface pool allocation helpers so that to allow for a simple
form, e.g. gst_vaapi_surface_pool_new(format, width, height); and a
somewhat more elaborated/flexible form with optional allocation flags
and precise GstVideoInfo specification.
This is an API/ABI change, and SONAME version needs to be bumped.
Add gst_vaapi_display_has_opengl() helper function to help determining
whether the display can support OpenGL context to be bound to it, i.e.
if the class is of type GST_VAAPI_DISPLAY_TYPE_GLX.
Make gst_vaapi_display_get_display_type() return the actual VA display
type. Conversely, add a gst_vaapi_display_get_class_type() function to
return the type of the GstVaapiDisplay instance. The former is used to
identify the display server onto which the application is running, and
the latter to identify the original object class.
GstVaapiTexture is a generic abstraction that could be moved to the
core libgstvaapi library. While doing this, no extra dependency needs
to be added. This means that a GstVaapitextureClass is now available
for any specific code that needs to be added, e.g. creation of the
underlying GL texture objects, or backend dependent ways to upload
a surface to the texture object.
Generic OpenGL data types (GLuint, GLenum) are also replaced with a
plain guint.
https://bugzilla.gnome.org/show_bug.cgi?id=736715
The gst_vaapi_texture_put_surface() function is missing a crop_rect
argument that would be used during transfer for cropping the source
surface to the desired dimensions.
Note: from a user point-of-view, he should create the GstVaapiTexture
object with the cropped size. That's the default behaviour in software
decoding pipelines that we need to cope with.
This is an API/ABI change, and SONAME version needs to be bumped.
https://bugzilla.gnome.org/show_bug.cgi?id=736712
dvbsuboverlay signals no subtitles present by not setting
GstVideoOverlayCompositionMeta on a buffer.
Detect this, and remove subtitles whenever we have no overlay composition to
hand.
Signed-off-by: Simon Farnsworth <simon.farnsworth@onelan.co.uk>
Otherwise restarting may fail because the state of vaapipluginbase and
vaapipostproc don't match. e.g. gst_vaapipostproc_set_caps() will skip
initailization and not call gst_vaapi_plugin_base_set_caps()
Make sure that the GstVaapiPluginBase instance receives the new src
pad caps whenever they get updated from within the GstVaapiDecoder
decode routines.
This also ensures that downstream elements receive correctly sized
SW decoded buffers if needed.
https://bugs.tizen.org/jira/browse/TC-114
Split GstVideoDecoder::set_format() handler to first update the sink
pad caps and reset the active VA decoder instance based on those, and
then update the src pad caps whenever possible, e.g. when the caps
specify a valid video resolution.
In some occasions, a buffer pool is created for pre-initialization
purposes regardless of whether a valid image size is available or
not. However, during actual decode stage, the vaapidecode element
is expected to update the srcpad caps with the new dimensions, thus
also triggering a reset of the underlying bufferpool.
If the best downstream capsfeature turns out to be GLMemory, then make
sure to propagate RGBA video format in caps to that element. This fixes
the following pipeline: ... ! vaapipostproc ! glimagesink.
When an explicit output video format is selected, from an src pad
capsfilter, make sure that the downstream element actually supports
that format. In particular, fix crash with the following pipelines:
... ! vaapipostproc ! video/x-raw,format=XXX ! xvimagesink ; where
XXX is a format not supported by xvimagesink.
While doing so, also reduce the set of src pad filter caps to the
actual set of allowed src pad caps.
The ensure_surface() and ensure_image() functions shall only relate
to the underlying backing store. The actual current flags are to be
updated only through ensure_{surface,image}_is_current() or very other
particular cases in GstMemory hooks.
If the surface proxy is updated into the GstVaapiVideoMemory, then
it is assumed it is the most current representation of the current
video frame. Likewise, make a few more arrangements to have the
"current " flags set more consistently.
When interacting with SW elements, the buffers and underlying video
memory could be mapped as read/write. However, we need to use those
buffers again as plain VA surfaces, we have to make sure the VA image
is thus committed back to VA surface memory.
This fixes pipelines involving avdec_* and vaapi{postproc,sink}.
Hence vaapisink can display buffers decoded by gst-libav, or HW decoded
buffers can be further processed in-place, e.g. with a textoverlay.
https://bugzilla.gnome.org/show_bug.cgi?id=704078
[ported to current git master branch, amended commit message]
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
In the current implementation, gst_video_info_set_format() would reset
the whole GstVideoInfo structure first, prior to setting video format
and size. So, coleteral information like framerate or pixel-aspect-
ratio are lost.
Provide and use a unique gst_video_info_change_format() for overcome
this issue, i.e. only have it change the format and video size, and
copy over the rest of the fields.
https://bugzilla.gnome.org/show_bug.cgi?id=734665
This is for helping decodebin to autoplug the vaapidecode element.
Decodebin is selecting decoder elements only based on rank and caps.
Without overriding the autoplug-* signals there is no way to autoplug
HW decoders inside decodebin. An easier soulution is to raise the
rank of vaapidecode, so that it gets selected first.
https://bugzilla.gnome.org/show_bug.cgi?id=739332
Added the same Klass specifications used in other upstream
video postprocessing elements like videoconvert, videoscale,
videobalance and deinterlace.
An example use case is for this is to help the playsink
to autoplug the hardware accelerated deinterlacer.
This is a workaround until auto-plugging is fixed when
format=ENCODED + memory:VASurface caps feature are provided.
Use the downstream negotiated video format as the output video format
if the user didn't ask for the colorspace conversion explicitly.
Usecase: This will help to connect elements like videoscale, videorate etc
to vaapipostproc.
https://bugzilla.gnome.org/show_bug.cgi?id=739443
Add new "scale-method" property to expose the scaling mode to use during
video processing. Note that this is only a hint, and the actual behaviour
may differ from implementation (VA driver) to implementation.
Attach the codec_data to out_caps only if downstream needed.
For eg: h264 encoder doesn't need to stuff codec_data to the
src caps if the negotiated caps has a stream format of byte-stream.
https://bugzilla.gnome.org/show_bug.cgi?id=734902
Fix arguments to XkbKeycodeToKeysym() for converting an X11 keycode
to a KeySym. In particular, there is no such Window argument. Also
make sure to check for, and use, the correct <X11/XKBlib.h> header
where that new function is defined. Otherwise, default to the older
XKeycodeToKeysym() function.
Really use the motion event coordinates to propagate the "mouse-move"
event to upper layer, instead of those from a button event. Those are
technically the same though.
When we copy a buffer because we're moving it into VA-API memory, we
need to copy flags. Otherwise, interlaced YUV buffers from a capture
source (e.g. V4L2) don't get flagged as interlaced.
https://bugzilla.gnome.org/show_bug.cgi?id=726270
Signed-off-by: Simon Farnsworth <simon.farnsworth@onelan.co.uk>
[reversed order of gst_buffer_copy_into() flags to match <1.0 code]
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Allow implicit conversions to raw video formats, while still keeping
VA surfaces underneath. This allows for chaining the vaapipostproc
element to a software-only element that takes care of maps/unmaps.
e.g. xvimagesink.
https://bugzilla.gnome.org/show_bug.cgi?id=720174
Use pooled GstVaapiVideoMeta information, i.e. always allocate that on
video buffer allocation. Also optimize copy of additional metadata info
into the resulting video buffer: only copy the video cropping info and
the source surface proxy.
https://bugzilla.gnome.org/show_bug.cgi?id=720311
Signed-off-by: Sreerenj Balachandran <sreerenj.balachandran@intel.com>
[fixed proxy leak, fixed double free on error, optimized meta copy]
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
If no explicit output surface format is supplied try to keep the one
supplied through the sink pad caps. This avoids a useless copy, even
if things are kept in GPU memory.
This is a performance regression from git commit dfa70b9.
If only advanced deinterlacing is requested, i.e. deinterlacing is
the only active algorithm to apply with source and output surface
formats being the same, then make sure to enable VPP processing.
Otherwise, allow fallback to bob-deinterlacing with simple rendering
flags alteration.
Add a default decide_allocation() hook to GstVaapiPluginBase. The caps
feature argument can be used to force a bufferpool with a specific kind
of memory.
Add GST_VAAPI_VIDEO_BUFFER_POOL_ACQUIRE_FLAG_NO_ALLOC params flag that
can be used to disable early allocations of vaapi video metas on buffers,
thus delagating that to the bufferpool user.
Default to I420 format for output surfaces so that to match the usual
GStreamer pipelines. Though, internally, we could still opt for NV12
surface formats, i.e. default format=ENCODED is a hint for that, thus
delegating the decision to the VA driver.
Use the new gst_caps_has_vaapi_surface() helper function to detect
whether the sink pad caps contain native VA surfaces, or not, i.e.
no raw video caps.
Also rename is_raw_yuv to get_va_surfaces to make the variable more
explicit as we just want a way to differentiate raw video caps from
VA surfaces actually.
The "discontinuity" tracking code, whereby lost frames are tentatively
detected, is inoperant if the sink pad buffer timestamps are not right
to begin with.
This is a temporary workaround until the following bug is fixed:
https://bugzilla.gnome.org/show_bug.cgi?id=734386
In order to make the discontinuity detection code useful, we need to
detect the lost frames in the history as early as the previous frame.
This is because some VA implementations only support one reference
frame for advanced deinterlacing.
In practice, turn the condition for detecting new frame that is beyond
the previous frame from field_duration*2 to field_duration*3, i.e.
nothing received for the past frame and a half because of possible
rounding errors when calculating the field-duration either in this
element (vaapipostproc), or from the upstream element (parser element).
This is a regression introduced with commit faefd62.
https://bugzilla.gnome.org/show_bug.cgi?id=734135
Introduce new gst_caps_has_vaapi_surface() helper function to detect
whether the supplied caps has VA surfaces. With GStreamer >= 1.2, this
implies a check for memory:VASurface caps features, and format=ENCODED
for earlier versions of GStreamer.
Simplify the creation and installation process of properties, by first
accumulating them into a g_properties[] array, and next calling into
g_object_class_install_properties().
Also add missing docs and flags to some properties.
Move code around in a more logical way. Introduce GST_VAAPISINK_CAST()
helper macro and use it wherever we know the object is a GstBaseSink or
any base class. Drop explicit initializers for values that have defaults
set to zero.
Introduce new backends vtable so that to have clean separation between
display dependent code and common base code. That's a "soft" separation,
we don't really need dedicated objects.
https://bugzilla.gnome.org/show_bug.cgi?id=722248
Support for X11 "synchronous" mode was never implemented, and was only
to be useful for debugging. Drop that altogether, that's not going to
be useful in practice.
https://bugzilla.gnome.org/show_bug.cgi?id=733985
Rendering with GLX in vaapisink is kind of useless nowadays, including
OpenGL related fancy effects. Plain VA/GLX interfaces are also getting
deprecated in favor of EGL, or more direct buffer sharing with actual
GL textures.
Should testing of interop with GLX be needed, one could still be using
the modern cluttersink or glimagesink elements.
https://bugzilla.gnome.org/show_bug.cgi?id=733984
Allow dynamic changes to the window, e.g. performed by the user, and
make sure to refresh its contents, while preserving aspect ratio.
In practice, Expose and ConfigureNotify events are tracked in X11
display mode by default. This occurs in a separte event thread, and
this is similar to what xvimagesink does. Any of those events will
trigger a reconfiguration of the window "soft" size, subsequently
the render-rect when necessary, and finally _expose() the result.
The default of handle_events=true can be changed programatically via
gst_x_overlay_handle_events().
Thanks to Fabrice Bellet for rebasing the patch.
https://bugzilla.gnome.org/show_bug.cgi?id=711478
[dropped XInitThreads(), cleaned up the code a little]
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
The gst_vaapidecode_decode_loop() function is called within a separate
task to fetch and output all frames that were decoded so far. So, if
the decoder_loop_status is forcibly set to EOS when _finish() is called,
then we are bound to exist the task without submitting the pending
frames.
If the downstream element error'ed out, then the gst_pad_push() would
propagate up an error and so we will get it right for cutting off
_finish() early in that case.
This is a regression from 6003596.
https://bugzilla.gnome.org/show_bug.cgi?id=733897
Fixes a hang/race on shutdown where _decode_loop() had already completed
its execution and _finish() was waiting on a GCond for decode_loop()
to complete. Also fixes the possible race where _finish() is called
but _decode_loop() endlessly returns before signalling completion
iff the decoder instance returns GST_FLOW_OK.
Found with: ... ! vaapidecode ! {glimagesink,cluttersink}
https://bugzilla.gnome.org/show_bug.cgi?id=733897
[factored out GST_VIDEO_DECODER_STREAM_UNLOCK() call]
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Now that we always track the currently active video buffer, it is
not necessary to automatically increase its reference since this is
implicitly performed in ::show_frame() through the get_input_buffer()
helper from GstVaapiPluginBase class.
This is a regression from a26df80.
Rework the logics behind the configuration of an adequate bufferpool,
especially when OpenGL meta or additional capsfeatures are needed.
Besides, for GStreamer >= 1.4, the first capsfeatures that gets matched,
and that is not system memory, is now selected by default.
Make sure to propagate memory:VASurface capsfeature to srcpad caps
only for GStreamer >= 1.5 as the plug-in elements in GStreamer 1.4
core currently miss additional patches available in 1.5-git (1.6).
This is a temporary workaround.
If a multiview stream is decoded, multiple view components are submitted
as is downstream. It is the responsibility of the sink element to display
the required view components. By default, always select the frame buffer
that matches the view-id of the very first frame to be displayed.
However, introduce a "view-id" property to allow the selection of a
specific view component of interest to display.
Always record the VA surface that is currently being rendered, no matter
the fact we are using texturedblit or overlay. That's because in some
occasions, we need to refresh or resize the displayed contents based on
new events. e.g. user-resized window.
Besides, it's simpler to track the last video buffer in GstVaapiSink than
through the base sink "last-sample".
Add a "display-name" property to vaapisink so that the end user could
select the desired output. Keep "display-name" in-line with the existing
"display" (GstVaapiDisplayXXX type).
So, for X11 or GLX, the "display-name" is the usual display name as we
know for XOpenDisplay(); for Wayland, the "display-name" is the name used
for wl_display_connect(); and for DRM, the "display-name" is actually the
DRI device name.
https://bugzilla.gnome.org/show_bug.cgi?id=722247
Always add VideoAlignment bufferpool option if the downstream element
expects its own pool to be used but does not offer it through a proper
propose_allocation() implementation for instance, and that the ALLOCATION
query does not expose the availability of the Video Meta API.
This fixes propagation of video buffer stride information to Firefox.
Make sure to always prefer native internal formats for the VA surfaces
that get allocated. Also disable "direct-rendering" mode in this case.
This is needed so that to make sure that anything that gets out of the
decoder, or anything that gets into the encoder, is in native format
for the hardware, and thus the driver doesn't need to perform implicit
conversions in there. Interop with SW elements is still available with
fast implementations of VA imaging APIs.
Forbid shares of GstMemory instances, and rather make copy of it. This
effectively copies the GstMemory structure and enclosed metadata, but
this does not copy the VA surface contents itself. It should though.
This fixes preroll and makes sure to not download garbage for the first
frame when a SW rendering sink is used.
Use an image pool to hold VA images to be used for downloads/uploads
of contents for the associated surface.
This is an optmization for size. So, instead of creating as many VA
images as there are buffers (then VA surfaces) allocated, we only
maintain a minimal set of live VA images, thus preserving memory
resources.
Disable read-write mappings if "direct-rendering" is not supported.
Since the ordering of read and write operations is not specified,
this would require to always download the VA surface on _map(), then
commit the temporary VA image back to the VA surface on _unmap().
Some SW decoding plug-in elements still use R/W mappings though.
https://bugzilla.gnome.org/show_bug.cgi?id=733242
Fix error messages introduced in the previous commit for the _map()
imaplementation. Also use the new get_image_data() helper function
to determine the base pixels data buffer from a GstVaapiImage when
updating the video info structure from it.
Allow raw pixels of the whole frame to be mapped read-only. i.e. in
cases where the buffer pool is allocated without VideoMeta API, thus
individual planes cannot be mapped.
This is initial support for Firefox >= 30.
https://bugzilla.gnome.org/show_bug.cgi?id=731886
While creating the vaapi video allocator, make sure the associated
surface pool has correct format instead of defaulting to NV12 video
format even though there is no direct rendering support.
https://bugzilla.gnome.org/show_bug.cgi?id=732691
Make sure to always update the VA surface pointer whenever the proxy
changes. This used to only work when the VA surface is written to, in
interop with SW element ("upload" feature), and this now fixes cases
when the VA surface is needed for reading, in interop with SW element
("download" feature).
Always expose I420 format by default when the VA surface could be
mapped for interoperability with non harware accelerated elements.
However, the default behaviour remains the auto-plugging of vaapi
elements, down to the sink.
Side effect: "direct-rendering" mode is also disabled most of the
times as plain memcpy() from uncached speculative write combining
memory is not going to be efficient enough.
Fix support for VA surface download capability in vaapidecode element
for GStreamer >= 1.2. This is a fix to supporting libva-vdpau-driver,
but also the libva-intel-driver while performing hardware accelerated
conversions from the native VA surface format (NV12) to the desired
output VA image format.
For instance, this fixes pipelines involving vaapidecode ! xvimagesink.
https://bugzilla.gnome.org/show_bug.cgi?id=733243
When playbin/decodebin builds the pipeline, it puts decoders and sinks
into different bins and forwards the queries from bins to bins. So in
the initials steps the pipeline is built iteratively by playbin and
looks like this :
[filesrc]
[filesrc] -> [typefind]
[filesrc] -> [typefind] -> [demuxer]
[filesrc] -> [typefind] -> [demuxer] -> [decoder]
At this point the decoder is asked for its SRC caps and it will make a
choice based on what gst_pad_peer_query_caps() returns. The problem is
that the caps returns at that point includes caps features like ANY,
essentially because playbin can plug in additional elements like
videoscale, videoconv or deinterlace.
This patch adds a another call to
gst_vaapi_find_preferred_caps_feature() when the decoder decides its
allocation, to make sure we asks the downstream elements when the
entire pipeline has been built.
https://bugzilla.gnome.org/show_bug.cgi?id=731645
A compiler change showed me that tmp_rect went out of scope before
it was used. Move it to the beginning of the function instead.
https://bugzilla.gnome.org/show_bug.cgi?id=726363
Signed-off-by: Simon Farnsworth <simon.farnsworth@onelan.co.uk>
[added guards for GStreamer 0.10 builds]
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
We can avoid scanning for start codes again if the bitstream is fed
in NALU chunks. Currently, we always scan for start codes, and keep
track of remaining bits in a GstAdapter, even if, in practice, we
are likely receiving one GstBuffer per NAL unit. i.e. h264parse with
"nal" alignment.
https://bugzilla.gnome.org/show_bug.cgi?id=723284
[use gst_adapter_available_fast() to determine the top buffer size]
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Avoid reaching an assert if dynamic framerates (0/1) are used. One
way to solve this problem is to just stick field_duration to zero.
However, this means that, in presence of interlaced streams, the
very first field will never be displayed if precise presentation
timestamps are honoured.
https://bugzilla.gnome.org/show_bug.cgi?id=729604
ensure_srcpad_buffer_pool() tries to avoid unnecessarily deleting and
recreating filter_pool. Unfortunately, this also meant it didn't create
it if it did not exist.
Fix it to always create the buffer pool if it does not exist.
https://bugzilla.gnome.org/show_bug.cgi?id=723834
Signed-off-by: Simon Farnsworth <simon.farnsworth@onelan.co.uk>
Reset deinterlacer state, i.e. past reference frames used for advanced
deinterlacing, when there is some discontinuity detected in the course
of processing source buffers.
This fixes support for advanced deinterlacing when a seek occurred.
https://bugzilla.gnome.org/show_bug.cgi?id=720375
[fixed type of pts_diff variable, fetch previous buffer PTS from the
history buffer, reduce heuristic for detecting discontinuity]
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Apply video cropping regions stored in GstVideoCropMeta, or in older
GstVaapiSurfaceProxy representation, to VPP pipelines. In non-VPP modes,
the crop meta are already propagated to the output buffers.
https://bugzilla.gnome.org/show_bug.cgi?id=720730
deinterlace-mode didn't behave in the way you'd expect if you have
past experience of the deinterlace element. There were two bugs:
1. "auto" mode wouldn't deinterlace "interleaved" buffers, only "mixed".
2. "force" mode wouldn't deinterlace "mixed" buffers flagged as progressive.
Fix these up, and add assertions and error messages to detect cases that
aren't handled.
https://bugzilla.gnome.org/show_bug.cgi?id=726361
Signed-off-by: Simon Farnsworth <simon.farnsworth@onelan.co.uk>
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
gst_video_info_set_format() does not preserve video info properties. In
order to keep important information in the caps such as interlace mode,
framerate, pixel aspect ratio, ... we need to manually copy back those
properties after setting the new video format.
https://bugzilla.gnome.org/show_bug.cgi?id=722276
It can happen that there is a pool provided that does not advertise
the vappivideometa. We should unref that pool before using our own.
Discovered with vaapidecode ! {glimagesink,cluttersink}
https://bugzilla.gnome.org/show_bug.cgi?id=724957
[fixed compilation by adding the missing semi-colon]
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Parse any pending data until a complete frame is obtained. This is a
memory optimization to avoid expansion of video packets stuffed into
the GstAdapter, and a fix to EOS condition to detect there is actually
pending data that needs to be decoded, and subsequently output.
https://bugzilla.gnome.org/show_bug.cgi?id=731831
Force early initializatin of the GstVaapiDisplay so that to make sure
that the sink element display object is presented first to upstream
elements, as it will be correctly featuring the requested display type
by the user.
Otherwise, we might end up in situations where a VA/X11 display is
initialized in vaapidecode, then we try VA/DRM display in vaapisink
(as requested by the "display" property), but this would cause a failure
because we cannot acquire a DRM display that was previously acquired
through another backend (e.g. VA/X11).
When a new display is settled through GstElement::set_context() (>= 1.2),
or GstVideoContext::set_context() (<= 1.0), then we shall also update the
associated display type.
The built-in video parsers elements are built into a single DSO named
libgstvaapi_parse.so. The various video parsers could be accessed as
vaapiparse_CODEC.
For now, this only includes a modified version of h264parse so that to
support H.264 MVC encoded streams.
Add initial support for Subset SPS, Prefix NAL and Slice Extension NAL
for non-base-view streams encoding, and the usual SPS, PPS and Slice
NALs for base-view encoding.
The H.264 Stereo High profile encoding mode will be turned on when the
"num-views" parameter is set to 2. The source (raw) YUV frames will be
considered as Left/Right view, alternatively.
Each of the two views has its own frames reordering pool and reference
frames list management system. Inter-view references are not supported
yet, so the views are encoded independently from each other.
Signed-off-by: Li Xiaowei <xiaowei.a.li@intel.com>
[limited to Stereo High profile per the definition of MAX_NUM_VIEWS]
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Fix generation of source tarballs when certain conditionals are not
met. e.g. always include all buildable codecparsers sources in the
distribution tarball, fix plug-in element sources set to include X11
and encoder bits.
This maps GstVideoColorimetry information in vaapisink's sinkpad caps
to GST_VAAPI_COLOR_STANDARD_* flags, if per-buffer information was not
available.
https://bugzilla.gnome.org/show_bug.cgi?id=722255
[factored out code, added SMPTE240M, handle per-buffer flags]
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Add support for hue, saturation, brightness and constrat adjustments.
Also fix cap info local copy to match the really expected cap subtype
of interest.
https://bugzilla.gnome.org/show_bug.cgi?id=720376
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Fix copy/paste error when submitting the "sharpen" value to the
GstVaapiFilter instance.
https://bugzilla.gnome.org/show_bug.cgi?id=720375
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
When gst_vaapisink_video_overlay_set_window_handle() is called early,
before the pipeline has been set to PLAYING, the display has not yet
been initialized and _PLUGIN_BASE_DISPLAY_TYPE() is not yet
up-to-date. For this reason the foreign XID is not attached.
Now _ensure_display() is called earlier.
https://bugzilla.gnome.org/show_bug.cgi?id=722244
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Fix vaapidecode to correctly report caps features downstream, when
a custom pipeline is built manually.
https://bugzilla.gnome.org/show_bug.cgi?id=719372
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Since vaapidecode provides buffer that can be mapped as regular memory,
those caps should be added to the template caps. That only applies to
GStreamer >= 1.2.
https://bugzilla.gnome.org/show_bug.cgi?id=720608
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
vaapidecode hangs when pipeline is stopped without any EOS, e.g. when
<Ctrl>+C is pressed, thus causing the srcpad task to keep running and
locked. This fixes a deadlock on state change from PAUSED to READY.
https://bugzilla.gnome.org/show_bug.cgi?id=720584
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
vaapiencode might hang when the pipeline is stopped without any EOS,
e.g. when <Ctrl>+C is pressed, thus causing the srcpad task to keep
running and locked. This fixes a possible deadlock on state change
from PAUSED to READY.
https://bugzilla.gnome.org/show_bug.cgi?id=720584
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Add gst_vaapi_video_format_new_template_caps_with_features() helper
function to add the supplied caps feature string on GStreamer >= 1.2.
Add gst_vaapi_find_preferred_caps_feature() helper function to discover
the "best" caps feature to use for the supplied pad. In practice, we
will always favor memory:VASurface first, then meta:GLTextureUploadMeta,
and finally the system memory caps.
https://bugzilla.gnome.org/show_bug.cgi?id=719372
The GLTextureUpload function is not in charge of doing the overlay
composition if any.
https://bugzilla.gnome.org/show_bug.cgi?id=721859
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Allow user to precise the largest profile to use for encoding due
to target decoder constraints. For instance, if CABAC entropy coding
mode is requested by "constrained-baseline" profile only is desired,
then an error is returned during codec configuration.
Also make sure that the suitable profile we derived actually matches
what the HW can cope with.
https://bugzilla.gnome.org/show_bug.cgi?id=719694
Various clean-ups to improve consistency and readability: rename some
variables, drop unused macro definitions, drop initialization of vars
that are zero-initialized from the base class, drop un-necessary casts.
Add interface to communicate the encoder resolution and related info
like framerate, interlaced vs. progressive, etc. This new interface
supersedes gst_vaapi_encoder_set_format() and doesn't use any GstCaps
but rather use GstVideoCodecState.
Note that gst_vaapi_encoder_set_codec_state() is also a synchronization
point for codec config. This means that the encoder is reconfigured
there to match the latest properties.
Don't try to destroy an encoder, in GstVideoEncoder::finish() handler,
if it was not created in the first place. Return "not-negotiated" error
since this means we did not even reach GstVideoEncoder::set_format(),
where the encoder could have been created.
This fixes a crash when the vaapiencode_* plug-in elements get deallocated
and that we failed to negotiate either pad.
https://bugzilla.gnome.org/show_bug.cgi?id=719704
The specified caps in gst_video_encoder_set_output_state() function
arguments should not contain any resolution, pixel-aspect-ratio,
framerate, codec-data et al. Those rather should be set through the
returned GstVideoCodecState. This means that output caps creation
could be delayed until before gst_video_encoder_finish_frame() is
called.
This greatly simplifies the GstVideoEncoder::set_format() callback
by the way.
Make base GstVaapiEncode class an abstract type so that we cannot
create an instance from it without going through any of the codec
specific derived class.
Rename a few member functions to make them more consistent:
- alloc_encoder(): now reduced to allocate the encoder object only;
- alloc_buffer(): allocate buffer from srcpad, and copy bitstream.
Update MPEG-2 and H.264 encode elements to cope with the new core
libgstvaapi properties API. i.e. all configurable properties are now
directly handled at the GstVaapiEncoder level.
Besides, this also makes sure to not use or modify the GstVaapiEncoder
private definitions directly. Private data need to remain private.
https://bugzilla.gnome.org/show_bug.cgi?id=719529
Add gst_vaapi_encoder_set_bitrate() interface to allow the user control
the bitrate for encoding. Currently, changing this parameter is only
valid before the first frame is encoded. Should the value be modified
afterwards, then GST_VAAPI_ENCODER_STATUS_ERROR_OPERATION_FAILED is
returned.
https://bugzilla.gnome.org/show_bug.cgi?id=719529
Add gst_vaapi_encoder_set_rate_control() interface to request a new
rate control mode for encoding. Changing the rate control mode is
only valid prior to encoding the very first frame. Afterwards, an
error ("operation-failed") is issued.
https://bugzilla.gnome.org/show_bug.cgi?id=719529
Fixes a crash when multiple vaapidecode elements are finalized since
the debug category is created once in the class init method.
This is a regression from git commit 7e58d60.
https://bugzilla.gnome.org/show_bug.cgi?id=721390
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Fix formts for various GST_DEBUG et al. invocations. More precisely,
make size_t arguments use the %zu format specifier accordingly; force
XID formats to be a 32-bit unsigned integer; and fix the format used
for gst_vaapi_create_surface_with_format() error cases since we have
been using strings nowadays.
Use standard GstVideoInfo related functions to build the output caps,
thus directly preserving additional fields as needed, instead of
manually copying them over through gst_vaapi_append_surface_caps().
Also ensure that the input caps are fixated first.
Add new helper functions to build video template caps.
- gst_vaapi_video_format_new_template_caps():
create GstCaps with size, frame rate and PAR to full range
- gst_vaapi_video_format_new_template_caps_from_list():
try to create a "simplified" list from the supplied formats
Add new helper functions to build GValues from GstVideoFormat:
- gst_vaapi_value_set_format():
build a GValue from the supplied video format
- gst_vaapi_value_set_format_list():
build a GValue list from the supplied array of video formats
Replace gst_vaapi_display_get_{decode,encode}_caps() APIs with more
more convenient APIs that return an array of GstVaapiProfile instead
of GstCaps: gst_vaapi_display_get_{decode,encode}_profiles().
Replace gst_vaapi_display_get_{image,subpicture}_caps() APIs, that
returned GstCaps, with more convenient APIs that return an array of
GstVideoFormat: gst_vaapi_display_get_{image,subpicture}_formats().
Makes the copies of a buffer reference their own GLTextureUploadMeta
user data and prevent the original buffer accessing already freed
memory if its copies has been released and freed.
https://bugzilla.gnome.org/show_bug.cgi?id=720336
[Propagate the original meta texture to the copy too]
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Factor out propose_allocation() hooks, creation of video buffer pool
for the sink pad, conversion from raw YUV buffers to VA surface backed
buffers. Update vaapidecode, vaapiencode and vaapipostproc to cope
with the new GstVaapiPluginBase abilities.
Fix display creation code to check that any display obtained from a
neighbour actually has the type we expect. Note: if display type is
set to "any", we can then accept any VA display type.
Move common VA display creation code to GstVaapiPluginBase, with the
default display type remaining "any". Also add a "display-changed"
hook so that subclasses could perform additional tasks when/if the
VA display changed, due to a new display type request for instance.
All plug-ins are updated to cope with the new internal APIs.
Introduce a new GstVaapiPluginBase object that will contain all common
data structures and perform all common tasks. First step is to have a
single place to hold VA displays.
While we are at it, also make sure to store and subsequently release
the appropriate debug category for the subclasses.
The GLTextureUploadMeta implementation assumed that for each upload()
sequence, the supplied texture id is always the same as the one that
was previously cached into the underlying GstVaapiTexture. Cope with
any texture id change the expense to recreate the underlying VA/GLX
resources.
https://bugzilla.gnome.org/show_bug.cgi?id=719643
Requesting the GLTextureUpload meta on buffers in the bufferpool
prevents such metas from being de-allocated when buffers are released
in the sink.
This is particulary useful in terms of performance when using the
GLTextureUploadMeta API since the GstVaapiTexture associated with
the target texture is stored in the meta.
https://bugzilla.gnome.org/show_bug.cgi?id=712558
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Make GstVideoGLTextureUploadMeta::upload() implementation more robust
when the GstVaapiTexture associated with the supplied texture id could
not be created.
Clean public APIs up so that to better align with the decoder APIs.
Most importantly, gst_vaapi_encoder_get_buffer() is changed to only
return the VA coded buffer proxy. Also provide useful documentation
for the public APIs.
Refactor the GstVaapiCodedBuffer APIs so that to more clearly separate
public and private interfaces. Besides, the map/unmap APIs should not
be exposed as is but appropriate accessors should be provided instead.
* GstVaapiCodedBuffer: VA coded buffer abstraction
- gst_vaapi_coded_buffer_get_size(): get coded buffer size.
- gst_vaapi_coded_buffer_copy_into(): copy coded buffer into GstBuffer
* GstVaapiCodedBufferPool: pool of VA coded buffer objects
- gst_vaapi_coded_buffer_pool_new(): create a pool of coded buffers of
the specified max size, and bound to the supplied encoder
* GstVaapiCodedBufferProxy: pool-allocated VA coded buffer object proxy
- gst_vaapi_coded_buffer_proxy_new_from_pool(): create coded buf from pool
- gst_vaapi_coded_buffer_proxy_get_buffer(): get underlying coded buffer
- gst_vaapi_coded_buffer_proxy_get_buffer_size(): get coded buffer size
Rationale: more optimized transfer functions might be provided in the
future, thus rendering the map/unmap mechanism obsolete or sub-optimal.
https://bugzilla.gnome.org/show_bug.cgi?id=719775
Fix GstElement::set_context() implementation for all plug-in elements
to avoid leaking an extra reference to the VA display, thus preventing
correct cleanup of VA resources in GStreamer 1.2 builds.
Return earlier if the creation of a VA display failed. Likewise, simplify
gst_vaapi_video_context_propagate() now that we are guaranteed to have a
valid VA display.
When GstVideoMeta maps were used, the supporting functions incorrectly
used gst_buffer_get_memory() instead of gst_buffer_peek_memory(), thus
always increasing the associated GstMemory reference count and giving
zero chance to actually release that, and subsequently the VA display.
Simplify GstVaapiVideoMeta to only hold a surface proxy, which is
now allocated from a surface pool. This also means that the local
reference to the VA surface is also gone, as it could be extracted
from the associated surface proxy.
Drop the following functions that are not longer used:
- gst_vaapi_video_buffer_new_with_surface()
- gst_vaapi_video_meta_new_with_surface()
- gst_vaapi_video_meta_set_surface()
- gst_vaapi_video_meta_set_surface_from_pool()
Fix gst_vaapi_video_meta_new_from_pool() to allocate VA surface proxies
from surface pools instead of plain VA surfaces. This is to simplify
allocations now that surface proxies are created from a surface pool.
Optimize gst_vaapiencode_handle_frame() to avoid extra memory allocation,
and in particular the GstVaapiEncObjUserData object. i.e. directly use
the VA surface proxy from the source buffer. This also makes the user
data attached to the GstVideoCodecFrame more consistent between both
the decoder and encoder plug-in elements.
Simplify gst_vaapiencode_push_frame(), while also removing the call
to gst_video_encoder_negotiate() since this is implicit in _finish()
if caps changed. Also fixed memory leaks that occured on error.
Constify pointers wherever possible. Drop unused variables, and use
consistent variable names. Fix gst_vaapiencode_h264_allocate_buffer()
to correctly report errors, especially when in-place conversion from
bytestream to avcC format failed.
Move "rate-control" mode and "bitrate" properties to the GstVaapiEncode
base class. The actual range of supported rate control modes is currently
implemented as a plug-in element hook. This ought to be determined from
the GstVaapiEncoder object instead, i.e. from libgstvaapi.
Align the plug-in debug category to its actual name. i.e. enable debug
logs through vaapiencode_<CODEC> where <CODEC> is mpeg2, h264, etc. Fix
the plug-in element description to make it more consistent with other
VA-API plug-ins.
Add a GST_VAAPIENCODE_CAST() helper to avoid run-time checks against
the GObject type system. We are guaranteed to only deal with the same
plug-in element object.
Allow vaapiencode plug-in elements to encode from raw YUV buffers.
The most efficient way to do so is to let the vaapiencode elements
allocate a buffer pool, and subsequently buffers from it. This means
that upstream elements are expected to honour downstream pools.
If upstream elements insist on providing their own allocated buffers
to the vaapiencode elements, then it possibly would be more efficient
to insert a vaapipostproc element before the vaapiencode element.
This is because vaapipostproc currently has better support than other
elements for "foreign" raw YUV buffers.
Add GstVaapiEncodeMPEG2 element object. The actual plug-in element
is called "vaapiencode_mpeg2".
Valid properties:
- rate-control: rate control mode (default: cqp - constant QP)
- bitrate: desired bitrate in kbps (default: auto-calculated)
- key-period: maximal distance between two key frames (default: 30)
- max-bframes: number of B-frames between I and P (default: 2)
- quantizer: constant quantizer (default: 8)
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
