Commit graph

1871 commits

Author SHA1 Message Date
Gwenole Beauchesne
cafb25c4d1 vaapidecode: add support for VideoAlignment bufferpool option.
Always add VideoAlignment bufferpool option if the downstream element
expects its own pool to be used but does not offer it through a proper
propose_allocation() implementation for instance, and that the ALLOCATION
query does not expose the availability of the Video Meta API.

This fixes propagation of video buffer stride information to Firefox.
2014-07-25 15:52:06 +02:00
Gwenole Beauchesne
1d1be0ae76 vaapivideomemory: always prefer native VA surface formats.
Make sure to always prefer native internal formats for the VA surfaces
that get allocated. Also disable "direct-rendering" mode in this case.

This is needed so that to make sure that anything that gets out of the
decoder, or anything that gets into the encoder, is in native format
for the hardware, and thus the driver doesn't need to perform implicit
conversions in there. Interop with SW elements is still available with
fast implementations of VA imaging APIs.
2014-07-25 15:44:58 +02:00
Gwenole Beauchesne
e0e869f536 vaapivideomemory: disallow memory shares across buffers, use a copy.
Forbid shares of GstMemory instances, and rather make copy of it. This
effectively copies the GstMemory structure and enclosed metadata, but
this does not copy the VA surface contents itself. It should though.

This fixes preroll and makes sure to not download garbage for the first
frame when a SW rendering sink is used.
2014-07-24 11:58:29 +02:00
Gwenole Beauchesne
3a762284a2 vaapivideomemory: use an image pool to cache objects.
Use an image pool to hold VA images to be used for downloads/uploads
of contents for the associated surface.

This is an optmization for size. So, instead of creating as many VA
images as there are buffers (then VA surfaces) allocated, we only
maintain a minimal set of live VA images, thus preserving memory
resources.
2014-07-24 06:55:26 +02:00
Gwenole Beauchesne
c3643b42a4 vaapivideomemory: forbid R/W mappings if non direct-rendering mode.
Disable read-write mappings if "direct-rendering" is not supported.
Since the ordering of read and write operations is not specified,
this would require to always download the VA surface on _map(), then
commit the temporary VA image back to the VA surface on _unmap().

Some SW decoding plug-in elements still use R/W mappings though.

https://bugzilla.gnome.org/show_bug.cgi?id=733242
2014-07-24 00:14:09 +02:00
Gwenole Beauchesne
f2ce28e4b7 vaapivideomemory: minor code clean-ups.
Fix error messages introduced in the previous commit for the _map()
imaplementation. Also use the new get_image_data() helper function
to determine the base pixels data buffer from a GstVaapiImage when
updating the video info structure from it.
2014-07-23 23:53:15 +02:00
Gwenole Beauchesne
9cad85a936 vaapivideomemory: add support for raw pixels mappings.
Allow raw pixels of the whole frame to be mapped read-only. i.e. in
cases where the buffer pool is allocated without VideoMeta API, thus
individual planes cannot be mapped.

This is initial support for Firefox >= 30.

https://bugzilla.gnome.org/show_bug.cgi?id=731886
2014-07-23 19:14:47 +02:00
Sreerenj Balachandran
9cb3acc813 vaapivideomemory: fix determination of the surface pool format.
While creating the vaapi video allocator, make sure the associated
surface pool has correct format instead of defaulting to NV12 video
format even though there is no direct rendering support.

https://bugzilla.gnome.org/show_bug.cgi?id=732691
2014-07-23 18:48:32 +02:00
Gwenole Beauchesne
b07de8a7f9 vaapivideomemory: fix association of surface to proxy.
Make sure to always update the VA surface pointer whenever the proxy
changes. This used to only work when the VA surface is written to, in
interop with SW element ("upload" feature), and this now fixes cases
when the VA surface is needed for reading, in interop with SW element
("download" feature).
2014-07-23 18:01:21 +02:00
Gwenole Beauchesne
dac20cecb4 plugins: expose I420 format for interop with SW elements.
Always expose I420 format by default when the VA surface could be
mapped for interoperability with non harware accelerated elements.
However, the default behaviour remains the auto-plugging of vaapi
elements, down to the sink.

Side effect: "direct-rendering" mode is also disabled most of the
times as plain memcpy() from uncached speculative write combining
memory is not going to be efficient enough.
2014-07-23 16:25:08 +02:00
Gwenole Beauchesne
dc6d529830 plugins: allow download capability to vaapidecode element.
Fix support for VA surface download capability in vaapidecode element
for GStreamer >= 1.2. This is a fix to supporting libva-vdpau-driver,
but also the libva-intel-driver while performing hardware accelerated
conversions from the native VA surface format (NV12) to the desired
output VA image format.

For instance, this fixes pipelines involving vaapidecode ! xvimagesink.

https://bugzilla.gnome.org/show_bug.cgi?id=733243
2014-07-22 19:06:52 +02:00
Fabrice Bellet
00ca734ebf decoder: mpeg4: fix picture decoder return value for skipped frames.
The picture decoder should return GST_VAAPI_DECODER_STATUS_DROP_FRAME
when a frame should be skipped, so the stream processing is not stalled.

https://bugzilla.gnome.org/show_bug.cgi?id=733324
2014-07-21 10:17:00 +02:00
Jan Schmidt
22dc8c4251 decoder: mpeg2: respect any input PTS provided for a frame.
The timestamp generator in gstvaapidecoder_mpeg2.c always interpolated
frame timestamps within a GOP, even when it's been fed input PTS for
every frame.

That leads to incorrect output timestamps in some situations - for example
live playback where input timestamps have been scaled based on arrival time
from the network and don't exactly match the framerate.

https://bugzilla.gnome.org/show_bug.cgi?id=732719
2014-07-04 07:38:50 +02:00
Sreerenj Balachandran
99bf1b1f98 GstVaapiObject: make gst_vaapi_object_new() more robust.
Forbid GstVaapiObject to be created without an associated klass spec.
It is mandatory that the subclass implements an adequate .finalize()
hook, so it shall provide a valid GstVaapiObjectClass.

https://bugzilla.gnome.org/show_bug.cgi?id=722757

[made non-NULL klass argument to gst_vaapi_object_new() a requirement]
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2014-07-03 23:50:11 +02:00
Sreerenj Balachandran
502952d080 GstVaapiObject: initialize the derived object data with init() hook.
Call the subclass .init() function in gst_vaapi_object_new(), if
needed. The default behaviour is to zero initialize the subclass
object data, then the .init() function can be used to initialize
fields to non-default values, e.g. VA object ids to VA_INVALID_ID.

Also fix the gst_vaapi_object_new() description, which was merely
copied from GstVaapiMiniObject.

https://bugzilla.gnome.org/show_bug.cgi?id=722757

[changed to always zero initialize the subclass]
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2014-07-03 23:38:25 +02:00
Lionel Landwerlin
334a0ca8bd vaapidecode: make decoder work with playbin
When playbin/decodebin builds the pipeline, it puts decoders and sinks
into different bins and forwards the queries from bins to bins. So in
the initials steps the pipeline is built iteratively by playbin and
looks like this :

[filesrc]

[filesrc] -> [typefind]

[filesrc] -> [typefind] -> [demuxer]

[filesrc] -> [typefind] -> [demuxer] -> [decoder]

At this point the decoder is asked for its SRC caps and it will make a
choice based on what gst_pad_peer_query_caps() returns. The problem is
that the caps returns at that point includes caps features like ANY,
essentially because playbin can plug in additional elements like
videoscale, videoconv or deinterlace.

This patch adds a another call to
gst_vaapi_find_preferred_caps_feature() when the decoder decides its
allocation, to make sure we asks the downstream elements when the
entire pipeline has been built.

https://bugzilla.gnome.org/show_bug.cgi?id=731645
2014-07-03 23:07:51 +02:00
Simon Farnsworth
4d2de696a9 vaapipostproc: don't let tmp_rect go out of scope.
A compiler change showed me that tmp_rect went out of scope before
it was used. Move it to the beginning of the function instead.

https://bugzilla.gnome.org/show_bug.cgi?id=726363

Signed-off-by: Simon Farnsworth <simon.farnsworth@onelan.co.uk>
[added guards for GStreamer 0.10 builds]
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2014-07-03 22:59:36 +02:00
Gwenole Beauchesne
04b82421e5 AUTHORS: updates. 2014-07-03 22:51:41 +02:00
Gwenole Beauchesne
5fe46af4d3 README: updates.
Drop references to deprecated plugins (vaapiupload, vaapidownload),
mention that support for GStreamer 0.10 is deprecated, make overview
more descriptive in certain aspects.
2014-07-03 22:35:29 +02:00
Gwenole Beauchesne
9a3e4f9d3e NEWS: updates. 2014-07-03 22:21:39 +02:00
Gwenole Beauchesne
1175ecb412 build: mention that support for GStreamer 0.10 is deprecated. 2014-07-03 19:43:05 +02:00
Gwenole Beauchesne
700fd242cc build: fix for GStreamer 0.10. 2014-07-03 19:43:04 +02:00
Gwenole Beauchesne
65f897df6a build: fix for GStreamer 1.0.x. 2014-07-03 18:03:52 +02:00
Gwenole Beauchesne
b5f1bdd59a decoder: h264: detect incorrectly paired fields in frames.
When a DPB flush is required, e.g. at a natural and of stream or issued
explicitly through an IDR, try to detect any frame left in the DPB that
is interlaced but does not contain two decoded fields. In that case, mark
the picture as having a single field only.

This avoids a hang while decoding tv_cut.mkv.
2014-07-03 14:56:26 +02:00
Gwenole Beauchesne
a3e49d6d13 decoder: h264: simplify the DPB output process.
Simplify the dpb_output() function to exclusively rely on the frame store
buffer to output, since this is now always provided. Besides, also fix
cases where split fields would not be displayed.

This is a regression from f48b1e0.
2014-07-03 14:56:23 +02:00
Gwenole Beauchesne
e6cdacee65 h264parse: drop patches merged upstream.
0003-h264parse-fix-and-optimize-NAL-collection-function.patch
0005-h264parse-introduce-new-state-tracking-variables.patch
0006-h264parse-improve-conditions-for-skipping-NAL-units.patch
0007-h264parse-fix-collection-of-access-units-to-preserve.patch
2014-07-01 17:20:44 +02:00
Gwenole Beauchesne
cba9b97300 decoder: h264: fix memory leak in PPS.
Cope with latest changes from codecparsers/h264. It is now required
to explicitly clear the GstH264PPS structure as it could contain
additional allocations (slice_group_ids).
2014-07-01 17:18:08 +02:00
Gwenole Beauchesne
4263effee5 codecparsers: update to gst-vaapi-branch commit 2d53b69.
c4ace00 h264parse: fix collection of access units to preserve config headers
0f9f7c9 h264parse: improve conditions for skipping NAL units
9ffb25c h264parse: introduce new state tracking variables
64955d3 h264parse: fix and optimize NAL collection function
13cd2a3 h264: clarifications and documentation fixes
53e7dd1 h264: fix identification of EOSEQ and EOS NALs
18f0de0 h264: fix memory leak in GstH264PPS
fdcb54c h264: fix typo in GstH264VUIParams description
fd4dae9 vp8: move up built-in range decoder private data
2014-07-01 17:15:47 +02:00
Gwenole Beauchesne
702de9ad2f decoder: propagate MVC metadata ("view-id", head of multiview set).
Add new GstVaapiSurfaceProxy flag FFB, which means "first frame in
bundle", and really expresses the first view component of a multi
view coded frame. e.g. in H.264 MVC, the surface proxy has flag FFB
set if VOIdx = 0.

Likewise, new API is exposed to retrieve the associated "view-id".
2014-06-30 19:13:25 +02:00
Gwenole Beauchesne
70a48e0962 decoder: propagate "one-field" flags.
Allow decoders to set the "one-field" attribute when the decoded frame
genuinely has a single field, or if the second field was mis-decoded but
we still want to display the first field.
2014-06-30 19:13:25 +02:00
Gwenole Beauchesne
f040f4f8b4 decoder: output decoded frames only once.
Make sure to output the decoded picture, and push the associated
GstVideoCodecFrame, only once. The frame fully represents what needs
to be output, included for interlaced streams. Otherwise, the base
GstVideoDecoder class would release the frame twice.

Anyway, the general process is to output decoded frames only when
they are complete. By complete, we mean a full frame was decoded or
both fields of a frame were decoded.
2014-06-30 19:13:25 +02:00
Gwenole Beauchesne
8bdef56cd4 decoder: h264: decode current picture earlier.
Slightly optimize decoding process by submitting the current VA surface
for decoding earlier to the hardware, and perform the reference picture
marking process and DPB update process afterwards.

This is a minor optimization to let the video decode engine kick in work
earlier, thus improving parallel resources utilization.
2014-06-30 19:13:25 +02:00
Gwenole Beauchesne
f48b1e0cd6 decoder: h264: fix output of second field when first field is not in DPB.
Fix decoding of interlaced streams where a first field (e.g. B-slice)
was immediately output and the current decoded field is to be paired
with that former frame, which is no longer in DPB.

https://bugzilla.gnome.org/show_bug.cgi?id=701340
2014-06-30 19:13:25 +02:00
Gwenole Beauchesne
a208a80c29 decoder: h264: slightly optimize the process to detect new pictures.
Optimize the process to detect new pictures or start of new access
units by checking if the previous NAL unit was the end of a picture,
or the end of the previous access unit.
2014-06-30 11:51:59 +02:00
Gwenole Beauchesne
8ed129b8bd decoder: h264: handle access unit ("au") optimization.
Optimize parsing when buffers are supplied with access unit alignment.
This helps determining faster when the end of an access unit is reached.
2014-06-30 11:51:57 +02:00
Gwenole Beauchesne
850d3d6a4d decoder: h264: fix tracking of DPB size changes.
Add support for MVC streams with multiple SPS and subset SPS headers
emitted regularly, e.g. at around every I-frame. Track the maximum
number of views in ensure_context() and really reset the DPB size to
the expected value, always. i.e. even if it decreased. dpb_reset()
only cares of ensuring the DPB allocation.
2014-06-28 07:41:00 +02:00
Gwenole Beauchesne
9169c520cb decoder: h264: fix the DPB compaction process.
Fix the compaction process when the DPB is cleared for a specific
view, i.e. fix the process of filling in the holes resulting from
removing frame buffers matching the current picture.
2014-06-27 20:44:24 +02:00
Sreerenj Balachandran
95d7f95971 encoder: h264: generate new SPS only when codec config changed.
It is not necessary to periodically send SPS or subset SPS headers.
This is up to the upper layer (e.g. transport layer) to decide on
if/how to periodically submit those. For now, only generate new SPS
or subset SPS headers when the codec config changed.

Note: the upper layer could readily determine the config headers
(SPS/PPS) through the gst_vaapi_encoder_h264_get_codec_data() function.

https://bugzilla.gnome.org/show_bug.cgi?id=732083

Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2014-06-27 20:26:21 +02:00
Gwenole Beauchesne
0d971bce26 encoder: h264: track encoder config changes.
Track and report when encoder configuration changed. For now, this covers
resolution, profile/level and bitrate changes.
2014-06-27 20:26:18 +02:00
Gwenole Beauchesne
a12662fd3e encoder: h264: add pixel-aspect-ratio to VUI parameters.
Report sample aspect ratio (SAR) as present, and make it match what
we have obtained from the user as pixel-aspect-ratio (PAR). i.e. the
VUI parameter aspect_ratio_info_present_flag now defaults to TRUE.
2014-06-27 13:15:13 +02:00
Sreerenj Balachandran
a4bd8450f7 encoder: h264: fix number of anchor and non-anchor reference pictures.
Set the value of num_anchor_refs_l0, num_anchor_refs_l1, num_non_anchor_refs_l0,
and num_non_anchor_refs_l1 to zero since the inter-view prediction is not yet
supported.
2014-06-27 11:51:32 +02:00
Sreerenj Balachandran
8ebf60124a encoder: h264: fix timing_info_present_flag value in subset SPS.
When the seq_parameter_set_data() syntax structure is present in a subset
sequence parameter set and vui_parameters_present_flag is equal to 1, then
timing_info_present_flag shall be equal to 0 (H.7.4.2.1.1).
2014-06-27 11:48:40 +02:00
Sreerenj Balachandran
619a79943a encoder: h264: add cpbBrNalFactor values for MVC profiles. 2014-06-27 11:48:36 +02:00
Gwenole Beauchesne
1650471da3 h264parse: fix detection of access unit boundaries for MVC.
The gst_h264_parse_collect_nal() function is a misnomer. In reality,
this function is used to determine access unit boundaries, i.e. that
is the key function for alignment=au output format generation.
2014-06-26 14:51:32 +02:00
Gwenole Beauchesne
92c7d9e6a5 h264parse: fix collection of access units to preserve config headers.
Always use a GstAdapter when collecting access units (alignment="au")
in either byte-stream or avcC format. This is required to properly
preserve config headers like SPS and PPS when invalid or broken NAL
units are subsequently parsed.

More precisely, this fixes scenario like:
<SPS> <PPS> <invalid-NAL> <slice>

where we used to reset the output frame buffer when an invalid or
broken NAL is parsed, i.e. SPS and PPS NAL units were lost, thus
preventing the next slice unit to be decoded, should this also
represent any valid data.

https://bugzilla.gnome.org/show_bug.cgi?id=732203
2014-06-26 14:49:53 +02:00
Gwenole Beauchesne
2905209d9b h264parse: improve conditions for skipping NAL units.
Carefully track cases when skipping broken or invalid NAL units is
necessary. In particular, always allow NAL units to be processed
and let that gst_h264_parse_process_nal() function decide on whether
the current NAL needs to be dropped or not.

This fixes parsing of streams with SEI NAL buffering_period() message
inserted between SPS and PPS, or SPS-Ext NAL following a traditional
SPS NAL unit, among other cases too.

Practical examples from the H.264 AVC conformance suite include
alphaconformanceG, CVSE2_Sony_B, CVSE3_Sony_H, CVSEFDFT3_Sony_E
when parsing in stream-format=byte-stream,alignment=au mode.

https://bugzilla.gnome.org/show_bug.cgi?id=732203
2014-06-26 14:48:08 +02:00
Gwenole Beauchesne
6d6caf17c9 h264parse: introduce new state tracking variables.
Improve parser state tracking by introducing new flags reflecting
it: "got-sps", "got-pps" and "got-slice". This is an addition for
robustness purposes.

Older have_sps and have_pps variables are kept because they have
a different meaning. i.e. they are used for deciding on when to
submit updated caps or not, and rather mean "have new SPS/PPS to
be submitted?"
2014-06-26 14:45:34 +02:00
Gwenole Beauchesne
c767be53d6 h264parse: default to byte-stream/nalu format (Annex B).
Always default to stream-format=byte-stream,alignment=nalu if avcC
format was not detected. This is the natural stream format specified
in the standard (Annex.B): a series of NAL units prefixed with the
usual start code.

https://bugzilla.gnome.org/show_bug.cgi?id=732167
2014-06-26 14:39:30 +02:00
Gwenole Beauchesne
8b473972e0 h264parse: fix and optimize NAL collection function.
Use gst_h264_parser_identify_nalu_unchecked() to identify the next
NAL unit. We don't want to parse the full NAL unit, but only the
header bytes and possibly the first RBSP byte for identifying the
first_mb_in_slice syntax element.

Also fix check for failure when returning from that function. The
only success condition for that is GST_H264_PARSER_OK, so use it.

https://bugzilla.gnome.org/show_bug.cgi?id=732154
2014-06-26 14:38:32 +02:00
Sreerenj Balachandran
173f32d8e5 encoder: h264: fix NAL unit types in packed headers.
Submit Prefix NAL headers (nal_unit_type = 14) before every packed
slice header (nal_unit_type = 1 or 5) only for the base view. In non
base views, a Coded Slice Extension NAL header (nal_unit_type = 20)
is required, with an appropriate nal_unit_header_mvc_extension() in
the NAL header bytes.

https://bugzilla.gnome.org/show_bug.cgi?id=732083
2014-06-26 11:08:32 +02:00