When playbin/decodebin builds the pipeline, it puts decoders and sinks
into different bins and forwards the queries from bins to bins. So in
the initials steps the pipeline is built iteratively by playbin and
looks like this :
[filesrc]
[filesrc] -> [typefind]
[filesrc] -> [typefind] -> [demuxer]
[filesrc] -> [typefind] -> [demuxer] -> [decoder]
At this point the decoder is asked for its SRC caps and it will make a
choice based on what gst_pad_peer_query_caps() returns. The problem is
that the caps returns at that point includes caps features like ANY,
essentially because playbin can plug in additional elements like
videoscale, videoconv or deinterlace.
This patch adds a another call to
gst_vaapi_find_preferred_caps_feature() when the decoder decides its
allocation, to make sure we asks the downstream elements when the
entire pipeline has been built.
https://bugzilla.gnome.org/show_bug.cgi?id=731645
A compiler change showed me that tmp_rect went out of scope before
it was used. Move it to the beginning of the function instead.
https://bugzilla.gnome.org/show_bug.cgi?id=726363
Signed-off-by: Simon Farnsworth <simon.farnsworth@onelan.co.uk>
[added guards for GStreamer 0.10 builds]
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Drop references to deprecated plugins (vaapiupload, vaapidownload),
mention that support for GStreamer 0.10 is deprecated, make overview
more descriptive in certain aspects.
When a DPB flush is required, e.g. at a natural and of stream or issued
explicitly through an IDR, try to detect any frame left in the DPB that
is interlaced but does not contain two decoded fields. In that case, mark
the picture as having a single field only.
This avoids a hang while decoding tv_cut.mkv.
Simplify the dpb_output() function to exclusively rely on the frame store
buffer to output, since this is now always provided. Besides, also fix
cases where split fields would not be displayed.
This is a regression from f48b1e0.
Cope with latest changes from codecparsers/h264. It is now required
to explicitly clear the GstH264PPS structure as it could contain
additional allocations (slice_group_ids).
c4ace00 h264parse: fix collection of access units to preserve config headers
0f9f7c9 h264parse: improve conditions for skipping NAL units
9ffb25c h264parse: introduce new state tracking variables
64955d3 h264parse: fix and optimize NAL collection function
13cd2a3 h264: clarifications and documentation fixes
53e7dd1 h264: fix identification of EOSEQ and EOS NALs
18f0de0 h264: fix memory leak in GstH264PPS
fdcb54c h264: fix typo in GstH264VUIParams description
fd4dae9 vp8: move up built-in range decoder private data
Add new GstVaapiSurfaceProxy flag FFB, which means "first frame in
bundle", and really expresses the first view component of a multi
view coded frame. e.g. in H.264 MVC, the surface proxy has flag FFB
set if VOIdx = 0.
Likewise, new API is exposed to retrieve the associated "view-id".
Allow decoders to set the "one-field" attribute when the decoded frame
genuinely has a single field, or if the second field was mis-decoded but
we still want to display the first field.
Make sure to output the decoded picture, and push the associated
GstVideoCodecFrame, only once. The frame fully represents what needs
to be output, included for interlaced streams. Otherwise, the base
GstVideoDecoder class would release the frame twice.
Anyway, the general process is to output decoded frames only when
they are complete. By complete, we mean a full frame was decoded or
both fields of a frame were decoded.
Slightly optimize decoding process by submitting the current VA surface
for decoding earlier to the hardware, and perform the reference picture
marking process and DPB update process afterwards.
This is a minor optimization to let the video decode engine kick in work
earlier, thus improving parallel resources utilization.
Fix decoding of interlaced streams where a first field (e.g. B-slice)
was immediately output and the current decoded field is to be paired
with that former frame, which is no longer in DPB.
https://bugzilla.gnome.org/show_bug.cgi?id=701340
Optimize the process to detect new pictures or start of new access
units by checking if the previous NAL unit was the end of a picture,
or the end of the previous access unit.
Add support for MVC streams with multiple SPS and subset SPS headers
emitted regularly, e.g. at around every I-frame. Track the maximum
number of views in ensure_context() and really reset the DPB size to
the expected value, always. i.e. even if it decreased. dpb_reset()
only cares of ensuring the DPB allocation.
Fix the compaction process when the DPB is cleared for a specific
view, i.e. fix the process of filling in the holes resulting from
removing frame buffers matching the current picture.
It is not necessary to periodically send SPS or subset SPS headers.
This is up to the upper layer (e.g. transport layer) to decide on
if/how to periodically submit those. For now, only generate new SPS
or subset SPS headers when the codec config changed.
Note: the upper layer could readily determine the config headers
(SPS/PPS) through the gst_vaapi_encoder_h264_get_codec_data() function.
https://bugzilla.gnome.org/show_bug.cgi?id=732083
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Report sample aspect ratio (SAR) as present, and make it match what
we have obtained from the user as pixel-aspect-ratio (PAR). i.e. the
VUI parameter aspect_ratio_info_present_flag now defaults to TRUE.
Set the value of num_anchor_refs_l0, num_anchor_refs_l1, num_non_anchor_refs_l0,
and num_non_anchor_refs_l1 to zero since the inter-view prediction is not yet
supported.
When the seq_parameter_set_data() syntax structure is present in a subset
sequence parameter set and vui_parameters_present_flag is equal to 1, then
timing_info_present_flag shall be equal to 0 (H.7.4.2.1.1).
The gst_h264_parse_collect_nal() function is a misnomer. In reality,
this function is used to determine access unit boundaries, i.e. that
is the key function for alignment=au output format generation.
Always use a GstAdapter when collecting access units (alignment="au")
in either byte-stream or avcC format. This is required to properly
preserve config headers like SPS and PPS when invalid or broken NAL
units are subsequently parsed.
More precisely, this fixes scenario like:
<SPS> <PPS> <invalid-NAL> <slice>
where we used to reset the output frame buffer when an invalid or
broken NAL is parsed, i.e. SPS and PPS NAL units were lost, thus
preventing the next slice unit to be decoded, should this also
represent any valid data.
https://bugzilla.gnome.org/show_bug.cgi?id=732203
Carefully track cases when skipping broken or invalid NAL units is
necessary. In particular, always allow NAL units to be processed
and let that gst_h264_parse_process_nal() function decide on whether
the current NAL needs to be dropped or not.
This fixes parsing of streams with SEI NAL buffering_period() message
inserted between SPS and PPS, or SPS-Ext NAL following a traditional
SPS NAL unit, among other cases too.
Practical examples from the H.264 AVC conformance suite include
alphaconformanceG, CVSE2_Sony_B, CVSE3_Sony_H, CVSEFDFT3_Sony_E
when parsing in stream-format=byte-stream,alignment=au mode.
https://bugzilla.gnome.org/show_bug.cgi?id=732203
Improve parser state tracking by introducing new flags reflecting
it: "got-sps", "got-pps" and "got-slice". This is an addition for
robustness purposes.
Older have_sps and have_pps variables are kept because they have
a different meaning. i.e. they are used for deciding on when to
submit updated caps or not, and rather mean "have new SPS/PPS to
be submitted?"
Always default to stream-format=byte-stream,alignment=nalu if avcC
format was not detected. This is the natural stream format specified
in the standard (Annex.B): a series of NAL units prefixed with the
usual start code.
https://bugzilla.gnome.org/show_bug.cgi?id=732167
Use gst_h264_parser_identify_nalu_unchecked() to identify the next
NAL unit. We don't want to parse the full NAL unit, but only the
header bytes and possibly the first RBSP byte for identifying the
first_mb_in_slice syntax element.
Also fix check for failure when returning from that function. The
only success condition for that is GST_H264_PARSER_OK, so use it.
https://bugzilla.gnome.org/show_bug.cgi?id=732154
Submit Prefix NAL headers (nal_unit_type = 14) before every packed
slice header (nal_unit_type = 1 or 5) only for the base view. In non
base views, a Coded Slice Extension NAL header (nal_unit_type = 20)
is required, with an appropriate nal_unit_header_mvc_extension() in
the NAL header bytes.
https://bugzilla.gnome.org/show_bug.cgi?id=732083
Fix search for a picture in the DPB that has a lower POC value than
the current picture. The dpb_find_lowest_poc() function will return
a picture with the lowest POC in DPB and that is marked as "needed
for output", but an additional check against the actual POC value
of the current picture is needed.
This is a regression from 1c46990.
https://bugzilla.gnome.org/show_bug.cgi?id=732130
Fix dpb_clear() to clear previous frame buffers only if they actually
exist to begin with. If the decoder bailed out early, e.g. when it
does not support a specific profile, that array of previous frames
might not be allocated beforehand.
We can avoid scanning for start codes again if the bitstream is fed
in NALU chunks. Currently, we always scan for start codes, and keep
track of remaining bits in a GstAdapter, even if, in practice, we
are likely receiving one GstBuffer per NAL unit. i.e. h264parse with
"nal" alignment.
https://bugzilla.gnome.org/show_bug.cgi?id=723284
[use gst_adapter_available_fast() to determine the top buffer size]
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
The `vaapipostproc' element could never determine if the H.264 stream
was interlaced, and thus always assumed it to be progressive. Fix the
H.264 decoder to report interlace-mode accordingly, thus allowing the
vaapipostproc element to automatically enable deinterlacing.
Avoid reaching an assert if dynamic framerates (0/1) are used. One
way to solve this problem is to just stick field_duration to zero.
However, this means that, in presence of interlaced streams, the
very first field will never be displayed if precise presentation
timestamps are honoured.
https://bugzilla.gnome.org/show_bug.cgi?id=729604
ensure_srcpad_buffer_pool() tries to avoid unnecessarily deleting and
recreating filter_pool. Unfortunately, this also meant it didn't create
it if it did not exist.
Fix it to always create the buffer pool if it does not exist.
https://bugzilla.gnome.org/show_bug.cgi?id=723834
Signed-off-by: Simon Farnsworth <simon.farnsworth@onelan.co.uk>
Reset deinterlacer state, i.e. past reference frames used for advanced
deinterlacing, when there is some discontinuity detected in the course
of processing source buffers.
This fixes support for advanced deinterlacing when a seek occurred.
https://bugzilla.gnome.org/show_bug.cgi?id=720375
[fixed type of pts_diff variable, fetch previous buffer PTS from the
history buffer, reduce heuristic for detecting discontinuity]
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Apply video cropping regions stored in GstVideoCropMeta, or in older
GstVaapiSurfaceProxy representation, to VPP pipelines. In non-VPP modes,
the crop meta are already propagated to the output buffers.
https://bugzilla.gnome.org/show_bug.cgi?id=720730
deinterlace-mode didn't behave in the way you'd expect if you have
past experience of the deinterlace element. There were two bugs:
1. "auto" mode wouldn't deinterlace "interleaved" buffers, only "mixed".
2. "force" mode wouldn't deinterlace "mixed" buffers flagged as progressive.
Fix these up, and add assertions and error messages to detect cases that
aren't handled.
https://bugzilla.gnome.org/show_bug.cgi?id=726361
Signed-off-by: Simon Farnsworth <simon.farnsworth@onelan.co.uk>
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
gst_video_info_set_format() does not preserve video info properties. In
order to keep important information in the caps such as interlace mode,
framerate, pixel aspect ratio, ... we need to manually copy back those
properties after setting the new video format.
https://bugzilla.gnome.org/show_bug.cgi?id=722276
It can happen that there is a pool provided that does not advertise
the vappivideometa. We should unref that pool before using our own.
Discovered with vaapidecode ! {glimagesink,cluttersink}
https://bugzilla.gnome.org/show_bug.cgi?id=724957
[fixed compilation by adding the missing semi-colon]
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Parse any pending data until a complete frame is obtained. This is a
memory optimization to avoid expansion of video packets stuffed into
the GstAdapter, and a fix to EOS condition to detect there is actually
pending data that needs to be decoded, and subsequently output.
https://bugzilla.gnome.org/show_bug.cgi?id=731831
The packed slice header and packed raw data need to be paired with
the submission of VAEncSliceHeaderParameterBuffer. So handle them
on a per-slice basis insted of a per-picture basis.
[removed useless initializer]
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
