Commit graph

1677 commits

Author SHA1 Message Date
Gwenole Beauchesne
92c7d9e6a5 h264parse: fix collection of access units to preserve config headers.
Always use a GstAdapter when collecting access units (alignment="au")
in either byte-stream or avcC format. This is required to properly
preserve config headers like SPS and PPS when invalid or broken NAL
units are subsequently parsed.

More precisely, this fixes scenario like:
<SPS> <PPS> <invalid-NAL> <slice>

where we used to reset the output frame buffer when an invalid or
broken NAL is parsed, i.e. SPS and PPS NAL units were lost, thus
preventing the next slice unit to be decoded, should this also
represent any valid data.

https://bugzilla.gnome.org/show_bug.cgi?id=732203
2014-06-26 14:49:53 +02:00
Gwenole Beauchesne
2905209d9b h264parse: improve conditions for skipping NAL units.
Carefully track cases when skipping broken or invalid NAL units is
necessary. In particular, always allow NAL units to be processed
and let that gst_h264_parse_process_nal() function decide on whether
the current NAL needs to be dropped or not.

This fixes parsing of streams with SEI NAL buffering_period() message
inserted between SPS and PPS, or SPS-Ext NAL following a traditional
SPS NAL unit, among other cases too.

Practical examples from the H.264 AVC conformance suite include
alphaconformanceG, CVSE2_Sony_B, CVSE3_Sony_H, CVSEFDFT3_Sony_E
when parsing in stream-format=byte-stream,alignment=au mode.

https://bugzilla.gnome.org/show_bug.cgi?id=732203
2014-06-26 14:48:08 +02:00
Gwenole Beauchesne
6d6caf17c9 h264parse: introduce new state tracking variables.
Improve parser state tracking by introducing new flags reflecting
it: "got-sps", "got-pps" and "got-slice". This is an addition for
robustness purposes.

Older have_sps and have_pps variables are kept because they have
a different meaning. i.e. they are used for deciding on when to
submit updated caps or not, and rather mean "have new SPS/PPS to
be submitted?"
2014-06-26 14:45:34 +02:00
Gwenole Beauchesne
c767be53d6 h264parse: default to byte-stream/nalu format (Annex B).
Always default to stream-format=byte-stream,alignment=nalu if avcC
format was not detected. This is the natural stream format specified
in the standard (Annex.B): a series of NAL units prefixed with the
usual start code.

https://bugzilla.gnome.org/show_bug.cgi?id=732167
2014-06-26 14:39:30 +02:00
Gwenole Beauchesne
8b473972e0 h264parse: fix and optimize NAL collection function.
Use gst_h264_parser_identify_nalu_unchecked() to identify the next
NAL unit. We don't want to parse the full NAL unit, but only the
header bytes and possibly the first RBSP byte for identifying the
first_mb_in_slice syntax element.

Also fix check for failure when returning from that function. The
only success condition for that is GST_H264_PARSER_OK, so use it.

https://bugzilla.gnome.org/show_bug.cgi?id=732154
2014-06-26 14:38:32 +02:00
Sreerenj Balachandran
173f32d8e5 encoder: h264: fix NAL unit types in packed headers.
Submit Prefix NAL headers (nal_unit_type = 14) before every packed
slice header (nal_unit_type = 1 or 5) only for the base view. In non
base views, a Coded Slice Extension NAL header (nal_unit_type = 20)
is required, with an appropriate nal_unit_header_mvc_extension() in
the NAL header bytes.

https://bugzilla.gnome.org/show_bug.cgi?id=732083
2014-06-26 11:08:32 +02:00
Sreerenj Balachandran
a7c27bb7d5 encoder: h264: add missing field in packed Subset SPS header.
Write the missing num_level_values_signalled_minus1 syntax element
into the packed header for subset sequence parameter set.

https://bugzilla.gnome.org/show_bug.cgi?id=732083
2014-06-26 11:00:47 +02:00
Gwenole Beauchesne
bea0d07042 decoder: h264: fix marking of non-reference picture into DPB.
Fix search for a picture in the DPB that has a lower POC value than
the current picture. The dpb_find_lowest_poc() function will return
a picture with the lowest POC in DPB and that is marked as "needed
for output", but an additional check against the actual POC value
of the current picture is needed.

This is a regression from 1c46990.

https://bugzilla.gnome.org/show_bug.cgi?id=732130
2014-06-25 22:38:35 +02:00
Gwenole Beauchesne
8db72147c7 decoder: h264: fix DPB clear when no decoding actually started.
Fix dpb_clear() to clear previous frame buffers only if they actually
exist to begin with. If the decoder bailed out early, e.g. when it
does not support a specific profile, that array of previous frames
might not be allocated beforehand.
2014-06-19 17:08:47 +02:00
Sreerenj Balachandran
cb9f98f0d5 decoder: h264: add support for NALU "alignment" optimization.
We can avoid scanning for start codes again if the bitstream is fed
in NALU chunks. Currently, we always scan for start codes, and keep
track of remaining bits in a GstAdapter, even if, in practice, we
are likely receiving one GstBuffer per NAL unit. i.e. h264parse with
"nal" alignment.

https://bugzilla.gnome.org/show_bug.cgi?id=723284

[use gst_adapter_available_fast() to determine the top buffer size]
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2014-06-19 15:29:12 +02:00
Gwenole Beauchesne
95c781c34f decoder: h264: fix caps to report interlace-mode accordingly.
The `vaapipostproc' element could never determine if the H.264 stream
was interlaced, and thus always assumed it to be progressive. Fix the
H.264 decoder to report interlace-mode accordingly, thus allowing the
vaapipostproc element to automatically enable deinterlacing.
2014-06-19 15:13:04 +02:00
Gwenole Beauchesne
9d417815ab vaapipostproc: don't crash with dynamic framerate (0/1).
Avoid reaching an assert if dynamic framerates (0/1) are used. One
way to solve this problem is to just stick field_duration to zero.
However, this means that, in presence of interlaced streams, the
very first field will never be displayed if precise presentation
timestamps are honoured.

https://bugzilla.gnome.org/show_bug.cgi?id=729604
2014-06-19 14:51:01 +02:00
Simon Farnsworth
1a2c06a81c vaapipostproc: create filter surface pool if it does not exist yet.
ensure_srcpad_buffer_pool() tries to avoid unnecessarily deleting and
recreating filter_pool. Unfortunately, this also meant it didn't create
it if it did not exist.

Fix it to always create the buffer pool if it does not exist.

https://bugzilla.gnome.org/show_bug.cgi?id=723834

Signed-off-by: Simon Farnsworth <simon.farnsworth@onelan.co.uk>
2014-06-18 17:30:12 +02:00
Zhao, Halley
faefd62e9b vaapipostproc: reset deinterlacer state when there is a discontinuity.
Reset deinterlacer state, i.e. past reference frames used for advanced
deinterlacing, when there is some discontinuity detected in the course
of processing source buffers.

This fixes support for advanced deinterlacing when a seek occurred.

https://bugzilla.gnome.org/show_bug.cgi?id=720375

[fixed type of pts_diff variable, fetch previous buffer PTS from the
 history buffer, reduce heuristic for detecting discontinuity]
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2014-06-18 17:24:48 +02:00
Gwenole Beauchesne
b3401dbb29 vaapipostproc: add support for crop regions in VPP mode.
Apply video cropping regions stored in GstVideoCropMeta, or in older
GstVaapiSurfaceProxy representation, to VPP pipelines. In non-VPP modes,
the crop meta are already propagated to the output buffers.

https://bugzilla.gnome.org/show_bug.cgi?id=720730
2014-06-18 16:24:52 +02:00
Simon Farnsworth
927fd2e1a8 vaapipostproc: make deinterlace-mode behave as expected.
deinterlace-mode didn't behave in the way you'd expect if you have
past experience of the deinterlace element. There were two bugs:

 1. "auto" mode wouldn't deinterlace "interleaved" buffers, only "mixed".
 2. "force" mode wouldn't deinterlace "mixed" buffers flagged as progressive.

Fix these up, and add assertions and error messages to detect cases that
aren't handled.

https://bugzilla.gnome.org/show_bug.cgi?id=726361

Signed-off-by: Simon Farnsworth <simon.farnsworth@onelan.co.uk>
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2014-06-18 16:24:23 +02:00
Matthieu Bouron
ead38a4d77 vaapidecode: do not discard video info props when the format changed.
gst_video_info_set_format() does not preserve video info properties. In
order to keep important information in the caps such as interlace mode,
framerate, pixel aspect ratio, ... we need to manually copy back those
properties after setting the new video format.

https://bugzilla.gnome.org/show_bug.cgi?id=722276
2014-06-18 13:59:09 +02:00
Matthew Waters
3fbef25e13 vaapidecode: plug a memory leak.
It can happen that there is a pool provided that does not advertise
the vappivideometa. We should unref that pool before using our own.

Discovered with vaapidecode ! {glimagesink,cluttersink}

https://bugzilla.gnome.org/show_bug.cgi?id=724957

[fixed compilation by adding the missing semi-colon]
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2014-06-18 13:58:49 +02:00
Gwenole Beauchesne
abfb5dd06c vaapidecode: parse source data until a frame is obtained.
Parse any pending data until a complete frame is obtained. This is a
memory optimization to avoid expansion of video packets stuffed into
the GstAdapter, and a fix to EOS condition to detect there is actually
pending data that needs to be decoded, and subsequently output.

https://bugzilla.gnome.org/show_bug.cgi?id=731831
2014-06-18 13:47:36 +02:00
Sreerenj Balachandran
781abad2c7 encoder: h264: fix multiple slices support in packed headers mode.
Handle packedi slice headers and packed raw data on a per-slice basis,
which is necessary for multi slice encoding.
2014-06-17 16:05:11 +02:00
Sreerenj Balachandran
786b68ac21 encoder: add infrastructure for per-slice handling of packed headers.
The packed slice header and packed raw data need to be paired with
the submission of VAEncSliceHeaderParameterBuffer. So handle them
on a per-slice basis insted of a per-picture basis.

[removed useless initializer]
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2014-06-17 16:03:33 +02:00
Gwenole Beauchesne
5e5d62cac7 vaapisink: fix initialization with "drm" display type.
Force early initializatin of the GstVaapiDisplay so that to make sure
that the sink element display object is presented first to upstream
elements, as it will be correctly featuring the requested display type
by the user.

Otherwise, we might end up in situations where a VA/X11 display is
initialized in vaapidecode, then we try VA/DRM display in vaapisink
(as requested by the "display" property), but this would cause a failure
because we cannot acquire a DRM display that was previously acquired
through another backend (e.g. VA/X11).
2014-06-13 17:42:35 +02:00
Gwenole Beauchesne
eef863f82f plugins: fix initialization with foreign context.
When a new display is settled through GstElement::set_context() (>= 1.2),
or GstVideoContext::set_context() (<= 1.0), then we shall also update the
associated display type.
2014-06-13 17:42:29 +02:00
Gwenole Beauchesne
3af6b0b8a0 plugins: add built-in video parsers as "vaapiparse" element.
The built-in video parsers elements are built into a single DSO named
libgstvaapi_parse.so. The various video parsers could be accessed as
vaapiparse_CODEC.

For now, this only includes a modified version of h264parse so that to
support H.264 MVC encoded streams.
2014-06-13 17:17:07 +02:00
Gwenole Beauchesne
799d7b7d0e decoder: h264: cope with new gst_h264_quant_matrix_*() interfaces.
New gst_h264_quant_matrix_*_get_raster_from_zigzag() were renamed
from gst_h264_video_quant_matrix_*_get_raster_from_zigzag().
2014-06-13 11:36:56 +02:00
Gwenole Beauchesne
7fd8aa3ed0 codecparsers: update to gst-vaapi-branch commit d6325ac.
7d8d045 h264parse: use new gst_h264_video_calculate_framerate()
d2f965a h264parse: set field_pic_flag when parsing a slice header
24c15b8 Import h264parse
a9283e5 bytereader: Use concistant derefence method
a8252c6 bytereader: Use pointer instead of index access
b1bebfc Import GstBitReader and GstByteReader
2f58788 h264: recognize SVC NAL units
4335da5 h264: fix SPS copy code for MVC
cf9b6dc h264: fix quantization matrix conversion routine names
b11ce2a h264: add gst_h264_video_calculate_framerate()
126dc6f add C++ guards for MPEG-4 and VP8 parsers
2014-06-13 11:34:07 +02:00
Gwenole Beauchesne
c65aec1aa6 decoder: h264: factor out DPB pruning for MVC.
Factor out the removal process of unused inter-view only reference
pictures from the DPB, prior to the possible insertion of the current
picture.

Ideally, the compiler could still opt for generating two loops. But
at least, the code is now clearer for maintenance.
2014-06-10 18:33:59 +02:00
Gwenole Beauchesne
63fde28dfd decoder: h264: clean-ups.
Fix GST_VAAPI_PICTURE_IS_{INTER_VIEW,ANCHOR}() definitions to use
the base GST_VAAPI_PICTURE_FLAG_IS_SET() macro.
2014-06-10 18:33:59 +02:00
Gwenole Beauchesne
428b038dba decoder: h264: improve pruning of unused MVC inter-view frames.
Improve process for the removal of pictures from DPB before possible
insertion of the current picture (C.4.4) for H.264 MVC inter-view only
reference components. In particular, handle cases where picture to be
inserted is not the last one of the access unit and if it was already
output and is no longer marked as used for reference, including for
decoding next view components within the same access unit.
2014-06-10 18:33:59 +02:00
Gwenole Beauchesne
42ab3e0669 decoder: h264: improve DPB bumping process for MVC.
While invoking the DPB bumping process in presence of many views,
it could be necessary to output previous pictures that are ready,
in a whole. i.e. emitting all view components from the very first
view order index zero to the very last one in its original access
unit; and not starting from the view order index of the picture
that caused the DPB bumping process to be invoked.

As a reminder, the maximum number of frames in DPB for MultiView
High profile with more than 2 views is not necessarily a multiple
of the number of views.

This fixes decoding of MVCNV-4.264.
2014-06-10 18:33:59 +02:00
Gwenole Beauchesne
e8fe78824b decoder: h264: fix inter-view references array growth.
Let the utility layer handle dynamic growth of the inter-view pictures
array. By definition, setting a new size to the array will effectively
grow the array, but would also fill in the newly created elements with
empty entries (NULL), thus also increasing the reported length, which
is not correct.
2014-06-06 17:58:50 +02:00
Gwenole Beauchesne
60af04901f decoder: h264: reduce ReferenceFrames entries to the essential set.
When decoding Multiview High profile streams with a large number of
views, it is not possible to make the VAPictureParameterBufferH264.
ReferenceFrames[] array hold the complete DPB, with all possibly
active pictures to be used for inter-view prediction in the current
access unit.

So reduce the scope of the ReferenceFrames[] array to only include
the set of reference pictures that are going to be used for decoding
the current picture. Basically, this is a union of all RefPicListX[]
array, for all slices constituting the decoded picture.
2014-06-06 17:34:02 +02:00
Gwenole Beauchesne
e670e36007 decoder: h264: fix MVC inter-view prediction process.
The inter-view reference components and inter-view only reference
components that are included in the reference picture lists shall
be considered as not being marked as "used for short-term reference"
or "used for long-term reference". This means that reference flags
should all be removed from VAPictureH264.flags.

This fixes decoding of MVCNV-2.264.
2014-06-04 19:10:44 +02:00
Gwenole Beauchesne
f36486a1e2 decoder: h264: fix detection of profile changes for MVC.
If the VA driver exposes ad-hoc H.264 MVC profiles, then we have to
be careful to detect profiles changes and not reset the underlying
VA context erroneously. In MVC situations, we could indeed get a
profile_idc change for every SPS that gets activated, alternatively
(base-view -> non-base view -> base-view, etc.).

An improved fix would be to characterize the exact profile to use
once and for all when SPS NAL units are parsed. This would also
allow for fallbacks to a base-view decoding only mode.
2014-06-04 19:08:18 +02:00
Gwenole Beauchesne
885ebf4c72 encoder: h264: drop extraneous definitions.
Re-use definitions from the codecparser headers instead of duplicating
them here again. That covers NALU definitions and slice types.
2014-06-03 14:30:39 +02:00
Sreerenj Balachandran
e1976df1a1 encoder: h264: remove unnecessary calcualtion of max_pic_order_cnt.
https://bugzilla.gnome.org/show_bug.cgi?id=727418
2014-06-03 13:53:46 +02:00
Sreerenj Balachandran
6df5c74081 encoder: h264: don't allow CABAC with Extended profile.
The H.264 specification does not support CABAC entropy coding for the
Extended profile.

https://bugzilla.gnome.org/show_bug.cgi?id=727418
2014-06-03 13:53:43 +02:00
Sreerenj Balachandran
dba440b164 encoder: h264: use packed headers mode for MVC encoding.
Exclusively use VA drivers that support raw packed headers for encoding.
i.e. simply submit packed headers Subset SPS and Prefix NAL units. This
provides for better compatibility accross the various VA drivers and HW
generations since no particular API is needed beyond what readily exists.
2014-06-02 18:25:13 +02:00
Sreerenj Balachandran
0de9fd6347 encoder: h264: add support for packed slice headers.
https://bugzilla.gnome.org/show_bug.cgi?id=722905
2014-06-02 18:25:13 +02:00
Sreerenj Balachandran
73355a680f encoder: h264: store subset sps to generate the codec-data
Store the SubsetSPS nal unit which we need for MVC specific
codec_data generation.
2014-06-02 18:25:13 +02:00
Sreerenj Balachandran
f0924ba0d1 encoder: h264: fix MVC pipeline hang while encoding with B-frames.
Since we are encoding each view independently from each other, we
need a higher number of pre-allocated surfaces to be used as the
reconstructed frames. For Stereo High profile encoding, this means
to effectively double the number of frames to be stored in the DPB.
2014-06-02 18:25:13 +02:00
Li Xiaowei
7bdf3fa177 encoder: h264: add initial support for H.264 Stereo High profile.
Add initial support for Subset SPS, Prefix NAL and Slice Extension NAL
for non-base-view streams encoding, and the usual SPS, PPS and Slice
NALs for base-view encoding.

The H.264 Stereo High profile encoding mode will be turned on when the
"num-views" parameter is set to 2. The source (raw) YUV frames will be
considered as Left/Right view, alternatively.

Each of the two views has its own frames reordering pool and reference
frames list management system. Inter-view references are not supported
yet, so the views are encoded independently from each other.

Signed-off-by: Li Xiaowei <xiaowei.a.li@intel.com>
[limited to Stereo High profile per the definition of MAX_NUM_VIEWS]
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2014-06-02 18:25:13 +02:00
Li Xiaowei
5323570a35 encoder: h264: wrap pools for refs and frames reordering.
Create structures to maintain the reference frames list (RefPool) and
frames reordering (ReorderPool) logic.

This is a prerequisite for H.264 MVC support.

Signed-off-by: Li Xiaowei <xiaowei.a.li@intel.com>
2014-06-02 18:25:13 +02:00
Li Xiaowei
4beee390e5 encoder: h264: add provisional support for subset SPS headers.
Add provisions to write subset SPS headers to the bitstream in view
to supporting the H.264 MVC specification.

This assumes the libva "staging" branch is in use.

Signed-off-by: Li Xiaowei <xiaowei.a.li@intel.com>
2014-06-02 18:25:13 +02:00
Li Xiaowei
fb7ecb15e2 utils: add H.264 MVC profiles.
Add "MultiView High" and "Stereo High" definitions.

Signed-off-by: Li Xiaowei <xiaowei.a.li@intel.com>
[require VA-API >= 0.35.2 for MVC profiles]
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2014-06-02 18:25:12 +02:00
Gwenole Beauchesne
800bfc1add utils: only enable VP8 profiles for newer VA-API versions.
VP8 decoding API appeared in VA-API >= 0.35.0. So, disable mappings
involving VP8 codec on earlier versions of the API.
2014-06-02 18:25:12 +02:00
Gwenole Beauchesne
f0d3d9cd3c decoder: h264: compute view ids only once per slice.
Optimize lookups of view ids / view order indices by caching the result
of the calculatiosn right into the GstVaapiParserInfoH264 struct. This
terribly simplifies is_new_access_unit() and find_first_field() functions.
2014-05-22 10:13:34 +02:00
Gwenole Beauchesne
abcc8bb7a2 decoder: h264: add support for MVC interlaced streams.
Fix support for MVC Stereo High profile streams with interlaced frames.
Also improve the detection logic of the first field.
2014-05-21 20:04:35 +02:00
Gwenole Beauchesne
5d76afb40d decoder: h264: add MVC profiles compatibility logic.
Add safe fallbacks for MVC profiles:
- all MultiView High profile streams with 2 views at most can be decoded
  with a Stereo High profile compliant decoder ;
- all Stereo High profile streams with only progressive views can be
  decoded with a MultiView High profile compliant decoder ;
- all drivers that support slice-level decoding could normally support
  MVC profiles when the DPB holds at most 16 frames.
2014-05-21 20:04:35 +02:00
Gwenole Beauchesne
8c5e89c298 decoder: h264: add initial support for MVC.
https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-21 20:04:34 +02:00