Commit graph

1697 commits

Author SHA1 Message Date
Gwenole Beauchesne
e8fe78824b decoder: h264: fix inter-view references array growth.
Let the utility layer handle dynamic growth of the inter-view pictures
array. By definition, setting a new size to the array will effectively
grow the array, but would also fill in the newly created elements with
empty entries (NULL), thus also increasing the reported length, which
is not correct.
2014-06-06 17:58:50 +02:00
Gwenole Beauchesne
60af04901f decoder: h264: reduce ReferenceFrames entries to the essential set.
When decoding Multiview High profile streams with a large number of
views, it is not possible to make the VAPictureParameterBufferH264.
ReferenceFrames[] array hold the complete DPB, with all possibly
active pictures to be used for inter-view prediction in the current
access unit.

So reduce the scope of the ReferenceFrames[] array to only include
the set of reference pictures that are going to be used for decoding
the current picture. Basically, this is a union of all RefPicListX[]
array, for all slices constituting the decoded picture.
2014-06-06 17:34:02 +02:00
Gwenole Beauchesne
e670e36007 decoder: h264: fix MVC inter-view prediction process.
The inter-view reference components and inter-view only reference
components that are included in the reference picture lists shall
be considered as not being marked as "used for short-term reference"
or "used for long-term reference". This means that reference flags
should all be removed from VAPictureH264.flags.

This fixes decoding of MVCNV-2.264.
2014-06-04 19:10:44 +02:00
Gwenole Beauchesne
f36486a1e2 decoder: h264: fix detection of profile changes for MVC.
If the VA driver exposes ad-hoc H.264 MVC profiles, then we have to
be careful to detect profiles changes and not reset the underlying
VA context erroneously. In MVC situations, we could indeed get a
profile_idc change for every SPS that gets activated, alternatively
(base-view -> non-base view -> base-view, etc.).

An improved fix would be to characterize the exact profile to use
once and for all when SPS NAL units are parsed. This would also
allow for fallbacks to a base-view decoding only mode.
2014-06-04 19:08:18 +02:00
Gwenole Beauchesne
885ebf4c72 encoder: h264: drop extraneous definitions.
Re-use definitions from the codecparser headers instead of duplicating
them here again. That covers NALU definitions and slice types.
2014-06-03 14:30:39 +02:00
Sreerenj Balachandran
e1976df1a1 encoder: h264: remove unnecessary calcualtion of max_pic_order_cnt.
https://bugzilla.gnome.org/show_bug.cgi?id=727418
2014-06-03 13:53:46 +02:00
Sreerenj Balachandran
6df5c74081 encoder: h264: don't allow CABAC with Extended profile.
The H.264 specification does not support CABAC entropy coding for the
Extended profile.

https://bugzilla.gnome.org/show_bug.cgi?id=727418
2014-06-03 13:53:43 +02:00
Sreerenj Balachandran
dba440b164 encoder: h264: use packed headers mode for MVC encoding.
Exclusively use VA drivers that support raw packed headers for encoding.
i.e. simply submit packed headers Subset SPS and Prefix NAL units. This
provides for better compatibility accross the various VA drivers and HW
generations since no particular API is needed beyond what readily exists.
2014-06-02 18:25:13 +02:00
Sreerenj Balachandran
0de9fd6347 encoder: h264: add support for packed slice headers.
https://bugzilla.gnome.org/show_bug.cgi?id=722905
2014-06-02 18:25:13 +02:00
Sreerenj Balachandran
73355a680f encoder: h264: store subset sps to generate the codec-data
Store the SubsetSPS nal unit which we need for MVC specific
codec_data generation.
2014-06-02 18:25:13 +02:00
Sreerenj Balachandran
f0924ba0d1 encoder: h264: fix MVC pipeline hang while encoding with B-frames.
Since we are encoding each view independently from each other, we
need a higher number of pre-allocated surfaces to be used as the
reconstructed frames. For Stereo High profile encoding, this means
to effectively double the number of frames to be stored in the DPB.
2014-06-02 18:25:13 +02:00
Li Xiaowei
7bdf3fa177 encoder: h264: add initial support for H.264 Stereo High profile.
Add initial support for Subset SPS, Prefix NAL and Slice Extension NAL
for non-base-view streams encoding, and the usual SPS, PPS and Slice
NALs for base-view encoding.

The H.264 Stereo High profile encoding mode will be turned on when the
"num-views" parameter is set to 2. The source (raw) YUV frames will be
considered as Left/Right view, alternatively.

Each of the two views has its own frames reordering pool and reference
frames list management system. Inter-view references are not supported
yet, so the views are encoded independently from each other.

Signed-off-by: Li Xiaowei <xiaowei.a.li@intel.com>
[limited to Stereo High profile per the definition of MAX_NUM_VIEWS]
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2014-06-02 18:25:13 +02:00
Li Xiaowei
5323570a35 encoder: h264: wrap pools for refs and frames reordering.
Create structures to maintain the reference frames list (RefPool) and
frames reordering (ReorderPool) logic.

This is a prerequisite for H.264 MVC support.

Signed-off-by: Li Xiaowei <xiaowei.a.li@intel.com>
2014-06-02 18:25:13 +02:00
Li Xiaowei
4beee390e5 encoder: h264: add provisional support for subset SPS headers.
Add provisions to write subset SPS headers to the bitstream in view
to supporting the H.264 MVC specification.

This assumes the libva "staging" branch is in use.

Signed-off-by: Li Xiaowei <xiaowei.a.li@intel.com>
2014-06-02 18:25:13 +02:00
Li Xiaowei
fb7ecb15e2 utils: add H.264 MVC profiles.
Add "MultiView High" and "Stereo High" definitions.

Signed-off-by: Li Xiaowei <xiaowei.a.li@intel.com>
[require VA-API >= 0.35.2 for MVC profiles]
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2014-06-02 18:25:12 +02:00
Gwenole Beauchesne
800bfc1add utils: only enable VP8 profiles for newer VA-API versions.
VP8 decoding API appeared in VA-API >= 0.35.0. So, disable mappings
involving VP8 codec on earlier versions of the API.
2014-06-02 18:25:12 +02:00
Gwenole Beauchesne
f0d3d9cd3c decoder: h264: compute view ids only once per slice.
Optimize lookups of view ids / view order indices by caching the result
of the calculatiosn right into the GstVaapiParserInfoH264 struct. This
terribly simplifies is_new_access_unit() and find_first_field() functions.
2014-05-22 10:13:34 +02:00
Gwenole Beauchesne
abcc8bb7a2 decoder: h264: add support for MVC interlaced streams.
Fix support for MVC Stereo High profile streams with interlaced frames.
Also improve the detection logic of the first field.
2014-05-21 20:04:35 +02:00
Gwenole Beauchesne
5d76afb40d decoder: h264: add MVC profiles compatibility logic.
Add safe fallbacks for MVC profiles:
- all MultiView High profile streams with 2 views at most can be decoded
  with a Stereo High profile compliant decoder ;
- all Stereo High profile streams with only progressive views can be
  decoded with a MultiView High profile compliant decoder ;
- all drivers that support slice-level decoding could normally support
  MVC profiles when the DPB holds at most 16 frames.
2014-05-21 20:04:35 +02:00
Gwenole Beauchesne
8c5e89c298 decoder: h264: add initial support for MVC.
https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-21 20:04:34 +02:00
Gwenole Beauchesne
42b112f8bd decoder: h264: dynamically allocate the DPB.
Dynamically allocate the Decoded Picture Buffer (DPB) and add provisions
for supporting the MVC allocation requirements.
2014-05-21 19:59:52 +02:00
Gwenole Beauchesne
ae1d8267d9 decoder: h264: fix detection of access unit boundaries.
In order to have a stricter conforming implementation, we need to carefully
detect access unit boundaries. Additional operations could be necessary to
perform at those boundaries.
2014-05-21 19:59:52 +02:00
Sreerenj Balachandran
a6215a0757 decoder: h264: detect the first VCL NAL unit of a picture for MVC.
Detect the first VCL NAL unit of a picture for MVC, based on the
view_id as per H.7.4.1.2.4. Note that we only need to detect new
view components.

Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2014-05-21 19:59:52 +02:00
Sreerenj Balachandran
e4166f5a91 decoder: h264: properly handle Prefix NAL units.
Always cache the previous NAL unit so that we could check whether
there is a Prefix NAL unit immediately preceding the current slice
or IDR NAL unit. In that case, the NAL unit metadata is copied into
the current NAL unit. Otherwise, some default values are inferred,
tentatively. e.g. view_id shall be set to 0 and inter_view_flag to 1.

[infer default values for slice if previous NAL was not a Prefix]
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2014-05-21 19:59:52 +02:00
Xiaowei Li
79118904a0 decoder: h264: add support for MVC base views.
Allow decoding for base views of MVC encoded streams. For now, just skip
the slice extension and prefix NAL units, and skip non-base view frames.

Signed-off-by: Xiaowei Li <xiaowei.a.li@intel.com>
[fixed memory leak, improved check for MVC NAL units]
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2014-05-21 19:59:52 +02:00
Gwenole Beauchesne
1c46990ecd decoder: h264: simplify storage of decoded picture into DPB.
Factor out process by which the decoded picture with the lowest POC
is found, and possibly output. Likewise, the storage and marking of
a reference decoded, or non-reference decoded picture, into the DPB
could also be simplified as they mostly share the same operations.
2014-05-21 19:59:52 +02:00
Gwenole Beauchesne
a571350dee decoder: h264: minor clean-ups.
Make init_picture_ref_lists() more consistent with other functions
related to the reference marking process by supplying the current
picture as argument.
2014-05-21 19:59:52 +02:00
Gwenole Beauchesne
c0993182bd display: add utility function to query VA driver name.
Add gst_vaapi_display_get_vendor_string() helper function to query
the underlying VA driver name. The display object owns the resulting
string, so it shall not be deallocated.

That function is thread-safe. It could be used for debugging purposes,
for instance.
2014-05-21 19:59:52 +02:00
Gwenole Beauchesne
71053228b3 display: make cache maintenance really MT-safe.
Make sure to initialize one GstVaapiDisplay at a time, even in threaded
environments. This makes sure the display cache is also consistent
during the whole display creation process. In the former implementation,
there were risks that display cache got updated in another thread.
2014-05-21 19:59:52 +02:00
Gwenole Beauchesne
446aa9acf8 context: allow dynamic growth of VA surfaces pool.
Add support for dynamic growth of the VA surfaces pool. For decoding,
this implies the recreation of the underlying VA context, as per the
requirement from VA-API. Besides, only increases are supported, not
shrinks.
2014-05-21 19:59:52 +02:00
Gwenole Beauchesne
906a561c26 context: reset VA context if VA surfaces set changed.
It is a requirement from VA-API specification that the VA context got
from vaCreateContext(), for decoding purposes, binds the supplied set
of VA surfaces. This means that if the set of VA surfaces is to be
changed for the current decode session, then the VA context needs to
be recreated with the new set of VA surfaces.
2014-05-21 19:59:52 +02:00
Gwenole Beauchesne
c12d80eb88 decoder: h264: fix assignment of LongTermFrameIdx.
Complement fix committed as e95a42e.

The H.264 AVC standard has to say: if the field is part of a reference
frame or a complementary reference field pair, and the other field of
the same reference frame or complementary reference field pair is also
marked as "used for long-term reference", the reference frame or
complementary reference field pair is also marked as "used for long-term
reference" and assigned LongTermFrameIdx equal to long_term_frame_idx.

This fixes decoding of MR9_BT_B in strict mode.

https://bugs.freedesktop.org/show_bug.cgi?id=64624
https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-05-12 19:29:37 +02:00
Gwenole Beauchesne
85a60441fa decoder: h264: properly support grayscale formats.
Request the correct chroma format for decoding grayscale streams.
i.e. make lookups of the VA chroma format more generic, thus possibly
supporting more formats in the future.

This means that, if a VA driver doesn't support grayscale formats,
it is now going to fail. We cannot safely assume that maybe grayscale
was implemented on top of some YUV 4:2:0 with the chroma components
all set to 0x80.
2014-05-10 06:29:25 +02:00
Simon Farnsworth
c4cd8e2b36 build: fix source file modes.
A few source files are marked executable in error - fix them

https://bugzilla.gnome.org/show_bug.cgi?id=723748

Signed-off-by: Simon Farnsworth <simon.farnsworth@onelan.co.uk>
2014-05-09 14:41:57 +02:00
Sreerenj Balachandran
ee6d1b7bd1 build: fix conditional compilation of VP8 decoder.
https://bugzilla.gnome.org/show_bug.cgi?id=729170

[added check for VASliceParameterBufferBase fields]
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2014-05-09 14:41:52 +02:00
Gwenole Beauchesne
8d16aa87b2 build: fix make dist for codecparsers. 2014-05-01 11:23:20 +02:00
Gwenole Beauchesne
7f5b9edb14 codecparsers: update to gst-vaapi-branch commit eaa3f7e.
h264: fix parsing of slice groups for map type = 2
2014-04-28 09:42:13 +02:00
Gwenole Beauchesne
72accf53b2 codecparsers: update to gst-vaapi branch commit f44edfc.
h264: fix derivation of default scaling lists
2014-04-26 22:35:49 +02:00
Cong Zhong
e95a42ea6e decoder: h264: fix long-term reference picture marking process.
Fix reference picture marking process with memory_management_control_op
set to 3 and 6, i.e. assign LongTermFrameIdx to a short-term reference
picture, or the current picture.

This fixes decoding of FRExt_MMCO4_Sony_B.

https://bugs.freedesktop.org/show_bug.cgi?id=64624
https://bugzilla.gnome.org/show_bug.cgi?id=724518

[squashed, edited to use GST_VAAPI_PICTURE_IS_COMPLETE() macro]
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2014-04-26 22:16:19 +02:00
Gwenole Beauchesne
fac59d6fcf decoder: h264: fix initialization of RefPicLists for multiple slices.
The initialization of reference picture lists (8.2.4.2) applies to all
slices. So, the RefPicList0/1 lists need to be constructed prior to
each slice submission to the HW decoder.

This fixes decoding of video sequences where frames are encoded with
multiple slices of different types, e.g. 4 slices in this order I, P,
I, and P. More precisely, CABAST3_Sony_E and CABASTBR3_Sony_B.

https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-26 20:21:46 +02:00
Zhong Cong
3a6f6f97ab decoder: h264: skip SPS extension and auxiliary slice NALs.
When NAL units of type 13 (SPS extension) or type 19 (auxiliary slice)
are present in a video, decoders shall perform the (optional) decoding
process specified for these NAL units or shall ignore them (7.4.1).

Implement option 2 (skip) for now, as alpha composition is not
supported yet during the decoding process.

This fixes decoding of the primary coded video in alphaconformanceG.

https://bugzilla.gnome.org/show_bug.cgi?id=703928
https://bugzilla.gnome.org/show_bug.cgi?id=728869
https://bugzilla.gnome.org/show_bug.cgi?id=724518

[skip NAL units earlier, i.e. at parsing time]
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2014-04-26 14:38:59 +02:00
Li Xiaowei
3f14745d61 decoder: h264: fix slice data bit offset with MVC NAL units.
When MVC slice NAL units (coded slice extension and prefix NAL) are
present, the number of NAL header bytes is 3, not 1 as usual.

Signed-off-by: Li Xiaowei <xiaowei.a.li@intel.com>
Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2014-04-25 19:24:07 +02:00
Gwenole Beauchesne
5a3e51472d decoder: h264: fix activation of picture and sequence parameters.
At the time the first VCL NAL unit of a primary coded picture is found,
and if that NAL unit was parsed to be an SPS or PPS, then the entries
in the parser may have been overriden. This means that, when the picture
is to be decoded, slice_hdr->pps could point to an invalid (the next)
PPS entry.

So, one way to solve this problem is to not use the parser PPS and
SPS info but rather maintain our own activation chain in the decoder.

https://bugzilla.gnome.org/show_bug.cgi?id=724519
https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 19:24:07 +02:00
Gwenole Beauchesne
a365dfd269 decoder: h264: retain SEI messages until the end of frame.
Retain the SEI messages that were parsed from the access unit until we
have completely decoded the current frame. This is done so that we can
peek at that data whenever necessary during decoding. e.g. for exposing
3D stereoscopic information at a later stage.
2014-04-25 19:24:07 +02:00
Gwenole Beauchesne
fa7f9cd08c decoder: h264: add support for grayscale encoded clips.
Fix support for grayscale encoded video clips, and possibly others if
the underlying driver supports the non-YUV 4:2:0 formats. i.e. defer
the decision that a surface with the desired chroma format is not
supported to the actual VA driver implementation.

https://bugzilla.gnome.org/show_bug.cgi?id=728144
2014-04-25 19:24:07 +02:00
Gwenole Beauchesne
dbf32a2521 decoder: default to YUV 4:2:0 VA surfaces.
Cope with context changes to support non-YUV 4:2:0 VA surfaces. Still,
make sure all codecs use YUV 4:2:0 output format for now, by default.
2014-04-25 19:24:07 +02:00
Gwenole Beauchesne
1b79049317 decoder: re-indent GstVaapiDecoder base object. 2014-04-25 19:24:07 +02:00
Gwenole Beauchesne
14e9d383e3 encoder: derive chroma type from video format.
Cope with previous VA context change to derive the correct surface chroma
type from the input video format.
2014-04-25 19:24:07 +02:00
Gwenole Beauchesne
519f0e6b8a context: add support for non-YUV 4:2:0 formats.
Don't force allocation of VA surfaces in YUV 4:2:0 format. Rather, allow
for the upper layer to specify the desired chroma type. If the chroma
type field is not set (or yields zero), then YUV 4:2:0 format is used
by default.
2014-04-25 19:24:07 +02:00
Gwenole Beauchesne
f2b0d02168 vp8: fix per-segment deblocking filter level in relative mode.
Fix possible bug when a per-segment deblocking filter level value
needs to be set in non-absolute mode, i.e. when the loop filter update
value is negative in delta mode.

Also clamp the resulting filter level value to 0..63 range.
2014-04-22 19:57:04 +02:00