mirrors/gstreamer
mirrors/gstreamer
1
1
Fork 0
mirror of https://gitlab.freedesktop.org/gstreamer/gstreamer.git synced 2024-12-26 02:00:33 +00:00
Code Issues 1 Projects Releases Wiki Activity
gstreamer/gst-libs/gst/vaapi/gstvaapidecoder_h264.c

4285 lines
144 KiB
C
Raw Normal View History

Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
/*
* gstvaapidecoder_h264.c - H.264 decoder
*
legal: update copyright notice dates.
2014-01-22 17:54:14 +00:00
* Copyright (C) 2011-2014 Intel Corporation
legal: add per-file authorship information. Credit original authors on a per-file basis as we cannot expect people to know all country-specific rules, or bother browsing through the git history.
2013-11-22 04:57:18 +00:00
* Author: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public License
* as published by the Free Software Foundation; either version 2.1
* of the License, or (at your option) any later version.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with this library; if not, write to the Free
* Software Foundation, Inc., 51 Franklin Street, Fifth Floor,
* Boston, MA 02110-1301 USA
*/
/**
* SECTION:gstvaapidecoder_h264
* @short_description: H.264 decoder
*/
Add header for system-dependent definitions.
2012-01-30 17:12:59 +00:00
#include "sysdeps.h"
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
#include <string.h>
h264: fix presentation timestamps.
2012-01-23 14:20:51 +00:00
#include <gst/base/gstadapter.h>
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
#include <gst/codecparsers/gsth264parser.h>
#include "gstvaapidecoder_h264.h"
#include "gstvaapidecoder_objects.h"
#include "gstvaapidecoder_priv.h"
#include "gstvaapidisplay_priv.h"
#include "gstvaapiobject_priv.h"
utils: h264: expose levels in public header. Instal <gst/vaapi/gstvaapiutils_h264.h> header but only expose the H.264 levels in there. The additional helper functions are meant to be private for now.
2014-01-10 17:27:20 +00:00
#include "gstvaapiutils_h264_priv.h"
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
#define DEBUG 1
#include "gstvaapidebug.h"
h264: add flag to compile with strict DPB ordering mode. Allow build with strict DPB ordering mode whereby evicted entries are replaced by the next entries, in order instead of optimizing it away with the last entry in the DPB. This is only useful for debugging purpose, against a reference SW decoder for example.
2012-10-16 14:52:04 +00:00
/* Defined to 1 if strict ordering of DPB is needed. Only useful for debug */
#define USE_STRICT_DPB_ORDERING 0
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
typedef struct _GstVaapiDecoderH264Private GstVaapiDecoderH264Private;
typedef struct _GstVaapiDecoderH264Class GstVaapiDecoderH264Class;
h264: introduce new frame store structure. The frame store represents a Decoded Picture Buffer entry, which can hold up to two fields. So far, the frame store is only used to hold full frames.
2012-11-02 14:14:58 +00:00
typedef struct _GstVaapiFrameStore GstVaapiFrameStore;
typedef struct _GstVaapiFrameStoreClass GstVaapiFrameStoreClass;
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
typedef struct _GstVaapiParserInfoH264 GstVaapiParserInfoH264;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
typedef struct _GstVaapiPictureH264 GstVaapiPictureH264;
h264: introduce per-field POC in GstVaapiPictureH264. Use the POC member available in the GstVaapiPicture base class and get rid of the dependency on the local VAPictureH264 TopFieldOrderCnt and BottomFieldOrderCnt. Rather, use a simple field_poc[] array initialized to INT_MAX, so that to simplify picture POC calculation for non frame pictures.
2012-10-31 10:52:03 +00:00
// Used for field_poc[]
#define TOP_FIELD 0
#define BOTTOM_FIELD 1
h264: add codec specific decoder unit. Introduce new GstVaapiDecoderUnitH264 object, which holds the standard NAL unit header (GstH264NalUnit) and additional parsed header info. Besides, we now parse headers as early as in the _parse() function so that to avoid un-necessary creation of sub-buffers in _decode() for NAL units that are not slices. This is a performance win by ~+1.1% only.
2012-12-06 16:25:01 +00:00
/* ------------------------------------------------------------------------- */
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
/* --- H.264 Parser Info --- */
h264: add codec specific decoder unit. Introduce new GstVaapiDecoderUnitH264 object, which holds the standard NAL unit header (GstH264NalUnit) and additional parsed header info. Besides, we now parse headers as early as in the _parse() function so that to avoid un-necessary creation of sub-buffers in _decode() for NAL units that are not slices. This is a performance win by ~+1.1% only.
2012-12-06 16:25:01 +00:00
/* ------------------------------------------------------------------------- */
decoder: h264: fix detection of access unit boundaries. In order to have a stricter conforming implementation, we need to carefully detect access unit boundaries. Additional operations could be necessary to perform at those boundaries.
2014-05-01 17:33:40 +00:00
/*
* Extended decoder unit flags:
*
* @GST_VAAPI_DECODER_UNIT_AU_START: marks the start of an access unit.
* @GST_VAAPI_DECODER_UNIT_AU_END: marks the end of an access unit.
*/
enum {
/* This flag does not strictly follow the definitions (7.4.1.2.3)
for detecting the start of an access unit as we are only
interested in knowing if the current slice is the first one or
the last one in the current access unit */
GST_VAAPI_DECODER_UNIT_FLAG_AU_START = (
GST_VAAPI_DECODER_UNIT_FLAG_LAST << 0),
GST_VAAPI_DECODER_UNIT_FLAG_AU_END = (
GST_VAAPI_DECODER_UNIT_FLAG_LAST << 1),
GST_VAAPI_DECODER_UNIT_FLAGS_AU = (
GST_VAAPI_DECODER_UNIT_FLAG_AU_START |
GST_VAAPI_DECODER_UNIT_FLAG_AU_END),
};
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
#define GST_VAAPI_PARSER_INFO_H264(obj) \
((GstVaapiParserInfoH264 *)(obj))
struct _GstVaapiParserInfoH264 {
libs: refine GstVaapiMiniObject. Drop support for user-defined data since this capability was not used so far and GstVaapiMiniObject represents the smallest reference counted object type. Add missing GST_VAAPI_MINI_OBJECT_CLASS() helper macro. Besides, since GstVaapiMiniObject is a libgstvaapi internal object, it is also possible to further simplify the layout of the object. i.e. merge GstVaapiMiniObjectBase into GstVaapiMiniObject.
2013-04-30 08:28:30 +00:00
GstVaapiMiniObject parent_instance;
h264: add codec specific decoder unit. Introduce new GstVaapiDecoderUnitH264 object, which holds the standard NAL unit header (GstH264NalUnit) and additional parsed header info. Besides, we now parse headers as early as in the _parse() function so that to avoid un-necessary creation of sub-buffers in _decode() for NAL units that are not slices. This is a performance win by ~+1.1% only.
2012-12-06 16:25:01 +00:00
GstH264NalUnit nalu;
union {
GstH264SPS sps;
GstH264PPS pps;
decoder: h264: retain SEI messages until the end of frame. Retain the SEI messages that were parsed from the access unit until we have completely decoded the current frame. This is done so that we can peek at that data whenever necessary during decoding. e.g. for exposing 3D stereoscopic information at a later stage.
2014-04-25 14:24:01 +00:00
GArray *sei;
h264: add codec specific decoder unit. Introduce new GstVaapiDecoderUnitH264 object, which holds the standard NAL unit header (GstH264NalUnit) and additional parsed header info. Besides, we now parse headers as early as in the _parse() function so that to avoid un-necessary creation of sub-buffers in _decode() for NAL units that are not slices. This is a performance win by ~+1.1% only.
2012-12-06 16:25:01 +00:00
GstH264SliceHdr slice_hdr;
} data;
decoder: h264: fix robustness patch for bytestream format. Fix parser and decoder state to sync at the right locations. This is because we could reset the parser state, while the decoder state was not copied yet, e.g. when parsing several NAL units from multiple frames whereas the current frame was not decoded yet. This is a regression brought in by commit 6fe5496.
2014-02-04 17:35:28 +00:00
guint state;
decoder: h264: compute view ids only once per slice. Optimize lookups of view ids / view order indices by caching the result of the calculatiosn right into the GstVaapiParserInfoH264 struct. This terribly simplifies is_new_access_unit() and find_first_field() functions.
2014-05-22 08:04:46 +00:00
guint flags; // Same as decoder unit flags (persistent)
guint view_id; // View ID of slice
guint voc; // View order index (VOIdx) of slice
h264: add codec specific decoder unit. Introduce new GstVaapiDecoderUnitH264 object, which holds the standard NAL unit header (GstH264NalUnit) and additional parsed header info. Besides, we now parse headers as early as in the _parse() function so that to avoid un-necessary creation of sub-buffers in _decode() for NAL units that are not slices. This is a performance win by ~+1.1% only.
2012-12-06 16:25:01 +00:00
};
decoder: h264: retain SEI messages until the end of frame. Retain the SEI messages that were parsed from the access unit until we have completely decoded the current frame. This is done so that we can peek at that data whenever necessary during decoding. e.g. for exposing 3D stereoscopic information at a later stage.
2014-04-25 14:24:01 +00:00
static void
gst_vaapi_parser_info_h264_finalize(GstVaapiParserInfoH264 *pi)
{
switch (pi->nalu.type) {
decoder: h264: add support for MVC base views. Allow decoding for base views of MVC encoded streams. For now, just skip the slice extension and prefix NAL units, and skip non-base view frames. Signed-off-by: Xiaowei Li <xiaowei.a.li@intel.com> [fixed memory leak, improved check for MVC NAL units] Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2013-02-28 07:59:55 +00:00
case GST_H264_NAL_SPS:
case GST_H264_NAL_SUBSET_SPS:
gst_h264_sps_clear(&pi->data.sps);
break;
decoder: h264: fix memory leak in PPS. Cope with latest changes from codecparsers/h264. It is now required to explicitly clear the GstH264PPS structure as it could contain additional allocations (slice_group_ids).
2014-07-01 15:18:08 +00:00
case GST_H264_NAL_PPS:
gst_h264_pps_clear(&pi->data.pps);
break;
decoder: h264: retain SEI messages until the end of frame. Retain the SEI messages that were parsed from the access unit until we have completely decoded the current frame. This is done so that we can peek at that data whenever necessary during decoding. e.g. for exposing 3D stereoscopic information at a later stage.
2014-04-25 14:24:01 +00:00
case GST_H264_NAL_SEI:
if (pi->data.sei) {
g_array_unref(pi->data.sei);
pi->data.sei = NULL;
}
break;
}
}
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
static inline const GstVaapiMiniObjectClass *
gst_vaapi_parser_info_h264_class(void)
h264: add codec specific decoder unit. Introduce new GstVaapiDecoderUnitH264 object, which holds the standard NAL unit header (GstH264NalUnit) and additional parsed header info. Besides, we now parse headers as early as in the _parse() function so that to avoid un-necessary creation of sub-buffers in _decode() for NAL units that are not slices. This is a performance win by ~+1.1% only.
2012-12-06 16:25:01 +00:00
{
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
static const GstVaapiMiniObjectClass GstVaapiParserInfoH264Class = {
decoder: h264: retain SEI messages until the end of frame. Retain the SEI messages that were parsed from the access unit until we have completely decoded the current frame. This is done so that we can peek at that data whenever necessary during decoding. e.g. for exposing 3D stereoscopic information at a later stage.
2014-04-25 14:24:01 +00:00
.size = sizeof(GstVaapiParserInfoH264),
.finalize = (GDestroyNotify)gst_vaapi_parser_info_h264_finalize
h264: add codec specific decoder unit. Introduce new GstVaapiDecoderUnitH264 object, which holds the standard NAL unit header (GstH264NalUnit) and additional parsed header info. Besides, we now parse headers as early as in the _parse() function so that to avoid un-necessary creation of sub-buffers in _decode() for NAL units that are not slices. This is a performance win by ~+1.1% only.
2012-12-06 16:25:01 +00:00
};
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
return &GstVaapiParserInfoH264Class;
}
h264: add codec specific decoder unit. Introduce new GstVaapiDecoderUnitH264 object, which holds the standard NAL unit header (GstH264NalUnit) and additional parsed header info. Besides, we now parse headers as early as in the _parse() function so that to avoid un-necessary creation of sub-buffers in _decode() for NAL units that are not slices. This is a performance win by ~+1.1% only.
2012-12-06 16:25:01 +00:00
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
static inline GstVaapiParserInfoH264 *
gst_vaapi_parser_info_h264_new(void)
{
return (GstVaapiParserInfoH264 *)
gst_vaapi_mini_object_new(gst_vaapi_parser_info_h264_class());
h264: add codec specific decoder unit. Introduce new GstVaapiDecoderUnitH264 object, which holds the standard NAL unit header (GstH264NalUnit) and additional parsed header info. Besides, we now parse headers as early as in the _parse() function so that to avoid un-necessary creation of sub-buffers in _decode() for NAL units that are not slices. This is a performance win by ~+1.1% only.
2012-12-06 16:25:01 +00:00
}
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
#define gst_vaapi_parser_info_h264_ref(pi) \
gst_vaapi_mini_object_ref(GST_VAAPI_MINI_OBJECT(pi))
#define gst_vaapi_parser_info_h264_unref(pi) \
gst_vaapi_mini_object_unref(GST_VAAPI_MINI_OBJECT(pi))
#define gst_vaapi_parser_info_h264_replace(old_pi_ptr, new_pi) \
gst_vaapi_mini_object_replace((GstVaapiMiniObject **)(old_pi_ptr), \
(GstVaapiMiniObject *)(new_pi))
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
/* ------------------------------------------------------------------------- */
/* --- H.264 Pictures --- */
/* ------------------------------------------------------------------------- */
h264: introduce GST_VAAPI_PICTURE_FLAG_IDR flag.
2012-10-31 10:33:40 +00:00
/*
* Extended picture flags:
*
* @GST_VAAPI_PICTURE_FLAG_IDR: flag that specifies an IDR picture
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
* @GST_VAAPI_PICTURE_FLAG_INTER_VIEW: flag that indicates the picture
* may be used for inter-view prediction
* @GST_VAAPI_PICTURE_FLAG_ANCHOR: flag that specifies an anchor picture,
* i.e. a picture that is decoded with only inter-view prediction,
* and not inter prediction
decoder: h264: fix detection of access unit boundaries. In order to have a stricter conforming implementation, we need to carefully detect access unit boundaries. Additional operations could be necessary to perform at those boundaries.
2014-05-01 17:33:40 +00:00
* @GST_VAAPI_PICTURE_FLAG_AU_START: flag that marks the start of an
* access unit (AU)
* @GST_VAAPI_PICTURE_FLAG_AU_END: flag that marks the end of an
* access unit (AU)
h264: introduce GST_VAAPI_PICTURE_{SHORT,LONG}_TERM_REFERENCE flags. Further get rid of GstVaapiPictureH264-local VAPictureH264.flags for reference bits, thus simplifying the reference picture marking process to only track a single set of reference flags. Also introduce a new long_term_frame_idx member.
2012-10-31 10:45:14 +00:00
* @GST_VAAPI_PICTURE_FLAG_SHORT_TERM_REFERENCE: flag that specifies
* "used for short-term reference"
* @GST_VAAPI_PICTURE_FLAG_LONG_TERM_REFERENCE: flag that specifies
* "used for long-term reference"
* @GST_VAAPI_PICTURE_FLAGS_REFERENCE: mask covering any kind of
* reference picture (short-term reference or long-term reference)
h264: introduce GST_VAAPI_PICTURE_FLAG_IDR flag.
2012-10-31 10:33:40 +00:00
*/
enum {
decoder: h264: minor clean-ups. Make init_picture_ref_lists() more consistent with other functions related to the reference marking process by supplying the current picture as argument.
2014-05-02 20:40:16 +00:00
GST_VAAPI_PICTURE_FLAG_IDR = (GST_VAAPI_PICTURE_FLAG_LAST << 0),
GST_VAAPI_PICTURE_FLAG_REFERENCE2 = (GST_VAAPI_PICTURE_FLAG_LAST << 1),
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
GST_VAAPI_PICTURE_FLAG_INTER_VIEW = (GST_VAAPI_PICTURE_FLAG_LAST << 2),
GST_VAAPI_PICTURE_FLAG_ANCHOR = (GST_VAAPI_PICTURE_FLAG_LAST << 3),
decoder: h264: fix detection of access unit boundaries. In order to have a stricter conforming implementation, we need to carefully detect access unit boundaries. Additional operations could be necessary to perform at those boundaries.
2014-05-01 17:33:40 +00:00
GST_VAAPI_PICTURE_FLAG_AU_START = (GST_VAAPI_PICTURE_FLAG_LAST << 4),
GST_VAAPI_PICTURE_FLAG_AU_END = (GST_VAAPI_PICTURE_FLAG_LAST << 5),
h264: introduce GST_VAAPI_PICTURE_{SHORT,LONG}_TERM_REFERENCE flags. Further get rid of GstVaapiPictureH264-local VAPictureH264.flags for reference bits, thus simplifying the reference picture marking process to only track a single set of reference flags. Also introduce a new long_term_frame_idx member.
2012-10-31 10:45:14 +00:00
GST_VAAPI_PICTURE_FLAG_SHORT_TERM_REFERENCE = (
GST_VAAPI_PICTURE_FLAG_REFERENCE),
GST_VAAPI_PICTURE_FLAG_LONG_TERM_REFERENCE = (
decoder: h264: minor clean-ups. Make init_picture_ref_lists() more consistent with other functions related to the reference marking process by supplying the current picture as argument.
2014-05-02 20:40:16 +00:00
GST_VAAPI_PICTURE_FLAG_REFERENCE | GST_VAAPI_PICTURE_FLAG_REFERENCE2),
h264: introduce GST_VAAPI_PICTURE_{SHORT,LONG}_TERM_REFERENCE flags. Further get rid of GstVaapiPictureH264-local VAPictureH264.flags for reference bits, thus simplifying the reference picture marking process to only track a single set of reference flags. Also introduce a new long_term_frame_idx member.
2012-10-31 10:45:14 +00:00
GST_VAAPI_PICTURE_FLAGS_REFERENCE = (
GST_VAAPI_PICTURE_FLAG_SHORT_TERM_REFERENCE |
GST_VAAPI_PICTURE_FLAG_LONG_TERM_REFERENCE),
h264: introduce GST_VAAPI_PICTURE_FLAG_IDR flag.
2012-10-31 10:33:40 +00:00
};
#define GST_VAAPI_PICTURE_IS_IDR(picture) \
(GST_VAAPI_PICTURE_FLAG_IS_SET(picture, GST_VAAPI_PICTURE_FLAG_IDR))
h264: introduce GST_VAAPI_PICTURE_{SHORT,LONG}_TERM_REFERENCE flags. Further get rid of GstVaapiPictureH264-local VAPictureH264.flags for reference bits, thus simplifying the reference picture marking process to only track a single set of reference flags. Also introduce a new long_term_frame_idx member.
2012-10-31 10:45:14 +00:00
#define GST_VAAPI_PICTURE_IS_SHORT_TERM_REFERENCE(picture) \
((GST_VAAPI_PICTURE_FLAGS(picture) & \
GST_VAAPI_PICTURE_FLAGS_REFERENCE) == \
GST_VAAPI_PICTURE_FLAG_SHORT_TERM_REFERENCE)
#define GST_VAAPI_PICTURE_IS_LONG_TERM_REFERENCE(picture) \
((GST_VAAPI_PICTURE_FLAGS(picture) & \
GST_VAAPI_PICTURE_FLAGS_REFERENCE) == \
GST_VAAPI_PICTURE_FLAG_LONG_TERM_REFERENCE)
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
#define GST_VAAPI_PICTURE_IS_INTER_VIEW(picture) \
decoder: h264: clean-ups. Fix GST_VAAPI_PICTURE_IS_{INTER_VIEW,ANCHOR}() definitions to use the base GST_VAAPI_PICTURE_FLAG_IS_SET() macro.
2014-06-10 15:42:58 +00:00
(GST_VAAPI_PICTURE_FLAG_IS_SET(picture, GST_VAAPI_PICTURE_FLAG_INTER_VIEW))
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
#define GST_VAAPI_PICTURE_IS_ANCHOR(picture) \
decoder: h264: clean-ups. Fix GST_VAAPI_PICTURE_IS_{INTER_VIEW,ANCHOR}() definitions to use the base GST_VAAPI_PICTURE_FLAG_IS_SET() macro.
2014-06-10 15:42:58 +00:00
(GST_VAAPI_PICTURE_FLAG_IS_SET(picture, GST_VAAPI_PICTURE_FLAG_ANCHOR))
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
decoder: h264: minor clean-ups. Make init_picture_ref_lists() more consistent with other functions related to the reference marking process by supplying the current picture as argument.
2014-05-02 20:40:16 +00:00
#define GST_VAAPI_PICTURE_H264(picture) \
((GstVaapiPictureH264 *)(picture))
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
struct _GstVaapiPictureH264 {
GstVaapiPicture base;
h264: drop GstVaapiSliceH264 object. Use standard GstVaapiSlice object from now on since we already have parsed and recorded the slice headers (GstH264SliceHdr decode units).
2012-12-07 16:45:03 +00:00
GstH264SliceHdr *last_slice_hdr;
h264: add picture structure for reference picture marking process. Introduce new `structure' field to the H.264 specific picture structure so that to simplify the reference picture marking process. That local picture structure is derived from the original picture structure, as defined by the syntax elements field_pic_flag and bottom_field_flag.
2012-11-13 13:04:31 +00:00
guint structure;
h264: introduce per-field POC in GstVaapiPictureH264. Use the POC member available in the GstVaapiPicture base class and get rid of the dependency on the local VAPictureH264 TopFieldOrderCnt and BottomFieldOrderCnt. Rather, use a simple field_poc[] array initialized to INT_MAX, so that to simplify picture POC calculation for non frame pictures.
2012-10-31 10:52:03 +00:00
gint32 field_poc[2];
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
gint32 frame_num; // Original frame_num from slice_header()
gint32 frame_num_wrap; // Temporary for ref pic marking: FrameNumWrap
h264: introduce GST_VAAPI_PICTURE_{SHORT,LONG}_TERM_REFERENCE flags. Further get rid of GstVaapiPictureH264-local VAPictureH264.flags for reference bits, thus simplifying the reference picture marking process to only track a single set of reference flags. Also introduce a new long_term_frame_idx member.
2012-10-31 10:45:14 +00:00
gint32 long_term_frame_idx; // Temporary for ref pic marking: LongTermFrameIdx
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
gint32 pic_num; // Temporary for ref pic marking: PicNum
gint32 long_term_pic_num; // Temporary for ref pic marking: LongTermPicNum
h264: add initial support for interlaced streams. Decoded frames are only output when they are complete, i.e. when both fields are decoded. This also means that the "interlaced" caps is not propagated to vaapipostproc or vaapisink elements. Another limitation is that interlaced bitstreams with MMCO are unlikely to work.
2012-11-13 16:14:39 +00:00
GstVaapiPictureH264 *other_field; // Temporary for ref pic marking: other field in the same frame store
h264: handle Decoded Picture Buffer (DPB).
2012-01-24 14:38:14 +00:00
guint output_flag : 1;
guint output_needed : 1;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
};
Port codec objects to GstVaapiMiniObject.
2012-12-03 10:19:08 +00:00
GST_VAAPI_CODEC_DEFINE_TYPE(GstVaapiPictureH264, gst_vaapi_picture_h264);
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
Port codec objects to GstVaapiMiniObject.
2012-12-03 10:19:08 +00:00
void
gst_vaapi_picture_h264_destroy(GstVaapiPictureH264 *picture)
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
{
Port codec objects to GstVaapiMiniObject.
2012-12-03 10:19:08 +00:00
gst_vaapi_picture_destroy(GST_VAAPI_PICTURE(picture));
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
}
Port codec objects to GstVaapiMiniObject.
2012-12-03 10:19:08 +00:00
gboolean
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
gst_vaapi_picture_h264_create(
GstVaapiPictureH264 *picture,
const GstVaapiCodecObjectConstructorArgs *args
)
{
Port codec objects to GstVaapiMiniObject.
2012-12-03 10:19:08 +00:00
if (!gst_vaapi_picture_create(GST_VAAPI_PICTURE(picture), args))
return FALSE;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
h264: introduce per-field POC in GstVaapiPictureH264. Use the POC member available in the GstVaapiPicture base class and get rid of the dependency on the local VAPictureH264 TopFieldOrderCnt and BottomFieldOrderCnt. Rather, use a simple field_poc[] array initialized to INT_MAX, so that to simplify picture POC calculation for non frame pictures.
2012-10-31 10:52:03 +00:00
picture->field_poc[0] = G_MAXINT32;
picture->field_poc[1] = G_MAXINT32;
h264: handle Decoded Picture Buffer (DPB).
2012-01-24 14:38:14 +00:00
picture->output_needed = FALSE;
Port codec objects to GstVaapiMiniObject.
2012-12-03 10:19:08 +00:00
return TRUE;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
}
static inline GstVaapiPictureH264 *
gst_vaapi_picture_h264_new(GstVaapiDecoderH264 *decoder)
{
h264: minor clean-ups. Drop explicit initialization of most fields that are implicitly set to zero. Drop helper macros for casting to GstVaapiPictureH264 or GstVaapiFrameStore. Also remove some useless checks for NULL pointers.
2012-12-13 10:48:06 +00:00
return (GstVaapiPictureH264 *)gst_vaapi_codec_object_new(
Port codec objects to GstVaapiMiniObject.
2012-12-03 10:19:08 +00:00
&GstVaapiPictureH264Class,
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
GST_VAAPI_CODEC_BASE(decoder),
NULL, sizeof(VAPictureParameterBufferH264),
Port codec objects to GstVaapiMiniObject.
2012-12-03 10:19:08 +00:00
NULL, 0,
h264: minor clean-ups. Drop explicit initialization of most fields that are implicitly set to zero. Drop helper macros for casting to GstVaapiPictureH264 or GstVaapiFrameStore. Also remove some useless checks for NULL pointers.
2012-12-13 10:48:06 +00:00
0);
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
}
h264: split remove_reference_at() into finer units. Split remove_reference_at() into a function that actually removes the specified entry from the short-term or long-term reference picture array, and a function that sets reference flags to the desired value, possibly zero. The latters marks the picture as "unused for reference".
2012-11-13 15:35:30 +00:00
static inline void
gst_vaapi_picture_h264_set_reference(
GstVaapiPictureH264 *picture,
h264: fix interlaced stream decoding with MMCO. Fix decoding of interlaced streams when adaptive_ref_pic_marking_mode_flag is equal to 1, i.e. when memory management control operations are used. In particular, when field_pic_flag is set to 0, the new reference flags shall be applied to both fields.
2012-11-14 09:27:12 +00:00
guint reference_flags,
gboolean other_field
h264: split remove_reference_at() into finer units. Split remove_reference_at() into a function that actually removes the specified entry from the short-term or long-term reference picture array, and a function that sets reference flags to the desired value, possibly zero. The latters marks the picture as "unused for reference".
2012-11-13 15:35:30 +00:00
)
{
h264: minor clean-ups. Drop explicit initialization of most fields that are implicitly set to zero. Drop helper macros for casting to GstVaapiPictureH264 or GstVaapiFrameStore. Also remove some useless checks for NULL pointers.
2012-12-13 10:48:06 +00:00
if (!picture)
return;
h264: split remove_reference_at() into finer units. Split remove_reference_at() into a function that actually removes the specified entry from the short-term or long-term reference picture array, and a function that sets reference flags to the desired value, possibly zero. The latters marks the picture as "unused for reference".
2012-11-13 15:35:30 +00:00
GST_VAAPI_PICTURE_FLAG_UNSET(picture, GST_VAAPI_PICTURE_FLAGS_REFERENCE);
GST_VAAPI_PICTURE_FLAG_SET(picture, reference_flags);
h264: fix interlaced stream decoding with MMCO. Fix decoding of interlaced streams when adaptive_ref_pic_marking_mode_flag is equal to 1, i.e. when memory management control operations are used. In particular, when field_pic_flag is set to 0, the new reference flags shall be applied to both fields.
2012-11-14 09:27:12 +00:00
if (!other_field || !(picture = picture->other_field))
return;
GST_VAAPI_PICTURE_FLAG_UNSET(picture, GST_VAAPI_PICTURE_FLAGS_REFERENCE);
GST_VAAPI_PICTURE_FLAG_SET(picture, reference_flags);
h264: split remove_reference_at() into finer units. Split remove_reference_at() into a function that actually removes the specified entry from the short-term or long-term reference picture array, and a function that sets reference flags to the desired value, possibly zero. The latters marks the picture as "unused for reference".
2012-11-13 15:35:30 +00:00
}
h264: add initial support for interlaced streams. Decoded frames are only output when they are complete, i.e. when both fields are decoded. This also means that the "interlaced" caps is not propagated to vaapipostproc or vaapisink elements. Another limitation is that interlaced bitstreams with MMCO are unlikely to work.
2012-11-13 16:14:39 +00:00
static inline GstVaapiPictureH264 *
gst_vaapi_picture_h264_new_field(GstVaapiPictureH264 *picture)
{
h264: minor clean-ups. Drop explicit initialization of most fields that are implicitly set to zero. Drop helper macros for casting to GstVaapiPictureH264 or GstVaapiFrameStore. Also remove some useless checks for NULL pointers.
2012-12-13 10:48:06 +00:00
g_return_val_if_fail(picture, NULL);
h264: add initial support for interlaced streams. Decoded frames are only output when they are complete, i.e. when both fields are decoded. This also means that the "interlaced" caps is not propagated to vaapipostproc or vaapisink elements. Another limitation is that interlaced bitstreams with MMCO are unlikely to work.
2012-11-13 16:14:39 +00:00
h264: minor clean-ups. Drop explicit initialization of most fields that are implicitly set to zero. Drop helper macros for casting to GstVaapiPictureH264 or GstVaapiFrameStore. Also remove some useless checks for NULL pointers.
2012-12-13 10:48:06 +00:00
return (GstVaapiPictureH264 *)gst_vaapi_picture_new_field(&picture->base);
h264: add initial support for interlaced streams. Decoded frames are only output when they are complete, i.e. when both fields are decoded. This also means that the "interlaced" caps is not propagated to vaapipostproc or vaapisink elements. Another limitation is that interlaced bitstreams with MMCO are unlikely to work.
2012-11-13 16:14:39 +00:00
}
h264: introduce new frame store structure. The frame store represents a Decoded Picture Buffer entry, which can hold up to two fields. So far, the frame store is only used to hold full frames.
2012-11-02 14:14:58 +00:00
/* ------------------------------------------------------------------------- */
/* --- Frame Buffers (DPB) --- */
/* ------------------------------------------------------------------------- */
struct _GstVaapiFrameStore {
/*< private >*/
Port GstVaapiFrameStore to GstVaapiMiniObject.
2012-12-03 13:09:01 +00:00
GstVaapiMiniObject parent_instance;
h264: introduce new frame store structure. The frame store represents a Decoded Picture Buffer entry, which can hold up to two fields. So far, the frame store is only used to hold full frames.
2012-11-02 14:14:58 +00:00
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
guint view_id;
h264: introduce new frame store structure. The frame store represents a Decoded Picture Buffer entry, which can hold up to two fields. So far, the frame store is only used to hold full frames.
2012-11-02 14:14:58 +00:00
guint structure;
GstVaapiPictureH264 *buffers[2];
guint num_buffers;
guint output_needed;
};
static void
Port GstVaapiFrameStore to GstVaapiMiniObject.
2012-12-03 13:09:01 +00:00
gst_vaapi_frame_store_finalize(gpointer object)
h264: introduce new frame store structure. The frame store represents a Decoded Picture Buffer entry, which can hold up to two fields. So far, the frame store is only used to hold full frames.
2012-11-02 14:14:58 +00:00
{
Port GstVaapiFrameStore to GstVaapiMiniObject.
2012-12-03 13:09:01 +00:00
GstVaapiFrameStore * const fs = object;
h264: introduce new frame store structure. The frame store represents a Decoded Picture Buffer entry, which can hold up to two fields. So far, the frame store is only used to hold full frames.
2012-11-02 14:14:58 +00:00
guint i;
for (i = 0; i < fs->num_buffers; i++)
gst_vaapi_picture_replace(&fs->buffers[i], NULL);
}
static GstVaapiFrameStore *
gst_vaapi_frame_store_new(GstVaapiPictureH264 *picture)
{
GstVaapiFrameStore *fs;
Port GstVaapiFrameStore to GstVaapiMiniObject.
2012-12-03 13:09:01 +00:00
static const GstVaapiMiniObjectClass GstVaapiFrameStoreClass = {
sizeof(GstVaapiFrameStore),
gst_vaapi_frame_store_finalize
};
fs = (GstVaapiFrameStore *)
gst_vaapi_mini_object_new(&GstVaapiFrameStoreClass);
h264: introduce new frame store structure. The frame store represents a Decoded Picture Buffer entry, which can hold up to two fields. So far, the frame store is only used to hold full frames.
2012-11-02 14:14:58 +00:00
if (!fs)
return NULL;
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
fs->view_id = picture->base.view_id;
h264: add picture structure for reference picture marking process. Introduce new `structure' field to the H.264 specific picture structure so that to simplify the reference picture marking process. That local picture structure is derived from the original picture structure, as defined by the syntax elements field_pic_flag and bottom_field_flag.
2012-11-13 13:04:31 +00:00
fs->structure = picture->structure;
h264: introduce new frame store structure. The frame store represents a Decoded Picture Buffer entry, which can hold up to two fields. So far, the frame store is only used to hold full frames.
2012-11-02 14:14:58 +00:00
fs->buffers[0] = gst_vaapi_picture_ref(picture);
Port GstVaapiFrameStore to GstVaapiMiniObject.
2012-12-03 13:09:01 +00:00
fs->buffers[1] = NULL;
h264: introduce new frame store structure. The frame store represents a Decoded Picture Buffer entry, which can hold up to two fields. So far, the frame store is only used to hold full frames.
2012-11-02 14:14:58 +00:00
fs->num_buffers = 1;
decoder: h264: simplify the DPB output process. Simplify the dpb_output() function to exclusively rely on the frame store buffer to output, since this is now always provided. Besides, also fix cases where split fields would not be displayed. This is a regression from f48b1e0.
2014-07-03 09:13:33 +00:00
fs->output_needed = 0;
if (picture->output_flag) {
picture->output_needed = TRUE;
fs->output_needed++;
}
h264: introduce new frame store structure. The frame store represents a Decoded Picture Buffer entry, which can hold up to two fields. So far, the frame store is only used to hold full frames.
2012-11-02 14:14:58 +00:00
return fs;
}
h264: add initial support for interlaced streams. Decoded frames are only output when they are complete, i.e. when both fields are decoded. This also means that the "interlaced" caps is not propagated to vaapipostproc or vaapisink elements. Another limitation is that interlaced bitstreams with MMCO are unlikely to work.
2012-11-13 16:14:39 +00:00
static gboolean
gst_vaapi_frame_store_add(GstVaapiFrameStore *fs, GstVaapiPictureH264 *picture)
{
guint field;
g_return_val_if_fail(fs->num_buffers == 1, FALSE);
g_return_val_if_fail(!GST_VAAPI_PICTURE_IS_FRAME(picture), FALSE);
h264: minor clean-ups. Drop explicit initialization of most fields that are implicitly set to zero. Drop helper macros for casting to GstVaapiPictureH264 or GstVaapiFrameStore. Also remove some useless checks for NULL pointers.
2012-12-13 10:48:06 +00:00
g_return_val_if_fail(!GST_VAAPI_PICTURE_IS_FIRST_FIELD(picture), FALSE);
h264: add initial support for interlaced streams. Decoded frames are only output when they are complete, i.e. when both fields are decoded. This also means that the "interlaced" caps is not propagated to vaapipostproc or vaapisink elements. Another limitation is that interlaced bitstreams with MMCO are unlikely to work.
2012-11-13 16:14:39 +00:00
gst_vaapi_picture_replace(&fs->buffers[fs->num_buffers++], picture);
if (picture->output_flag) {
picture->output_needed = TRUE;
fs->output_needed++;
}
fs->structure = GST_VAAPI_PICTURE_STRUCTURE_FRAME;
field = picture->structure == GST_VAAPI_PICTURE_STRUCTURE_TOP_FIELD ?
TOP_FIELD : BOTTOM_FIELD;
g_return_val_if_fail(fs->buffers[0]->field_poc[field] == G_MAXINT32, FALSE);
fs->buffers[0]->field_poc[field] = picture->field_poc[field];
g_return_val_if_fail(picture->field_poc[!field] == G_MAXINT32, FALSE);
picture->field_poc[!field] = fs->buffers[0]->field_poc[!field];
return TRUE;
}
static gboolean
gst_vaapi_frame_store_split_fields(GstVaapiFrameStore *fs)
{
GstVaapiPictureH264 * const first_field = fs->buffers[0];
GstVaapiPictureH264 *second_field;
g_return_val_if_fail(fs->num_buffers == 1, FALSE);
decoder: h264: fix detection of top-field-first (TFF) flag. Use the SEI pic_timing() message to track the pic_struct variable when present, or infer it from the regular slice header flags field_pic_flag and bottom_field_flag. This fixes temporal sequence ordering when the output pictures are to be displayed. https://bugzilla.gnome.org/show_bug.cgi?id=739291
2014-11-13 14:00:21 +00:00
first_field->base.structure = GST_VAAPI_PICTURE_IS_TFF(first_field) ?
GST_VAAPI_PICTURE_STRUCTURE_TOP_FIELD :
GST_VAAPI_PICTURE_STRUCTURE_BOTTOM_FIELD;
h264: add initial support for interlaced streams. Decoded frames are only output when they are complete, i.e. when both fields are decoded. This also means that the "interlaced" caps is not propagated to vaapipostproc or vaapisink elements. Another limitation is that interlaced bitstreams with MMCO are unlikely to work.
2012-11-13 16:14:39 +00:00
GST_VAAPI_PICTURE_FLAG_SET(first_field, GST_VAAPI_PICTURE_FLAG_INTERLACED);
second_field = gst_vaapi_picture_h264_new_field(first_field);
if (!second_field)
return FALSE;
gst_vaapi_picture_replace(&fs->buffers[fs->num_buffers++], second_field);
gst_vaapi_picture_unref(second_field);
second_field->frame_num = first_field->frame_num;
second_field->field_poc[0] = first_field->field_poc[0];
second_field->field_poc[1] = first_field->field_poc[1];
second_field->output_flag = first_field->output_flag;
if (second_field->output_flag) {
second_field->output_needed = TRUE;
fs->output_needed++;
}
return TRUE;
}
h264: introduce new frame store structure. The frame store represents a Decoded Picture Buffer entry, which can hold up to two fields. So far, the frame store is only used to hold full frames.
2012-11-02 14:14:58 +00:00
static inline gboolean
gst_vaapi_frame_store_has_frame(GstVaapiFrameStore *fs)
{
return fs->structure == GST_VAAPI_PICTURE_STRUCTURE_FRAME;
}
decoder: h264: simplify the DPB output process. Simplify the dpb_output() function to exclusively rely on the frame store buffer to output, since this is now always provided. Besides, also fix cases where split fields would not be displayed. This is a regression from f48b1e0.
2014-07-03 09:13:33 +00:00
static inline gboolean
gst_vaapi_frame_store_is_complete(GstVaapiFrameStore *fs)
{
return gst_vaapi_frame_store_has_frame(fs) ||
GST_VAAPI_PICTURE_IS_ONEFIELD(fs->buffers[0]);
}
h264: introduce new frame store structure. The frame store represents a Decoded Picture Buffer entry, which can hold up to two fields. So far, the frame store is only used to hold full frames.
2012-11-02 14:14:58 +00:00
static inline gboolean
gst_vaapi_frame_store_has_reference(GstVaapiFrameStore *fs)
{
guint i;
for (i = 0; i < fs->num_buffers; i++) {
if (GST_VAAPI_PICTURE_IS_REFERENCE(fs->buffers[i]))
return TRUE;
}
return FALSE;
}
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
static gboolean
gst_vaapi_frame_store_has_inter_view(GstVaapiFrameStore *fs)
{
guint i;
for (i = 0; i < fs->num_buffers; i++) {
if (GST_VAAPI_PICTURE_IS_INTER_VIEW(fs->buffers[i]))
return TRUE;
}
return FALSE;
}
h264: introduce new frame store structure. The frame store represents a Decoded Picture Buffer entry, which can hold up to two fields. So far, the frame store is only used to hold full frames.
2012-11-02 14:14:58 +00:00
#define gst_vaapi_frame_store_ref(fs) \
Port GstVaapiFrameStore to GstVaapiMiniObject.
2012-12-03 13:09:01 +00:00
gst_vaapi_mini_object_ref(GST_VAAPI_MINI_OBJECT(fs))
h264: introduce new frame store structure. The frame store represents a Decoded Picture Buffer entry, which can hold up to two fields. So far, the frame store is only used to hold full frames.
2012-11-02 14:14:58 +00:00
#define gst_vaapi_frame_store_unref(fs) \
Port GstVaapiFrameStore to GstVaapiMiniObject.
2012-12-03 13:09:01 +00:00
gst_vaapi_mini_object_unref(GST_VAAPI_MINI_OBJECT(fs))
h264: introduce new frame store structure. The frame store represents a Decoded Picture Buffer entry, which can hold up to two fields. So far, the frame store is only used to hold full frames.
2012-11-02 14:14:58 +00:00
Port GstVaapiFrameStore to GstVaapiMiniObject.
2012-12-03 13:09:01 +00:00
#define gst_vaapi_frame_store_replace(old_fs_p, new_fs) \
gst_vaapi_mini_object_replace((GstVaapiMiniObject **)(old_fs_p), \
(GstVaapiMiniObject *)(new_fs))
h264: introduce new frame store structure. The frame store represents a Decoded Picture Buffer entry, which can hold up to two fields. So far, the frame store is only used to hold full frames.
2012-11-02 14:14:58 +00:00
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
/* ------------------------------------------------------------------------- */
/* --- H.264 Decoder --- */
/* ------------------------------------------------------------------------- */
h264: minor clean-ups. Drop explicit initialization of most fields that are implicitly set to zero. Drop helper macros for casting to GstVaapiPictureH264 or GstVaapiFrameStore. Also remove some useless checks for NULL pointers.
2012-12-13 10:48:06 +00:00
#define GST_VAAPI_DECODER_H264_CAST(decoder) \
((GstVaapiDecoderH264 *)(decoder))
h264: improve robustness when packets are missing. Improve robustness when some expected packets where not received yet or that were not correctly decoded. For example, don't try to decode a picture if there was no valid frame headers parsed so far. https://bugs.freedesktop.org/show_bug.cgi?id=57902
2013-12-10 15:14:27 +00:00
typedef enum {
GST_H264_VIDEO_STATE_GOT_SPS = 1 << 0,
GST_H264_VIDEO_STATE_GOT_PPS = 1 << 1,
GST_H264_VIDEO_STATE_GOT_SLICE = 1 << 2,
GST_H264_VIDEO_STATE_VALID_PICTURE_HEADERS = (
GST_H264_VIDEO_STATE_GOT_SPS |
GST_H264_VIDEO_STATE_GOT_PPS),
GST_H264_VIDEO_STATE_VALID_PICTURE = (
GST_H264_VIDEO_STATE_VALID_PICTURE_HEADERS |
GST_H264_VIDEO_STATE_GOT_SLICE)
} GstH264VideoState;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
struct _GstVaapiDecoderH264Private {
GstH264NalParser *parser;
h264: improve robustness when packets are missing. Improve robustness when some expected packets where not received yet or that were not correctly decoded. For example, don't try to decode a picture if there was no valid frame headers parsed so far. https://bugs.freedesktop.org/show_bug.cgi?id=57902
2013-12-10 15:14:27 +00:00
guint parser_state;
guint decoder_state;
decoder: h264: add support for NALU "alignment" optimization. We can avoid scanning for start codes again if the bitstream is fed in NALU chunks. Currently, we always scan for start codes, and keep track of remaining bits in a GstAdapter, even if, in practice, we are likely receiving one GstBuffer per NAL unit. i.e. h264parse with "nal" alignment. https://bugzilla.gnome.org/show_bug.cgi?id=723284 [use gst_adapter_available_fast() to determine the top buffer size] Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2014-02-06 06:30:10 +00:00
GstVaapiStreamAlignH264 stream_alignment;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
GstVaapiPictureH264 *current_picture;
decoder: h264: fix activation of picture and sequence parameters. At the time the first VCL NAL unit of a primary coded picture is found, and if that NAL unit was parsed to be an SPS or PPS, then the entries in the parser may have been overriden. This means that, when the picture is to be decoded, slice_hdr->pps could point to an invalid (the next) PPS entry. So, one way to solve this problem is to not use the parser PPS and SPS info but rather maintain our own activation chain in the decoder. https://bugzilla.gnome.org/show_bug.cgi?id=724519 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 17:11:03 +00:00
GstVaapiParserInfoH264 *sps[GST_H264_MAX_SPS_COUNT];
GstVaapiParserInfoH264 *active_sps;
GstVaapiParserInfoH264 *pps[GST_H264_MAX_PPS_COUNT];
GstVaapiParserInfoH264 *active_pps;
decoder: h264: properly handle Prefix NAL units. Always cache the previous NAL unit so that we could check whether there is a Prefix NAL unit immediately preceding the current slice or IDR NAL unit. In that case, the NAL unit metadata is copied into the current NAL unit. Otherwise, some default values are inferred, tentatively. e.g. view_id shall be set to 0 and inter_view_flag to 1. [infer default values for slice if previous NAL was not a Prefix] Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2013-10-31 11:32:55 +00:00
GstVaapiParserInfoH264 *prev_pi;
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
GstVaapiParserInfoH264 *prev_slice_pi;
decoder: h264: add support for MVC interlaced streams. Fix support for MVC Stereo High profile streams with interlaced frames. Also improve the detection logic of the first field.
2014-05-21 15:57:00 +00:00
GstVaapiFrameStore **prev_frames;
guint prev_frames_alloc;
decoder: h264: dynamically allocate the DPB. Dynamically allocate the Decoded Picture Buffer (DPB) and add provisions for supporting the MVC allocation requirements.
2014-05-01 17:16:09 +00:00
GstVaapiFrameStore **dpb;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
guint dpb_count;
h264: handle Decoded Picture Buffer (DPB).
2012-01-24 14:38:14 +00:00
guint dpb_size;
decoder: h264: dynamically allocate the DPB. Dynamically allocate the Decoded Picture Buffer (DPB) and add provisions for supporting the MVC allocation requirements.
2014-05-01 17:16:09 +00:00
guint dpb_size_max;
guint max_views;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
GstVaapiProfile profile;
h264: fix activation order of picture and sequence parameters. Delay ensure_context() until we actually need a VA context for allocating new VA surfaces, and then GstVaapiPictures, but also when a real activation of a new picture parameter set occurs, thus also implying an activation of the related sequence parameter set. The most important thing was to drop the global pps and sps pointers since they may not have matched the currently activated picture parameter or sequence parameter sets at the specified decode point. Anoter positive side-effect is that this cleans up all occurrences of decode_current_picture() to only keep those useful in decode_picture(), before a new picture is allocated, or in decode_sequence_end() when an end-of-stream or end-of-sequence condition occurred.
2012-10-26 14:12:05 +00:00
GstVaapiEntrypoint entrypoint;
GstVaapiChromaType chroma_type;
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
GPtrArray *inter_views;
h264: add initial support for interlaced streams. Decoded frames are only output when they are complete, i.e. when both fields are decoded. This also means that the "interlaced" caps is not propagated to vaapipostproc or vaapisink elements. Another limitation is that interlaced bitstreams with MMCO are unlikely to work.
2012-11-13 16:14:39 +00:00
GstVaapiPictureH264 *short_ref[32];
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
guint short_ref_count;
h264: add initial support for interlaced streams. Decoded frames are only output when they are complete, i.e. when both fields are decoded. This also means that the "interlaced" caps is not propagated to vaapipostproc or vaapisink elements. Another limitation is that interlaced bitstreams with MMCO are unlikely to work.
2012-11-13 16:14:39 +00:00
GstVaapiPictureH264 *long_ref[32];
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
guint long_ref_count;
GstVaapiPictureH264 *RefPicList0[32];
guint RefPicList0_count;
GstVaapiPictureH264 *RefPicList1[32];
guint RefPicList1_count;
h264: handle codec-data. Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2011-11-25 12:37:00 +00:00
guint nal_length_size;
h264: fix picture size in macroblocks. The picture size signalled by sps->{width,height} is the actual size with cropping applied, not the original size derived from pic_width_in_mbs_minus1 and pic_height_in_map_units_minus1. VA driver expects that original size, uncropped. There is another issue pending: frame cropping information needs to be taken care of.
2012-11-16 17:00:10 +00:00
guint mb_width;
guint mb_height;
decoder: h264: fix detection of top-field-first (TFF) flag. Use the SEI pic_timing() message to track the pic_struct variable when present, or infer it from the regular slice header flags field_pic_flag and bottom_field_flag. This fixes temporal sequence ordering when the output pictures are to be displayed. https://bugzilla.gnome.org/show_bug.cgi?id=739291
2014-11-13 14:00:21 +00:00
guint pic_structure; // pic_struct (from SEI pic_timing() or inferred)
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
gint32 field_poc[2]; // 0:TopFieldOrderCnt / 1:BottomFieldOrderCnt
gint32 poc_msb; // PicOrderCntMsb
gint32 poc_lsb; // pic_order_cnt_lsb (from slice_header())
gint32 prev_poc_msb; // prevPicOrderCntMsb
gint32 prev_poc_lsb; // prevPicOrderCntLsb
gint32 frame_num_offset; // FrameNumOffset
gint32 frame_num; // frame_num (from slice_header())
gint32 prev_frame_num; // prevFrameNum
h264: simplify code when MMCO is 5. Remove exit_picture() and exit_picture_poc() since PicOrderCnt(CurrPic) is now updated accordingly to the standard. Besides, MMCO = 5 specific operations are moved up to exec_ref_pic_marking_adaptive_mmco_5().
2012-10-23 08:33:50 +00:00
gboolean prev_pic_has_mmco5; // prevMmco5Pic
decoder: h264: fix picture ordering count type 0 with previous MMCO5. Fix the decoding process for picture order count type 0 when the previous picture had a memory_management_control_operation = 5. In particular, fix the actual variable type for prev_pic_structure to hold the full bits of the picture structure. In practice, this used to work though, due to the underlying type used to express a gboolean.
2014-11-13 14:05:19 +00:00
guint prev_pic_structure; // previous picture structure
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
guint is_opened : 1;
h264: minor clean-ups. Drop explicit initialization of most fields that are implicitly set to zero. Drop helper macros for casting to GstVaapiPictureH264 or GstVaapiFrameStore. Also remove some useless checks for NULL pointers.
2012-12-13 10:48:06 +00:00
guint is_avcC : 1;
h264: create VA context earlier when SPS is parsed.
2012-02-08 17:07:14 +00:00
guint has_context : 1;
h264: add initial support for interlaced streams. Decoded frames are only output when they are complete, i.e. when both fields are decoded. This also means that the "interlaced" caps is not propagated to vaapipostproc or vaapisink elements. Another limitation is that interlaced bitstreams with MMCO are unlikely to work.
2012-11-13 16:14:39 +00:00
guint progressive_sequence : 1;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
};
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
/**
* GstVaapiDecoderH264:
*
* A decoder based on H264.
*/
struct _GstVaapiDecoderH264 {
/*< private >*/
GstVaapiDecoder parent_instance;
GstVaapiDecoderH264Private priv;
};
/**
* GstVaapiDecoderH264Class:
*
* A decoder class based on H264.
*/
struct _GstVaapiDecoderH264Class {
/*< private >*/
GstVaapiDecoderClass parent_class;
};
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
static gboolean
h264: simplify reference picture marking process. ... to build the short_ref[] and long_ref[] lists from the DPB, instead of maintaining them separately. This avoids refs/unrefs while making it possible to generate the list based on the actual picture structure. This also ensures that the list of generated ReferenceFrames[] actually matches what reference frames are available in the DPB. i.e. short_ref[] and long_ref[] entries are implied from the DPB, so there is no risk of having "dangling" references.
2012-10-31 13:24:09 +00:00
exec_ref_pic_marking(GstVaapiDecoderH264 *decoder, GstVaapiPictureH264 *picture);
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
decoder: h264: improve pruning of unused MVC inter-view frames. Improve process for the removal of pictures from DPB before possible insertion of the current picture (C.4.4) for H.264 MVC inter-view only reference components. In particular, handle cases where picture to be inserted is not the last one of the access unit and if it was already output and is no longer marked as used for reference, including for decoding next view components within the same access unit.
2014-06-10 14:07:51 +00:00
static gboolean
is_inter_view_reference_for_next_pictures(GstVaapiDecoderH264 *decoder,
GstVaapiPictureH264 *picture);
static inline gboolean
is_inter_view_reference_for_next_frames(GstVaapiDecoderH264 *decoder,
GstVaapiFrameStore *fs)
{
return is_inter_view_reference_for_next_pictures(decoder, fs->buffers[0]);
}
decoder: h264: dynamically allocate the DPB. Dynamically allocate the Decoded Picture Buffer (DPB) and add provisions for supporting the MVC allocation requirements.
2014-05-01 17:16:09 +00:00
/* Determines if the supplied profile is one of the MVC set */
static gboolean
is_mvc_profile(GstH264Profile profile)
{
return profile == GST_H264_PROFILE_MULTIVIEW_HIGH ||
profile == GST_H264_PROFILE_STEREO_HIGH;
}
decoder: h264: add support for MVC interlaced streams. Fix support for MVC Stereo High profile streams with interlaced frames. Also improve the detection logic of the first field.
2014-05-21 15:57:00 +00:00
/* Determines the view_id from the supplied NAL unit */
static inline guint
get_view_id(GstH264NalUnit *nalu)
{
return GST_H264_IS_MVC_NALU(nalu) ? nalu->extension.mvc.view_id : 0;
}
decoder: h264: fix detection of access unit boundaries. In order to have a stricter conforming implementation, we need to carefully detect access unit boundaries. Additional operations could be necessary to perform at those boundaries.
2014-05-01 17:33:40 +00:00
/* Determines the view order index (VOIdx) from the supplied view_id */
static gint
get_view_order_index(GstH264SPS *sps, guint16 view_id)
{
GstH264SPSExtMVC *mvc;
gint i;
if (!sps || sps->extension_type != GST_H264_NAL_EXTENSION_MVC)
return 0;
mvc = &sps->extension.mvc;
for (i = 0; i <= mvc->num_views_minus1; i++) {
if (mvc->view[i].view_id == view_id)
return i;
}
GST_ERROR("failed to find VOIdx from view_id (%d)", view_id);
return -1;
}
decoder: h264: dynamically allocate the DPB. Dynamically allocate the Decoded Picture Buffer (DPB) and add provisions for supporting the MVC allocation requirements.
2014-05-01 17:16:09 +00:00
/* Determines NumViews */
static guint
get_num_views(GstH264SPS *sps)
{
return 1 + (sps->extension_type == GST_H264_NAL_EXTENSION_MVC ?
sps->extension.mvc.num_views_minus1 : 0);
}
decoder: cope with new GstVaapiContextInfo based API. Update decoders to report the maximum number of reference frames to use.
2012-09-10 16:26:51 +00:00
/* Get number of reference frames to use */
static guint
get_max_dec_frame_buffering(GstH264SPS *sps)
{
decoder: h264: dynamically allocate the DPB. Dynamically allocate the Decoded Picture Buffer (DPB) and add provisions for supporting the MVC allocation requirements.
2014-05-01 17:16:09 +00:00
guint num_views, max_dpb_frames;
utils: add helpers for H.264 levels. - gst_vaapi_utils_h264_get_level(): Returns GstVaapiLevelH264 from H.264 level_idc value - gst_vaapi_utils_h264_get_level_idc(): Returns H.264 level_idc value from GstVaapiLevelH264 - gst_vaapi_utils_h264_get_level_limits(): Returns level limits as specified in Table A-1 of the H.264 standard - gst_vaapi_utils_h264_get_level_limits_table(): Returns the Table A-1 specification
2013-12-06 18:05:52 +00:00
guint max_dec_frame_buffering, PicSizeMbs;
GstVaapiLevelH264 level;
const GstVaapiH264LevelLimits *level_limits;
decoder: cope with new GstVaapiContextInfo based API. Update decoders to report the maximum number of reference frames to use.
2012-09-10 16:26:51 +00:00
/* Table A-1 - Level limits */
utils: add helpers for H.264 levels. - gst_vaapi_utils_h264_get_level(): Returns GstVaapiLevelH264 from H.264 level_idc value - gst_vaapi_utils_h264_get_level_idc(): Returns H.264 level_idc value from GstVaapiLevelH264 - gst_vaapi_utils_h264_get_level_limits(): Returns level limits as specified in Table A-1 of the H.264 standard - gst_vaapi_utils_h264_get_level_limits_table(): Returns the Table A-1 specification
2013-12-06 18:05:52 +00:00
if (G_UNLIKELY(sps->level_idc == 11 && sps->constraint_set3_flag))
level = GST_VAAPI_LEVEL_H264_L1b;
else
level = gst_vaapi_utils_h264_get_level(sps->level_idc);
level_limits = gst_vaapi_utils_h264_get_level_limits(level);
decoder: h264: dynamically allocate the DPB. Dynamically allocate the Decoded Picture Buffer (DPB) and add provisions for supporting the MVC allocation requirements.
2014-05-01 17:16:09 +00:00
if (G_UNLIKELY(!level_limits)) {
GST_FIXME("unsupported level_idc value (%d)", sps->level_idc);
max_dec_frame_buffering = 16;
}
else {
PicSizeMbs = ((sps->pic_width_in_mbs_minus1 + 1) *
(sps->pic_height_in_map_units_minus1 + 1) *
(sps->frame_mbs_only_flag ? 1 : 2));
max_dec_frame_buffering = level_limits->MaxDpbMbs / PicSizeMbs;
}
if (is_mvc_profile(sps->profile_idc))
max_dec_frame_buffering <<= 1;
decoder: cope with new GstVaapiContextInfo based API. Update decoders to report the maximum number of reference frames to use.
2012-09-10 16:26:51 +00:00
/* VUI parameters */
if (sps->vui_parameters_present_flag) {
GstH264VUIParams * const vui_params = &sps->vui_parameters;
if (vui_params->bitstream_restriction_flag)
max_dec_frame_buffering = vui_params->max_dec_frame_buffering;
else {
switch (sps->profile_idc) {
case 44: // CAVLC 4:4:4 Intra profile
h264: use new profile definitions from codecparsers.
2013-02-07 14:42:55 +00:00
case GST_H264_PROFILE_SCALABLE_HIGH:
case GST_H264_PROFILE_HIGH:
case GST_H264_PROFILE_HIGH10:
case GST_H264_PROFILE_HIGH_422:
case GST_H264_PROFILE_HIGH_444:
decoder: cope with new GstVaapiContextInfo based API. Update decoders to report the maximum number of reference frames to use.
2012-09-10 16:26:51 +00:00
if (sps->constraint_set3_flag)
max_dec_frame_buffering = 0;
break;
}
}
}
decoder: h264: dynamically allocate the DPB. Dynamically allocate the Decoded Picture Buffer (DPB) and add provisions for supporting the MVC allocation requirements.
2014-05-01 17:16:09 +00:00
num_views = get_num_views(sps);
max_dpb_frames = 16 * (num_views > 1 ? g_bit_storage(num_views - 1) : 1);
if (max_dec_frame_buffering > max_dpb_frames)
max_dec_frame_buffering = max_dpb_frames;
decoder: cope with new GstVaapiContextInfo based API. Update decoders to report the maximum number of reference frames to use.
2012-09-10 16:26:51 +00:00
else if (max_dec_frame_buffering < sps->num_ref_frames)
max_dec_frame_buffering = sps->num_ref_frames;
return MAX(1, max_dec_frame_buffering);
}
h264: split remove_reference_at() into finer units. Split remove_reference_at() into a function that actually removes the specified entry from the short-term or long-term reference picture array, and a function that sets reference flags to the desired value, possibly zero. The latters marks the picture as "unused for reference".
2012-11-13 15:35:30 +00:00
static void
array_remove_index_fast(void *array, guint *array_length_ptr, guint index)
{
gpointer * const entries = array;
guint num_entries = *array_length_ptr;
g_return_if_fail(index < num_entries);
if (index != --num_entries)
entries[index] = entries[num_entries];
entries[num_entries] = NULL;
*array_length_ptr = num_entries;
}
#if 1
static inline void
array_remove_index(void *array, guint *array_length_ptr, guint index)
{
array_remove_index_fast(array, array_length_ptr, index);
}
#else
static void
array_remove_index(void *array, guint *array_length_ptr, guint index)
{
gpointer * const entries = array;
const guint num_entries = *array_length_ptr - 1;
guint i;
g_return_if_fail(index <= num_entries);
for (i = index; i < num_entries; i++)
entries[i] = entries[i + 1];
entries[num_entries] = NULL;
*array_length_ptr = num_entries;
}
#endif
#define ARRAY_REMOVE_INDEX(array, index) \
array_remove_index(array, &array##_count, index)
h264: handle Decoded Picture Buffer (DPB).
2012-01-24 14:38:14 +00:00
static void
dpb_remove_index(GstVaapiDecoderH264 *decoder, guint index)
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
h264: introduce new frame store structure. The frame store represents a Decoded Picture Buffer entry, which can hold up to two fields. So far, the frame store is only used to hold full frames.
2012-11-02 14:14:58 +00:00
guint i, num_frames = --priv->dpb_count;
h264: handle Decoded Picture Buffer (DPB).
2012-01-24 14:38:14 +00:00
h264: add flag to compile with strict DPB ordering mode. Allow build with strict DPB ordering mode whereby evicted entries are replaced by the next entries, in order instead of optimizing it away with the last entry in the DPB. This is only useful for debugging purpose, against a reference SW decoder for example.
2012-10-16 14:52:04 +00:00
if (USE_STRICT_DPB_ORDERING) {
h264: introduce new frame store structure. The frame store represents a Decoded Picture Buffer entry, which can hold up to two fields. So far, the frame store is only used to hold full frames.
2012-11-02 14:14:58 +00:00
for (i = index; i < num_frames; i++)
gst_vaapi_frame_store_replace(&priv->dpb[i], priv->dpb[i + 1]);
h264: add flag to compile with strict DPB ordering mode. Allow build with strict DPB ordering mode whereby evicted entries are replaced by the next entries, in order instead of optimizing it away with the last entry in the DPB. This is only useful for debugging purpose, against a reference SW decoder for example.
2012-10-16 14:52:04 +00:00
}
h264: introduce new frame store structure. The frame store represents a Decoded Picture Buffer entry, which can hold up to two fields. So far, the frame store is only used to hold full frames.
2012-11-02 14:14:58 +00:00
else if (index != num_frames)
gst_vaapi_frame_store_replace(&priv->dpb[index], priv->dpb[num_frames]);
gst_vaapi_frame_store_replace(&priv->dpb[num_frames], NULL);
h264: handle Decoded Picture Buffer (DPB).
2012-01-24 14:38:14 +00:00
}
h264: introduce new frame store structure. The frame store represents a Decoded Picture Buffer entry, which can hold up to two fields. So far, the frame store is only used to hold full frames.
2012-11-02 14:14:58 +00:00
static gboolean
decoder: h264: simplify the DPB output process. Simplify the dpb_output() function to exclusively rely on the frame store buffer to output, since this is now always provided. Besides, also fix cases where split fields would not be displayed. This is a regression from f48b1e0.
2014-07-03 09:13:33 +00:00
dpb_output(GstVaapiDecoderH264 *decoder, GstVaapiFrameStore *fs)
h264: handle Decoded Picture Buffer (DPB).
2012-01-24 14:38:14 +00:00
{
decoder: h264: simplify the DPB output process. Simplify the dpb_output() function to exclusively rely on the frame store buffer to output, since this is now always provided. Besides, also fix cases where split fields would not be displayed. This is a regression from f48b1e0.
2014-07-03 09:13:33 +00:00
GstVaapiPictureH264 *picture;
h264: introduce new frame store structure. The frame store represents a Decoded Picture Buffer entry, which can hold up to two fields. So far, the frame store is only used to hold full frames.
2012-11-02 14:14:58 +00:00
decoder: h264: simplify the DPB output process. Simplify the dpb_output() function to exclusively rely on the frame store buffer to output, since this is now always provided. Besides, also fix cases where split fields would not be displayed. This is a regression from f48b1e0.
2014-07-03 09:13:33 +00:00
g_return_val_if_fail(fs != NULL, FALSE);
decoder: h264: fix output of second field when first field is not in DPB. Fix decoding of interlaced streams where a first field (e.g. B-slice) was immediately output and the current decoded field is to be paired with that former frame, which is no longer in DPB. https://bugzilla.gnome.org/show_bug.cgi?id=701340
2014-06-30 14:09:17 +00:00
decoder: h264: simplify the DPB output process. Simplify the dpb_output() function to exclusively rely on the frame store buffer to output, since this is now always provided. Besides, also fix cases where split fields would not be displayed. This is a regression from f48b1e0.
2014-07-03 09:13:33 +00:00
if (!gst_vaapi_frame_store_is_complete(fs))
decoder: h264: fix output of second field when first field is not in DPB. Fix decoding of interlaced streams where a first field (e.g. B-slice) was immediately output and the current decoded field is to be paired with that former frame, which is no longer in DPB. https://bugzilla.gnome.org/show_bug.cgi?id=701340
2014-06-30 14:09:17 +00:00
return TRUE;
decoder: h264: simplify the DPB output process. Simplify the dpb_output() function to exclusively rely on the frame store buffer to output, since this is now always provided. Besides, also fix cases where split fields would not be displayed. This is a regression from f48b1e0.
2014-07-03 09:13:33 +00:00
picture = fs->buffers[0];
g_return_val_if_fail(picture != NULL, FALSE);
picture->output_needed = FALSE;
if (fs->num_buffers > 1) {
picture = fs->buffers[1];
g_return_val_if_fail(picture != NULL, FALSE);
picture->output_needed = FALSE;
}
fs->output_needed = 0;
h264: handle Decoded Picture Buffer (DPB).
2012-01-24 14:38:14 +00:00
return gst_vaapi_picture_output(GST_VAAPI_PICTURE_CAST(picture));
}
h264: introduce new frame store structure. The frame store represents a Decoded Picture Buffer entry, which can hold up to two fields. So far, the frame store is only used to hold full frames.
2012-11-02 14:14:58 +00:00
static inline void
dpb_evict(GstVaapiDecoderH264 *decoder, GstVaapiPictureH264 *picture, guint i)
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
GstVaapiFrameStore * const fs = priv->dpb[i];
h264: introduce new frame store structure. The frame store represents a Decoded Picture Buffer entry, which can hold up to two fields. So far, the frame store is only used to hold full frames.
2012-11-02 14:14:58 +00:00
if (!fs->output_needed && !gst_vaapi_frame_store_has_reference(fs))
dpb_remove_index(decoder, i);
}
decoder: h264: add support for MVC interlaced streams. Fix support for MVC Stereo High profile streams with interlaced frames. Also improve the detection logic of the first field.
2014-05-21 15:57:00 +00:00
/* Finds the frame store holding the supplied picture */
static gint
dpb_find_picture(GstVaapiDecoderH264 *decoder, GstVaapiPictureH264 *picture)
{
GstVaapiDecoderH264Private * const priv = &decoder->priv;
gint i, j;
for (i = 0; i < priv->dpb_count; i++) {
GstVaapiFrameStore * const fs = priv->dpb[i];
for (j = 0; j < fs->num_buffers; j++) {
if (fs->buffers[j] == picture)
return i;
}
}
return -1;
}
decoder: h264: simplify storage of decoded picture into DPB. Factor out process by which the decoded picture with the lowest POC is found, and possibly output. Likewise, the storage and marking of a reference decoded, or non-reference decoded picture, into the DPB could also be simplified as they mostly share the same operations.
2014-05-04 12:49:28 +00:00
/* Finds the picture with the lowest POC that needs to be output */
static gint
dpb_find_lowest_poc(GstVaapiDecoderH264 *decoder, GstVaapiPictureH264 *picture,
GstVaapiPictureH264 **found_picture_ptr)
h264: handle Decoded Picture Buffer (DPB).
2012-01-24 14:38:14 +00:00
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
h264: introduce new frame store structure. The frame store represents a Decoded Picture Buffer entry, which can hold up to two fields. So far, the frame store is only used to hold full frames.
2012-11-02 14:14:58 +00:00
GstVaapiPictureH264 *found_picture = NULL;
guint i, j, found_index;
h264: handle Decoded Picture Buffer (DPB).
2012-01-24 14:38:14 +00:00
for (i = 0; i < priv->dpb_count; i++) {
h264: introduce new frame store structure. The frame store represents a Decoded Picture Buffer entry, which can hold up to two fields. So far, the frame store is only used to hold full frames.
2012-11-02 14:14:58 +00:00
GstVaapiFrameStore * const fs = priv->dpb[i];
if (!fs->output_needed)
continue;
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
if (picture && picture->base.view_id != fs->view_id)
continue;
h264: introduce new frame store structure. The frame store represents a Decoded Picture Buffer entry, which can hold up to two fields. So far, the frame store is only used to hold full frames.
2012-11-02 14:14:58 +00:00
for (j = 0; j < fs->num_buffers; j++) {
decoder: h264: simplify storage of decoded picture into DPB. Factor out process by which the decoded picture with the lowest POC is found, and possibly output. Likewise, the storage and marking of a reference decoded, or non-reference decoded picture, into the DPB could also be simplified as they mostly share the same operations.
2014-05-04 12:49:28 +00:00
GstVaapiPictureH264 * const pic = fs->buffers[j];
if (!pic->output_needed)
h264: introduce new frame store structure. The frame store represents a Decoded Picture Buffer entry, which can hold up to two fields. So far, the frame store is only used to hold full frames.
2012-11-02 14:14:58 +00:00
continue;
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
if (!found_picture || found_picture->base.poc > pic->base.poc ||
(found_picture->base.poc == pic->base.poc &&
found_picture->base.voc > pic->base.voc))
found_picture = pic, found_index = i;
}
}
if (found_picture_ptr)
*found_picture_ptr = found_picture;
return found_picture ? found_index : -1;
}
/* Finds the picture with the lowest VOC that needs to be output */
static gint
dpb_find_lowest_voc(GstVaapiDecoderH264 *decoder, GstVaapiPictureH264 *picture,
GstVaapiPictureH264 **found_picture_ptr)
{
GstVaapiDecoderH264Private * const priv = &decoder->priv;
GstVaapiPictureH264 *found_picture = NULL;
guint i, j, found_index;
for (i = 0; i < priv->dpb_count; i++) {
GstVaapiFrameStore * const fs = priv->dpb[i];
if (!fs->output_needed || fs->view_id == picture->base.view_id)
continue;
for (j = 0; j < fs->num_buffers; j++) {
GstVaapiPictureH264 * const pic = fs->buffers[j];
if (!pic->output_needed || pic->base.poc != picture->base.poc)
continue;
if (!found_picture || found_picture->base.voc > pic->base.voc)
decoder: h264: simplify storage of decoded picture into DPB. Factor out process by which the decoded picture with the lowest POC is found, and possibly output. Likewise, the storage and marking of a reference decoded, or non-reference decoded picture, into the DPB could also be simplified as they mostly share the same operations.
2014-05-04 12:49:28 +00:00
found_picture = pic, found_index = i;
h264: introduce new frame store structure. The frame store represents a Decoded Picture Buffer entry, which can hold up to two fields. So far, the frame store is only used to hold full frames.
2012-11-02 14:14:58 +00:00
}
h264: handle Decoded Picture Buffer (DPB).
2012-01-24 14:38:14 +00:00
}
decoder: h264: simplify storage of decoded picture into DPB. Factor out process by which the decoded picture with the lowest POC is found, and possibly output. Likewise, the storage and marking of a reference decoded, or non-reference decoded picture, into the DPB could also be simplified as they mostly share the same operations.
2014-05-04 12:49:28 +00:00
if (found_picture_ptr)
*found_picture_ptr = found_picture;
return found_picture ? found_index : -1;
}
static gboolean
decoder: h264: improve DPB bumping process for MVC. While invoking the DPB bumping process in presence of many views, it could be necessary to output previous pictures that are ready, in a whole. i.e. emitting all view components from the very first view order index zero to the very last one in its original access unit; and not starting from the view order index of the picture that caused the DPB bumping process to be invoked. As a reminder, the maximum number of frames in DPB for MultiView High profile with more than 2 views is not necessarily a multiple of the number of views. This fixes decoding of MVCNV-4.264.
2014-06-03 15:36:38 +00:00
dpb_output_other_views(GstVaapiDecoderH264 *decoder,
GstVaapiPictureH264 *picture, guint voc)
decoder: h264: simplify storage of decoded picture into DPB. Factor out process by which the decoded picture with the lowest POC is found, and possibly output. Likewise, the storage and marking of a reference decoded, or non-reference decoded picture, into the DPB could also be simplified as they mostly share the same operations.
2014-05-04 12:49:28 +00:00
{
GstVaapiDecoderH264Private * const priv = &decoder->priv;
GstVaapiPictureH264 *found_picture;
gint found_index;
gboolean success;
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
if (priv->max_views == 1)
decoder: h264: improve DPB bumping process for MVC. While invoking the DPB bumping process in presence of many views, it could be necessary to output previous pictures that are ready, in a whole. i.e. emitting all view components from the very first view order index zero to the very last one in its original access unit; and not starting from the view order index of the picture that caused the DPB bumping process to be invoked. As a reminder, the maximum number of frames in DPB for MultiView High profile with more than 2 views is not necessarily a multiple of the number of views. This fixes decoding of MVCNV-4.264.
2014-06-03 15:36:38 +00:00
return TRUE;
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
/* Emit all other view components that were in the same access
unit than the picture we have just found */
decoder: h264: improve DPB bumping process for MVC. While invoking the DPB bumping process in presence of many views, it could be necessary to output previous pictures that are ready, in a whole. i.e. emitting all view components from the very first view order index zero to the very last one in its original access unit; and not starting from the view order index of the picture that caused the DPB bumping process to be invoked. As a reminder, the maximum number of frames in DPB for MultiView High profile with more than 2 views is not necessarily a multiple of the number of views. This fixes decoding of MVCNV-4.264.
2014-06-03 15:36:38 +00:00
found_picture = picture;
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
for (;;) {
found_index = dpb_find_lowest_voc(decoder, found_picture,
&found_picture);
decoder: h264: improve DPB bumping process for MVC. While invoking the DPB bumping process in presence of many views, it could be necessary to output previous pictures that are ready, in a whole. i.e. emitting all view components from the very first view order index zero to the very last one in its original access unit; and not starting from the view order index of the picture that caused the DPB bumping process to be invoked. As a reminder, the maximum number of frames in DPB for MultiView High profile with more than 2 views is not necessarily a multiple of the number of views. This fixes decoding of MVCNV-4.264.
2014-06-03 15:36:38 +00:00
if (found_index < 0 || found_picture->base.voc >= voc)
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
break;
decoder: h264: simplify the DPB output process. Simplify the dpb_output() function to exclusively rely on the frame store buffer to output, since this is now always provided. Besides, also fix cases where split fields would not be displayed. This is a regression from f48b1e0.
2014-07-03 09:13:33 +00:00
success = dpb_output(decoder, priv->dpb[found_index]);
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
dpb_evict(decoder, found_picture, found_index);
decoder: h264: improve DPB bumping process for MVC. While invoking the DPB bumping process in presence of many views, it could be necessary to output previous pictures that are ready, in a whole. i.e. emitting all view components from the very first view order index zero to the very last one in its original access unit; and not starting from the view order index of the picture that caused the DPB bumping process to be invoked. As a reminder, the maximum number of frames in DPB for MultiView High profile with more than 2 views is not necessarily a multiple of the number of views. This fixes decoding of MVCNV-4.264.
2014-06-03 15:36:38 +00:00
if (!success)
return FALSE;
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
}
decoder: h264: improve DPB bumping process for MVC. While invoking the DPB bumping process in presence of many views, it could be necessary to output previous pictures that are ready, in a whole. i.e. emitting all view components from the very first view order index zero to the very last one in its original access unit; and not starting from the view order index of the picture that caused the DPB bumping process to be invoked. As a reminder, the maximum number of frames in DPB for MultiView High profile with more than 2 views is not necessarily a multiple of the number of views. This fixes decoding of MVCNV-4.264.
2014-06-03 15:36:38 +00:00
return TRUE;
}
static gboolean
dpb_bump(GstVaapiDecoderH264 *decoder, GstVaapiPictureH264 *picture)
{
GstVaapiDecoderH264Private * const priv = &decoder->priv;
GstVaapiPictureH264 *found_picture;
gint found_index;
gboolean success;
found_index = dpb_find_lowest_poc(decoder, picture, &found_picture);
if (found_index < 0)
return FALSE;
if (picture && picture->base.poc != found_picture->base.poc)
dpb_output_other_views(decoder, found_picture, found_picture->base.voc);
decoder: h264: simplify the DPB output process. Simplify the dpb_output() function to exclusively rely on the frame store buffer to output, since this is now always provided. Besides, also fix cases where split fields would not be displayed. This is a regression from f48b1e0.
2014-07-03 09:13:33 +00:00
success = dpb_output(decoder, priv->dpb[found_index]);
decoder: h264: improve DPB bumping process for MVC. While invoking the DPB bumping process in presence of many views, it could be necessary to output previous pictures that are ready, in a whole. i.e. emitting all view components from the very first view order index zero to the very last one in its original access unit; and not starting from the view order index of the picture that caused the DPB bumping process to be invoked. As a reminder, the maximum number of frames in DPB for MultiView High profile with more than 2 views is not necessarily a multiple of the number of views. This fixes decoding of MVCNV-4.264.
2014-06-03 15:36:38 +00:00
dpb_evict(decoder, found_picture, found_index);
if (priv->max_views == 1)
return success;
if (picture && picture->base.poc != found_picture->base.poc)
dpb_output_other_views(decoder, found_picture, G_MAXUINT32);
h264: handle Decoded Picture Buffer (DPB).
2012-01-24 14:38:14 +00:00
return success;
}
static void
decoder: h264: simplify storage of decoded picture into DPB. Factor out process by which the decoded picture with the lowest POC is found, and possibly output. Likewise, the storage and marking of a reference decoded, or non-reference decoded picture, into the DPB could also be simplified as they mostly share the same operations.
2014-05-04 12:49:28 +00:00
dpb_clear(GstVaapiDecoderH264 *decoder, GstVaapiPictureH264 *picture)
h264: handle Decoded Picture Buffer (DPB).
2012-01-24 14:38:14 +00:00
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
guint i, n;
h264: handle Decoded Picture Buffer (DPB).
2012-01-24 14:38:14 +00:00
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
for (i = 0; i < priv->dpb_count; i++) {
if (picture && picture->base.view_id != priv->dpb[i]->view_id)
continue;
h264: introduce new frame store structure. The frame store represents a Decoded Picture Buffer entry, which can hold up to two fields. So far, the frame store is only used to hold full frames.
2012-11-02 14:14:58 +00:00
gst_vaapi_frame_store_replace(&priv->dpb[i], NULL);
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
}
decoder: h264: fix the DPB compaction process. Fix the compaction process when the DPB is cleared for a specific view, i.e. fix the process of filling in the holes resulting from removing frame buffers matching the current picture.
2014-06-27 18:44:24 +00:00
/* Compact the resulting DPB, i.e. remove holes */
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
for (i = 0, n = 0; i < priv->dpb_count; i++) {
decoder: h264: fix the DPB compaction process. Fix the compaction process when the DPB is cleared for a specific view, i.e. fix the process of filling in the holes resulting from removing frame buffers matching the current picture.
2014-06-27 18:44:24 +00:00
if (priv->dpb[i]) {
if (i != n) {
priv->dpb[n] = priv->dpb[i];
priv->dpb[i] = NULL;
}
n++;
}
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
}
priv->dpb_count = n;
h264: introduce new frame store structure. The frame store represents a Decoded Picture Buffer entry, which can hold up to two fields. So far, the frame store is only used to hold full frames.
2012-11-02 14:14:58 +00:00
decoder: h264: add support for MVC interlaced streams. Fix support for MVC Stereo High profile streams with interlaced frames. Also improve the detection logic of the first field.
2014-05-21 15:57:00 +00:00
/* Clear previous frame buffers only if this is a "flush-all" operation,
or if the picture is the first one in the access unit */
decoder: h264: fix DPB clear when no decoding actually started. Fix dpb_clear() to clear previous frame buffers only if they actually exist to begin with. If the decoder bailed out early, e.g. when it does not support a specific profile, that array of previous frames might not be allocated beforehand.
2014-06-19 15:08:47 +00:00
if (priv->prev_frames && (!picture ||
GST_VAAPI_PICTURE_FLAG_IS_SET(picture,
GST_VAAPI_PICTURE_FLAG_AU_START))) {
decoder: h264: add support for MVC interlaced streams. Fix support for MVC Stereo High profile streams with interlaced frames. Also improve the detection logic of the first field.
2014-05-21 15:57:00 +00:00
for (i = 0; i < priv->max_views; i++)
decoder: h264: fix the DPB compaction process. Fix the compaction process when the DPB is cleared for a specific view, i.e. fix the process of filling in the holes resulting from removing frame buffers matching the current picture.
2014-06-27 18:44:24 +00:00
gst_vaapi_frame_store_replace(&priv->prev_frames[i], NULL);
decoder: h264: add support for MVC interlaced streams. Fix support for MVC Stereo High profile streams with interlaced frames. Also improve the detection logic of the first field.
2014-05-21 15:57:00 +00:00
}
h264: simplify reference picture marking process. ... to build the short_ref[] and long_ref[] lists from the DPB, instead of maintaining them separately. This avoids refs/unrefs while making it possible to generate the list based on the actual picture structure. This also ensures that the list of generated ReferenceFrames[] actually matches what reference frames are available in the DPB. i.e. short_ref[] and long_ref[] entries are implied from the DPB, so there is no risk of having "dangling" references.
2012-10-31 13:24:09 +00:00
}
static void
decoder: h264: simplify storage of decoded picture into DPB. Factor out process by which the decoded picture with the lowest POC is found, and possibly output. Likewise, the storage and marking of a reference decoded, or non-reference decoded picture, into the DPB could also be simplified as they mostly share the same operations.
2014-05-04 12:49:28 +00:00
dpb_flush(GstVaapiDecoderH264 *decoder, GstVaapiPictureH264 *picture)
h264: simplify reference picture marking process. ... to build the short_ref[] and long_ref[] lists from the DPB, instead of maintaining them separately. This avoids refs/unrefs while making it possible to generate the list based on the actual picture structure. This also ensures that the list of generated ReferenceFrames[] actually matches what reference frames are available in the DPB. i.e. short_ref[] and long_ref[] entries are implied from the DPB, so there is no risk of having "dangling" references.
2012-10-31 13:24:09 +00:00
{
decoder: h264: detect incorrectly paired fields in frames. When a DPB flush is required, e.g. at a natural and of stream or issued explicitly through an IDR, try to detect any frame left in the DPB that is interlaced but does not contain two decoded fields. In that case, mark the picture as having a single field only. This avoids a hang while decoding tv_cut.mkv.
2014-07-03 11:48:48 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
guint i;
/* Detect broken frames and mark them as having a single field if
needed */
for (i = 0; i < priv->dpb_count; i++) {
GstVaapiFrameStore * const fs = priv->dpb[i];
if (!fs->output_needed || gst_vaapi_frame_store_is_complete(fs))
continue;
GST_VAAPI_PICTURE_FLAG_SET(fs->buffers[0],
GST_VAAPI_PICTURE_FLAG_ONEFIELD);
}
/* Output any frame remaining in DPB */
decoder: h264: simplify storage of decoded picture into DPB. Factor out process by which the decoded picture with the lowest POC is found, and possibly output. Likewise, the storage and marking of a reference decoded, or non-reference decoded picture, into the DPB could also be simplified as they mostly share the same operations.
2014-05-04 12:49:28 +00:00
while (dpb_bump(decoder, picture))
h264: handle Decoded Picture Buffer (DPB).
2012-01-24 14:38:14 +00:00
;
decoder: h264: simplify storage of decoded picture into DPB. Factor out process by which the decoded picture with the lowest POC is found, and possibly output. Likewise, the storage and marking of a reference decoded, or non-reference decoded picture, into the DPB could also be simplified as they mostly share the same operations.
2014-05-04 12:49:28 +00:00
dpb_clear(decoder, picture);
h264: handle Decoded Picture Buffer (DPB).
2012-01-24 14:38:14 +00:00
}
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
static void
dpb_prune_mvc(GstVaapiDecoderH264 *decoder, GstVaapiPictureH264 *picture)
{
GstVaapiDecoderH264Private * const priv = &decoder->priv;
decoder: h264: factor out DPB pruning for MVC. Factor out the removal process of unused inter-view only reference pictures from the DPB, prior to the possible insertion of the current picture. Ideally, the compiler could still opt for generating two loops. But at least, the code is now clearer for maintenance.
2014-06-10 16:30:21 +00:00
const gboolean is_last_picture = /* in the access unit */
GST_VAAPI_PICTURE_FLAG_IS_SET(picture, GST_VAAPI_PICTURE_FLAG_AU_END);
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
guint i;
decoder: h264: improve pruning of unused MVC inter-view frames. Improve process for the removal of pictures from DPB before possible insertion of the current picture (C.4.4) for H.264 MVC inter-view only reference components. In particular, handle cases where picture to be inserted is not the last one of the access unit and if it was already output and is no longer marked as used for reference, including for decoding next view components within the same access unit.
2014-06-10 14:07:51 +00:00
// Remove all unused inter-view only reference components of the current AU
decoder: h264: factor out DPB pruning for MVC. Factor out the removal process of unused inter-view only reference pictures from the DPB, prior to the possible insertion of the current picture. Ideally, the compiler could still opt for generating two loops. But at least, the code is now clearer for maintenance.
2014-06-10 16:30:21 +00:00
i = 0;
while (i < priv->dpb_count) {
GstVaapiFrameStore * const fs = priv->dpb[i];
if (fs->view_id != picture->base.view_id &&
!fs->output_needed && !gst_vaapi_frame_store_has_reference(fs) &&
(is_last_picture ||
!is_inter_view_reference_for_next_frames(decoder, fs)))
dpb_remove_index(decoder, i);
else
i++;
decoder: h264: improve pruning of unused MVC inter-view frames. Improve process for the removal of pictures from DPB before possible insertion of the current picture (C.4.4) for H.264 MVC inter-view only reference components. In particular, handle cases where picture to be inserted is not the last one of the access unit and if it was already output and is no longer marked as used for reference, including for decoding next view components within the same access unit.
2014-06-10 14:07:51 +00:00
}
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
}
h264: handle Decoded Picture Buffer (DPB).
2012-01-24 14:38:14 +00:00
static gboolean
dpb_add(GstVaapiDecoderH264 *decoder, GstVaapiPictureH264 *picture)
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
h264: introduce new frame store structure. The frame store represents a Decoded Picture Buffer entry, which can hold up to two fields. So far, the frame store is only used to hold full frames.
2012-11-02 14:14:58 +00:00
GstVaapiFrameStore *fs;
decoder: h264: simplify storage of decoded picture into DPB. Factor out process by which the decoded picture with the lowest POC is found, and possibly output. Likewise, the storage and marking of a reference decoded, or non-reference decoded picture, into the DPB could also be simplified as they mostly share the same operations.
2014-05-04 12:49:28 +00:00
guint i;
h264: handle Decoded Picture Buffer (DPB).
2012-01-24 14:38:14 +00:00
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
if (priv->max_views > 1)
dpb_prune_mvc(decoder, picture);
h264: handle Decoded Picture Buffer (DPB).
2012-01-24 14:38:14 +00:00
// Remove all unused pictures
h264: minor clean-ups. Move DPB flush up if the current picture to decode is an IDR. Besides, don't bother to check for IDR pictures in dpb_add() function since an explicit DPB flush was already performed in this case.
2012-10-31 15:37:14 +00:00
if (!GST_VAAPI_PICTURE_IS_IDR(picture)) {
h264: handle Decoded Picture Buffer (DPB).
2012-01-24 14:38:14 +00:00
i = 0;
while (i < priv->dpb_count) {
h264: introduce new frame store structure. The frame store represents a Decoded Picture Buffer entry, which can hold up to two fields. So far, the frame store is only used to hold full frames.
2012-11-02 14:14:58 +00:00
GstVaapiFrameStore * const fs = priv->dpb[i];
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
if (fs->view_id == picture->base.view_id &&
!fs->output_needed && !gst_vaapi_frame_store_has_reference(fs))
h264: handle Decoded Picture Buffer (DPB).
2012-01-24 14:38:14 +00:00
dpb_remove_index(decoder, i);
else
i++;
}
}
h264: add initial support for interlaced streams. Decoded frames are only output when they are complete, i.e. when both fields are decoded. This also means that the "interlaced" caps is not propagated to vaapipostproc or vaapisink elements. Another limitation is that interlaced bitstreams with MMCO are unlikely to work.
2012-11-13 16:14:39 +00:00
// Check if picture is the second field and the first field is still in DPB
decoder: h264: add support for MVC interlaced streams. Fix support for MVC Stereo High profile streams with interlaced frames. Also improve the detection logic of the first field.
2014-05-21 15:57:00 +00:00
if (GST_VAAPI_PICTURE_IS_INTERLACED(picture) &&
!GST_VAAPI_PICTURE_IS_FIRST_FIELD(picture)) {
const gint found_index = dpb_find_picture(decoder,
GST_VAAPI_PICTURE_H264(picture->base.parent_picture));
if (found_index >= 0)
return gst_vaapi_frame_store_add(priv->dpb[found_index], picture);
decoder: h264: fix output of second field when first field is not in DPB. Fix decoding of interlaced streams where a first field (e.g. B-slice) was immediately output and the current decoded field is to be paired with that former frame, which is no longer in DPB. https://bugzilla.gnome.org/show_bug.cgi?id=701340
2014-06-30 14:09:17 +00:00
// ... also check the previous picture that was immediately output
fs = priv->prev_frames[picture->base.voc];
if (fs && &fs->buffers[0]->base == picture->base.parent_picture) {
if (!gst_vaapi_frame_store_add(fs, picture))
return FALSE;
decoder: h264: simplify the DPB output process. Simplify the dpb_output() function to exclusively rely on the frame store buffer to output, since this is now always provided. Besides, also fix cases where split fields would not be displayed. This is a regression from f48b1e0.
2014-07-03 09:13:33 +00:00
return dpb_output(decoder, fs);
decoder: h264: fix output of second field when first field is not in DPB. Fix decoding of interlaced streams where a first field (e.g. B-slice) was immediately output and the current decoded field is to be paired with that former frame, which is no longer in DPB. https://bugzilla.gnome.org/show_bug.cgi?id=701340
2014-06-30 14:09:17 +00:00
}
decoder: h264: add support for MVC interlaced streams. Fix support for MVC Stereo High profile streams with interlaced frames. Also improve the detection logic of the first field.
2014-05-21 15:57:00 +00:00
}
h264: add initial support for interlaced streams. Decoded frames are only output when they are complete, i.e. when both fields are decoded. This also means that the "interlaced" caps is not propagated to vaapipostproc or vaapisink elements. Another limitation is that interlaced bitstreams with MMCO are unlikely to work.
2012-11-13 16:14:39 +00:00
// Create new frame store, and split fields if necessary
h264: introduce new frame store structure. The frame store represents a Decoded Picture Buffer entry, which can hold up to two fields. So far, the frame store is only used to hold full frames.
2012-11-02 14:14:58 +00:00
fs = gst_vaapi_frame_store_new(picture);
if (!fs)
return FALSE;
decoder: h264: add support for MVC interlaced streams. Fix support for MVC Stereo High profile streams with interlaced frames. Also improve the detection logic of the first field.
2014-05-21 15:57:00 +00:00
gst_vaapi_frame_store_replace(&priv->prev_frames[picture->base.voc], fs);
h264: introduce new frame store structure. The frame store represents a Decoded Picture Buffer entry, which can hold up to two fields. So far, the frame store is only used to hold full frames.
2012-11-02 14:14:58 +00:00
gst_vaapi_frame_store_unref(fs);
h264: add initial support for interlaced streams. Decoded frames are only output when they are complete, i.e. when both fields are decoded. This also means that the "interlaced" caps is not propagated to vaapipostproc or vaapisink elements. Another limitation is that interlaced bitstreams with MMCO are unlikely to work.
2012-11-13 16:14:39 +00:00
if (!priv->progressive_sequence && gst_vaapi_frame_store_has_frame(fs)) {
if (!gst_vaapi_frame_store_split_fields(fs))
return FALSE;
}
h264: handle Decoded Picture Buffer (DPB).
2012-01-24 14:38:14 +00:00
// C.4.5.1 - Storage and marking of a reference decoded picture into the DPB
if (GST_VAAPI_PICTURE_IS_REFERENCE(picture)) {
while (priv->dpb_count == priv->dpb_size) {
decoder: h264: simplify storage of decoded picture into DPB. Factor out process by which the decoded picture with the lowest POC is found, and possibly output. Likewise, the storage and marking of a reference decoded, or non-reference decoded picture, into the DPB could also be simplified as they mostly share the same operations.
2014-05-04 12:49:28 +00:00
if (!dpb_bump(decoder, picture))
h264: handle Decoded Picture Buffer (DPB).
2012-01-24 14:38:14 +00:00
return FALSE;
}
}
// C.4.5.2 - Storage and marking of a non-reference decoded picture into the DPB
else {
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
const gboolean StoreInterViewOnlyRefFlag =
!GST_VAAPI_PICTURE_FLAG_IS_SET(picture,
GST_VAAPI_PICTURE_FLAG_AU_END) &&
GST_VAAPI_PICTURE_FLAG_IS_SET(picture,
GST_VAAPI_PICTURE_FLAG_INTER_VIEW);
if (!picture->output_flag && !StoreInterViewOnlyRefFlag)
h264: handle Decoded Picture Buffer (DPB).
2012-01-24 14:38:14 +00:00
return TRUE;
while (priv->dpb_count == priv->dpb_size) {
decoder: h264: fix marking of non-reference picture into DPB. Fix search for a picture in the DPB that has a lower POC value than the current picture. The dpb_find_lowest_poc() function will return a picture with the lowest POC in DPB and that is marked as "needed for output", but an additional check against the actual POC value of the current picture is needed. This is a regression from 1c46990. https://bugzilla.gnome.org/show_bug.cgi?id=732130
2014-06-25 20:26:32 +00:00
GstVaapiPictureH264 *found_picture;
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
if (!StoreInterViewOnlyRefFlag) {
decoder: h264: fix marking of non-reference picture into DPB. Fix search for a picture in the DPB that has a lower POC value than the current picture. The dpb_find_lowest_poc() function will return a picture with the lowest POC in DPB and that is marked as "needed for output", but an additional check against the actual POC value of the current picture is needed. This is a regression from 1c46990. https://bugzilla.gnome.org/show_bug.cgi?id=732130
2014-06-25 20:26:32 +00:00
if (dpb_find_lowest_poc(decoder, picture, &found_picture) < 0 ||
found_picture->base.poc > picture->base.poc)
decoder: h264: simplify the DPB output process. Simplify the dpb_output() function to exclusively rely on the frame store buffer to output, since this is now always provided. Besides, also fix cases where split fields would not be displayed. This is a regression from f48b1e0.
2014-07-03 09:13:33 +00:00
return dpb_output(decoder, fs);
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
}
decoder: h264: simplify storage of decoded picture into DPB. Factor out process by which the decoded picture with the lowest POC is found, and possibly output. Likewise, the storage and marking of a reference decoded, or non-reference decoded picture, into the DPB could also be simplified as they mostly share the same operations.
2014-05-04 12:49:28 +00:00
if (!dpb_bump(decoder, picture))
h264: handle Decoded Picture Buffer (DPB).
2012-01-24 14:38:14 +00:00
return FALSE;
}
decoder: h264: simplify storage of decoded picture into DPB. Factor out process by which the decoded picture with the lowest POC is found, and possibly output. Likewise, the storage and marking of a reference decoded, or non-reference decoded picture, into the DPB could also be simplified as they mostly share the same operations.
2014-05-04 12:49:28 +00:00
}
gst_vaapi_frame_store_replace(&priv->dpb[priv->dpb_count++], fs);
h264: handle Decoded Picture Buffer (DPB).
2012-01-24 14:38:14 +00:00
return TRUE;
}
decoder: h264: dynamically allocate the DPB. Dynamically allocate the Decoded Picture Buffer (DPB) and add provisions for supporting the MVC allocation requirements.
2014-05-01 17:16:09 +00:00
static gboolean
dpb_reset(GstVaapiDecoderH264 *decoder, guint dpb_size)
h264: handle Decoded Picture Buffer (DPB).
2012-01-24 14:38:14 +00:00
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
h264: handle Decoded Picture Buffer (DPB).
2012-01-24 14:38:14 +00:00
decoder: h264: dynamically allocate the DPB. Dynamically allocate the Decoded Picture Buffer (DPB) and add provisions for supporting the MVC allocation requirements.
2014-05-01 17:16:09 +00:00
if (dpb_size > priv->dpb_size_max) {
priv->dpb = g_try_realloc_n(priv->dpb, dpb_size, sizeof(*priv->dpb));
if (!priv->dpb)
return FALSE;
memset(&priv->dpb[priv->dpb_size_max], 0,
(dpb_size - priv->dpb_size_max) * sizeof(*priv->dpb));
priv->dpb_size_max = dpb_size;
}
decoder: h264: fix tracking of DPB size changes. Add support for MVC streams with multiple SPS and subset SPS headers emitted regularly, e.g. at around every I-frame. Track the maximum number of views in ensure_context() and really reset the DPB size to the expected value, always. i.e. even if it decreased. dpb_reset() only cares of ensuring the DPB allocation.
2014-06-28 05:25:35 +00:00
priv->dpb_size = dpb_size;
decoder: h264: dynamically allocate the DPB. Dynamically allocate the Decoded Picture Buffer (DPB) and add provisions for supporting the MVC allocation requirements.
2014-05-01 17:16:09 +00:00
h264: handle Decoded Picture Buffer (DPB).
2012-01-24 14:38:14 +00:00
GST_DEBUG("DPB size %u", priv->dpb_size);
decoder: h264: dynamically allocate the DPB. Dynamically allocate the Decoded Picture Buffer (DPB) and add provisions for supporting the MVC allocation requirements.
2014-05-01 17:16:09 +00:00
return TRUE;
h264: handle Decoded Picture Buffer (DPB).
2012-01-24 14:38:14 +00:00
}
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
static void
unref_inter_view(GstVaapiPictureH264 *picture)
{
if (!picture)
return;
GST_VAAPI_PICTURE_FLAG_UNSET(picture, GST_VAAPI_PICTURE_FLAG_INTER_VIEW);
gst_vaapi_picture_unref(picture);
}
/* Resets MVC resources */
static gboolean
mvc_reset(GstVaapiDecoderH264 *decoder)
{
GstVaapiDecoderH264Private * const priv = &decoder->priv;
decoder: h264: add support for MVC interlaced streams. Fix support for MVC Stereo High profile streams with interlaced frames. Also improve the detection logic of the first field.
2014-05-21 15:57:00 +00:00
guint i;
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
// Resize array of inter-view references
decoder: h264: fix inter-view references array growth. Let the utility layer handle dynamic growth of the inter-view pictures array. By definition, setting a new size to the array will effectively grow the array, but would also fill in the newly created elements with empty entries (NULL), thus also increasing the reported length, which is not correct.
2014-06-06 15:56:06 +00:00
if (!priv->inter_views) {
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
priv->inter_views = g_ptr_array_new_full(priv->max_views,
(GDestroyNotify)unref_inter_view);
if (!priv->inter_views)
return FALSE;
}
decoder: h264: add support for MVC interlaced streams. Fix support for MVC Stereo High profile streams with interlaced frames. Also improve the detection logic of the first field.
2014-05-21 15:57:00 +00:00
// Resize array of previous frame buffers
for (i = priv->max_views; i < priv->prev_frames_alloc; i++)
decoder: h264: fix the DPB compaction process. Fix the compaction process when the DPB is cleared for a specific view, i.e. fix the process of filling in the holes resulting from removing frame buffers matching the current picture.
2014-06-27 18:44:24 +00:00
gst_vaapi_frame_store_replace(&priv->prev_frames[i], NULL);
decoder: h264: add support for MVC interlaced streams. Fix support for MVC Stereo High profile streams with interlaced frames. Also improve the detection logic of the first field.
2014-05-21 15:57:00 +00:00
priv->prev_frames = g_try_realloc_n(priv->prev_frames, priv->max_views,
sizeof(*priv->prev_frames));
if (!priv->prev_frames) {
priv->prev_frames_alloc = 0;
return FALSE;
}
for (i = priv->prev_frames_alloc; i < priv->max_views; i++)
priv->prev_frames[i] = NULL;
priv->prev_frames_alloc = priv->max_views;
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
return TRUE;
}
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
static GstVaapiDecoderStatus
get_status(GstH264ParserResult result)
{
GstVaapiDecoderStatus status;
switch (result) {
case GST_H264_PARSER_OK:
status = GST_VAAPI_DECODER_STATUS_SUCCESS;
break;
case GST_H264_PARSER_NO_NAL_END:
status = GST_VAAPI_DECODER_STATUS_ERROR_NO_DATA;
break;
case GST_H264_PARSER_ERROR:
status = GST_VAAPI_DECODER_STATUS_ERROR_BITSTREAM_PARSER;
break;
default:
status = GST_VAAPI_DECODER_STATUS_ERROR_UNKNOWN;
break;
}
return status;
}
static void
gst_vaapi_decoder_h264_close(GstVaapiDecoderH264 *decoder)
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
gst_vaapi_picture_replace(&priv->current_picture, NULL);
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
gst_vaapi_parser_info_h264_replace(&priv->prev_slice_pi, NULL);
decoder: h264: properly handle Prefix NAL units. Always cache the previous NAL unit so that we could check whether there is a Prefix NAL unit immediately preceding the current slice or IDR NAL unit. In that case, the NAL unit metadata is copied into the current NAL unit. Otherwise, some default values are inferred, tentatively. e.g. view_id shall be set to 0 and inter_view_flag to 1. [infer default values for slice if previous NAL was not a Prefix] Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2013-10-31 11:32:55 +00:00
gst_vaapi_parser_info_h264_replace(&priv->prev_pi, NULL);
h264: simplify reference picture marking process. ... to build the short_ref[] and long_ref[] lists from the DPB, instead of maintaining them separately. This avoids refs/unrefs while making it possible to generate the list based on the actual picture structure. This also ensures that the list of generated ReferenceFrames[] actually matches what reference frames are available in the DPB. i.e. short_ref[] and long_ref[] entries are implied from the DPB, so there is no risk of having "dangling" references.
2012-10-31 13:24:09 +00:00
decoder: h264: simplify storage of decoded picture into DPB. Factor out process by which the decoded picture with the lowest POC is found, and possibly output. Likewise, the storage and marking of a reference decoded, or non-reference decoded picture, into the DPB could also be simplified as they mostly share the same operations.
2014-05-04 12:49:28 +00:00
dpb_clear(decoder, NULL);
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
if (priv->inter_views) {
g_ptr_array_unref(priv->inter_views);
priv->inter_views = NULL;
}
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
if (priv->parser) {
gst_h264_nal_parser_free(priv->parser);
priv->parser = NULL;
}
}
static gboolean
h264: initial port to new GstVaapiDecoder API
2012-12-06 13:02:17 +00:00
gst_vaapi_decoder_h264_open(GstVaapiDecoderH264 *decoder)
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
gst_vaapi_decoder_h264_close(decoder);
priv->parser = gst_h264_nal_parser_new();
if (!priv->parser)
return FALSE;
return TRUE;
}
static void
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
gst_vaapi_decoder_h264_destroy(GstVaapiDecoder *base_decoder)
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264 * const decoder =
GST_VAAPI_DECODER_H264_CAST(base_decoder);
decoder: h264: fix activation of picture and sequence parameters. At the time the first VCL NAL unit of a primary coded picture is found, and if that NAL unit was parsed to be an SPS or PPS, then the entries in the parser may have been overriden. This means that, when the picture is to be decoded, slice_hdr->pps could point to an invalid (the next) PPS entry. So, one way to solve this problem is to not use the parser PPS and SPS info but rather maintain our own activation chain in the decoder. https://bugzilla.gnome.org/show_bug.cgi?id=724519 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 17:11:03 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
guint i;
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
gst_vaapi_decoder_h264_close(decoder);
decoder: h264: fix activation of picture and sequence parameters. At the time the first VCL NAL unit of a primary coded picture is found, and if that NAL unit was parsed to be an SPS or PPS, then the entries in the parser may have been overriden. This means that, when the picture is to be decoded, slice_hdr->pps could point to an invalid (the next) PPS entry. So, one way to solve this problem is to not use the parser PPS and SPS info but rather maintain our own activation chain in the decoder. https://bugzilla.gnome.org/show_bug.cgi?id=724519 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 17:11:03 +00:00
decoder: h264: dynamically allocate the DPB. Dynamically allocate the Decoded Picture Buffer (DPB) and add provisions for supporting the MVC allocation requirements.
2014-05-01 17:16:09 +00:00
g_free(priv->dpb);
priv->dpb = NULL;
priv->dpb_size = 0;
decoder: h264: add support for MVC interlaced streams. Fix support for MVC Stereo High profile streams with interlaced frames. Also improve the detection logic of the first field.
2014-05-21 15:57:00 +00:00
g_free(priv->prev_frames);
priv->prev_frames = NULL;
priv->prev_frames_alloc = 0;
decoder: h264: fix activation of picture and sequence parameters. At the time the first VCL NAL unit of a primary coded picture is found, and if that NAL unit was parsed to be an SPS or PPS, then the entries in the parser may have been overriden. This means that, when the picture is to be decoded, slice_hdr->pps could point to an invalid (the next) PPS entry. So, one way to solve this problem is to not use the parser PPS and SPS info but rather maintain our own activation chain in the decoder. https://bugzilla.gnome.org/show_bug.cgi?id=724519 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 17:11:03 +00:00
for (i = 0; i < G_N_ELEMENTS(priv->pps); i++)
gst_vaapi_parser_info_h264_replace(&priv->pps[i], NULL);
gst_vaapi_parser_info_h264_replace(&priv->active_pps, NULL);
for (i = 0; i < G_N_ELEMENTS(priv->sps); i++)
gst_vaapi_parser_info_h264_replace(&priv->sps[i], NULL);
gst_vaapi_parser_info_h264_replace(&priv->active_sps, NULL);
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
}
static gboolean
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
gst_vaapi_decoder_h264_create(GstVaapiDecoder *base_decoder)
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264 * const decoder =
GST_VAAPI_DECODER_H264_CAST(base_decoder);
GstVaapiDecoderH264Private * const priv = &decoder->priv;
priv->profile = GST_VAAPI_PROFILE_UNKNOWN;
priv->entrypoint = GST_VAAPI_ENTRYPOINT_VLD;
priv->chroma_type = GST_VAAPI_CHROMA_TYPE_YUV420;
priv->prev_pic_structure = GST_VAAPI_PICTURE_STRUCTURE_FRAME;
priv->progressive_sequence = TRUE;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
return TRUE;
}
decoder: h264: fix activation of picture and sequence parameters. At the time the first VCL NAL unit of a primary coded picture is found, and if that NAL unit was parsed to be an SPS or PPS, then the entries in the parser may have been overriden. This means that, when the picture is to be decoded, slice_hdr->pps could point to an invalid (the next) PPS entry. So, one way to solve this problem is to not use the parser PPS and SPS info but rather maintain our own activation chain in the decoder. https://bugzilla.gnome.org/show_bug.cgi?id=724519 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 17:11:03 +00:00
/* Activates the supplied PPS */
static GstH264PPS *
ensure_pps(GstVaapiDecoderH264 *decoder, GstH264PPS *pps)
{
GstVaapiDecoderH264Private * const priv = &decoder->priv;
GstVaapiParserInfoH264 * const pi = priv->pps[pps->id];
gst_vaapi_parser_info_h264_replace(&priv->active_pps, pi);
return pi ? &pi->data.pps : NULL;
}
/* Returns the active PPS */
static inline GstH264PPS *
get_pps(GstVaapiDecoderH264 *decoder)
{
GstVaapiParserInfoH264 * const pi = decoder->priv.active_pps;
return pi ? &pi->data.pps : NULL;
}
/* Activate the supplied SPS */
static GstH264SPS *
ensure_sps(GstVaapiDecoderH264 *decoder, GstH264SPS *sps)
{
GstVaapiDecoderH264Private * const priv = &decoder->priv;
GstVaapiParserInfoH264 * const pi = priv->sps[sps->id];
gst_vaapi_parser_info_h264_replace(&priv->active_sps, pi);
return pi ? &pi->data.sps : NULL;
}
/* Returns the active SPS */
static inline GstH264SPS *
get_sps(GstVaapiDecoderH264 *decoder)
{
GstVaapiParserInfoH264 * const pi = decoder->priv.active_sps;
return pi ? &pi->data.sps : NULL;
}
decoder: h264: fix decoding of BA3_SVA_C.264. Conformance test Base_Ext_Main_profiles/BA3_SVA_C.264 complys with extended profile specifications. However, the SPS header has the constraint_set1_flag syntax element set to 1. This means that, if a Main profile compliant decoder is available, then it should be able to decode this stream. This changes makes it possible to fall-back from Extended profile to Main profile if constraint_set1_flag is set to 1. https://bugzilla.gnome.org/show_bug.cgi?id=720190
2013-12-10 13:20:55 +00:00
static void
fill_profiles(GstVaapiProfile profiles[16], guint *n_profiles_ptr,
GstVaapiProfile profile)
{
guint n_profiles = *n_profiles_ptr;
profiles[n_profiles++] = profile;
switch (profile) {
case GST_VAAPI_PROFILE_H264_MAIN:
profiles[n_profiles++] = GST_VAAPI_PROFILE_H264_HIGH;
break;
default:
break;
}
*n_profiles_ptr = n_profiles;
}
decoder: h264: add MVC profiles compatibility logic. Add safe fallbacks for MVC profiles: - all MultiView High profile streams with 2 views at most can be decoded with a Stereo High profile compliant decoder ; - all Stereo High profile streams with only progressive views can be decoded with a MultiView High profile compliant decoder ; - all drivers that support slice-level decoding could normally support MVC profiles when the DPB holds at most 16 frames.
2014-05-20 16:08:15 +00:00
/* Fills in compatible profiles for MVC decoding */
static void
fill_profiles_mvc(GstVaapiDecoderH264 *decoder, GstVaapiProfile profiles[16],
guint *n_profiles_ptr, guint dpb_size)
{
const gchar * const vendor_string =
gst_vaapi_display_get_vendor_string(GST_VAAPI_DECODER_DISPLAY(decoder));
gboolean add_high_profile = FALSE;
struct map {
const gchar *str;
guint str_len;
};
const struct map *m;
// Drivers that support slice level decoding
if (vendor_string && dpb_size <= 16) {
static const struct map drv_names[] = {
{ "Intel i965 driver", 17 },
{ NULL, 0 }
};
for (m = drv_names; m->str != NULL && !add_high_profile; m++) {
if (g_ascii_strncasecmp(vendor_string, m->str, m->str_len) == 0)
add_high_profile = TRUE;
}
}
if (add_high_profile)
fill_profiles(profiles, n_profiles_ptr, GST_VAAPI_PROFILE_H264_HIGH);
}
h264: fix activation order of picture and sequence parameters. Delay ensure_context() until we actually need a VA context for allocating new VA surfaces, and then GstVaapiPictures, but also when a real activation of a new picture parameter set occurs, thus also implying an activation of the related sequence parameter set. The most important thing was to drop the global pps and sps pointers since they may not have matched the currently activated picture parameter or sequence parameter sets at the specified decode point. Anoter positive side-effect is that this cleans up all occurrences of decode_current_picture() to only keep those useful in decode_picture(), before a new picture is allocated, or in decode_sequence_end() when an end-of-stream or end-of-sequence condition occurred.
2012-10-26 14:12:05 +00:00
static GstVaapiProfile
decoder: h264: add MVC profiles compatibility logic. Add safe fallbacks for MVC profiles: - all MultiView High profile streams with 2 views at most can be decoded with a Stereo High profile compliant decoder ; - all Stereo High profile streams with only progressive views can be decoded with a MultiView High profile compliant decoder ; - all drivers that support slice-level decoding could normally support MVC profiles when the DPB holds at most 16 frames.
2014-05-20 16:08:15 +00:00
get_profile(GstVaapiDecoderH264 *decoder, GstH264SPS *sps, guint dpb_size)
h264: fix activation order of picture and sequence parameters. Delay ensure_context() until we actually need a VA context for allocating new VA surfaces, and then GstVaapiPictures, but also when a real activation of a new picture parameter set occurs, thus also implying an activation of the related sequence parameter set. The most important thing was to drop the global pps and sps pointers since they may not have matched the currently activated picture parameter or sequence parameter sets at the specified decode point. Anoter positive side-effect is that this cleans up all occurrences of decode_current_picture() to only keep those useful in decode_picture(), before a new picture is allocated, or in decode_sequence_end() when an end-of-stream or end-of-sequence condition occurred.
2012-10-26 14:12:05 +00:00
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
h264: fix activation order of picture and sequence parameters. Delay ensure_context() until we actually need a VA context for allocating new VA surfaces, and then GstVaapiPictures, but also when a real activation of a new picture parameter set occurs, thus also implying an activation of the related sequence parameter set. The most important thing was to drop the global pps and sps pointers since they may not have matched the currently activated picture parameter or sequence parameter sets at the specified decode point. Anoter positive side-effect is that this cleans up all occurrences of decode_current_picture() to only keep those useful in decode_picture(), before a new picture is allocated, or in decode_sequence_end() when an end-of-stream or end-of-sequence condition occurred.
2012-10-26 14:12:05 +00:00
GstVaapiDisplay * const display = GST_VAAPI_DECODER_DISPLAY(decoder);
decoder: h264: add support for constrained baseline profile. Recognize streams marked as conforming to the "Constrained Baseline Profile". If VA driver supports that as is, fine. Otherwise, fallback to baseline, main or high profile. Constrained Baseline Profile conveys coding tools that are common to baseline profile and main profile. https://bugzilla.gnome.org/show_bug.cgi?id=719947 [Added fallbacks to main and high profiles] Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2013-12-06 07:08:26 +00:00
GstVaapiProfile profile, profiles[4];
h264: fix activation order of picture and sequence parameters. Delay ensure_context() until we actually need a VA context for allocating new VA surfaces, and then GstVaapiPictures, but also when a real activation of a new picture parameter set occurs, thus also implying an activation of the related sequence parameter set. The most important thing was to drop the global pps and sps pointers since they may not have matched the currently activated picture parameter or sequence parameter sets at the specified decode point. Anoter positive side-effect is that this cleans up all occurrences of decode_current_picture() to only keep those useful in decode_picture(), before a new picture is allocated, or in decode_sequence_end() when an end-of-stream or end-of-sequence condition occurred.
2012-10-26 14:12:05 +00:00
guint i, n_profiles = 0;
utils: add new H.264 helper functions. * Profiles: - gst_vaapi_utils_h264_get_profile(): Returns GstVaapiProfile from H.264 profile_idc value - gst_vaapi_utils_h264_get_profile_idc(): Returns H.264 profile_idc value from GstVaapiProfile * Chroma formats: - gst_vaapi_utils_h264_get_chroma_type(): Returns GstVaapiChromaType from H.264 chroma_format_idc value - gst_vaapi_utils_h264_get_chroma_format_idc(): Returns H.264 chroma_format_idc value from GstVaapiChromaType
2013-12-06 16:21:52 +00:00
profile = gst_vaapi_utils_h264_get_profile(sps->profile_idc);
h264: fix activation order of picture and sequence parameters. Delay ensure_context() until we actually need a VA context for allocating new VA surfaces, and then GstVaapiPictures, but also when a real activation of a new picture parameter set occurs, thus also implying an activation of the related sequence parameter set. The most important thing was to drop the global pps and sps pointers since they may not have matched the currently activated picture parameter or sequence parameter sets at the specified decode point. Anoter positive side-effect is that this cleans up all occurrences of decode_current_picture() to only keep those useful in decode_picture(), before a new picture is allocated, or in decode_sequence_end() when an end-of-stream or end-of-sequence condition occurred.
2012-10-26 14:12:05 +00:00
if (!profile)
return GST_VAAPI_PROFILE_UNKNOWN;
decoder: h264: fix decoding of BA3_SVA_C.264. Conformance test Base_Ext_Main_profiles/BA3_SVA_C.264 complys with extended profile specifications. However, the SPS header has the constraint_set1_flag syntax element set to 1. This means that, if a Main profile compliant decoder is available, then it should be able to decode this stream. This changes makes it possible to fall-back from Extended profile to Main profile if constraint_set1_flag is set to 1. https://bugzilla.gnome.org/show_bug.cgi?id=720190
2013-12-10 13:20:55 +00:00
fill_profiles(profiles, &n_profiles, profile);
h264: fix activation order of picture and sequence parameters. Delay ensure_context() until we actually need a VA context for allocating new VA surfaces, and then GstVaapiPictures, but also when a real activation of a new picture parameter set occurs, thus also implying an activation of the related sequence parameter set. The most important thing was to drop the global pps and sps pointers since they may not have matched the currently activated picture parameter or sequence parameter sets at the specified decode point. Anoter positive side-effect is that this cleans up all occurrences of decode_current_picture() to only keep those useful in decode_picture(), before a new picture is allocated, or in decode_sequence_end() when an end-of-stream or end-of-sequence condition occurred.
2012-10-26 14:12:05 +00:00
switch (profile) {
decoder: h264: fix decoding of BA3_SVA_C.264. Conformance test Base_Ext_Main_profiles/BA3_SVA_C.264 complys with extended profile specifications. However, the SPS header has the constraint_set1_flag syntax element set to 1. This means that, if a Main profile compliant decoder is available, then it should be able to decode this stream. This changes makes it possible to fall-back from Extended profile to Main profile if constraint_set1_flag is set to 1. https://bugzilla.gnome.org/show_bug.cgi?id=720190
2013-12-10 13:20:55 +00:00
case GST_VAAPI_PROFILE_H264_BASELINE:
if (sps->constraint_set1_flag) { // A.2.2 (main profile)
fill_profiles(profiles, &n_profiles,
GST_VAAPI_PROFILE_H264_CONSTRAINED_BASELINE);
fill_profiles(profiles, &n_profiles,
GST_VAAPI_PROFILE_H264_MAIN);
}
break;
case GST_VAAPI_PROFILE_H264_EXTENDED:
if (sps->constraint_set1_flag) { // A.2.2 (main profile)
fill_profiles(profiles, &n_profiles,
GST_VAAPI_PROFILE_H264_MAIN);
}
h264: fix activation order of picture and sequence parameters. Delay ensure_context() until we actually need a VA context for allocating new VA surfaces, and then GstVaapiPictures, but also when a real activation of a new picture parameter set occurs, thus also implying an activation of the related sequence parameter set. The most important thing was to drop the global pps and sps pointers since they may not have matched the currently activated picture parameter or sequence parameter sets at the specified decode point. Anoter positive side-effect is that this cleans up all occurrences of decode_current_picture() to only keep those useful in decode_picture(), before a new picture is allocated, or in decode_sequence_end() when an end-of-stream or end-of-sequence condition occurred.
2012-10-26 14:12:05 +00:00
break;
decoder: h264: add MVC profiles compatibility logic. Add safe fallbacks for MVC profiles: - all MultiView High profile streams with 2 views at most can be decoded with a Stereo High profile compliant decoder ; - all Stereo High profile streams with only progressive views can be decoded with a MultiView High profile compliant decoder ; - all drivers that support slice-level decoding could normally support MVC profiles when the DPB holds at most 16 frames.
2014-05-20 16:08:15 +00:00
case GST_VAAPI_PROFILE_H264_MULTIVIEW_HIGH:
if (priv->max_views == 2) {
fill_profiles(profiles, &n_profiles,
GST_VAAPI_PROFILE_H264_STEREO_HIGH);
}
fill_profiles_mvc(decoder, profiles, &n_profiles, dpb_size);
break;
case GST_VAAPI_PROFILE_H264_STEREO_HIGH:
if (sps->frame_mbs_only_flag) {
fill_profiles(profiles, &n_profiles,
GST_VAAPI_PROFILE_H264_MULTIVIEW_HIGH);
}
fill_profiles_mvc(decoder, profiles, &n_profiles, dpb_size);
break;
h264: fix activation order of picture and sequence parameters. Delay ensure_context() until we actually need a VA context for allocating new VA surfaces, and then GstVaapiPictures, but also when a real activation of a new picture parameter set occurs, thus also implying an activation of the related sequence parameter set. The most important thing was to drop the global pps and sps pointers since they may not have matched the currently activated picture parameter or sequence parameter sets at the specified decode point. Anoter positive side-effect is that this cleans up all occurrences of decode_current_picture() to only keep those useful in decode_picture(), before a new picture is allocated, or in decode_sequence_end() when an end-of-stream or end-of-sequence condition occurred.
2012-10-26 14:12:05 +00:00
default:
break;
}
/* If the preferred profile (profiles[0]) matches one that we already
found, then just return it now instead of searching for it again */
if (profiles[0] == priv->profile)
return priv->profile;
for (i = 0; i < n_profiles; i++) {
if (gst_vaapi_display_has_decoder(display, profiles[i], priv->entrypoint))
return profiles[i];
}
return GST_VAAPI_PROFILE_UNKNOWN;
}
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
static GstVaapiDecoderStatus
ensure_context(GstVaapiDecoderH264 *decoder, GstH264SPS *sps)
{
h264: fix activation order of picture and sequence parameters. Delay ensure_context() until we actually need a VA context for allocating new VA surfaces, and then GstVaapiPictures, but also when a real activation of a new picture parameter set occurs, thus also implying an activation of the related sequence parameter set. The most important thing was to drop the global pps and sps pointers since they may not have matched the currently activated picture parameter or sequence parameter sets at the specified decode point. Anoter positive side-effect is that this cleans up all occurrences of decode_current_picture() to only keep those useful in decode_picture(), before a new picture is allocated, or in decode_sequence_end() when an end-of-stream or end-of-sequence condition occurred.
2012-10-26 14:12:05 +00:00
GstVaapiDecoder * const base_decoder = GST_VAAPI_DECODER_CAST(decoder);
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
h264: fix activation order of picture and sequence parameters. Delay ensure_context() until we actually need a VA context for allocating new VA surfaces, and then GstVaapiPictures, but also when a real activation of a new picture parameter set occurs, thus also implying an activation of the related sequence parameter set. The most important thing was to drop the global pps and sps pointers since they may not have matched the currently activated picture parameter or sequence parameter sets at the specified decode point. Anoter positive side-effect is that this cleans up all occurrences of decode_current_picture() to only keep those useful in decode_picture(), before a new picture is allocated, or in decode_sequence_end() when an end-of-stream or end-of-sequence condition occurred.
2012-10-26 14:12:05 +00:00
GstVaapiContextInfo info;
GstVaapiProfile profile;
GstVaapiChromaType chroma_type;
gboolean reset_context = FALSE;
decoder: h264: fix tracking of DPB size changes. Add support for MVC streams with multiple SPS and subset SPS headers emitted regularly, e.g. at around every I-frame. Track the maximum number of views in ensure_context() and really reset the DPB size to the expected value, always. i.e. even if it decreased. dpb_reset() only cares of ensuring the DPB allocation.
2014-06-28 05:25:35 +00:00
guint mb_width, mb_height, dpb_size, num_views;
num_views = get_num_views(sps);
if (priv->max_views < num_views) {
priv->max_views = num_views;
GST_DEBUG("maximum number of views changed to %u", num_views);
}
decoder: h264: dynamically allocate the DPB. Dynamically allocate the Decoded Picture Buffer (DPB) and add provisions for supporting the MVC allocation requirements.
2014-05-01 17:16:09 +00:00
dpb_size = get_max_dec_frame_buffering(sps);
if (priv->dpb_size < dpb_size) {
GST_DEBUG("DPB size increased");
reset_context = TRUE;
}
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
decoder: h264: add MVC profiles compatibility logic. Add safe fallbacks for MVC profiles: - all MultiView High profile streams with 2 views at most can be decoded with a Stereo High profile compliant decoder ; - all Stereo High profile streams with only progressive views can be decoded with a MultiView High profile compliant decoder ; - all drivers that support slice-level decoding could normally support MVC profiles when the DPB holds at most 16 frames.
2014-05-20 16:08:15 +00:00
profile = get_profile(decoder, sps, dpb_size);
h264: fix activation order of picture and sequence parameters. Delay ensure_context() until we actually need a VA context for allocating new VA surfaces, and then GstVaapiPictures, but also when a real activation of a new picture parameter set occurs, thus also implying an activation of the related sequence parameter set. The most important thing was to drop the global pps and sps pointers since they may not have matched the currently activated picture parameter or sequence parameter sets at the specified decode point. Anoter positive side-effect is that this cleans up all occurrences of decode_current_picture() to only keep those useful in decode_picture(), before a new picture is allocated, or in decode_sequence_end() when an end-of-stream or end-of-sequence condition occurred.
2012-10-26 14:12:05 +00:00
if (!profile) {
GST_ERROR("unsupported profile_idc %u", sps->profile_idc);
return GST_VAAPI_DECODER_STATUS_ERROR_UNSUPPORTED_PROFILE;
}
decoder: h264: fix detection of profile changes for MVC. If the VA driver exposes ad-hoc H.264 MVC profiles, then we have to be careful to detect profiles changes and not reset the underlying VA context erroneously. In MVC situations, we could indeed get a profile_idc change for every SPS that gets activated, alternatively (base-view -> non-base view -> base-view, etc.). An improved fix would be to characterize the exact profile to use once and for all when SPS NAL units are parsed. This would also allow for fallbacks to a base-view decoding only mode.
2014-06-04 17:03:18 +00:00
if (!priv->profile || (priv->profile != profile && priv->max_views == 1)) {
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
GST_DEBUG("profile changed");
reset_context = TRUE;
h264: fix activation order of picture and sequence parameters. Delay ensure_context() until we actually need a VA context for allocating new VA surfaces, and then GstVaapiPictures, but also when a real activation of a new picture parameter set occurs, thus also implying an activation of the related sequence parameter set. The most important thing was to drop the global pps and sps pointers since they may not have matched the currently activated picture parameter or sequence parameter sets at the specified decode point. Anoter positive side-effect is that this cleans up all occurrences of decode_current_picture() to only keep those useful in decode_picture(), before a new picture is allocated, or in decode_sequence_end() when an end-of-stream or end-of-sequence condition occurred.
2012-10-26 14:12:05 +00:00
priv->profile = profile;
}
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
utils: add new H.264 helper functions. * Profiles: - gst_vaapi_utils_h264_get_profile(): Returns GstVaapiProfile from H.264 profile_idc value - gst_vaapi_utils_h264_get_profile_idc(): Returns H.264 profile_idc value from GstVaapiProfile * Chroma formats: - gst_vaapi_utils_h264_get_chroma_type(): Returns GstVaapiChromaType from H.264 chroma_format_idc value - gst_vaapi_utils_h264_get_chroma_format_idc(): Returns H.264 chroma_format_idc value from GstVaapiChromaType
2013-12-06 16:21:52 +00:00
chroma_type = gst_vaapi_utils_h264_get_chroma_type(sps->chroma_format_idc);
decoder: h264: add support for grayscale encoded clips. Fix support for grayscale encoded video clips, and possibly others if the underlying driver supports the non-YUV 4:2:0 formats. i.e. defer the decision that a surface with the desired chroma format is not supported to the actual VA driver implementation. https://bugzilla.gnome.org/show_bug.cgi?id=728144
2014-04-25 12:23:11 +00:00
if (!chroma_type) {
h264: fix activation order of picture and sequence parameters. Delay ensure_context() until we actually need a VA context for allocating new VA surfaces, and then GstVaapiPictures, but also when a real activation of a new picture parameter set occurs, thus also implying an activation of the related sequence parameter set. The most important thing was to drop the global pps and sps pointers since they may not have matched the currently activated picture parameter or sequence parameter sets at the specified decode point. Anoter positive side-effect is that this cleans up all occurrences of decode_current_picture() to only keep those useful in decode_picture(), before a new picture is allocated, or in decode_sequence_end() when an end-of-stream or end-of-sequence condition occurred.
2012-10-26 14:12:05 +00:00
GST_ERROR("unsupported chroma_format_idc %u", sps->chroma_format_idc);
return GST_VAAPI_DECODER_STATUS_ERROR_UNSUPPORTED_CHROMA_FORMAT;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
}
h264: fix activation order of picture and sequence parameters. Delay ensure_context() until we actually need a VA context for allocating new VA surfaces, and then GstVaapiPictures, but also when a real activation of a new picture parameter set occurs, thus also implying an activation of the related sequence parameter set. The most important thing was to drop the global pps and sps pointers since they may not have matched the currently activated picture parameter or sequence parameter sets at the specified decode point. Anoter positive side-effect is that this cleans up all occurrences of decode_current_picture() to only keep those useful in decode_picture(), before a new picture is allocated, or in decode_sequence_end() when an end-of-stream or end-of-sequence condition occurred.
2012-10-26 14:12:05 +00:00
if (priv->chroma_type != chroma_type) {
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
GST_DEBUG("chroma format changed");
h264: fix activation order of picture and sequence parameters. Delay ensure_context() until we actually need a VA context for allocating new VA surfaces, and then GstVaapiPictures, but also when a real activation of a new picture parameter set occurs, thus also implying an activation of the related sequence parameter set. The most important thing was to drop the global pps and sps pointers since they may not have matched the currently activated picture parameter or sequence parameter sets at the specified decode point. Anoter positive side-effect is that this cleans up all occurrences of decode_current_picture() to only keep those useful in decode_picture(), before a new picture is allocated, or in decode_sequence_end() when an end-of-stream or end-of-sequence condition occurred.
2012-10-26 14:12:05 +00:00
reset_context = TRUE;
priv->chroma_type = chroma_type;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
}
h264: fix picture size in macroblocks. The picture size signalled by sps->{width,height} is the actual size with cropping applied, not the original size derived from pic_width_in_mbs_minus1 and pic_height_in_map_units_minus1. VA driver expects that original size, uncropped. There is another issue pending: frame cropping information needs to be taken care of.
2012-11-16 17:00:10 +00:00
mb_width = sps->pic_width_in_mbs_minus1 + 1;
mb_height = (sps->pic_height_in_map_units_minus1 + 1) <<
!sps->frame_mbs_only_flag;
if (priv->mb_width != mb_width || priv->mb_height != mb_height) {
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
GST_DEBUG("size changed");
h264: fix picture size in macroblocks. The picture size signalled by sps->{width,height} is the actual size with cropping applied, not the original size derived from pic_width_in_mbs_minus1 and pic_height_in_map_units_minus1. VA driver expects that original size, uncropped. There is another issue pending: frame cropping information needs to be taken care of.
2012-11-16 17:00:10 +00:00
reset_context = TRUE;
priv->mb_width = mb_width;
priv->mb_height = mb_height;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
}
h264: add initial support for interlaced streams. Decoded frames are only output when they are complete, i.e. when both fields are decoded. This also means that the "interlaced" caps is not propagated to vaapipostproc or vaapisink elements. Another limitation is that interlaced bitstreams with MMCO are unlikely to work.
2012-11-13 16:14:39 +00:00
priv->progressive_sequence = sps->frame_mbs_only_flag;
gst_vaapi_decoder_set_interlaced(base_decoder, !priv->progressive_sequence);
h264: fix activation order of picture and sequence parameters. Delay ensure_context() until we actually need a VA context for allocating new VA surfaces, and then GstVaapiPictures, but also when a real activation of a new picture parameter set occurs, thus also implying an activation of the related sequence parameter set. The most important thing was to drop the global pps and sps pointers since they may not have matched the currently activated picture parameter or sequence parameter sets at the specified decode point. Anoter positive side-effect is that this cleans up all occurrences of decode_current_picture() to only keep those useful in decode_picture(), before a new picture is allocated, or in decode_sequence_end() when an end-of-stream or end-of-sequence condition occurred.
2012-10-26 14:12:05 +00:00
gst_vaapi_decoder_set_pixel_aspect_ratio(
base_decoder,
sps->vui_parameters.par_n,
sps->vui_parameters.par_d
);
decoder: cope with new GstVaapiContextInfo based API. Update decoders to report the maximum number of reference frames to use.
2012-09-10 16:26:51 +00:00
h264: fix activation order of picture and sequence parameters. Delay ensure_context() until we actually need a VA context for allocating new VA surfaces, and then GstVaapiPictures, but also when a real activation of a new picture parameter set occurs, thus also implying an activation of the related sequence parameter set. The most important thing was to drop the global pps and sps pointers since they may not have matched the currently activated picture parameter or sequence parameter sets at the specified decode point. Anoter positive side-effect is that this cleans up all occurrences of decode_current_picture() to only keep those useful in decode_picture(), before a new picture is allocated, or in decode_sequence_end() when an end-of-stream or end-of-sequence condition occurred.
2012-10-26 14:12:05 +00:00
if (!reset_context && priv->has_context)
return GST_VAAPI_DECODER_STATUS_SUCCESS;
decoder: cope with new GstVaapiContextInfo based API. Update decoders to report the maximum number of reference frames to use.
2012-09-10 16:26:51 +00:00
h264: fix picture size in macroblocks. The picture size signalled by sps->{width,height} is the actual size with cropping applied, not the original size derived from pic_width_in_mbs_minus1 and pic_height_in_map_units_minus1. VA driver expects that original size, uncropped. There is another issue pending: frame cropping information needs to be taken care of.
2012-11-16 17:00:10 +00:00
/* XXX: fix surface size when cropping is implemented */
h264: fix activation order of picture and sequence parameters. Delay ensure_context() until we actually need a VA context for allocating new VA surfaces, and then GstVaapiPictures, but also when a real activation of a new picture parameter set occurs, thus also implying an activation of the related sequence parameter set. The most important thing was to drop the global pps and sps pointers since they may not have matched the currently activated picture parameter or sequence parameter sets at the specified decode point. Anoter positive side-effect is that this cleans up all occurrences of decode_current_picture() to only keep those useful in decode_picture(), before a new picture is allocated, or in decode_sequence_end() when an end-of-stream or end-of-sequence condition occurred.
2012-10-26 14:12:05 +00:00
info.profile = priv->profile;
info.entrypoint = priv->entrypoint;
decoder: h264: properly support grayscale formats. Request the correct chroma format for decoding grayscale streams. i.e. make lookups of the VA chroma format more generic, thus possibly supporting more formats in the future. This means that, if a VA driver doesn't support grayscale formats, it is now going to fail. We cannot safely assume that maybe grayscale was implemented on top of some YUV 4:2:0 with the chroma components all set to 0x80.
2014-05-10 04:23:29 +00:00
info.chroma_type = priv->chroma_type;
h264: fix picture size in macroblocks. The picture size signalled by sps->{width,height} is the actual size with cropping applied, not the original size derived from pic_width_in_mbs_minus1 and pic_height_in_map_units_minus1. VA driver expects that original size, uncropped. There is another issue pending: frame cropping information needs to be taken care of.
2012-11-16 17:00:10 +00:00
info.width = sps->width;
info.height = sps->height;
decoder: h264: dynamically allocate the DPB. Dynamically allocate the Decoded Picture Buffer (DPB) and add provisions for supporting the MVC allocation requirements.
2014-05-01 17:16:09 +00:00
info.ref_frames = dpb_size;
h264: handle Decoded Picture Buffer (DPB).
2012-01-24 14:38:14 +00:00
h264: fix activation order of picture and sequence parameters. Delay ensure_context() until we actually need a VA context for allocating new VA surfaces, and then GstVaapiPictures, but also when a real activation of a new picture parameter set occurs, thus also implying an activation of the related sequence parameter set. The most important thing was to drop the global pps and sps pointers since they may not have matched the currently activated picture parameter or sequence parameter sets at the specified decode point. Anoter positive side-effect is that this cleans up all occurrences of decode_current_picture() to only keep those useful in decode_picture(), before a new picture is allocated, or in decode_sequence_end() when an end-of-stream or end-of-sequence condition occurred.
2012-10-26 14:12:05 +00:00
if (!gst_vaapi_decoder_ensure_context(GST_VAAPI_DECODER(decoder), &info))
return GST_VAAPI_DECODER_STATUS_ERROR_UNKNOWN;
priv->has_context = TRUE;
/* Reset DPB */
decoder: h264: dynamically allocate the DPB. Dynamically allocate the Decoded Picture Buffer (DPB) and add provisions for supporting the MVC allocation requirements.
2014-05-01 17:16:09 +00:00
if (!dpb_reset(decoder, dpb_size))
return GST_VAAPI_DECODER_STATUS_ERROR_ALLOCATION_FAILED;
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
/* Reset MVC data */
if (!mvc_reset(decoder))
return GST_VAAPI_DECODER_STATUS_ERROR_ALLOCATION_FAILED;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
return GST_VAAPI_DECODER_STATUS_SUCCESS;
}
h264: fix scaling list generation. ... aka fix regression from efaab79. In particular, ScalingList8x8[] array was partially copied to the VAIQMatrixBufferH264. While we are at it, also improve bounds checking and avoid copying 8x8 scaling lists if transform_8x8_mode_flag is set to 0.
2012-10-26 11:17:43 +00:00
static void
decoder: h264: fix activation of picture and sequence parameters. At the time the first VCL NAL unit of a primary coded picture is found, and if that NAL unit was parsed to be an SPS or PPS, then the entries in the parser may have been overriden. This means that, when the picture is to be decoded, slice_hdr->pps could point to an invalid (the next) PPS entry. So, one way to solve this problem is to not use the parser PPS and SPS info but rather maintain our own activation chain in the decoder. https://bugzilla.gnome.org/show_bug.cgi?id=724519 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 17:11:03 +00:00
fill_iq_matrix_4x4(VAIQMatrixBufferH264 *iq_matrix, const GstH264PPS *pps,
const GstH264SPS *sps)
h264: fix scaling list generation. ... aka fix regression from efaab79. In particular, ScalingList8x8[] array was partially copied to the VAIQMatrixBufferH264. While we are at it, also improve bounds checking and avoid copying 8x8 scaling lists if transform_8x8_mode_flag is set to 0.
2012-10-26 11:17:43 +00:00
{
decoder: h264: fix decoding of scaling lists. The GStreamer codecparser layer now parses the scaling lists in zigzag scan order, as expected, so that to match the original bitstream layout and specification. However, further convert the scaling lists into raster scan order to fit the existing practice in most VA drivers. https://bugzilla.gnome.org/show_bug.cgi?id=706406
2013-12-09 11:46:45 +00:00
guint i;
h264: fix scaling list generation. ... aka fix regression from efaab79. In particular, ScalingList8x8[] array was partially copied to the VAIQMatrixBufferH264. While we are at it, also improve bounds checking and avoid copying 8x8 scaling lists if transform_8x8_mode_flag is set to 0.
2012-10-26 11:17:43 +00:00
/* There are always 6 4x4 scaling lists */
g_assert(G_N_ELEMENTS(iq_matrix->ScalingList4x4) == 6);
g_assert(G_N_ELEMENTS(iq_matrix->ScalingList4x4[0]) == 16);
decoder: h264: fix decoding of scaling lists. The GStreamer codecparser layer now parses the scaling lists in zigzag scan order, as expected, so that to match the original bitstream layout and specification. However, further convert the scaling lists into raster scan order to fit the existing practice in most VA drivers. https://bugzilla.gnome.org/show_bug.cgi?id=706406
2013-12-09 11:46:45 +00:00
for (i = 0; i < G_N_ELEMENTS(iq_matrix->ScalingList4x4); i++)
decoder: h264: cope with new gst_h264_quant_matrix_*() interfaces. New gst_h264_quant_matrix_*_get_raster_from_zigzag() were renamed from gst_h264_video_quant_matrix_*_get_raster_from_zigzag().
2014-06-13 09:36:56 +00:00
gst_h264_quant_matrix_4x4_get_raster_from_zigzag(
decoder: h264: fix decoding of scaling lists. The GStreamer codecparser layer now parses the scaling lists in zigzag scan order, as expected, so that to match the original bitstream layout and specification. However, further convert the scaling lists into raster scan order to fit the existing practice in most VA drivers. https://bugzilla.gnome.org/show_bug.cgi?id=706406
2013-12-09 11:46:45 +00:00
iq_matrix->ScalingList4x4[i], pps->scaling_lists_4x4[i]);
h264: fix scaling list generation. ... aka fix regression from efaab79. In particular, ScalingList8x8[] array was partially copied to the VAIQMatrixBufferH264. While we are at it, also improve bounds checking and avoid copying 8x8 scaling lists if transform_8x8_mode_flag is set to 0.
2012-10-26 11:17:43 +00:00
}
static void
decoder: h264: fix activation of picture and sequence parameters. At the time the first VCL NAL unit of a primary coded picture is found, and if that NAL unit was parsed to be an SPS or PPS, then the entries in the parser may have been overriden. This means that, when the picture is to be decoded, slice_hdr->pps could point to an invalid (the next) PPS entry. So, one way to solve this problem is to not use the parser PPS and SPS info but rather maintain our own activation chain in the decoder. https://bugzilla.gnome.org/show_bug.cgi?id=724519 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 17:11:03 +00:00
fill_iq_matrix_8x8(VAIQMatrixBufferH264 *iq_matrix, const GstH264PPS *pps,
const GstH264SPS *sps)
h264: fix scaling list generation. ... aka fix regression from efaab79. In particular, ScalingList8x8[] array was partially copied to the VAIQMatrixBufferH264. While we are at it, also improve bounds checking and avoid copying 8x8 scaling lists if transform_8x8_mode_flag is set to 0.
2012-10-26 11:17:43 +00:00
{
decoder: h264: fix decoding of scaling lists. The GStreamer codecparser layer now parses the scaling lists in zigzag scan order, as expected, so that to match the original bitstream layout and specification. However, further convert the scaling lists into raster scan order to fit the existing practice in most VA drivers. https://bugzilla.gnome.org/show_bug.cgi?id=706406
2013-12-09 11:46:45 +00:00
guint i, n;
h264: fix scaling list generation. ... aka fix regression from efaab79. In particular, ScalingList8x8[] array was partially copied to the VAIQMatrixBufferH264. While we are at it, also improve bounds checking and avoid copying 8x8 scaling lists if transform_8x8_mode_flag is set to 0.
2012-10-26 11:17:43 +00:00
/* If chroma_format_idc != 3, there are up to 2 8x8 scaling lists */
if (!pps->transform_8x8_mode_flag)
return;
g_assert(G_N_ELEMENTS(iq_matrix->ScalingList8x8) >= 2);
g_assert(G_N_ELEMENTS(iq_matrix->ScalingList8x8[0]) == 64);
decoder: h264: fix decoding of scaling lists. The GStreamer codecparser layer now parses the scaling lists in zigzag scan order, as expected, so that to match the original bitstream layout and specification. However, further convert the scaling lists into raster scan order to fit the existing practice in most VA drivers. https://bugzilla.gnome.org/show_bug.cgi?id=706406
2013-12-09 11:46:45 +00:00
n = (sps->chroma_format_idc != 3) ? 2 : 6;
for (i = 0; i < n; i++) {
decoder: h264: cope with new gst_h264_quant_matrix_*() interfaces. New gst_h264_quant_matrix_*_get_raster_from_zigzag() were renamed from gst_h264_video_quant_matrix_*_get_raster_from_zigzag().
2014-06-13 09:36:56 +00:00
gst_h264_quant_matrix_8x8_get_raster_from_zigzag(
decoder: h264: fix decoding of scaling lists. The GStreamer codecparser layer now parses the scaling lists in zigzag scan order, as expected, so that to match the original bitstream layout and specification. However, further convert the scaling lists into raster scan order to fit the existing practice in most VA drivers. https://bugzilla.gnome.org/show_bug.cgi?id=706406
2013-12-09 11:46:45 +00:00
iq_matrix->ScalingList8x8[i], pps->scaling_lists_8x8[i]);
h264: fix scaling list generation. ... aka fix regression from efaab79. In particular, ScalingList8x8[] array was partially copied to the VAIQMatrixBufferH264. While we are at it, also improve bounds checking and avoid copying 8x8 scaling lists if transform_8x8_mode_flag is set to 0.
2012-10-26 11:17:43 +00:00
}
}
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
static GstVaapiDecoderStatus
h264: optimize handling of scaling lists. Don't copy scaling lists twice to an intermediate state. Rather, directly use the scaling lists from GstH264PPS since they would match those provided by SPS header, if necessary. i.e. if PPS-specific scaling lists are not available in the bitstream.
2012-10-23 12:50:14 +00:00
ensure_quant_matrix(GstVaapiDecoderH264 *decoder, GstVaapiPictureH264 *picture)
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
{
h264: optimize handling of scaling lists. Don't copy scaling lists twice to an intermediate state. Rather, directly use the scaling lists from GstH264PPS since they would match those provided by SPS header, if necessary. i.e. if PPS-specific scaling lists are not available in the bitstream.
2012-10-23 12:50:14 +00:00
GstVaapiPicture * const base_picture = &picture->base;
decoder: h264: fix activation of picture and sequence parameters. At the time the first VCL NAL unit of a primary coded picture is found, and if that NAL unit was parsed to be an SPS or PPS, then the entries in the parser may have been overriden. This means that, when the picture is to be decoded, slice_hdr->pps could point to an invalid (the next) PPS entry. So, one way to solve this problem is to not use the parser PPS and SPS info but rather maintain our own activation chain in the decoder. https://bugzilla.gnome.org/show_bug.cgi?id=724519 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 17:11:03 +00:00
GstH264PPS * const pps = get_pps(decoder);
GstH264SPS * const sps = get_sps(decoder);
h264: optimize handling of scaling lists. Don't copy scaling lists twice to an intermediate state. Rather, directly use the scaling lists from GstH264PPS since they would match those provided by SPS header, if necessary. i.e. if PPS-specific scaling lists are not available in the bitstream.
2012-10-23 12:50:14 +00:00
VAIQMatrixBufferH264 *iq_matrix;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
h264: optimize handling of scaling lists. Don't copy scaling lists twice to an intermediate state. Rather, directly use the scaling lists from GstH264PPS since they would match those provided by SPS header, if necessary. i.e. if PPS-specific scaling lists are not available in the bitstream.
2012-10-23 12:50:14 +00:00
base_picture->iq_matrix = GST_VAAPI_IQ_MATRIX_NEW(H264, decoder);
if (!base_picture->iq_matrix) {
GST_ERROR("failed to allocate IQ matrix");
return GST_VAAPI_DECODER_STATUS_ERROR_ALLOCATION_FAILED;
}
iq_matrix = base_picture->iq_matrix->param;
/* XXX: we can only support 4:2:0 or 4:2:2 since ScalingLists8x8[]
is not large enough to hold lists for 4:4:4 */
if (sps->chroma_format_idc == 3)
return GST_VAAPI_DECODER_STATUS_ERROR_UNSUPPORTED_CHROMA_FORMAT;
decoder: h264: fix activation of picture and sequence parameters. At the time the first VCL NAL unit of a primary coded picture is found, and if that NAL unit was parsed to be an SPS or PPS, then the entries in the parser may have been overriden. This means that, when the picture is to be decoded, slice_hdr->pps could point to an invalid (the next) PPS entry. So, one way to solve this problem is to not use the parser PPS and SPS info but rather maintain our own activation chain in the decoder. https://bugzilla.gnome.org/show_bug.cgi?id=724519 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 17:11:03 +00:00
fill_iq_matrix_4x4(iq_matrix, pps, sps);
fill_iq_matrix_8x8(iq_matrix, pps, sps);
h264: optimize handling of scaling lists. Don't copy scaling lists twice to an intermediate state. Rather, directly use the scaling lists from GstH264PPS since they would match those provided by SPS header, if necessary. i.e. if PPS-specific scaling lists are not available in the bitstream.
2012-10-23 12:50:14 +00:00
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
return GST_VAAPI_DECODER_STATUS_SUCCESS;
}
h264: improve robustness when packets are missing. Improve robustness when some expected packets where not received yet or that were not correctly decoded. For example, don't try to decode a picture if there was no valid frame headers parsed so far. https://bugs.freedesktop.org/show_bug.cgi?id=57902
2013-12-10 15:14:27 +00:00
static inline gboolean
is_valid_state(guint state, guint ref_state)
{
return (state & ref_state) == ref_state;
}
h264: fix activation order of picture and sequence parameters. Delay ensure_context() until we actually need a VA context for allocating new VA surfaces, and then GstVaapiPictures, but also when a real activation of a new picture parameter set occurs, thus also implying an activation of the related sequence parameter set. The most important thing was to drop the global pps and sps pointers since they may not have matched the currently activated picture parameter or sequence parameter sets at the specified decode point. Anoter positive side-effect is that this cleans up all occurrences of decode_current_picture() to only keep those useful in decode_picture(), before a new picture is allocated, or in decode_sequence_end() when an end-of-stream or end-of-sequence condition occurred.
2012-10-26 14:12:05 +00:00
static GstVaapiDecoderStatus
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
decode_current_picture(GstVaapiDecoderH264 *decoder)
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
GstVaapiPictureH264 * const picture = priv->current_picture;
h264: improve robustness when packets are missing. Improve robustness when some expected packets where not received yet or that were not correctly decoded. For example, don't try to decode a picture if there was no valid frame headers parsed so far. https://bugs.freedesktop.org/show_bug.cgi?id=57902
2013-12-10 15:14:27 +00:00
if (!is_valid_state(priv->decoder_state, GST_H264_VIDEO_STATE_VALID_PICTURE))
goto drop_frame;
priv->decoder_state = 0;
decoder: h264: fix detection of top-field-first (TFF) flag. Use the SEI pic_timing() message to track the pic_struct variable when present, or infer it from the regular slice header flags field_pic_flag and bottom_field_flag. This fixes temporal sequence ordering when the output pictures are to be displayed. https://bugzilla.gnome.org/show_bug.cgi?id=739291
2014-11-13 14:00:21 +00:00
priv->pic_structure = GST_H264_SEI_PIC_STRUCT_FRAME;
h264: improve robustness when packets are missing. Improve robustness when some expected packets where not received yet or that were not correctly decoded. For example, don't try to decode a picture if there was no valid frame headers parsed so far. https://bugs.freedesktop.org/show_bug.cgi?id=57902
2013-12-10 15:14:27 +00:00
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
if (!picture)
h264: fix activation order of picture and sequence parameters. Delay ensure_context() until we actually need a VA context for allocating new VA surfaces, and then GstVaapiPictures, but also when a real activation of a new picture parameter set occurs, thus also implying an activation of the related sequence parameter set. The most important thing was to drop the global pps and sps pointers since they may not have matched the currently activated picture parameter or sequence parameter sets at the specified decode point. Anoter positive side-effect is that this cleans up all occurrences of decode_current_picture() to only keep those useful in decode_picture(), before a new picture is allocated, or in decode_sequence_end() when an end-of-stream or end-of-sequence condition occurred.
2012-10-26 14:12:05 +00:00
return GST_VAAPI_DECODER_STATUS_SUCCESS;
decoder: h264: decode current picture earlier. Slightly optimize decoding process by submitting the current VA surface for decoding earlier to the hardware, and perform the reference picture marking process and DPB update process afterwards. This is a minor optimization to let the video decode engine kick in work earlier, thus improving parallel resources utilization.
2014-06-30 14:12:52 +00:00
if (!gst_vaapi_picture_decode(GST_VAAPI_PICTURE_CAST(picture)))
goto error;
h264: simplify reference picture marking process. ... to build the short_ref[] and long_ref[] lists from the DPB, instead of maintaining them separately. This avoids refs/unrefs while making it possible to generate the list based on the actual picture structure. This also ensures that the list of generated ReferenceFrames[] actually matches what reference frames are available in the DPB. i.e. short_ref[] and long_ref[] entries are implied from the DPB, so there is no risk of having "dangling" references.
2012-10-31 13:24:09 +00:00
if (!exec_ref_pic_marking(decoder, picture))
goto error;
if (!dpb_add(decoder, picture))
h264: fix activation order of picture and sequence parameters. Delay ensure_context() until we actually need a VA context for allocating new VA surfaces, and then GstVaapiPictures, but also when a real activation of a new picture parameter set occurs, thus also implying an activation of the related sequence parameter set. The most important thing was to drop the global pps and sps pointers since they may not have matched the currently activated picture parameter or sequence parameter sets at the specified decode point. Anoter positive side-effect is that this cleans up all occurrences of decode_current_picture() to only keep those useful in decode_picture(), before a new picture is allocated, or in decode_sequence_end() when an end-of-stream or end-of-sequence condition occurred.
2012-10-26 14:12:05 +00:00
goto error;
decoder: h264: add support for MVC interlaced streams. Fix support for MVC Stereo High profile streams with interlaced frames. Also improve the detection logic of the first field.
2014-05-21 15:57:00 +00:00
gst_vaapi_picture_replace(&priv->current_picture, NULL);
h264: fix activation order of picture and sequence parameters. Delay ensure_context() until we actually need a VA context for allocating new VA surfaces, and then GstVaapiPictures, but also when a real activation of a new picture parameter set occurs, thus also implying an activation of the related sequence parameter set. The most important thing was to drop the global pps and sps pointers since they may not have matched the currently activated picture parameter or sequence parameter sets at the specified decode point. Anoter positive side-effect is that this cleans up all occurrences of decode_current_picture() to only keep those useful in decode_picture(), before a new picture is allocated, or in decode_sequence_end() when an end-of-stream or end-of-sequence condition occurred.
2012-10-26 14:12:05 +00:00
return GST_VAAPI_DECODER_STATUS_SUCCESS;
error:
h264: add initial support for interlaced streams. Decoded frames are only output when they are complete, i.e. when both fields are decoded. This also means that the "interlaced" caps is not propagated to vaapipostproc or vaapisink elements. Another limitation is that interlaced bitstreams with MMCO are unlikely to work.
2012-11-13 16:14:39 +00:00
/* XXX: fix for cases where first field failed to be decoded */
h264: fix activation order of picture and sequence parameters. Delay ensure_context() until we actually need a VA context for allocating new VA surfaces, and then GstVaapiPictures, but also when a real activation of a new picture parameter set occurs, thus also implying an activation of the related sequence parameter set. The most important thing was to drop the global pps and sps pointers since they may not have matched the currently activated picture parameter or sequence parameter sets at the specified decode point. Anoter positive side-effect is that this cleans up all occurrences of decode_current_picture() to only keep those useful in decode_picture(), before a new picture is allocated, or in decode_sequence_end() when an end-of-stream or end-of-sequence condition occurred.
2012-10-26 14:12:05 +00:00
gst_vaapi_picture_replace(&priv->current_picture, NULL);
return GST_VAAPI_DECODER_STATUS_ERROR_UNKNOWN;
h264: improve robustness when packets are missing. Improve robustness when some expected packets where not received yet or that were not correctly decoded. For example, don't try to decode a picture if there was no valid frame headers parsed so far. https://bugs.freedesktop.org/show_bug.cgi?id=57902
2013-12-10 15:14:27 +00:00
drop_frame:
priv->decoder_state = 0;
decoder: h264: fix detection of top-field-first (TFF) flag. Use the SEI pic_timing() message to track the pic_struct variable when present, or infer it from the regular slice header flags field_pic_flag and bottom_field_flag. This fixes temporal sequence ordering when the output pictures are to be displayed. https://bugzilla.gnome.org/show_bug.cgi?id=739291
2014-11-13 14:00:21 +00:00
priv->pic_structure = GST_H264_SEI_PIC_STRUCT_FRAME;
h264: improve robustness when packets are missing. Improve robustness when some expected packets where not received yet or that were not correctly decoded. For example, don't try to decode a picture if there was no valid frame headers parsed so far. https://bugs.freedesktop.org/show_bug.cgi?id=57902
2013-12-10 15:14:27 +00:00
return GST_VAAPI_DECODER_STATUS_DROP_FRAME;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
}
static GstVaapiDecoderStatus
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
parse_sps(GstVaapiDecoderH264 *decoder, GstVaapiDecoderUnit *unit)
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
GstVaapiParserInfoH264 * const pi = unit->parsed_info;
GstH264SPS * const sps = &pi->data.sps;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
GstH264ParserResult result;
h264: add codec specific decoder unit. Introduce new GstVaapiDecoderUnitH264 object, which holds the standard NAL unit header (GstH264NalUnit) and additional parsed header info. Besides, we now parse headers as early as in the _parse() function so that to avoid un-necessary creation of sub-buffers in _decode() for NAL units that are not slices. This is a performance win by ~+1.1% only.
2012-12-06 16:25:01 +00:00
GST_DEBUG("parse SPS");
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
h264: improve robustness when packets are missing. Improve robustness when some expected packets where not received yet or that were not correctly decoded. For example, don't try to decode a picture if there was no valid frame headers parsed so far. https://bugs.freedesktop.org/show_bug.cgi?id=57902
2013-12-10 15:14:27 +00:00
priv->parser_state = 0;
h264: set default values for some header fields. The SPS, PPS and slice headers are not fully zero-initialized in the codecparsers/ library. Rather, the standard upstream behaviour is to initialize only certain syntax elements with some inferred values if they are not present in the bitstream. At the gstreamer-vaapi decoder level, we need to further initialize certain syntax elements with some sensible default values so that to not complicate VA drivers that just pass those verbatim to the HW, and also avoid an memset() of the whole decoder unit. Signed-off-by: Sreerenj Balachandran <sreerenj.balachandran@intel.com> Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2013-01-05 10:33:06 +00:00
/* Variables that don't have inferred values per the H.264
standard but that should get a default value anyway */
sps->log2_max_pic_order_cnt_lsb_minus4 = 0;
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
result = gst_h264_parser_parse_sps(priv->parser, &pi->nalu, sps, TRUE);
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
if (result != GST_H264_PARSER_OK)
return get_status(result);
h264: improve robustness when packets are missing. Improve robustness when some expected packets where not received yet or that were not correctly decoded. For example, don't try to decode a picture if there was no valid frame headers parsed so far. https://bugs.freedesktop.org/show_bug.cgi?id=57902
2013-12-10 15:14:27 +00:00
priv->parser_state |= GST_H264_VIDEO_STATE_GOT_SPS;
h264: fix activation order of picture and sequence parameters. Delay ensure_context() until we actually need a VA context for allocating new VA surfaces, and then GstVaapiPictures, but also when a real activation of a new picture parameter set occurs, thus also implying an activation of the related sequence parameter set. The most important thing was to drop the global pps and sps pointers since they may not have matched the currently activated picture parameter or sequence parameter sets at the specified decode point. Anoter positive side-effect is that this cleans up all occurrences of decode_current_picture() to only keep those useful in decode_picture(), before a new picture is allocated, or in decode_sequence_end() when an end-of-stream or end-of-sequence condition occurred.
2012-10-26 14:12:05 +00:00
return GST_VAAPI_DECODER_STATUS_SUCCESS;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
}
decoder: h264: add support for MVC base views. Allow decoding for base views of MVC encoded streams. For now, just skip the slice extension and prefix NAL units, and skip non-base view frames. Signed-off-by: Xiaowei Li <xiaowei.a.li@intel.com> [fixed memory leak, improved check for MVC NAL units] Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2013-02-28 07:59:55 +00:00
static GstVaapiDecoderStatus
parse_subset_sps(GstVaapiDecoderH264 *decoder, GstVaapiDecoderUnit *unit)
{
GstVaapiDecoderH264Private * const priv = &decoder->priv;
GstVaapiParserInfoH264 * const pi = unit->parsed_info;
GstH264SPS * const sps = &pi->data.sps;
GstH264ParserResult result;
GST_DEBUG("parse subset SPS");
/* Variables that don't have inferred values per the H.264
standard but that should get a default value anyway */
sps->log2_max_pic_order_cnt_lsb_minus4 = 0;
result = gst_h264_parser_parse_subset_sps(priv->parser, &pi->nalu, sps,
TRUE);
if (result != GST_H264_PARSER_OK)
return get_status(result);
priv->parser_state |= GST_H264_VIDEO_STATE_GOT_SPS;
return GST_VAAPI_DECODER_STATUS_SUCCESS;
}
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
static GstVaapiDecoderStatus
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
parse_pps(GstVaapiDecoderH264 *decoder, GstVaapiDecoderUnit *unit)
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
GstVaapiParserInfoH264 * const pi = unit->parsed_info;
GstH264PPS * const pps = &pi->data.pps;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
GstH264ParserResult result;
h264: add codec specific decoder unit. Introduce new GstVaapiDecoderUnitH264 object, which holds the standard NAL unit header (GstH264NalUnit) and additional parsed header info. Besides, we now parse headers as early as in the _parse() function so that to avoid un-necessary creation of sub-buffers in _decode() for NAL units that are not slices. This is a performance win by ~+1.1% only.
2012-12-06 16:25:01 +00:00
GST_DEBUG("parse PPS");
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
h264: improve robustness when packets are missing. Improve robustness when some expected packets where not received yet or that were not correctly decoded. For example, don't try to decode a picture if there was no valid frame headers parsed so far. https://bugs.freedesktop.org/show_bug.cgi?id=57902
2013-12-10 15:14:27 +00:00
priv->parser_state &= GST_H264_VIDEO_STATE_GOT_SPS;
h264: set default values for some header fields. The SPS, PPS and slice headers are not fully zero-initialized in the codecparsers/ library. Rather, the standard upstream behaviour is to initialize only certain syntax elements with some inferred values if they are not present in the bitstream. At the gstreamer-vaapi decoder level, we need to further initialize certain syntax elements with some sensible default values so that to not complicate VA drivers that just pass those verbatim to the HW, and also avoid an memset() of the whole decoder unit. Signed-off-by: Sreerenj Balachandran <sreerenj.balachandran@intel.com> Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2013-01-05 10:33:06 +00:00
/* Variables that don't have inferred values per the H.264
standard but that should get a default value anyway */
pps->slice_group_map_type = 0;
pps->slice_group_change_rate_minus1 = 0;
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
result = gst_h264_parser_parse_pps(priv->parser, &pi->nalu, pps);
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
if (result != GST_H264_PARSER_OK)
return get_status(result);
h264: improve robustness when packets are missing. Improve robustness when some expected packets where not received yet or that were not correctly decoded. For example, don't try to decode a picture if there was no valid frame headers parsed so far. https://bugs.freedesktop.org/show_bug.cgi?id=57902
2013-12-10 15:14:27 +00:00
priv->parser_state |= GST_H264_VIDEO_STATE_GOT_PPS;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
return GST_VAAPI_DECODER_STATUS_SUCCESS;
}
static GstVaapiDecoderStatus
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
parse_sei(GstVaapiDecoderH264 *decoder, GstVaapiDecoderUnit *unit)
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
GstVaapiParserInfoH264 * const pi = unit->parsed_info;
decoder: h264: retain SEI messages until the end of frame. Retain the SEI messages that were parsed from the access unit until we have completely decoded the current frame. This is done so that we can peek at that data whenever necessary during decoding. e.g. for exposing 3D stereoscopic information at a later stage.
2014-04-25 14:24:01 +00:00
GArray ** const sei_ptr = &pi->data.sei;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
GstH264ParserResult result;
h264: add codec specific decoder unit. Introduce new GstVaapiDecoderUnitH264 object, which holds the standard NAL unit header (GstH264NalUnit) and additional parsed header info. Besides, we now parse headers as early as in the _parse() function so that to avoid un-necessary creation of sub-buffers in _decode() for NAL units that are not slices. This is a performance win by ~+1.1% only.
2012-12-06 16:25:01 +00:00
GST_DEBUG("parse SEI");
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
decoder: h264: retain SEI messages until the end of frame. Retain the SEI messages that were parsed from the access unit until we have completely decoded the current frame. This is done so that we can peek at that data whenever necessary during decoding. e.g. for exposing 3D stereoscopic information at a later stage.
2014-04-25 14:24:01 +00:00
result = gst_h264_parser_parse_sei(priv->parser, &pi->nalu, sei_ptr);
h264: skip all Access Unit (AU) NALs. Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2012-02-21 07:11:20 +00:00
if (result != GST_H264_PARSER_OK) {
decoder: h264: cope with new gst_h264_parser_parse_sei() interface. The gst_h264_parse_parse_sei() function now returns an array of SEI messages, instead of a single SEI message. Reason: it is allowed to have several SEI messages packed into a single SEI NAL unit, instead of multiple NAL units.
2014-03-21 14:15:37 +00:00
GST_WARNING("failed to parse SEI messages");
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
return get_status(result);
h264: skip all Access Unit (AU) NALs. Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2012-02-21 07:11:20 +00:00
}
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
return GST_VAAPI_DECODER_STATUS_SUCCESS;
}
h264: add codec specific decoder unit. Introduce new GstVaapiDecoderUnitH264 object, which holds the standard NAL unit header (GstH264NalUnit) and additional parsed header info. Besides, we now parse headers as early as in the _parse() function so that to avoid un-necessary creation of sub-buffers in _decode() for NAL units that are not slices. This is a performance win by ~+1.1% only.
2012-12-06 16:25:01 +00:00
static GstVaapiDecoderStatus
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
parse_slice(GstVaapiDecoderH264 *decoder, GstVaapiDecoderUnit *unit)
h264: add codec specific decoder unit. Introduce new GstVaapiDecoderUnitH264 object, which holds the standard NAL unit header (GstH264NalUnit) and additional parsed header info. Besides, we now parse headers as early as in the _parse() function so that to avoid un-necessary creation of sub-buffers in _decode() for NAL units that are not slices. This is a performance win by ~+1.1% only.
2012-12-06 16:25:01 +00:00
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
GstVaapiParserInfoH264 * const pi = unit->parsed_info;
GstH264SliceHdr * const slice_hdr = &pi->data.slice_hdr;
decoder: h264: properly handle Prefix NAL units. Always cache the previous NAL unit so that we could check whether there is a Prefix NAL unit immediately preceding the current slice or IDR NAL unit. In that case, the NAL unit metadata is copied into the current NAL unit. Otherwise, some default values are inferred, tentatively. e.g. view_id shall be set to 0 and inter_view_flag to 1. [infer default values for slice if previous NAL was not a Prefix] Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2013-10-31 11:32:55 +00:00
GstH264NalUnit * const nalu = &pi->nalu;
decoder: h264: compute view ids only once per slice. Optimize lookups of view ids / view order indices by caching the result of the calculatiosn right into the GstVaapiParserInfoH264 struct. This terribly simplifies is_new_access_unit() and find_first_field() functions.
2014-05-22 08:04:46 +00:00
GstH264SPS *sps;
h264: add codec specific decoder unit. Introduce new GstVaapiDecoderUnitH264 object, which holds the standard NAL unit header (GstH264NalUnit) and additional parsed header info. Besides, we now parse headers as early as in the _parse() function so that to avoid un-necessary creation of sub-buffers in _decode() for NAL units that are not slices. This is a performance win by ~+1.1% only.
2012-12-06 16:25:01 +00:00
GstH264ParserResult result;
GST_DEBUG("parse slice");
h264: improve robustness when packets are missing. Improve robustness when some expected packets where not received yet or that were not correctly decoded. For example, don't try to decode a picture if there was no valid frame headers parsed so far. https://bugs.freedesktop.org/show_bug.cgi?id=57902
2013-12-10 15:14:27 +00:00
priv->parser_state &= (GST_H264_VIDEO_STATE_GOT_SPS|
GST_H264_VIDEO_STATE_GOT_PPS);
decoder: h264: properly handle Prefix NAL units. Always cache the previous NAL unit so that we could check whether there is a Prefix NAL unit immediately preceding the current slice or IDR NAL unit. In that case, the NAL unit metadata is copied into the current NAL unit. Otherwise, some default values are inferred, tentatively. e.g. view_id shall be set to 0 and inter_view_flag to 1. [infer default values for slice if previous NAL was not a Prefix] Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2013-10-31 11:32:55 +00:00
/* Propagate Prefix NAL unit info, if necessary */
switch (nalu->type) {
case GST_H264_NAL_SLICE:
case GST_H264_NAL_SLICE_IDR: {
GstVaapiParserInfoH264 * const prev_pi = priv->prev_pi;
if (prev_pi && prev_pi->nalu.type == GST_H264_NAL_PREFIX_UNIT) {
/* MVC sequences shall have a Prefix NAL unit immediately
preceding this NAL unit */
pi->nalu.extension_type = prev_pi->nalu.extension_type;
pi->nalu.extension = prev_pi->nalu.extension;
}
else {
/* In the very unlikely case there is no Prefix NAL unit
immediately preceding this NAL unit, try to infer some
defaults (H.7.4.1.1) */
GstH264NalUnitExtensionMVC * const mvc = &pi->nalu.extension.mvc;
mvc->non_idr_flag = !(nalu->type == GST_H264_NAL_SLICE_IDR);
nalu->idr_pic_flag = !mvc->non_idr_flag;
mvc->priority_id = 0;
mvc->view_id = 0;
mvc->temporal_id = 0;
mvc->anchor_pic_flag = 0;
mvc->inter_view_flag = 1;
}
break;
}
}
h264: set default values for some header fields. The SPS, PPS and slice headers are not fully zero-initialized in the codecparsers/ library. Rather, the standard upstream behaviour is to initialize only certain syntax elements with some inferred values if they are not present in the bitstream. At the gstreamer-vaapi decoder level, we need to further initialize certain syntax elements with some sensible default values so that to not complicate VA drivers that just pass those verbatim to the HW, and also avoid an memset() of the whole decoder unit. Signed-off-by: Sreerenj Balachandran <sreerenj.balachandran@intel.com> Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2013-01-05 10:33:06 +00:00
/* Variables that don't have inferred values per the H.264
standard but that should get a default value anyway */
slice_hdr->cabac_init_idc = 0;
slice_hdr->direct_spatial_mv_pred_flag = 0;
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
result = gst_h264_parser_parse_slice_hdr(priv->parser, &pi->nalu,
h264: add codec specific decoder unit. Introduce new GstVaapiDecoderUnitH264 object, which holds the standard NAL unit header (GstH264NalUnit) and additional parsed header info. Besides, we now parse headers as early as in the _parse() function so that to avoid un-necessary creation of sub-buffers in _decode() for NAL units that are not slices. This is a performance win by ~+1.1% only.
2012-12-06 16:25:01 +00:00
slice_hdr, TRUE, TRUE);
if (result != GST_H264_PARSER_OK)
return get_status(result);
decoder: h264: compute view ids only once per slice. Optimize lookups of view ids / view order indices by caching the result of the calculatiosn right into the GstVaapiParserInfoH264 struct. This terribly simplifies is_new_access_unit() and find_first_field() functions.
2014-05-22 08:04:46 +00:00
sps = slice_hdr->pps->sequence;
/* Update MVC data */
pi->view_id = get_view_id(&pi->nalu);
pi->voc = get_view_order_index(sps, pi->view_id);
decoder: h264: dynamically allocate the DPB. Dynamically allocate the Decoded Picture Buffer (DPB) and add provisions for supporting the MVC allocation requirements.
2014-05-01 17:16:09 +00:00
h264: improve robustness when packets are missing. Improve robustness when some expected packets where not received yet or that were not correctly decoded. For example, don't try to decode a picture if there was no valid frame headers parsed so far. https://bugs.freedesktop.org/show_bug.cgi?id=57902
2013-12-10 15:14:27 +00:00
priv->parser_state |= GST_H264_VIDEO_STATE_GOT_SLICE;
h264: add codec specific decoder unit. Introduce new GstVaapiDecoderUnitH264 object, which holds the standard NAL unit header (GstH264NalUnit) and additional parsed header info. Besides, we now parse headers as early as in the _parse() function so that to avoid un-necessary creation of sub-buffers in _decode() for NAL units that are not slices. This is a performance win by ~+1.1% only.
2012-12-06 16:25:01 +00:00
return GST_VAAPI_DECODER_STATUS_SUCCESS;
}
decoder: h264: fix activation of picture and sequence parameters. At the time the first VCL NAL unit of a primary coded picture is found, and if that NAL unit was parsed to be an SPS or PPS, then the entries in the parser may have been overriden. This means that, when the picture is to be decoded, slice_hdr->pps could point to an invalid (the next) PPS entry. So, one way to solve this problem is to not use the parser PPS and SPS info but rather maintain our own activation chain in the decoder. https://bugzilla.gnome.org/show_bug.cgi?id=724519 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 17:11:03 +00:00
static GstVaapiDecoderStatus
decode_sps(GstVaapiDecoderH264 *decoder, GstVaapiDecoderUnit *unit)
{
GstVaapiDecoderH264Private * const priv = &decoder->priv;
GstVaapiParserInfoH264 * const pi = unit->parsed_info;
GstH264SPS * const sps = &pi->data.sps;
GST_DEBUG("decode SPS");
gst_vaapi_parser_info_h264_replace(&priv->sps[sps->id], pi);
return GST_VAAPI_DECODER_STATUS_SUCCESS;
}
decoder: h264: add support for MVC base views. Allow decoding for base views of MVC encoded streams. For now, just skip the slice extension and prefix NAL units, and skip non-base view frames. Signed-off-by: Xiaowei Li <xiaowei.a.li@intel.com> [fixed memory leak, improved check for MVC NAL units] Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2013-02-28 07:59:55 +00:00
static GstVaapiDecoderStatus
decode_subset_sps(GstVaapiDecoderH264 *decoder, GstVaapiDecoderUnit *unit)
{
GstVaapiDecoderH264Private * const priv = &decoder->priv;
GstVaapiParserInfoH264 * const pi = unit->parsed_info;
GstH264SPS * const sps = &pi->data.sps;
GST_DEBUG("decode subset SPS");
gst_vaapi_parser_info_h264_replace(&priv->sps[sps->id], pi);
return GST_VAAPI_DECODER_STATUS_SUCCESS;
}
decoder: h264: fix activation of picture and sequence parameters. At the time the first VCL NAL unit of a primary coded picture is found, and if that NAL unit was parsed to be an SPS or PPS, then the entries in the parser may have been overriden. This means that, when the picture is to be decoded, slice_hdr->pps could point to an invalid (the next) PPS entry. So, one way to solve this problem is to not use the parser PPS and SPS info but rather maintain our own activation chain in the decoder. https://bugzilla.gnome.org/show_bug.cgi?id=724519 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 17:11:03 +00:00
static GstVaapiDecoderStatus
decode_pps(GstVaapiDecoderH264 *decoder, GstVaapiDecoderUnit *unit)
{
GstVaapiDecoderH264Private * const priv = &decoder->priv;
GstVaapiParserInfoH264 * const pi = unit->parsed_info;
GstH264PPS * const pps = &pi->data.pps;
GST_DEBUG("decode PPS");
gst_vaapi_parser_info_h264_replace(&priv->pps[pps->id], pi);
return GST_VAAPI_DECODER_STATUS_SUCCESS;
}
decoder: h264: fix detection of top-field-first (TFF) flag. Use the SEI pic_timing() message to track the pic_struct variable when present, or infer it from the regular slice header flags field_pic_flag and bottom_field_flag. This fixes temporal sequence ordering when the output pictures are to be displayed. https://bugzilla.gnome.org/show_bug.cgi?id=739291
2014-11-13 14:00:21 +00:00
static GstVaapiDecoderStatus
decode_sei(GstVaapiDecoderH264 *decoder, GstVaapiDecoderUnit *unit)
{
GstVaapiDecoderH264Private * const priv = &decoder->priv;
GstVaapiParserInfoH264 * const pi = unit->parsed_info;
guint i;
GST_DEBUG("decode SEI messages");
for (i = 0; i < pi->data.sei->len; i++) {
const GstH264SEIMessage * const sei =
&g_array_index(pi->data.sei, GstH264SEIMessage, i);
switch (sei->payloadType) {
case GST_H264_SEI_PIC_TIMING: {
const GstH264PicTiming * const pic_timing =
&sei->payload.pic_timing;
if (pic_timing->pic_struct_present_flag)
priv->pic_structure = pic_timing->pic_struct;
break;
}
default:
break;
}
}
return GST_VAAPI_DECODER_STATUS_SUCCESS;
}
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
static GstVaapiDecoderStatus
decode_sequence_end(GstVaapiDecoderH264 *decoder)
{
decoder: h264: fix tracking of DPB size changes. Add support for MVC streams with multiple SPS and subset SPS headers emitted regularly, e.g. at around every I-frame. Track the maximum number of views in ensure_context() and really reset the DPB size to the expected value, always. i.e. even if it decreased. dpb_reset() only cares of ensuring the DPB allocation.
2014-06-28 05:25:35 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
h264: fix activation order of picture and sequence parameters. Delay ensure_context() until we actually need a VA context for allocating new VA surfaces, and then GstVaapiPictures, but also when a real activation of a new picture parameter set occurs, thus also implying an activation of the related sequence parameter set. The most important thing was to drop the global pps and sps pointers since they may not have matched the currently activated picture parameter or sequence parameter sets at the specified decode point. Anoter positive side-effect is that this cleans up all occurrences of decode_current_picture() to only keep those useful in decode_picture(), before a new picture is allocated, or in decode_sequence_end() when an end-of-stream or end-of-sequence condition occurred.
2012-10-26 14:12:05 +00:00
GstVaapiDecoderStatus status;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
GST_DEBUG("decode sequence-end");
h264: fix activation order of picture and sequence parameters. Delay ensure_context() until we actually need a VA context for allocating new VA surfaces, and then GstVaapiPictures, but also when a real activation of a new picture parameter set occurs, thus also implying an activation of the related sequence parameter set. The most important thing was to drop the global pps and sps pointers since they may not have matched the currently activated picture parameter or sequence parameter sets at the specified decode point. Anoter positive side-effect is that this cleans up all occurrences of decode_current_picture() to only keep those useful in decode_picture(), before a new picture is allocated, or in decode_sequence_end() when an end-of-stream or end-of-sequence condition occurred.
2012-10-26 14:12:05 +00:00
status = decode_current_picture(decoder);
if (status != GST_VAAPI_DECODER_STATUS_SUCCESS)
return status;
decoder: h264: simplify storage of decoded picture into DPB. Factor out process by which the decoded picture with the lowest POC is found, and possibly output. Likewise, the storage and marking of a reference decoded, or non-reference decoded picture, into the DPB could also be simplified as they mostly share the same operations.
2014-05-04 12:49:28 +00:00
dpb_flush(decoder, NULL);
decoder: h264: fix tracking of DPB size changes. Add support for MVC streams with multiple SPS and subset SPS headers emitted regularly, e.g. at around every I-frame. Track the maximum number of views in ensure_context() and really reset the DPB size to the expected value, always. i.e. even if it decreased. dpb_reset() only cares of ensuring the DPB allocation.
2014-06-28 05:25:35 +00:00
/* Reset defaults, should there be a new sequence available next */
priv->max_views = 1;
h264: implement {start,end}_frame() hooks. Implement GstVaapiDecoder.start_frame() and end_frame() semantics so that to create new VA context earlier and submit VA pictures to the HW for decoding as soon as possible. i.e. don't wait for the next frame to start decoding the previous one.
2012-12-13 09:21:46 +00:00
return GST_VAAPI_DECODER_STATUS_SUCCESS;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
}
/* 8.2.1.1 - Decoding process for picture order count type 0 */
static void
init_picture_poc_0(
GstVaapiDecoderH264 *decoder,
GstVaapiPictureH264 *picture,
GstH264SliceHdr *slice_hdr
)
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
decoder: h264: fix activation of picture and sequence parameters. At the time the first VCL NAL unit of a primary coded picture is found, and if that NAL unit was parsed to be an SPS or PPS, then the entries in the parser may have been overriden. This means that, when the picture is to be decoded, slice_hdr->pps could point to an invalid (the next) PPS entry. So, one way to solve this problem is to not use the parser PPS and SPS info but rather maintain our own activation chain in the decoder. https://bugzilla.gnome.org/show_bug.cgi?id=724519 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 17:11:03 +00:00
GstH264SPS * const sps = get_sps(decoder);
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
const gint32 MaxPicOrderCntLsb = 1 << (sps->log2_max_pic_order_cnt_lsb_minus4 + 4);
h264: fill in GstVaapiPicture structure. ... and get rid of local VAPictureH264.flags fields in GstVaapiPictureH264.
2012-10-31 09:56:15 +00:00
gint32 temp_poc;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
GST_DEBUG("decode picture order count type 0");
h264: introduce GST_VAAPI_PICTURE_FLAG_IDR flag.
2012-10-31 10:33:40 +00:00
if (GST_VAAPI_PICTURE_IS_IDR(picture)) {
h264: simplify code when MMCO is 5. Remove exit_picture() and exit_picture_poc() since PicOrderCnt(CurrPic) is now updated accordingly to the standard. Besides, MMCO = 5 specific operations are moved up to exec_ref_pic_marking_adaptive_mmco_5().
2012-10-23 08:33:50 +00:00
priv->prev_poc_msb = 0;
priv->prev_poc_lsb = 0;
}
else if (priv->prev_pic_has_mmco5) {
priv->prev_poc_msb = 0;
h264: fill in GstVaapiPicture structure. ... and get rid of local VAPictureH264.flags fields in GstVaapiPictureH264.
2012-10-31 09:56:15 +00:00
priv->prev_poc_lsb =
(priv->prev_pic_structure == GST_VAAPI_PICTURE_STRUCTURE_BOTTOM_FIELD ?
0 : priv->field_poc[TOP_FIELD]);
h264: simplify code when MMCO is 5. Remove exit_picture() and exit_picture_poc() since PicOrderCnt(CurrPic) is now updated accordingly to the standard. Besides, MMCO = 5 specific operations are moved up to exec_ref_pic_marking_adaptive_mmco_5().
2012-10-23 08:33:50 +00:00
}
else {
priv->prev_poc_msb = priv->poc_msb;
priv->prev_poc_lsb = priv->poc_lsb;
}
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
// (8-3)
priv->poc_lsb = slice_hdr->pic_order_cnt_lsb;
if (priv->poc_lsb < priv->prev_poc_lsb &&
(priv->prev_poc_lsb - priv->poc_lsb) >= (MaxPicOrderCntLsb / 2))
priv->poc_msb = priv->prev_poc_msb + MaxPicOrderCntLsb;
else if (priv->poc_lsb > priv->prev_poc_lsb &&
(priv->poc_lsb - priv->prev_poc_lsb) > (MaxPicOrderCntLsb / 2))
priv->poc_msb = priv->prev_poc_msb - MaxPicOrderCntLsb;
else
priv->poc_msb = priv->prev_poc_msb;
h264: fill in GstVaapiPicture structure. ... and get rid of local VAPictureH264.flags fields in GstVaapiPictureH264.
2012-10-31 09:56:15 +00:00
temp_poc = priv->poc_msb + priv->poc_lsb;
h264: add picture structure for reference picture marking process. Introduce new `structure' field to the H.264 specific picture structure so that to simplify the reference picture marking process. That local picture structure is derived from the original picture structure, as defined by the syntax elements field_pic_flag and bottom_field_flag.
2012-11-13 13:04:31 +00:00
switch (picture->structure) {
h264: fill in GstVaapiPicture structure. ... and get rid of local VAPictureH264.flags fields in GstVaapiPictureH264.
2012-10-31 09:56:15 +00:00
case GST_VAAPI_PICTURE_STRUCTURE_FRAME:
// (8-4, 8-5)
priv->field_poc[TOP_FIELD] = temp_poc;
priv->field_poc[BOTTOM_FIELD] = temp_poc +
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
slice_hdr->delta_pic_order_cnt_bottom;
h264: fill in GstVaapiPicture structure. ... and get rid of local VAPictureH264.flags fields in GstVaapiPictureH264.
2012-10-31 09:56:15 +00:00
break;
case GST_VAAPI_PICTURE_STRUCTURE_TOP_FIELD:
// (8-4)
priv->field_poc[TOP_FIELD] = temp_poc;
break;
case GST_VAAPI_PICTURE_STRUCTURE_BOTTOM_FIELD:
// (8-5)
priv->field_poc[BOTTOM_FIELD] = temp_poc;
break;
}
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
}
/* 8.2.1.2 - Decoding process for picture order count type 1 */
static void
init_picture_poc_1(
GstVaapiDecoderH264 *decoder,
GstVaapiPictureH264 *picture,
GstH264SliceHdr *slice_hdr
)
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
decoder: h264: fix activation of picture and sequence parameters. At the time the first VCL NAL unit of a primary coded picture is found, and if that NAL unit was parsed to be an SPS or PPS, then the entries in the parser may have been overriden. This means that, when the picture is to be decoded, slice_hdr->pps could point to an invalid (the next) PPS entry. So, one way to solve this problem is to not use the parser PPS and SPS info but rather maintain our own activation chain in the decoder. https://bugzilla.gnome.org/show_bug.cgi?id=724519 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 17:11:03 +00:00
GstH264SPS * const sps = get_sps(decoder);
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
const gint32 MaxFrameNum = 1 << (sps->log2_max_frame_num_minus4 + 4);
h264: simplify code when MMCO is 5. Remove exit_picture() and exit_picture_poc() since PicOrderCnt(CurrPic) is now updated accordingly to the standard. Besides, MMCO = 5 specific operations are moved up to exec_ref_pic_marking_adaptive_mmco_5().
2012-10-23 08:33:50 +00:00
gint32 prev_frame_num_offset, abs_frame_num, expected_poc;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
guint i;
GST_DEBUG("decode picture order count type 1");
h264: simplify code when MMCO is 5. Remove exit_picture() and exit_picture_poc() since PicOrderCnt(CurrPic) is now updated accordingly to the standard. Besides, MMCO = 5 specific operations are moved up to exec_ref_pic_marking_adaptive_mmco_5().
2012-10-23 08:33:50 +00:00
if (priv->prev_pic_has_mmco5)
prev_frame_num_offset = 0;
else
prev_frame_num_offset = priv->frame_num_offset;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
// (8-6)
h264: introduce GST_VAAPI_PICTURE_FLAG_IDR flag.
2012-10-31 10:33:40 +00:00
if (GST_VAAPI_PICTURE_IS_IDR(picture))
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
priv->frame_num_offset = 0;
else if (priv->prev_frame_num > priv->frame_num)
h264: simplify code when MMCO is 5. Remove exit_picture() and exit_picture_poc() since PicOrderCnt(CurrPic) is now updated accordingly to the standard. Besides, MMCO = 5 specific operations are moved up to exec_ref_pic_marking_adaptive_mmco_5().
2012-10-23 08:33:50 +00:00
priv->frame_num_offset = prev_frame_num_offset + MaxFrameNum;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
else
h264: simplify code when MMCO is 5. Remove exit_picture() and exit_picture_poc() since PicOrderCnt(CurrPic) is now updated accordingly to the standard. Besides, MMCO = 5 specific operations are moved up to exec_ref_pic_marking_adaptive_mmco_5().
2012-10-23 08:33:50 +00:00
priv->frame_num_offset = prev_frame_num_offset;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
// (8-7)
if (sps->num_ref_frames_in_pic_order_cnt_cycle != 0)
abs_frame_num = priv->frame_num_offset + priv->frame_num;
else
abs_frame_num = 0;
if (!GST_VAAPI_PICTURE_IS_REFERENCE(picture) && abs_frame_num > 0)
abs_frame_num = abs_frame_num - 1;
if (abs_frame_num > 0) {
gint32 expected_delta_per_poc_cycle;
gint32 poc_cycle_cnt, frame_num_in_poc_cycle;
expected_delta_per_poc_cycle = 0;
for (i = 0; i < sps->num_ref_frames_in_pic_order_cnt_cycle; i++)
expected_delta_per_poc_cycle += sps->offset_for_ref_frame[i];
// (8-8)
poc_cycle_cnt = (abs_frame_num - 1) /
sps->num_ref_frames_in_pic_order_cnt_cycle;
frame_num_in_poc_cycle = (abs_frame_num - 1) %
sps->num_ref_frames_in_pic_order_cnt_cycle;
// (8-9)
expected_poc = poc_cycle_cnt * expected_delta_per_poc_cycle;
for (i = 0; i <= frame_num_in_poc_cycle; i++)
expected_poc += sps->offset_for_ref_frame[i];
}
else
expected_poc = 0;
if (!GST_VAAPI_PICTURE_IS_REFERENCE(picture))
expected_poc += sps->offset_for_non_ref_pic;
// (8-10)
h264: add picture structure for reference picture marking process. Introduce new `structure' field to the H.264 specific picture structure so that to simplify the reference picture marking process. That local picture structure is derived from the original picture structure, as defined by the syntax elements field_pic_flag and bottom_field_flag.
2012-11-13 13:04:31 +00:00
switch (picture->structure) {
h264: fill in GstVaapiPicture structure. ... and get rid of local VAPictureH264.flags fields in GstVaapiPictureH264.
2012-10-31 09:56:15 +00:00
case GST_VAAPI_PICTURE_STRUCTURE_FRAME:
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
priv->field_poc[TOP_FIELD] = expected_poc +
slice_hdr->delta_pic_order_cnt[0];
priv->field_poc[BOTTOM_FIELD] = priv->field_poc[TOP_FIELD] +
sps->offset_for_top_to_bottom_field +
slice_hdr->delta_pic_order_cnt[1];
h264: fill in GstVaapiPicture structure. ... and get rid of local VAPictureH264.flags fields in GstVaapiPictureH264.
2012-10-31 09:56:15 +00:00
break;
case GST_VAAPI_PICTURE_STRUCTURE_TOP_FIELD:
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
priv->field_poc[TOP_FIELD] = expected_poc +
slice_hdr->delta_pic_order_cnt[0];
h264: fill in GstVaapiPicture structure. ... and get rid of local VAPictureH264.flags fields in GstVaapiPictureH264.
2012-10-31 09:56:15 +00:00
break;
case GST_VAAPI_PICTURE_STRUCTURE_BOTTOM_FIELD:
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
priv->field_poc[BOTTOM_FIELD] = expected_poc +
h264: fill in GstVaapiPicture structure. ... and get rid of local VAPictureH264.flags fields in GstVaapiPictureH264.
2012-10-31 09:56:15 +00:00
sps->offset_for_top_to_bottom_field +
slice_hdr->delta_pic_order_cnt[0];
break;
}
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
}
/* 8.2.1.3 - Decoding process for picture order count type 2 */
static void
init_picture_poc_2(
GstVaapiDecoderH264 *decoder,
GstVaapiPictureH264 *picture,
GstH264SliceHdr *slice_hdr
)
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
decoder: h264: fix activation of picture and sequence parameters. At the time the first VCL NAL unit of a primary coded picture is found, and if that NAL unit was parsed to be an SPS or PPS, then the entries in the parser may have been overriden. This means that, when the picture is to be decoded, slice_hdr->pps could point to an invalid (the next) PPS entry. So, one way to solve this problem is to not use the parser PPS and SPS info but rather maintain our own activation chain in the decoder. https://bugzilla.gnome.org/show_bug.cgi?id=724519 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 17:11:03 +00:00
GstH264SPS * const sps = get_sps(decoder);
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
const gint32 MaxFrameNum = 1 << (sps->log2_max_frame_num_minus4 + 4);
h264: simplify code when MMCO is 5. Remove exit_picture() and exit_picture_poc() since PicOrderCnt(CurrPic) is now updated accordingly to the standard. Besides, MMCO = 5 specific operations are moved up to exec_ref_pic_marking_adaptive_mmco_5().
2012-10-23 08:33:50 +00:00
gint32 prev_frame_num_offset, temp_poc;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
GST_DEBUG("decode picture order count type 2");
h264: simplify code when MMCO is 5. Remove exit_picture() and exit_picture_poc() since PicOrderCnt(CurrPic) is now updated accordingly to the standard. Besides, MMCO = 5 specific operations are moved up to exec_ref_pic_marking_adaptive_mmco_5().
2012-10-23 08:33:50 +00:00
if (priv->prev_pic_has_mmco5)
prev_frame_num_offset = 0;
else
prev_frame_num_offset = priv->frame_num_offset;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
// (8-11)
h264: introduce GST_VAAPI_PICTURE_FLAG_IDR flag.
2012-10-31 10:33:40 +00:00
if (GST_VAAPI_PICTURE_IS_IDR(picture))
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
priv->frame_num_offset = 0;
else if (priv->prev_frame_num > priv->frame_num)
h264: simplify code when MMCO is 5. Remove exit_picture() and exit_picture_poc() since PicOrderCnt(CurrPic) is now updated accordingly to the standard. Besides, MMCO = 5 specific operations are moved up to exec_ref_pic_marking_adaptive_mmco_5().
2012-10-23 08:33:50 +00:00
priv->frame_num_offset = prev_frame_num_offset + MaxFrameNum;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
else
h264: simplify code when MMCO is 5. Remove exit_picture() and exit_picture_poc() since PicOrderCnt(CurrPic) is now updated accordingly to the standard. Besides, MMCO = 5 specific operations are moved up to exec_ref_pic_marking_adaptive_mmco_5().
2012-10-23 08:33:50 +00:00
priv->frame_num_offset = prev_frame_num_offset;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
// (8-12)
h264: introduce GST_VAAPI_PICTURE_FLAG_IDR flag.
2012-10-31 10:33:40 +00:00
if (GST_VAAPI_PICTURE_IS_IDR(picture))
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
temp_poc = 0;
else if (!GST_VAAPI_PICTURE_IS_REFERENCE(picture))
temp_poc = 2 * (priv->frame_num_offset + priv->frame_num) - 1;
else
temp_poc = 2 * (priv->frame_num_offset + priv->frame_num);
// (8-13)
h264: add picture structure for reference picture marking process. Introduce new `structure' field to the H.264 specific picture structure so that to simplify the reference picture marking process. That local picture structure is derived from the original picture structure, as defined by the syntax elements field_pic_flag and bottom_field_flag.
2012-11-13 13:04:31 +00:00
if (picture->structure != GST_VAAPI_PICTURE_STRUCTURE_BOTTOM_FIELD)
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
priv->field_poc[TOP_FIELD] = temp_poc;
h264: add picture structure for reference picture marking process. Introduce new `structure' field to the H.264 specific picture structure so that to simplify the reference picture marking process. That local picture structure is derived from the original picture structure, as defined by the syntax elements field_pic_flag and bottom_field_flag.
2012-11-13 13:04:31 +00:00
if (picture->structure != GST_VAAPI_PICTURE_STRUCTURE_TOP_FIELD)
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
priv->field_poc[BOTTOM_FIELD] = temp_poc;
}
/* 8.2.1 - Decoding process for picture order count */
static void
init_picture_poc(
GstVaapiDecoderH264 *decoder,
GstVaapiPictureH264 *picture,
GstH264SliceHdr *slice_hdr
)
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
decoder: h264: fix activation of picture and sequence parameters. At the time the first VCL NAL unit of a primary coded picture is found, and if that NAL unit was parsed to be an SPS or PPS, then the entries in the parser may have been overriden. This means that, when the picture is to be decoded, slice_hdr->pps could point to an invalid (the next) PPS entry. So, one way to solve this problem is to not use the parser PPS and SPS info but rather maintain our own activation chain in the decoder. https://bugzilla.gnome.org/show_bug.cgi?id=724519 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 17:11:03 +00:00
GstH264SPS * const sps = get_sps(decoder);
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
switch (sps->pic_order_cnt_type) {
case 0:
init_picture_poc_0(decoder, picture, slice_hdr);
break;
case 1:
init_picture_poc_1(decoder, picture, slice_hdr);
break;
case 2:
init_picture_poc_2(decoder, picture, slice_hdr);
break;
}
h264: add picture structure for reference picture marking process. Introduce new `structure' field to the H.264 specific picture structure so that to simplify the reference picture marking process. That local picture structure is derived from the original picture structure, as defined by the syntax elements field_pic_flag and bottom_field_flag.
2012-11-13 13:04:31 +00:00
if (picture->structure != GST_VAAPI_PICTURE_STRUCTURE_BOTTOM_FIELD)
h264: introduce per-field POC in GstVaapiPictureH264. Use the POC member available in the GstVaapiPicture base class and get rid of the dependency on the local VAPictureH264 TopFieldOrderCnt and BottomFieldOrderCnt. Rather, use a simple field_poc[] array initialized to INT_MAX, so that to simplify picture POC calculation for non frame pictures.
2012-10-31 10:52:03 +00:00
picture->field_poc[TOP_FIELD] = priv->field_poc[TOP_FIELD];
h264: add picture structure for reference picture marking process. Introduce new `structure' field to the H.264 specific picture structure so that to simplify the reference picture marking process. That local picture structure is derived from the original picture structure, as defined by the syntax elements field_pic_flag and bottom_field_flag.
2012-11-13 13:04:31 +00:00
if (picture->structure != GST_VAAPI_PICTURE_STRUCTURE_TOP_FIELD)
h264: introduce per-field POC in GstVaapiPictureH264. Use the POC member available in the GstVaapiPicture base class and get rid of the dependency on the local VAPictureH264 TopFieldOrderCnt and BottomFieldOrderCnt. Rather, use a simple field_poc[] array initialized to INT_MAX, so that to simplify picture POC calculation for non frame pictures.
2012-10-31 10:52:03 +00:00
picture->field_poc[BOTTOM_FIELD] = priv->field_poc[BOTTOM_FIELD];
picture->base.poc = MIN(picture->field_poc[0], picture->field_poc[1]);
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
}
static int
compare_picture_pic_num_dec(const void *a, const void *b)
{
const GstVaapiPictureH264 * const picA = *(GstVaapiPictureH264 **)a;
const GstVaapiPictureH264 * const picB = *(GstVaapiPictureH264 **)b;
return picB->pic_num - picA->pic_num;
}
static int
compare_picture_long_term_pic_num_inc(const void *a, const void *b)
{
const GstVaapiPictureH264 * const picA = *(GstVaapiPictureH264 **)a;
const GstVaapiPictureH264 * const picB = *(GstVaapiPictureH264 **)b;
return picA->long_term_pic_num - picB->long_term_pic_num;
}
static int
compare_picture_poc_dec(const void *a, const void *b)
{
const GstVaapiPictureH264 * const picA = *(GstVaapiPictureH264 **)a;
const GstVaapiPictureH264 * const picB = *(GstVaapiPictureH264 **)b;
h264: introduce per-field POC in GstVaapiPictureH264. Use the POC member available in the GstVaapiPicture base class and get rid of the dependency on the local VAPictureH264 TopFieldOrderCnt and BottomFieldOrderCnt. Rather, use a simple field_poc[] array initialized to INT_MAX, so that to simplify picture POC calculation for non frame pictures.
2012-10-31 10:52:03 +00:00
return picB->base.poc - picA->base.poc;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
}
static int
compare_picture_poc_inc(const void *a, const void *b)
{
const GstVaapiPictureH264 * const picA = *(GstVaapiPictureH264 **)a;
const GstVaapiPictureH264 * const picB = *(GstVaapiPictureH264 **)b;
h264: introduce per-field POC in GstVaapiPictureH264. Use the POC member available in the GstVaapiPicture base class and get rid of the dependency on the local VAPictureH264 TopFieldOrderCnt and BottomFieldOrderCnt. Rather, use a simple field_poc[] array initialized to INT_MAX, so that to simplify picture POC calculation for non frame pictures.
2012-10-31 10:52:03 +00:00
return picA->base.poc - picB->base.poc;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
}
static int
compare_picture_frame_num_wrap_dec(const void *a, const void *b)
{
const GstVaapiPictureH264 * const picA = *(GstVaapiPictureH264 **)a;
const GstVaapiPictureH264 * const picB = *(GstVaapiPictureH264 **)b;
return picB->frame_num_wrap - picA->frame_num_wrap;
}
static int
compare_picture_long_term_frame_idx_inc(const void *a, const void *b)
{
const GstVaapiPictureH264 * const picA = *(GstVaapiPictureH264 **)a;
const GstVaapiPictureH264 * const picB = *(GstVaapiPictureH264 **)b;
h264: introduce GST_VAAPI_PICTURE_{SHORT,LONG}_TERM_REFERENCE flags. Further get rid of GstVaapiPictureH264-local VAPictureH264.flags for reference bits, thus simplifying the reference picture marking process to only track a single set of reference flags. Also introduce a new long_term_frame_idx member.
2012-10-31 10:45:14 +00:00
return picA->long_term_frame_idx - picB->long_term_frame_idx;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
}
/* 8.2.4.1 - Decoding process for picture numbers */
static void
init_picture_refs_pic_num(
GstVaapiDecoderH264 *decoder,
GstVaapiPictureH264 *picture,
GstH264SliceHdr *slice_hdr
)
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
decoder: h264: fix activation of picture and sequence parameters. At the time the first VCL NAL unit of a primary coded picture is found, and if that NAL unit was parsed to be an SPS or PPS, then the entries in the parser may have been overriden. This means that, when the picture is to be decoded, slice_hdr->pps could point to an invalid (the next) PPS entry. So, one way to solve this problem is to not use the parser PPS and SPS info but rather maintain our own activation chain in the decoder. https://bugzilla.gnome.org/show_bug.cgi?id=724519 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 17:11:03 +00:00
GstH264SPS * const sps = get_sps(decoder);
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
const gint32 MaxFrameNum = 1 << (sps->log2_max_frame_num_minus4 + 4);
guint i;
GST_DEBUG("decode picture numbers");
for (i = 0; i < priv->short_ref_count; i++) {
GstVaapiPictureH264 * const pic = priv->short_ref[i];
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
// (H.8.2)
if (pic->base.view_id != picture->base.view_id)
continue;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
// (8-27)
if (pic->frame_num > priv->frame_num)
pic->frame_num_wrap = pic->frame_num - MaxFrameNum;
else
pic->frame_num_wrap = pic->frame_num;
// (8-28, 8-30, 8-31)
h264: fill in GstVaapiPicture structure. ... and get rid of local VAPictureH264.flags fields in GstVaapiPictureH264.
2012-10-31 09:56:15 +00:00
if (GST_VAAPI_PICTURE_IS_FRAME(picture))
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
pic->pic_num = pic->frame_num_wrap;
else {
h264: add picture structure for reference picture marking process. Introduce new `structure' field to the H.264 specific picture structure so that to simplify the reference picture marking process. That local picture structure is derived from the original picture structure, as defined by the syntax elements field_pic_flag and bottom_field_flag.
2012-11-13 13:04:31 +00:00
if (pic->structure == picture->structure)
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
pic->pic_num = 2 * pic->frame_num_wrap + 1;
else
pic->pic_num = 2 * pic->frame_num_wrap;
}
}
for (i = 0; i < priv->long_ref_count; i++) {
GstVaapiPictureH264 * const pic = priv->long_ref[i];
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
// (H.8.2)
if (pic->base.view_id != picture->base.view_id)
continue;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
// (8-29, 8-32, 8-33)
h264: fill in GstVaapiPicture structure. ... and get rid of local VAPictureH264.flags fields in GstVaapiPictureH264.
2012-10-31 09:56:15 +00:00
if (GST_VAAPI_PICTURE_IS_FRAME(picture))
h264: introduce GST_VAAPI_PICTURE_{SHORT,LONG}_TERM_REFERENCE flags. Further get rid of GstVaapiPictureH264-local VAPictureH264.flags for reference bits, thus simplifying the reference picture marking process to only track a single set of reference flags. Also introduce a new long_term_frame_idx member.
2012-10-31 10:45:14 +00:00
pic->long_term_pic_num = pic->long_term_frame_idx;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
else {
h264: add picture structure for reference picture marking process. Introduce new `structure' field to the H.264 specific picture structure so that to simplify the reference picture marking process. That local picture structure is derived from the original picture structure, as defined by the syntax elements field_pic_flag and bottom_field_flag.
2012-11-13 13:04:31 +00:00
if (pic->structure == picture->structure)
h264: introduce GST_VAAPI_PICTURE_{SHORT,LONG}_TERM_REFERENCE flags. Further get rid of GstVaapiPictureH264-local VAPictureH264.flags for reference bits, thus simplifying the reference picture marking process to only track a single set of reference flags. Also introduce a new long_term_frame_idx member.
2012-10-31 10:45:14 +00:00
pic->long_term_pic_num = 2 * pic->long_term_frame_idx + 1;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
else
h264: introduce GST_VAAPI_PICTURE_{SHORT,LONG}_TERM_REFERENCE flags. Further get rid of GstVaapiPictureH264-local VAPictureH264.flags for reference bits, thus simplifying the reference picture marking process to only track a single set of reference flags. Also introduce a new long_term_frame_idx member.
2012-10-31 10:45:14 +00:00
pic->long_term_pic_num = 2 * pic->long_term_frame_idx;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
}
}
}
#define SORT_REF_LIST(list, n, compare_func) \
qsort(list, n, sizeof(*(list)), compare_picture_##compare_func)
h264: add initial support for interlaced streams. Decoded frames are only output when they are complete, i.e. when both fields are decoded. This also means that the "interlaced" caps is not propagated to vaapipostproc or vaapisink elements. Another limitation is that interlaced bitstreams with MMCO are unlikely to work.
2012-11-13 16:14:39 +00:00
static void
init_picture_refs_fields_1(
guint picture_structure,
GstVaapiPictureH264 *RefPicList[32],
guint *RefPicList_count,
GstVaapiPictureH264 *ref_list[32],
guint ref_list_count
)
{
guint i, j, n;
i = 0;
j = 0;
n = *RefPicList_count;
do {
g_assert(n < 32);
for (; i < ref_list_count; i++) {
if (ref_list[i]->structure == picture_structure) {
RefPicList[n++] = ref_list[i++];
break;
}
}
for (; j < ref_list_count; j++) {
if (ref_list[j]->structure != picture_structure) {
RefPicList[n++] = ref_list[j++];
break;
}
}
} while (i < ref_list_count || j < ref_list_count);
*RefPicList_count = n;
}
static inline void
init_picture_refs_fields(
GstVaapiPictureH264 *picture,
GstVaapiPictureH264 *RefPicList[32],
guint *RefPicList_count,
GstVaapiPictureH264 *short_ref[32],
guint short_ref_count,
GstVaapiPictureH264 *long_ref[32],
guint long_ref_count
)
{
guint n = 0;
/* 8.2.4.2.5 - reference picture lists in fields */
init_picture_refs_fields_1(picture->structure, RefPicList, &n,
short_ref, short_ref_count);
init_picture_refs_fields_1(picture->structure, RefPicList, &n,
long_ref, long_ref_count);
*RefPicList_count = n;
}
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
/* Finds the inter-view reference picture with the supplied view id */
static GstVaapiPictureH264 *
find_inter_view_reference(GstVaapiDecoderH264 *decoder, guint16 view_id)
{
GPtrArray * const inter_views = decoder->priv.inter_views;
guint i;
for (i = 0; i < inter_views->len; i++) {
GstVaapiPictureH264 * const picture = g_ptr_array_index(inter_views, i);
if (picture->base.view_id == view_id)
return picture;
}
GST_WARNING("failed to find inter-view reference picture for view_id: %d",
view_id);
return NULL;
}
decoder: h264: reduce ReferenceFrames entries to the essential set. When decoding Multiview High profile streams with a large number of views, it is not possible to make the VAPictureParameterBufferH264. ReferenceFrames[] array hold the complete DPB, with all possibly active pictures to be used for inter-view prediction in the current access unit. So reduce the scope of the ReferenceFrames[] array to only include the set of reference pictures that are going to be used for decoding the current picture. Basically, this is a union of all RefPicListX[] array, for all slices constituting the decoded picture.
2014-06-03 15:36:38 +00:00
/* Checks whether the view id exists in the supplied list of view ids */
static gboolean
find_view_id(guint16 view_id, const guint16 *view_ids, guint num_view_ids)
{
guint i;
for (i = 0; i < num_view_ids; i++) {
if (view_ids[i] == view_id)
return TRUE;
}
return FALSE;
}
decoder: h264: improve pruning of unused MVC inter-view frames. Improve process for the removal of pictures from DPB before possible insertion of the current picture (C.4.4) for H.264 MVC inter-view only reference components. In particular, handle cases where picture to be inserted is not the last one of the access unit and if it was already output and is no longer marked as used for reference, including for decoding next view components within the same access unit.
2014-06-10 14:07:51 +00:00
static gboolean
find_view_id_in_view(guint16 view_id, const GstH264SPSExtMVCView *view,
gboolean is_anchor)
{
if (is_anchor)
return (find_view_id(view_id, view->anchor_ref_l0,
view->num_anchor_refs_l0) ||
find_view_id(view_id, view->anchor_ref_l1,
view->num_anchor_refs_l1));
return (find_view_id(view_id, view->non_anchor_ref_l0,
view->num_non_anchor_refs_l0) ||
find_view_id(view_id, view->non_anchor_ref_l1,
view->num_non_anchor_refs_l1));
}
decoder: h264: reduce ReferenceFrames entries to the essential set. When decoding Multiview High profile streams with a large number of views, it is not possible to make the VAPictureParameterBufferH264. ReferenceFrames[] array hold the complete DPB, with all possibly active pictures to be used for inter-view prediction in the current access unit. So reduce the scope of the ReferenceFrames[] array to only include the set of reference pictures that are going to be used for decoding the current picture. Basically, this is a union of all RefPicListX[] array, for all slices constituting the decoded picture.
2014-06-03 15:36:38 +00:00
/* Checks whether the inter-view reference picture with the supplied
view id is used for decoding the current view component picture */
static gboolean
is_inter_view_reference_for_picture(GstVaapiDecoderH264 *decoder,
guint16 view_id, GstVaapiPictureH264 *picture)
{
const GstH264SPS * const sps = get_sps(decoder);
decoder: h264: improve pruning of unused MVC inter-view frames. Improve process for the removal of pictures from DPB before possible insertion of the current picture (C.4.4) for H.264 MVC inter-view only reference components. In particular, handle cases where picture to be inserted is not the last one of the access unit and if it was already output and is no longer marked as used for reference, including for decoding next view components within the same access unit.
2014-06-10 14:07:51 +00:00
gboolean is_anchor;
decoder: h264: reduce ReferenceFrames entries to the essential set. When decoding Multiview High profile streams with a large number of views, it is not possible to make the VAPictureParameterBufferH264. ReferenceFrames[] array hold the complete DPB, with all possibly active pictures to be used for inter-view prediction in the current access unit. So reduce the scope of the ReferenceFrames[] array to only include the set of reference pictures that are going to be used for decoding the current picture. Basically, this is a union of all RefPicListX[] array, for all slices constituting the decoded picture.
2014-06-03 15:36:38 +00:00
if (!GST_VAAPI_PICTURE_IS_MVC(picture) ||
sps->extension_type != GST_H264_NAL_EXTENSION_MVC)
return FALSE;
decoder: h264: improve pruning of unused MVC inter-view frames. Improve process for the removal of pictures from DPB before possible insertion of the current picture (C.4.4) for H.264 MVC inter-view only reference components. In particular, handle cases where picture to be inserted is not the last one of the access unit and if it was already output and is no longer marked as used for reference, including for decoding next view components within the same access unit.
2014-06-10 14:07:51 +00:00
is_anchor = GST_VAAPI_PICTURE_IS_ANCHOR(picture);
return find_view_id_in_view(view_id,
&sps->extension.mvc.view[picture->base.voc], is_anchor);
}
decoder: h264: reduce ReferenceFrames entries to the essential set. When decoding Multiview High profile streams with a large number of views, it is not possible to make the VAPictureParameterBufferH264. ReferenceFrames[] array hold the complete DPB, with all possibly active pictures to be used for inter-view prediction in the current access unit. So reduce the scope of the ReferenceFrames[] array to only include the set of reference pictures that are going to be used for decoding the current picture. Basically, this is a union of all RefPicListX[] array, for all slices constituting the decoded picture.
2014-06-03 15:36:38 +00:00
decoder: h264: improve pruning of unused MVC inter-view frames. Improve process for the removal of pictures from DPB before possible insertion of the current picture (C.4.4) for H.264 MVC inter-view only reference components. In particular, handle cases where picture to be inserted is not the last one of the access unit and if it was already output and is no longer marked as used for reference, including for decoding next view components within the same access unit.
2014-06-10 14:07:51 +00:00
/* Checks whether the supplied inter-view reference picture is used
for decoding the next view component pictures */
static gboolean
is_inter_view_reference_for_next_pictures(GstVaapiDecoderH264 *decoder,
GstVaapiPictureH264 *picture)
{
const GstH264SPS * const sps = get_sps(decoder);
gboolean is_anchor;
guint i, num_views;
if (!GST_VAAPI_PICTURE_IS_MVC(picture) ||
sps->extension_type != GST_H264_NAL_EXTENSION_MVC)
return FALSE;
is_anchor = GST_VAAPI_PICTURE_IS_ANCHOR(picture);
num_views = sps->extension.mvc.num_views_minus1 + 1;
for (i = picture->base.voc + 1; i < num_views; i++) {
const GstH264SPSExtMVCView * const view = &sps->extension.mvc.view[i];
if (find_view_id_in_view(picture->base.view_id, view, is_anchor))
return TRUE;
}
return FALSE;
decoder: h264: reduce ReferenceFrames entries to the essential set. When decoding Multiview High profile streams with a large number of views, it is not possible to make the VAPictureParameterBufferH264. ReferenceFrames[] array hold the complete DPB, with all possibly active pictures to be used for inter-view prediction in the current access unit. So reduce the scope of the ReferenceFrames[] array to only include the set of reference pictures that are going to be used for decoding the current picture. Basically, this is a union of all RefPicListX[] array, for all slices constituting the decoded picture.
2014-06-03 15:36:38 +00:00
}
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
/* H.8.2.1 - Initialization process for inter-view prediction references */
static void
init_picture_refs_mvc_1(GstVaapiDecoderH264 *decoder,
GstVaapiPictureH264 **ref_list, guint *ref_list_count_ptr, guint num_refs,
const guint16 *view_ids, guint num_view_ids)
{
guint j, n;
n = *ref_list_count_ptr;
for (j = 0; j < num_view_ids && n < num_refs; j++) {
GstVaapiPictureH264 * const pic =
find_inter_view_reference(decoder, view_ids[j]);
if (pic)
ref_list[n++] = pic;
}
*ref_list_count_ptr = n;
}
static inline void
init_picture_refs_mvc(GstVaapiDecoderH264 *decoder,
GstVaapiPictureH264 *picture, GstH264SliceHdr *slice_hdr, guint list)
{
GstVaapiDecoderH264Private * const priv = &decoder->priv;
const GstH264SPS * const sps = get_sps(decoder);
const GstH264SPSExtMVCView *view;
GST_DEBUG("initialize reference picture list for inter-view prediction");
if (sps->extension_type != GST_H264_NAL_EXTENSION_MVC)
return;
view = &sps->extension.mvc.view[picture->base.voc];
#define INVOKE_INIT_PICTURE_REFS_MVC(ref_list, view_list) do { \
init_picture_refs_mvc_1(decoder, \
priv->RefPicList##ref_list, \
&priv->RefPicList##ref_list##_count, \
slice_hdr->num_ref_idx_l##ref_list##_active_minus1 + 1, \
view->view_list##_l##ref_list, \
view->num_##view_list##s_l##ref_list); \
} while (0)
if (list == 0) {
if (GST_VAAPI_PICTURE_IS_ANCHOR(picture))
INVOKE_INIT_PICTURE_REFS_MVC(0, anchor_ref);
else
INVOKE_INIT_PICTURE_REFS_MVC(0, non_anchor_ref);
}
else {
if (GST_VAAPI_PICTURE_IS_ANCHOR(picture))
INVOKE_INIT_PICTURE_REFS_MVC(1, anchor_ref);
else
INVOKE_INIT_PICTURE_REFS_MVC(1, non_anchor_ref);
}
#undef INVOKE_INIT_PICTURE_REFS_MVC
}
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
static void
init_picture_refs_p_slice(
GstVaapiDecoderH264 *decoder,
GstVaapiPictureH264 *picture,
GstH264SliceHdr *slice_hdr
)
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
GstVaapiPictureH264 **ref_list;
guint i;
GST_DEBUG("decode reference picture list for P and SP slices");
h264: fill in GstVaapiPicture structure. ... and get rid of local VAPictureH264.flags fields in GstVaapiPictureH264.
2012-10-31 09:56:15 +00:00
if (GST_VAAPI_PICTURE_IS_FRAME(picture)) {
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
/* 8.2.4.2.1 - P and SP slices in frames */
if (priv->short_ref_count > 0) {
ref_list = priv->RefPicList0;
for (i = 0; i < priv->short_ref_count; i++)
ref_list[i] = priv->short_ref[i];
SORT_REF_LIST(ref_list, i, pic_num_dec);
priv->RefPicList0_count += i;
}
if (priv->long_ref_count > 0) {
ref_list = &priv->RefPicList0[priv->RefPicList0_count];
for (i = 0; i < priv->long_ref_count; i++)
ref_list[i] = priv->long_ref[i];
SORT_REF_LIST(ref_list, i, long_term_pic_num_inc);
priv->RefPicList0_count += i;
}
}
else {
/* 8.2.4.2.2 - P and SP slices in fields */
GstVaapiPictureH264 *short_ref[32];
guint short_ref_count = 0;
GstVaapiPictureH264 *long_ref[32];
guint long_ref_count = 0;
if (priv->short_ref_count > 0) {
for (i = 0; i < priv->short_ref_count; i++)
short_ref[i] = priv->short_ref[i];
SORT_REF_LIST(short_ref, i, frame_num_wrap_dec);
short_ref_count = i;
}
if (priv->long_ref_count > 0) {
for (i = 0; i < priv->long_ref_count; i++)
long_ref[i] = priv->long_ref[i];
SORT_REF_LIST(long_ref, i, long_term_frame_idx_inc);
long_ref_count = i;
}
h264: add initial support for interlaced streams. Decoded frames are only output when they are complete, i.e. when both fields are decoded. This also means that the "interlaced" caps is not propagated to vaapipostproc or vaapisink elements. Another limitation is that interlaced bitstreams with MMCO are unlikely to work.
2012-11-13 16:14:39 +00:00
init_picture_refs_fields(
picture,
priv->RefPicList0, &priv->RefPicList0_count,
short_ref, short_ref_count,
long_ref, long_ref_count
);
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
}
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
if (GST_VAAPI_PICTURE_IS_MVC(picture)) {
/* RefPicList0 */
init_picture_refs_mvc(decoder, picture, slice_hdr, 0);
}
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
}
static void
init_picture_refs_b_slice(
GstVaapiDecoderH264 *decoder,
GstVaapiPictureH264 *picture,
GstH264SliceHdr *slice_hdr
)
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
GstVaapiPictureH264 **ref_list;
guint i, n;
GST_DEBUG("decode reference picture list for B slices");
h264: fill in GstVaapiPicture structure. ... and get rid of local VAPictureH264.flags fields in GstVaapiPictureH264.
2012-10-31 09:56:15 +00:00
if (GST_VAAPI_PICTURE_IS_FRAME(picture)) {
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
/* 8.2.4.2.3 - B slices in frames */
/* RefPicList0 */
if (priv->short_ref_count > 0) {
// 1. Short-term references
ref_list = priv->RefPicList0;
for (n = 0, i = 0; i < priv->short_ref_count; i++) {
h264: introduce per-field POC in GstVaapiPictureH264. Use the POC member available in the GstVaapiPicture base class and get rid of the dependency on the local VAPictureH264 TopFieldOrderCnt and BottomFieldOrderCnt. Rather, use a simple field_poc[] array initialized to INT_MAX, so that to simplify picture POC calculation for non frame pictures.
2012-10-31 10:52:03 +00:00
if (priv->short_ref[i]->base.poc < picture->base.poc)
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
ref_list[n++] = priv->short_ref[i];
}
SORT_REF_LIST(ref_list, n, poc_dec);
priv->RefPicList0_count += n;
ref_list = &priv->RefPicList0[priv->RefPicList0_count];
for (n = 0, i = 0; i < priv->short_ref_count; i++) {
h264: introduce per-field POC in GstVaapiPictureH264. Use the POC member available in the GstVaapiPicture base class and get rid of the dependency on the local VAPictureH264 TopFieldOrderCnt and BottomFieldOrderCnt. Rather, use a simple field_poc[] array initialized to INT_MAX, so that to simplify picture POC calculation for non frame pictures.
2012-10-31 10:52:03 +00:00
if (priv->short_ref[i]->base.poc >= picture->base.poc)
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
ref_list[n++] = priv->short_ref[i];
}
SORT_REF_LIST(ref_list, n, poc_inc);
priv->RefPicList0_count += n;
}
if (priv->long_ref_count > 0) {
// 2. Long-term references
ref_list = &priv->RefPicList0[priv->RefPicList0_count];
for (n = 0, i = 0; i < priv->long_ref_count; i++)
ref_list[n++] = priv->long_ref[i];
SORT_REF_LIST(ref_list, n, long_term_pic_num_inc);
priv->RefPicList0_count += n;
}
/* RefPicList1 */
if (priv->short_ref_count > 0) {
// 1. Short-term references
ref_list = priv->RefPicList1;
for (n = 0, i = 0; i < priv->short_ref_count; i++) {
h264: introduce per-field POC in GstVaapiPictureH264. Use the POC member available in the GstVaapiPicture base class and get rid of the dependency on the local VAPictureH264 TopFieldOrderCnt and BottomFieldOrderCnt. Rather, use a simple field_poc[] array initialized to INT_MAX, so that to simplify picture POC calculation for non frame pictures.
2012-10-31 10:52:03 +00:00
if (priv->short_ref[i]->base.poc > picture->base.poc)
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
ref_list[n++] = priv->short_ref[i];
}
SORT_REF_LIST(ref_list, n, poc_inc);
priv->RefPicList1_count += n;
ref_list = &priv->RefPicList1[priv->RefPicList1_count];
for (n = 0, i = 0; i < priv->short_ref_count; i++) {
h264: introduce per-field POC in GstVaapiPictureH264. Use the POC member available in the GstVaapiPicture base class and get rid of the dependency on the local VAPictureH264 TopFieldOrderCnt and BottomFieldOrderCnt. Rather, use a simple field_poc[] array initialized to INT_MAX, so that to simplify picture POC calculation for non frame pictures.
2012-10-31 10:52:03 +00:00
if (priv->short_ref[i]->base.poc <= picture->base.poc)
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
ref_list[n++] = priv->short_ref[i];
}
SORT_REF_LIST(ref_list, n, poc_dec);
priv->RefPicList1_count += n;
}
if (priv->long_ref_count > 0) {
// 2. Long-term references
ref_list = &priv->RefPicList1[priv->RefPicList1_count];
for (n = 0, i = 0; i < priv->long_ref_count; i++)
ref_list[n++] = priv->long_ref[i];
SORT_REF_LIST(ref_list, n, long_term_pic_num_inc);
priv->RefPicList1_count += n;
}
}
else {
/* 8.2.4.2.4 - B slices in fields */
GstVaapiPictureH264 *short_ref0[32];
guint short_ref0_count = 0;
GstVaapiPictureH264 *short_ref1[32];
guint short_ref1_count = 0;
GstVaapiPictureH264 *long_ref[32];
guint long_ref_count = 0;
/* refFrameList0ShortTerm */
if (priv->short_ref_count > 0) {
ref_list = short_ref0;
for (n = 0, i = 0; i < priv->short_ref_count; i++) {
h264: introduce per-field POC in GstVaapiPictureH264. Use the POC member available in the GstVaapiPicture base class and get rid of the dependency on the local VAPictureH264 TopFieldOrderCnt and BottomFieldOrderCnt. Rather, use a simple field_poc[] array initialized to INT_MAX, so that to simplify picture POC calculation for non frame pictures.
2012-10-31 10:52:03 +00:00
if (priv->short_ref[i]->base.poc <= picture->base.poc)
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
ref_list[n++] = priv->short_ref[i];
}
SORT_REF_LIST(ref_list, n, poc_dec);
short_ref0_count += n;
ref_list = &short_ref0[short_ref0_count];
for (n = 0, i = 0; i < priv->short_ref_count; i++) {
h264: introduce per-field POC in GstVaapiPictureH264. Use the POC member available in the GstVaapiPicture base class and get rid of the dependency on the local VAPictureH264 TopFieldOrderCnt and BottomFieldOrderCnt. Rather, use a simple field_poc[] array initialized to INT_MAX, so that to simplify picture POC calculation for non frame pictures.
2012-10-31 10:52:03 +00:00
if (priv->short_ref[i]->base.poc > picture->base.poc)
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
ref_list[n++] = priv->short_ref[i];
}
SORT_REF_LIST(ref_list, n, poc_inc);
short_ref0_count += n;
}
/* refFrameList1ShortTerm */
if (priv->short_ref_count > 0) {
ref_list = short_ref1;
for (n = 0, i = 0; i < priv->short_ref_count; i++) {
h264: introduce per-field POC in GstVaapiPictureH264. Use the POC member available in the GstVaapiPicture base class and get rid of the dependency on the local VAPictureH264 TopFieldOrderCnt and BottomFieldOrderCnt. Rather, use a simple field_poc[] array initialized to INT_MAX, so that to simplify picture POC calculation for non frame pictures.
2012-10-31 10:52:03 +00:00
if (priv->short_ref[i]->base.poc > picture->base.poc)
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
ref_list[n++] = priv->short_ref[i];
}
SORT_REF_LIST(ref_list, n, poc_inc);
short_ref1_count += n;
ref_list = &short_ref1[short_ref1_count];
for (n = 0, i = 0; i < priv->short_ref_count; i++) {
h264: introduce per-field POC in GstVaapiPictureH264. Use the POC member available in the GstVaapiPicture base class and get rid of the dependency on the local VAPictureH264 TopFieldOrderCnt and BottomFieldOrderCnt. Rather, use a simple field_poc[] array initialized to INT_MAX, so that to simplify picture POC calculation for non frame pictures.
2012-10-31 10:52:03 +00:00
if (priv->short_ref[i]->base.poc <= picture->base.poc)
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
ref_list[n++] = priv->short_ref[i];
}
SORT_REF_LIST(ref_list, n, poc_dec);
short_ref1_count += n;
}
/* refFrameListLongTerm */
if (priv->long_ref_count > 0) {
for (i = 0; i < priv->long_ref_count; i++)
long_ref[i] = priv->long_ref[i];
SORT_REF_LIST(long_ref, i, long_term_frame_idx_inc);
long_ref_count = i;
}
h264: add initial support for interlaced streams. Decoded frames are only output when they are complete, i.e. when both fields are decoded. This also means that the "interlaced" caps is not propagated to vaapipostproc or vaapisink elements. Another limitation is that interlaced bitstreams with MMCO are unlikely to work.
2012-11-13 16:14:39 +00:00
init_picture_refs_fields(
picture,
priv->RefPicList0, &priv->RefPicList0_count,
short_ref0, short_ref0_count,
long_ref, long_ref_count
);
init_picture_refs_fields(
picture,
priv->RefPicList1, &priv->RefPicList1_count,
short_ref1, short_ref1_count,
long_ref, long_ref_count
);
}
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
/* Check whether RefPicList1 is identical to RefPicList0, then
swap if necessary */
if (priv->RefPicList1_count > 1 &&
priv->RefPicList1_count == priv->RefPicList0_count &&
memcmp(priv->RefPicList0, priv->RefPicList1,
priv->RefPicList0_count * sizeof(priv->RefPicList0[0])) == 0) {
GstVaapiPictureH264 * const tmp = priv->RefPicList1[0];
priv->RefPicList1[0] = priv->RefPicList1[1];
priv->RefPicList1[1] = tmp;
}
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
if (GST_VAAPI_PICTURE_IS_MVC(picture)) {
/* RefPicList0 */
init_picture_refs_mvc(decoder, picture, slice_hdr, 0);
/* RefPicList1 */
init_picture_refs_mvc(decoder, picture, slice_hdr, 1);
}
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
}
#undef SORT_REF_LIST
static gint
find_short_term_reference(GstVaapiDecoderH264 *decoder, gint32 pic_num)
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
guint i;
for (i = 0; i < priv->short_ref_count; i++) {
if (priv->short_ref[i]->pic_num == pic_num)
return i;
}
GST_ERROR("found no short-term reference picture with PicNum = %d",
pic_num);
return -1;
}
static gint
find_long_term_reference(GstVaapiDecoderH264 *decoder, gint32 long_term_pic_num)
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
guint i;
for (i = 0; i < priv->long_ref_count; i++) {
if (priv->long_ref[i]->long_term_pic_num == long_term_pic_num)
return i;
}
GST_ERROR("found no long-term reference picture with LongTermPicNum = %d",
long_term_pic_num);
return -1;
}
static void
exec_picture_refs_modification_1(
GstVaapiDecoderH264 *decoder,
GstVaapiPictureH264 *picture,
GstH264SliceHdr *slice_hdr,
guint list
)
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
decoder: h264: fix activation of picture and sequence parameters. At the time the first VCL NAL unit of a primary coded picture is found, and if that NAL unit was parsed to be an SPS or PPS, then the entries in the parser may have been overriden. This means that, when the picture is to be decoded, slice_hdr->pps could point to an invalid (the next) PPS entry. So, one way to solve this problem is to not use the parser PPS and SPS info but rather maintain our own activation chain in the decoder. https://bugzilla.gnome.org/show_bug.cgi?id=724519 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 17:11:03 +00:00
GstH264SPS * const sps = get_sps(decoder);
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
GstH264RefPicListModification *ref_pic_list_modification;
guint num_ref_pic_list_modifications;
GstVaapiPictureH264 **ref_list;
guint *ref_list_count_ptr, ref_list_count, ref_list_idx = 0;
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
const guint16 *view_ids = NULL;
guint i, j, n, num_refs, num_view_ids = 0;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
gint found_ref_idx;
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
gint32 MaxPicNum, CurrPicNum, picNumPred, picViewIdxPred;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
GST_DEBUG("modification process of reference picture list %u", list);
if (list == 0) {
ref_pic_list_modification = slice_hdr->ref_pic_list_modification_l0;
num_ref_pic_list_modifications = slice_hdr->n_ref_pic_list_modification_l0;
ref_list = priv->RefPicList0;
ref_list_count_ptr = &priv->RefPicList0_count;
num_refs = slice_hdr->num_ref_idx_l0_active_minus1 + 1;
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
if (GST_VAAPI_PICTURE_IS_MVC(picture) &&
sps->extension_type == GST_H264_NAL_EXTENSION_MVC) {
const GstH264SPSExtMVCView * const view =
&sps->extension.mvc.view[picture->base.voc];
if (GST_VAAPI_PICTURE_IS_ANCHOR(picture)) {
view_ids = view->anchor_ref_l0;
num_view_ids = view->num_anchor_refs_l0;
}
else {
view_ids = view->non_anchor_ref_l0;
num_view_ids = view->num_non_anchor_refs_l0;
}
}
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
}
else {
ref_pic_list_modification = slice_hdr->ref_pic_list_modification_l1;
num_ref_pic_list_modifications = slice_hdr->n_ref_pic_list_modification_l1;
ref_list = priv->RefPicList1;
ref_list_count_ptr = &priv->RefPicList1_count;
num_refs = slice_hdr->num_ref_idx_l1_active_minus1 + 1;
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
if (GST_VAAPI_PICTURE_IS_MVC(picture) &&
sps->extension_type == GST_H264_NAL_EXTENSION_MVC) {
const GstH264SPSExtMVCView * const view =
&sps->extension.mvc.view[picture->base.voc];
if (GST_VAAPI_PICTURE_IS_ANCHOR(picture)) {
view_ids = view->anchor_ref_l1;
num_view_ids = view->num_anchor_refs_l1;
}
else {
view_ids = view->non_anchor_ref_l1;
num_view_ids = view->num_non_anchor_refs_l1;
}
}
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
}
ref_list_count = *ref_list_count_ptr;
h264: fill in GstVaapiPicture structure. ... and get rid of local VAPictureH264.flags fields in GstVaapiPictureH264.
2012-10-31 09:56:15 +00:00
if (!GST_VAAPI_PICTURE_IS_FRAME(picture)) {
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
MaxPicNum = 1 << (sps->log2_max_frame_num_minus4 + 5); // 2 * MaxFrameNum
CurrPicNum = 2 * slice_hdr->frame_num + 1; // 2 * frame_num + 1
}
else {
MaxPicNum = 1 << (sps->log2_max_frame_num_minus4 + 4); // MaxFrameNum
CurrPicNum = slice_hdr->frame_num; // frame_num
}
picNumPred = CurrPicNum;
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
picViewIdxPred = -1;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
for (i = 0; i < num_ref_pic_list_modifications; i++) {
GstH264RefPicListModification * const l = &ref_pic_list_modification[i];
if (l->modification_of_pic_nums_idc == 3)
break;
/* 8.2.4.3.1 - Short-term reference pictures */
if (l->modification_of_pic_nums_idc == 0 || l->modification_of_pic_nums_idc == 1) {
gint32 abs_diff_pic_num = l->value.abs_diff_pic_num_minus1 + 1;
gint32 picNum, picNumNoWrap;
// (8-34)
if (l->modification_of_pic_nums_idc == 0) {
picNumNoWrap = picNumPred - abs_diff_pic_num;
if (picNumNoWrap < 0)
picNumNoWrap += MaxPicNum;
}
// (8-35)
else {
picNumNoWrap = picNumPred + abs_diff_pic_num;
if (picNumNoWrap >= MaxPicNum)
picNumNoWrap -= MaxPicNum;
}
picNumPred = picNumNoWrap;
// (8-36)
picNum = picNumNoWrap;
if (picNum > CurrPicNum)
picNum -= MaxPicNum;
h264: fix modification process of reference picture lists. Construction of RefPicList0/1 could be off by one element. Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2012-02-29 08:08:46 +00:00
// (8-37)
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
for (j = num_refs; j > ref_list_idx; j--)
ref_list[j] = ref_list[j - 1];
found_ref_idx = find_short_term_reference(decoder, picNum);
ref_list[ref_list_idx++] =
found_ref_idx >= 0 ? priv->short_ref[found_ref_idx] : NULL;
n = ref_list_idx;
h264: fix modification process of reference picture lists. Construction of RefPicList0/1 could be off by one element. Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2012-02-29 08:08:46 +00:00
for (j = ref_list_idx; j <= num_refs; j++) {
gint32 PicNumF;
if (!ref_list[j])
continue;
h264: introduce GST_VAAPI_PICTURE_{SHORT,LONG}_TERM_REFERENCE flags. Further get rid of GstVaapiPictureH264-local VAPictureH264.flags for reference bits, thus simplifying the reference picture marking process to only track a single set of reference flags. Also introduce a new long_term_frame_idx member.
2012-10-31 10:45:14 +00:00
PicNumF =
GST_VAAPI_PICTURE_IS_SHORT_TERM_REFERENCE(ref_list[j]) ?
ref_list[j]->pic_num : MaxPicNum;
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
if (PicNumF != picNum ||
ref_list[j]->base.view_id != picture->base.view_id)
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
ref_list[n++] = ref_list[j];
}
}
/* 8.2.4.3.2 - Long-term reference pictures */
decoder: h264: minor clean-ups. Make init_picture_ref_lists() more consistent with other functions related to the reference marking process by supplying the current picture as argument.
2014-05-02 20:40:16 +00:00
else if (l->modification_of_pic_nums_idc == 2) {
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
for (j = num_refs; j > ref_list_idx; j--)
ref_list[j] = ref_list[j - 1];
found_ref_idx =
find_long_term_reference(decoder, l->value.long_term_pic_num);
ref_list[ref_list_idx++] =
found_ref_idx >= 0 ? priv->long_ref[found_ref_idx] : NULL;
n = ref_list_idx;
h264: fix modification process of reference picture lists. Construction of RefPicList0/1 could be off by one element. Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2012-02-29 08:08:46 +00:00
for (j = ref_list_idx; j <= num_refs; j++) {
gint32 LongTermPicNumF;
if (!ref_list[j])
continue;
h264: introduce GST_VAAPI_PICTURE_{SHORT,LONG}_TERM_REFERENCE flags. Further get rid of GstVaapiPictureH264-local VAPictureH264.flags for reference bits, thus simplifying the reference picture marking process to only track a single set of reference flags. Also introduce a new long_term_frame_idx member.
2012-10-31 10:45:14 +00:00
LongTermPicNumF =
GST_VAAPI_PICTURE_IS_LONG_TERM_REFERENCE(ref_list[j]) ?
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
ref_list[j]->long_term_pic_num : INT_MAX;
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
if (LongTermPicNumF != l->value.long_term_pic_num ||
ref_list[j]->base.view_id != picture->base.view_id)
ref_list[n++] = ref_list[j];
}
}
/* H.8.2.2.3 - Inter-view prediction reference pictures */
else if ((GST_VAAPI_PICTURE_IS_MVC(picture) &&
sps->extension_type == GST_H264_NAL_EXTENSION_MVC) &&
(l->modification_of_pic_nums_idc == 4 ||
l->modification_of_pic_nums_idc == 5)) {
gint32 abs_diff_view_idx = l->value.abs_diff_view_idx_minus1 + 1;
gint32 picViewIdx, targetViewId;
// (H-6)
if (l->modification_of_pic_nums_idc == 4) {
picViewIdx = picViewIdxPred - abs_diff_view_idx;
if (picViewIdx < 0)
picViewIdx += num_view_ids;
}
// (H-7)
else {
picViewIdx = picViewIdxPred + abs_diff_view_idx;
if (picViewIdx >= num_view_ids)
picViewIdx -= num_view_ids;
}
picViewIdxPred = picViewIdx;
// (H-8, H-9)
targetViewId = view_ids[picViewIdx];
// (H-10)
for (j = num_refs; j > ref_list_idx; j--)
ref_list[j] = ref_list[j - 1];
ref_list[ref_list_idx++] =
find_inter_view_reference(decoder, targetViewId);
n = ref_list_idx;
for (j = ref_list_idx; j <= num_refs; j++) {
if (!ref_list[j])
continue;
if (ref_list[j]->base.view_id != targetViewId ||
ref_list[j]->base.poc != picture->base.poc)
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
ref_list[n++] = ref_list[j];
}
}
}
#if DEBUG
for (i = 0; i < num_refs; i++)
if (!ref_list[i])
GST_ERROR("list %u entry %u is empty", list, i);
#endif
*ref_list_count_ptr = num_refs;
}
/* 8.2.4.3 - Modification process for reference picture lists */
static void
exec_picture_refs_modification(
GstVaapiDecoderH264 *decoder,
GstVaapiPictureH264 *picture,
GstH264SliceHdr *slice_hdr
)
{
GST_DEBUG("execute ref_pic_list_modification()");
/* RefPicList0 */
if (!GST_H264_IS_I_SLICE(slice_hdr) && !GST_H264_IS_SI_SLICE(slice_hdr) &&
slice_hdr->ref_pic_list_modification_flag_l0)
exec_picture_refs_modification_1(decoder, picture, slice_hdr, 0);
/* RefPicList1 */
if (GST_H264_IS_B_SLICE(slice_hdr) &&
slice_hdr->ref_pic_list_modification_flag_l1)
exec_picture_refs_modification_1(decoder, picture, slice_hdr, 1);
}
h264: simplify reference picture marking process. ... to build the short_ref[] and long_ref[] lists from the DPB, instead of maintaining them separately. This avoids refs/unrefs while making it possible to generate the list based on the actual picture structure. This also ensures that the list of generated ReferenceFrames[] actually matches what reference frames are available in the DPB. i.e. short_ref[] and long_ref[] entries are implied from the DPB, so there is no risk of having "dangling" references.
2012-10-31 13:24:09 +00:00
static void
decoder: h264: minor clean-ups. Make init_picture_ref_lists() more consistent with other functions related to the reference marking process by supplying the current picture as argument.
2014-05-02 20:40:16 +00:00
init_picture_ref_lists(GstVaapiDecoderH264 *decoder,
GstVaapiPictureH264 *picture)
h264: simplify reference picture marking process. ... to build the short_ref[] and long_ref[] lists from the DPB, instead of maintaining them separately. This avoids refs/unrefs while making it possible to generate the list based on the actual picture structure. This also ensures that the list of generated ReferenceFrames[] actually matches what reference frames are available in the DPB. i.e. short_ref[] and long_ref[] entries are implied from the DPB, so there is no risk of having "dangling" references.
2012-10-31 13:24:09 +00:00
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
h264: add initial support for interlaced streams. Decoded frames are only output when they are complete, i.e. when both fields are decoded. This also means that the "interlaced" caps is not propagated to vaapipostproc or vaapisink elements. Another limitation is that interlaced bitstreams with MMCO are unlikely to work.
2012-11-13 16:14:39 +00:00
guint i, j, short_ref_count, long_ref_count;
h264: simplify reference picture marking process. ... to build the short_ref[] and long_ref[] lists from the DPB, instead of maintaining them separately. This avoids refs/unrefs while making it possible to generate the list based on the actual picture structure. This also ensures that the list of generated ReferenceFrames[] actually matches what reference frames are available in the DPB. i.e. short_ref[] and long_ref[] entries are implied from the DPB, so there is no risk of having "dangling" references.
2012-10-31 13:24:09 +00:00
short_ref_count = 0;
long_ref_count = 0;
decoder: h264: minor clean-ups. Make init_picture_ref_lists() more consistent with other functions related to the reference marking process by supplying the current picture as argument.
2014-05-02 20:40:16 +00:00
if (GST_VAAPI_PICTURE_IS_FRAME(picture)) {
h264: introduce new frame store structure. The frame store represents a Decoded Picture Buffer entry, which can hold up to two fields. So far, the frame store is only used to hold full frames.
2012-11-02 14:14:58 +00:00
for (i = 0; i < priv->dpb_count; i++) {
GstVaapiFrameStore * const fs = priv->dpb[i];
decoder: h264: minor clean-ups. Make init_picture_ref_lists() more consistent with other functions related to the reference marking process by supplying the current picture as argument.
2014-05-02 20:40:16 +00:00
GstVaapiPictureH264 *pic;
h264: introduce new frame store structure. The frame store represents a Decoded Picture Buffer entry, which can hold up to two fields. So far, the frame store is only used to hold full frames.
2012-11-02 14:14:58 +00:00
if (!gst_vaapi_frame_store_has_frame(fs))
continue;
decoder: h264: minor clean-ups. Make init_picture_ref_lists() more consistent with other functions related to the reference marking process by supplying the current picture as argument.
2014-05-02 20:40:16 +00:00
pic = fs->buffers[0];
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
if (pic->base.view_id != picture->base.view_id)
continue;
decoder: h264: minor clean-ups. Make init_picture_ref_lists() more consistent with other functions related to the reference marking process by supplying the current picture as argument.
2014-05-02 20:40:16 +00:00
if (GST_VAAPI_PICTURE_IS_SHORT_TERM_REFERENCE(pic))
priv->short_ref[short_ref_count++] = pic;
else if (GST_VAAPI_PICTURE_IS_LONG_TERM_REFERENCE(pic))
priv->long_ref[long_ref_count++] = pic;
pic->structure = GST_VAAPI_PICTURE_STRUCTURE_FRAME;
pic->other_field = fs->buffers[1];
h264: add initial support for interlaced streams. Decoded frames are only output when they are complete, i.e. when both fields are decoded. This also means that the "interlaced" caps is not propagated to vaapipostproc or vaapisink elements. Another limitation is that interlaced bitstreams with MMCO are unlikely to work.
2012-11-13 16:14:39 +00:00
}
}
else {
for (i = 0; i < priv->dpb_count; i++) {
GstVaapiFrameStore * const fs = priv->dpb[i];
for (j = 0; j < fs->num_buffers; j++) {
decoder: h264: minor clean-ups. Make init_picture_ref_lists() more consistent with other functions related to the reference marking process by supplying the current picture as argument.
2014-05-02 20:40:16 +00:00
GstVaapiPictureH264 * const pic = fs->buffers[j];
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
if (pic->base.view_id != picture->base.view_id)
continue;
decoder: h264: minor clean-ups. Make init_picture_ref_lists() more consistent with other functions related to the reference marking process by supplying the current picture as argument.
2014-05-02 20:40:16 +00:00
if (GST_VAAPI_PICTURE_IS_SHORT_TERM_REFERENCE(pic))
priv->short_ref[short_ref_count++] = pic;
else if (GST_VAAPI_PICTURE_IS_LONG_TERM_REFERENCE(pic))
priv->long_ref[long_ref_count++] = pic;
pic->structure = pic->base.structure;
pic->other_field = fs->buffers[j ^ 1];
h264: add initial support for interlaced streams. Decoded frames are only output when they are complete, i.e. when both fields are decoded. This also means that the "interlaced" caps is not propagated to vaapipostproc or vaapisink elements. Another limitation is that interlaced bitstreams with MMCO are unlikely to work.
2012-11-13 16:14:39 +00:00
}
h264: introduce new frame store structure. The frame store represents a Decoded Picture Buffer entry, which can hold up to two fields. So far, the frame store is only used to hold full frames.
2012-11-02 14:14:58 +00:00
}
h264: simplify reference picture marking process. ... to build the short_ref[] and long_ref[] lists from the DPB, instead of maintaining them separately. This avoids refs/unrefs while making it possible to generate the list based on the actual picture structure. This also ensures that the list of generated ReferenceFrames[] actually matches what reference frames are available in the DPB. i.e. short_ref[] and long_ref[] entries are implied from the DPB, so there is no risk of having "dangling" references.
2012-10-31 13:24:09 +00:00
}
for (i = short_ref_count; i < priv->short_ref_count; i++)
priv->short_ref[i] = NULL;
priv->short_ref_count = short_ref_count;
for (i = long_ref_count; i < priv->long_ref_count; i++)
priv->long_ref[i] = NULL;
priv->long_ref_count = long_ref_count;
}
h264: minor clean-ups. Move DPB flush up if the current picture to decode is an IDR. Besides, don't bother to check for IDR pictures in dpb_add() function since an explicit DPB flush was already performed in this case.
2012-10-31 15:37:14 +00:00
static void
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
init_picture_refs(
GstVaapiDecoderH264 *decoder,
GstVaapiPictureH264 *picture,
GstH264SliceHdr *slice_hdr
)
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
guint i, num_refs;
decoder: h264: minor clean-ups. Make init_picture_ref_lists() more consistent with other functions related to the reference marking process by supplying the current picture as argument.
2014-05-02 20:40:16 +00:00
init_picture_ref_lists(decoder, picture);
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
init_picture_refs_pic_num(decoder, picture, slice_hdr);
priv->RefPicList0_count = 0;
priv->RefPicList1_count = 0;
decoder: h264: fix initialization of RefPicLists for multiple slices. The initialization of reference picture lists (8.2.4.2) applies to all slices. So, the RefPicList0/1 lists need to be constructed prior to each slice submission to the HW decoder. This fixes decoding of video sequences where frames are encoded with multiple slices of different types, e.g. 4 slices in this order I, P, I, and P. More precisely, CABAST3_Sony_E and CABASTBR3_Sony_B. https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-26 18:21:46 +00:00
switch (slice_hdr->type % 5) {
case GST_H264_P_SLICE:
case GST_H264_SP_SLICE:
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
init_picture_refs_p_slice(decoder, picture, slice_hdr);
break;
decoder: h264: fix initialization of RefPicLists for multiple slices. The initialization of reference picture lists (8.2.4.2) applies to all slices. So, the RefPicList0/1 lists need to be constructed prior to each slice submission to the HW decoder. This fixes decoding of video sequences where frames are encoded with multiple slices of different types, e.g. 4 slices in this order I, P, I, and P. More precisely, CABAST3_Sony_E and CABASTBR3_Sony_B. https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-26 18:21:46 +00:00
case GST_H264_B_SLICE:
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
init_picture_refs_b_slice(decoder, picture, slice_hdr);
break;
default:
break;
}
exec_picture_refs_modification(decoder, picture, slice_hdr);
decoder: h264: fix initialization of RefPicLists for multiple slices. The initialization of reference picture lists (8.2.4.2) applies to all slices. So, the RefPicList0/1 lists need to be constructed prior to each slice submission to the HW decoder. This fixes decoding of video sequences where frames are encoded with multiple slices of different types, e.g. 4 slices in this order I, P, I, and P. More precisely, CABAST3_Sony_E and CABASTBR3_Sony_B. https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-26 18:21:46 +00:00
switch (slice_hdr->type % 5) {
case GST_H264_B_SLICE:
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
num_refs = 1 + slice_hdr->num_ref_idx_l1_active_minus1;
for (i = priv->RefPicList1_count; i < num_refs; i++)
priv->RefPicList1[i] = NULL;
priv->RefPicList1_count = num_refs;
// fall-through
decoder: h264: fix initialization of RefPicLists for multiple slices. The initialization of reference picture lists (8.2.4.2) applies to all slices. So, the RefPicList0/1 lists need to be constructed prior to each slice submission to the HW decoder. This fixes decoding of video sequences where frames are encoded with multiple slices of different types, e.g. 4 slices in this order I, P, I, and P. More precisely, CABAST3_Sony_E and CABASTBR3_Sony_B. https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-26 18:21:46 +00:00
case GST_H264_P_SLICE:
case GST_H264_SP_SLICE:
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
num_refs = 1 + slice_hdr->num_ref_idx_l0_active_minus1;
for (i = priv->RefPicList0_count; i < num_refs; i++)
priv->RefPicList0[i] = NULL;
priv->RefPicList0_count = num_refs;
break;
default:
break;
}
}
static gboolean
init_picture(
GstVaapiDecoderH264 *decoder,
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
GstVaapiPictureH264 *picture, GstVaapiParserInfoH264 *pi)
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
GstVaapiPicture * const base_picture = &picture->base;
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
GstH264SliceHdr * const slice_hdr = &pi->data.slice_hdr;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
h264: simplify code when MMCO is 5. Remove exit_picture() and exit_picture_poc() since PicOrderCnt(CurrPic) is now updated accordingly to the standard. Besides, MMCO = 5 specific operations are moved up to exec_ref_pic_marking_adaptive_mmco_5().
2012-10-23 08:33:50 +00:00
priv->prev_frame_num = priv->frame_num;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
priv->frame_num = slice_hdr->frame_num;
picture->frame_num = priv->frame_num;
h264: execute reference picture marking process (sliding window).
2012-01-18 12:38:34 +00:00
picture->frame_num_wrap = priv->frame_num;
h264: handle Decoded Picture Buffer (DPB).
2012-01-24 14:38:14 +00:00
picture->output_flag = TRUE; /* XXX: conformant to Annex A only */
h264: initial port to new GstVaapiDecoder API
2012-12-06 13:02:17 +00:00
base_picture->pts = GST_VAAPI_DECODER_CODEC_FRAME(decoder)->pts;
decoder: h264: fix initialization of RefPicLists for multiple slices. The initialization of reference picture lists (8.2.4.2) applies to all slices. So, the RefPicList0/1 lists need to be constructed prior to each slice submission to the HW decoder. This fixes decoding of video sequences where frames are encoded with multiple slices of different types, e.g. 4 slices in this order I, P, I, and P. More precisely, CABAST3_Sony_E and CABASTBR3_Sony_B. https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-26 18:21:46 +00:00
base_picture->type = GST_VAAPI_PICTURE_TYPE_NONE;
decoder: h264: compute view ids only once per slice. Optimize lookups of view ids / view order indices by caching the result of the calculatiosn right into the GstVaapiParserInfoH264 struct. This terribly simplifies is_new_access_unit() and find_first_field() functions.
2014-05-22 08:04:46 +00:00
base_picture->view_id = pi->view_id;
base_picture->voc = pi->voc;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
/* Initialize extensions */
switch (pi->nalu.extension_type) {
case GST_H264_NAL_EXTENSION_MVC: {
GstH264NalUnitExtensionMVC * const mvc = &pi->nalu.extension.mvc;
GST_VAAPI_PICTURE_FLAG_SET(picture, GST_VAAPI_PICTURE_FLAG_MVC);
if (mvc->inter_view_flag)
GST_VAAPI_PICTURE_FLAG_SET(picture,
GST_VAAPI_PICTURE_FLAG_INTER_VIEW);
if (mvc->anchor_pic_flag)
GST_VAAPI_PICTURE_FLAG_SET(picture,
GST_VAAPI_PICTURE_FLAG_ANCHOR);
break;
}
}
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
/* Reset decoder state for IDR pictures */
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
if (pi->nalu.idr_pic_flag) {
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
GST_DEBUG("<IDR>");
h264: introduce GST_VAAPI_PICTURE_FLAG_IDR flag.
2012-10-31 10:33:40 +00:00
GST_VAAPI_PICTURE_FLAG_SET(picture, GST_VAAPI_PICTURE_FLAG_IDR);
decoder: h264: simplify storage of decoded picture into DPB. Factor out process by which the decoded picture with the lowest POC is found, and possibly output. Likewise, the storage and marking of a reference decoded, or non-reference decoded picture, into the DPB could also be simplified as they mostly share the same operations.
2014-05-04 12:49:28 +00:00
dpb_flush(decoder, picture);
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
}
h264: fill in GstVaapiPicture structure. ... and get rid of local VAPictureH264.flags fields in GstVaapiPictureH264.
2012-10-31 09:56:15 +00:00
/* Initialize picture structure */
decoder: h264: fix detection of top-field-first (TFF) flag. Use the SEI pic_timing() message to track the pic_struct variable when present, or infer it from the regular slice header flags field_pic_flag and bottom_field_flag. This fixes temporal sequence ordering when the output pictures are to be displayed. https://bugzilla.gnome.org/show_bug.cgi?id=739291
2014-11-13 14:00:21 +00:00
if (slice_hdr->field_pic_flag) {
h264: add initial support for interlaced streams. Decoded frames are only output when they are complete, i.e. when both fields are decoded. This also means that the "interlaced" caps is not propagated to vaapipostproc or vaapisink elements. Another limitation is that interlaced bitstreams with MMCO are unlikely to work.
2012-11-13 16:14:39 +00:00
GST_VAAPI_PICTURE_FLAG_SET(picture, GST_VAAPI_PICTURE_FLAG_INTERLACED);
decoder: h264: fix detection of top-field-first (TFF) flag. Use the SEI pic_timing() message to track the pic_struct variable when present, or infer it from the regular slice header flags field_pic_flag and bottom_field_flag. This fixes temporal sequence ordering when the output pictures are to be displayed. https://bugzilla.gnome.org/show_bug.cgi?id=739291
2014-11-13 14:00:21 +00:00
priv->pic_structure = slice_hdr->bottom_field_flag ?
GST_H264_SEI_PIC_STRUCT_BOTTOM_FIELD :
GST_H264_SEI_PIC_STRUCT_TOP_FIELD;
}
base_picture->structure = GST_VAAPI_PICTURE_STRUCTURE_FRAME;
switch (priv->pic_structure) {
case GST_H264_SEI_PIC_STRUCT_TOP_FIELD:
base_picture->structure = GST_VAAPI_PICTURE_STRUCTURE_TOP_FIELD;
if (GST_VAAPI_PICTURE_IS_FIRST_FIELD(picture))
GST_VAAPI_PICTURE_FLAG_SET(picture, GST_VAAPI_PICTURE_FLAG_TFF);
break;
case GST_H264_SEI_PIC_STRUCT_BOTTOM_FIELD:
base_picture->structure = GST_VAAPI_PICTURE_STRUCTURE_BOTTOM_FIELD;
break;
case GST_H264_SEI_PIC_STRUCT_TOP_BOTTOM_TOP:
decoder: h264: add initial support for repeat-first-field (RFF) flag. Use the SEI pic_timing() message to track and propagate down the repeat first field (RFF) flag. This is only initial support as there is one other condition that could induce the RFF flag, which is not handled yet.
2014-11-13 14:13:24 +00:00
GST_VAAPI_PICTURE_FLAG_SET(picture, GST_VAAPI_PICTURE_FLAG_RFF);
// fall-through
decoder: h264: fix detection of top-field-first (TFF) flag. Use the SEI pic_timing() message to track the pic_struct variable when present, or infer it from the regular slice header flags field_pic_flag and bottom_field_flag. This fixes temporal sequence ordering when the output pictures are to be displayed. https://bugzilla.gnome.org/show_bug.cgi?id=739291
2014-11-13 14:00:21 +00:00
case GST_H264_SEI_PIC_STRUCT_TOP_BOTTOM:
if (GST_VAAPI_PICTURE_IS_FIRST_FIELD(picture))
GST_VAAPI_PICTURE_FLAG_SET(picture, GST_VAAPI_PICTURE_FLAG_TFF);
break;
decoder: h264: add initial support for repeat-first-field (RFF) flag. Use the SEI pic_timing() message to track and propagate down the repeat first field (RFF) flag. This is only initial support as there is one other condition that could induce the RFF flag, which is not handled yet.
2014-11-13 14:13:24 +00:00
case GST_H264_SEI_PIC_STRUCT_BOTTOM_TOP_BOTTOM:
GST_VAAPI_PICTURE_FLAG_SET(picture, GST_VAAPI_PICTURE_FLAG_RFF);
break;
h264: add initial support for interlaced streams. Decoded frames are only output when they are complete, i.e. when both fields are decoded. This also means that the "interlaced" caps is not propagated to vaapipostproc or vaapisink elements. Another limitation is that interlaced bitstreams with MMCO are unlikely to work.
2012-11-13 16:14:39 +00:00
}
h264: add picture structure for reference picture marking process. Introduce new `structure' field to the H.264 specific picture structure so that to simplify the reference picture marking process. That local picture structure is derived from the original picture structure, as defined by the syntax elements field_pic_flag and bottom_field_flag.
2012-11-13 13:04:31 +00:00
picture->structure = base_picture->structure;
h264: fill in GstVaapiPicture structure. ... and get rid of local VAPictureH264.flags fields in GstVaapiPictureH264.
2012-10-31 09:56:15 +00:00
h264: minor clean-ups. Move DPB flush up if the current picture to decode is an IDR. Besides, don't bother to check for IDR pictures in dpb_add() function since an explicit DPB flush was already performed in this case.
2012-10-31 15:37:14 +00:00
/* Initialize reference flags */
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
if (pi->nalu.ref_idc) {
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
GstH264DecRefPicMarking * const dec_ref_pic_marking =
&slice_hdr->dec_ref_pic_marking;
h264: introduce GST_VAAPI_PICTURE_{SHORT,LONG}_TERM_REFERENCE flags. Further get rid of GstVaapiPictureH264-local VAPictureH264.flags for reference bits, thus simplifying the reference picture marking process to only track a single set of reference flags. Also introduce a new long_term_frame_idx member.
2012-10-31 10:45:14 +00:00
if (GST_VAAPI_PICTURE_IS_IDR(picture) &&
dec_ref_pic_marking->long_term_reference_flag)
GST_VAAPI_PICTURE_FLAG_SET(picture,
GST_VAAPI_PICTURE_FLAG_LONG_TERM_REFERENCE);
else
GST_VAAPI_PICTURE_FLAG_SET(picture,
GST_VAAPI_PICTURE_FLAG_SHORT_TERM_REFERENCE);
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
}
init_picture_poc(decoder, picture, slice_hdr);
return TRUE;
}
h264: execute reference picture marking process (sliding window).
2012-01-18 12:38:34 +00:00
/* 8.2.5.3 - Sliding window decoded reference picture marking process */
static gboolean
exec_ref_pic_marking_sliding_window(GstVaapiDecoderH264 *decoder)
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
decoder: h264: fix activation of picture and sequence parameters. At the time the first VCL NAL unit of a primary coded picture is found, and if that NAL unit was parsed to be an SPS or PPS, then the entries in the parser may have been overriden. This means that, when the picture is to be decoded, slice_hdr->pps could point to an invalid (the next) PPS entry. So, one way to solve this problem is to not use the parser PPS and SPS info but rather maintain our own activation chain in the decoder. https://bugzilla.gnome.org/show_bug.cgi?id=724519 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 17:11:03 +00:00
GstH264SPS * const sps = get_sps(decoder);
h264: split remove_reference_at() into finer units. Split remove_reference_at() into a function that actually removes the specified entry from the short-term or long-term reference picture array, and a function that sets reference flags to the desired value, possibly zero. The latters marks the picture as "unused for reference".
2012-11-13 15:35:30 +00:00
GstVaapiPictureH264 *ref_picture;
guint i, m, max_num_ref_frames;
h264: execute reference picture marking process (sliding window).
2012-01-18 12:38:34 +00:00
GST_DEBUG("reference picture marking process (sliding window)");
h264: add initial support for interlaced streams. Decoded frames are only output when they are complete, i.e. when both fields are decoded. This also means that the "interlaced" caps is not propagated to vaapipostproc or vaapisink elements. Another limitation is that interlaced bitstreams with MMCO are unlikely to work.
2012-11-13 16:14:39 +00:00
if (!GST_VAAPI_PICTURE_IS_FIRST_FIELD(priv->current_picture))
return TRUE;
h264: execute reference picture marking process (sliding window).
2012-01-18 12:38:34 +00:00
max_num_ref_frames = sps->num_ref_frames;
if (max_num_ref_frames == 0)
max_num_ref_frames = 1;
h264: add initial support for interlaced streams. Decoded frames are only output when they are complete, i.e. when both fields are decoded. This also means that the "interlaced" caps is not propagated to vaapipostproc or vaapisink elements. Another limitation is that interlaced bitstreams with MMCO are unlikely to work.
2012-11-13 16:14:39 +00:00
if (!GST_VAAPI_PICTURE_IS_FRAME(priv->current_picture))
max_num_ref_frames <<= 1;
h264: execute reference picture marking process (sliding window).
2012-01-18 12:38:34 +00:00
if (priv->short_ref_count + priv->long_ref_count < max_num_ref_frames)
return TRUE;
if (priv->short_ref_count < 1)
return FALSE;
h264: split remove_reference_at() into finer units. Split remove_reference_at() into a function that actually removes the specified entry from the short-term or long-term reference picture array, and a function that sets reference flags to the desired value, possibly zero. The latters marks the picture as "unused for reference".
2012-11-13 15:35:30 +00:00
for (m = 0, i = 1; i < priv->short_ref_count; i++) {
GstVaapiPictureH264 * const picture = priv->short_ref[i];
if (picture->frame_num_wrap < priv->short_ref[m]->frame_num_wrap)
m = i;
h264: execute reference picture marking process (sliding window).
2012-01-18 12:38:34 +00:00
}
h264: split remove_reference_at() into finer units. Split remove_reference_at() into a function that actually removes the specified entry from the short-term or long-term reference picture array, and a function that sets reference flags to the desired value, possibly zero. The latters marks the picture as "unused for reference".
2012-11-13 15:35:30 +00:00
ref_picture = priv->short_ref[m];
h264: fix interlaced stream decoding with MMCO. Fix decoding of interlaced streams when adaptive_ref_pic_marking_mode_flag is equal to 1, i.e. when memory management control operations are used. In particular, when field_pic_flag is set to 0, the new reference flags shall be applied to both fields.
2012-11-14 09:27:12 +00:00
gst_vaapi_picture_h264_set_reference(ref_picture, 0, TRUE);
h264: split remove_reference_at() into finer units. Split remove_reference_at() into a function that actually removes the specified entry from the short-term or long-term reference picture array, and a function that sets reference flags to the desired value, possibly zero. The latters marks the picture as "unused for reference".
2012-11-13 15:35:30 +00:00
ARRAY_REMOVE_INDEX(priv->short_ref, m);
h264: add initial support for interlaced streams. Decoded frames are only output when they are complete, i.e. when both fields are decoded. This also means that the "interlaced" caps is not propagated to vaapipostproc or vaapisink elements. Another limitation is that interlaced bitstreams with MMCO are unlikely to work.
2012-11-13 16:14:39 +00:00
h264: fix interlaced stream decoding with MMCO. Fix decoding of interlaced streams when adaptive_ref_pic_marking_mode_flag is equal to 1, i.e. when memory management control operations are used. In particular, when field_pic_flag is set to 0, the new reference flags shall be applied to both fields.
2012-11-14 09:27:12 +00:00
/* Both fields need to be marked as "unused for reference", so
remove the other field from the short_ref[] list as well */
if (!GST_VAAPI_PICTURE_IS_FRAME(priv->current_picture) && ref_picture->other_field) {
h264: add initial support for interlaced streams. Decoded frames are only output when they are complete, i.e. when both fields are decoded. This also means that the "interlaced" caps is not propagated to vaapipostproc or vaapisink elements. Another limitation is that interlaced bitstreams with MMCO are unlikely to work.
2012-11-13 16:14:39 +00:00
for (i = 0; i < priv->short_ref_count; i++) {
if (priv->short_ref[i] == ref_picture->other_field) {
ARRAY_REMOVE_INDEX(priv->short_ref, i);
break;
}
}
}
h264: execute reference picture marking process (sliding window).
2012-01-18 12:38:34 +00:00
return TRUE;
}
h264: move MMCO handlers out of the loop (cosmetics). This change only splits each individual MMCO handler into several functions dedicated for each operation. This is needed to perform further work later on.
2012-10-22 08:50:29 +00:00
static inline gint32
get_picNumX(GstVaapiPictureH264 *picture, GstH264RefPicMarking *ref_pic_marking)
{
gint32 pic_num;
h264: fill in GstVaapiPicture structure. ... and get rid of local VAPictureH264.flags fields in GstVaapiPictureH264.
2012-10-31 09:56:15 +00:00
if (GST_VAAPI_PICTURE_IS_FRAME(picture))
h264: move MMCO handlers out of the loop (cosmetics). This change only splits each individual MMCO handler into several functions dedicated for each operation. This is needed to perform further work later on.
2012-10-22 08:50:29 +00:00
pic_num = picture->frame_num_wrap;
else
pic_num = 2 * picture->frame_num_wrap + 1;
pic_num -= ref_pic_marking->difference_of_pic_nums_minus1 + 1;
return pic_num;
}
h264: simplify reference picture marking process. ... to build the short_ref[] and long_ref[] lists from the DPB, instead of maintaining them separately. This avoids refs/unrefs while making it possible to generate the list based on the actual picture structure. This also ensures that the list of generated ReferenceFrames[] actually matches what reference frames are available in the DPB. i.e. short_ref[] and long_ref[] entries are implied from the DPB, so there is no risk of having "dangling" references.
2012-10-31 13:24:09 +00:00
/* 8.2.5.4.1. Mark short-term reference picture as "unused for reference" */
h264: move MMCO handlers out of the loop (cosmetics). This change only splits each individual MMCO handler into several functions dedicated for each operation. This is needed to perform further work later on.
2012-10-22 08:50:29 +00:00
static void
exec_ref_pic_marking_adaptive_mmco_1(
GstVaapiDecoderH264 *decoder,
GstVaapiPictureH264 *picture,
GstH264RefPicMarking *ref_pic_marking
)
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
h264: move MMCO handlers out of the loop (cosmetics). This change only splits each individual MMCO handler into several functions dedicated for each operation. This is needed to perform further work later on.
2012-10-22 08:50:29 +00:00
gint32 i, picNumX;
picNumX = get_picNumX(picture, ref_pic_marking);
i = find_short_term_reference(decoder, picNumX);
if (i < 0)
return;
h264: split remove_reference_at() into finer units. Split remove_reference_at() into a function that actually removes the specified entry from the short-term or long-term reference picture array, and a function that sets reference flags to the desired value, possibly zero. The latters marks the picture as "unused for reference".
2012-11-13 15:35:30 +00:00
h264: fix interlaced stream decoding with MMCO. Fix decoding of interlaced streams when adaptive_ref_pic_marking_mode_flag is equal to 1, i.e. when memory management control operations are used. In particular, when field_pic_flag is set to 0, the new reference flags shall be applied to both fields.
2012-11-14 09:27:12 +00:00
gst_vaapi_picture_h264_set_reference(priv->short_ref[i], 0,
GST_VAAPI_PICTURE_IS_FRAME(picture));
h264: split remove_reference_at() into finer units. Split remove_reference_at() into a function that actually removes the specified entry from the short-term or long-term reference picture array, and a function that sets reference flags to the desired value, possibly zero. The latters marks the picture as "unused for reference".
2012-11-13 15:35:30 +00:00
ARRAY_REMOVE_INDEX(priv->short_ref, i);
h264: move MMCO handlers out of the loop (cosmetics). This change only splits each individual MMCO handler into several functions dedicated for each operation. This is needed to perform further work later on.
2012-10-22 08:50:29 +00:00
}
/* 8.2.5.4.2. Mark long-term reference picture as "unused for reference" */
static void
exec_ref_pic_marking_adaptive_mmco_2(
GstVaapiDecoderH264 *decoder,
GstVaapiPictureH264 *picture,
GstH264RefPicMarking *ref_pic_marking
)
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
h264: move MMCO handlers out of the loop (cosmetics). This change only splits each individual MMCO handler into several functions dedicated for each operation. This is needed to perform further work later on.
2012-10-22 08:50:29 +00:00
gint32 i;
h264: fix MMCO-based reference picture marking process. Fix adaptive memory control decoded reference picture marking process implementation for operations 2 to 6, thus also fixing support for long-term reference pictures.
2012-10-22 09:52:13 +00:00
i = find_long_term_reference(decoder, ref_pic_marking->long_term_pic_num);
h264: move MMCO handlers out of the loop (cosmetics). This change only splits each individual MMCO handler into several functions dedicated for each operation. This is needed to perform further work later on.
2012-10-22 08:50:29 +00:00
if (i < 0)
return;
h264: split remove_reference_at() into finer units. Split remove_reference_at() into a function that actually removes the specified entry from the short-term or long-term reference picture array, and a function that sets reference flags to the desired value, possibly zero. The latters marks the picture as "unused for reference".
2012-11-13 15:35:30 +00:00
h264: fix interlaced stream decoding with MMCO. Fix decoding of interlaced streams when adaptive_ref_pic_marking_mode_flag is equal to 1, i.e. when memory management control operations are used. In particular, when field_pic_flag is set to 0, the new reference flags shall be applied to both fields.
2012-11-14 09:27:12 +00:00
gst_vaapi_picture_h264_set_reference(priv->long_ref[i], 0,
GST_VAAPI_PICTURE_IS_FRAME(picture));
h264: split remove_reference_at() into finer units. Split remove_reference_at() into a function that actually removes the specified entry from the short-term or long-term reference picture array, and a function that sets reference flags to the desired value, possibly zero. The latters marks the picture as "unused for reference".
2012-11-13 15:35:30 +00:00
ARRAY_REMOVE_INDEX(priv->long_ref, i);
h264: move MMCO handlers out of the loop (cosmetics). This change only splits each individual MMCO handler into several functions dedicated for each operation. This is needed to perform further work later on.
2012-10-22 08:50:29 +00:00
}
/* 8.2.5.4.3. Assign LongTermFrameIdx to a short-term reference picture */
static void
exec_ref_pic_marking_adaptive_mmco_3(
GstVaapiDecoderH264 *decoder,
GstVaapiPictureH264 *picture,
GstH264RefPicMarking *ref_pic_marking
)
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
decoder: h264: fix assignment of LongTermFrameIdx. Complement fix committed as e95a42e. The H.264 AVC standard has to say: if the field is part of a reference frame or a complementary reference field pair, and the other field of the same reference frame or complementary reference field pair is also marked as "used for long-term reference", the reference frame or complementary reference field pair is also marked as "used for long-term reference" and assigned LongTermFrameIdx equal to long_term_frame_idx. This fixes decoding of MR9_BT_B in strict mode. https://bugs.freedesktop.org/show_bug.cgi?id=64624 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-05-12 17:23:04 +00:00
GstVaapiPictureH264 *ref_picture, *other_field;
h264: move MMCO handlers out of the loop (cosmetics). This change only splits each individual MMCO handler into several functions dedicated for each operation. This is needed to perform further work later on.
2012-10-22 08:50:29 +00:00
gint32 i, picNumX;
h264: fix MMCO-based reference picture marking process. Fix adaptive memory control decoded reference picture marking process implementation for operations 2 to 6, thus also fixing support for long-term reference pictures.
2012-10-22 09:52:13 +00:00
for (i = 0; i < priv->long_ref_count; i++) {
h264: introduce GST_VAAPI_PICTURE_{SHORT,LONG}_TERM_REFERENCE flags. Further get rid of GstVaapiPictureH264-local VAPictureH264.flags for reference bits, thus simplifying the reference picture marking process to only track a single set of reference flags. Also introduce a new long_term_frame_idx member.
2012-10-31 10:45:14 +00:00
if (priv->long_ref[i]->long_term_frame_idx == ref_pic_marking->long_term_frame_idx)
h264: fix MMCO-based reference picture marking process. Fix adaptive memory control decoded reference picture marking process implementation for operations 2 to 6, thus also fixing support for long-term reference pictures.
2012-10-22 09:52:13 +00:00
break;
}
h264: split remove_reference_at() into finer units. Split remove_reference_at() into a function that actually removes the specified entry from the short-term or long-term reference picture array, and a function that sets reference flags to the desired value, possibly zero. The latters marks the picture as "unused for reference".
2012-11-13 15:35:30 +00:00
if (i != priv->long_ref_count) {
h264: fix interlaced stream decoding with MMCO. Fix decoding of interlaced streams when adaptive_ref_pic_marking_mode_flag is equal to 1, i.e. when memory management control operations are used. In particular, when field_pic_flag is set to 0, the new reference flags shall be applied to both fields.
2012-11-14 09:27:12 +00:00
gst_vaapi_picture_h264_set_reference(priv->long_ref[i], 0, TRUE);
h264: split remove_reference_at() into finer units. Split remove_reference_at() into a function that actually removes the specified entry from the short-term or long-term reference picture array, and a function that sets reference flags to the desired value, possibly zero. The latters marks the picture as "unused for reference".
2012-11-13 15:35:30 +00:00
ARRAY_REMOVE_INDEX(priv->long_ref, i);
}
h264: fix MMCO-based reference picture marking process. Fix adaptive memory control decoded reference picture marking process implementation for operations 2 to 6, thus also fixing support for long-term reference pictures.
2012-10-22 09:52:13 +00:00
h264: move MMCO handlers out of the loop (cosmetics). This change only splits each individual MMCO handler into several functions dedicated for each operation. This is needed to perform further work later on.
2012-10-22 08:50:29 +00:00
picNumX = get_picNumX(picture, ref_pic_marking);
i = find_short_term_reference(decoder, picNumX);
if (i < 0)
return;
h264: fix MMCO-based reference picture marking process. Fix adaptive memory control decoded reference picture marking process implementation for operations 2 to 6, thus also fixing support for long-term reference pictures.
2012-10-22 09:52:13 +00:00
h264: split remove_reference_at() into finer units. Split remove_reference_at() into a function that actually removes the specified entry from the short-term or long-term reference picture array, and a function that sets reference flags to the desired value, possibly zero. The latters marks the picture as "unused for reference".
2012-11-13 15:35:30 +00:00
ref_picture = priv->short_ref[i];
ARRAY_REMOVE_INDEX(priv->short_ref, i);
priv->long_ref[priv->long_ref_count++] = ref_picture;
h264: fix MMCO-based reference picture marking process. Fix adaptive memory control decoded reference picture marking process implementation for operations 2 to 6, thus also fixing support for long-term reference pictures.
2012-10-22 09:52:13 +00:00
h264: split remove_reference_at() into finer units. Split remove_reference_at() into a function that actually removes the specified entry from the short-term or long-term reference picture array, and a function that sets reference flags to the desired value, possibly zero. The latters marks the picture as "unused for reference".
2012-11-13 15:35:30 +00:00
ref_picture->long_term_frame_idx = ref_pic_marking->long_term_frame_idx;
gst_vaapi_picture_h264_set_reference(ref_picture,
h264: fix interlaced stream decoding with MMCO. Fix decoding of interlaced streams when adaptive_ref_pic_marking_mode_flag is equal to 1, i.e. when memory management control operations are used. In particular, when field_pic_flag is set to 0, the new reference flags shall be applied to both fields.
2012-11-14 09:27:12 +00:00
GST_VAAPI_PICTURE_FLAG_LONG_TERM_REFERENCE,
decoder: h264: fix long-term reference picture marking process. Fix reference picture marking process with memory_management_control_op set to 3 and 6, i.e. assign LongTermFrameIdx to a short-term reference picture, or the current picture. This fixes decoding of FRExt_MMCO4_Sony_B. https://bugs.freedesktop.org/show_bug.cgi?id=64624 https://bugzilla.gnome.org/show_bug.cgi?id=724518 [squashed, edited to use GST_VAAPI_PICTURE_IS_COMPLETE() macro] Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2013-05-24 11:00:54 +00:00
GST_VAAPI_PICTURE_IS_COMPLETE(picture));
decoder: h264: fix assignment of LongTermFrameIdx. Complement fix committed as e95a42e. The H.264 AVC standard has to say: if the field is part of a reference frame or a complementary reference field pair, and the other field of the same reference frame or complementary reference field pair is also marked as "used for long-term reference", the reference frame or complementary reference field pair is also marked as "used for long-term reference" and assigned LongTermFrameIdx equal to long_term_frame_idx. This fixes decoding of MR9_BT_B in strict mode. https://bugs.freedesktop.org/show_bug.cgi?id=64624 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-05-12 17:23:04 +00:00
/* Assign LongTermFrameIdx to the other field if it was also
marked as "used for long-term reference */
other_field = ref_picture->other_field;
if (other_field && GST_VAAPI_PICTURE_IS_LONG_TERM_REFERENCE(other_field))
other_field->long_term_frame_idx = ref_pic_marking->long_term_frame_idx;
h264: fix MMCO-based reference picture marking process. Fix adaptive memory control decoded reference picture marking process implementation for operations 2 to 6, thus also fixing support for long-term reference pictures.
2012-10-22 09:52:13 +00:00
}
/* 8.2.5.4.4. Mark pictures with LongTermFramIdx > max_long_term_frame_idx
* as "unused for reference" */
static void
exec_ref_pic_marking_adaptive_mmco_4(
GstVaapiDecoderH264 *decoder,
GstVaapiPictureH264 *picture,
GstH264RefPicMarking *ref_pic_marking
)
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
h264: fix MMCO-based reference picture marking process. Fix adaptive memory control decoded reference picture marking process implementation for operations 2 to 6, thus also fixing support for long-term reference pictures.
2012-10-22 09:52:13 +00:00
gint32 i, long_term_frame_idx;
long_term_frame_idx = ref_pic_marking->max_long_term_frame_idx_plus1 - 1;
for (i = 0; i < priv->long_ref_count; i++) {
h264: introduce GST_VAAPI_PICTURE_{SHORT,LONG}_TERM_REFERENCE flags. Further get rid of GstVaapiPictureH264-local VAPictureH264.flags for reference bits, thus simplifying the reference picture marking process to only track a single set of reference flags. Also introduce a new long_term_frame_idx member.
2012-10-31 10:45:14 +00:00
if (priv->long_ref[i]->long_term_frame_idx <= long_term_frame_idx)
h264: fix MMCO-based reference picture marking process. Fix adaptive memory control decoded reference picture marking process implementation for operations 2 to 6, thus also fixing support for long-term reference pictures.
2012-10-22 09:52:13 +00:00
continue;
h264: fix interlaced stream decoding with MMCO. Fix decoding of interlaced streams when adaptive_ref_pic_marking_mode_flag is equal to 1, i.e. when memory management control operations are used. In particular, when field_pic_flag is set to 0, the new reference flags shall be applied to both fields.
2012-11-14 09:27:12 +00:00
gst_vaapi_picture_h264_set_reference(priv->long_ref[i], 0, FALSE);
h264: split remove_reference_at() into finer units. Split remove_reference_at() into a function that actually removes the specified entry from the short-term or long-term reference picture array, and a function that sets reference flags to the desired value, possibly zero. The latters marks the picture as "unused for reference".
2012-11-13 15:35:30 +00:00
ARRAY_REMOVE_INDEX(priv->long_ref, i);
h264: fix MMCO-based reference picture marking process. Fix adaptive memory control decoded reference picture marking process implementation for operations 2 to 6, thus also fixing support for long-term reference pictures.
2012-10-22 09:52:13 +00:00
i--;
}
h264: move MMCO handlers out of the loop (cosmetics). This change only splits each individual MMCO handler into several functions dedicated for each operation. This is needed to perform further work later on.
2012-10-22 08:50:29 +00:00
}
/* 8.2.5.4.5. Mark all reference pictures as "unused for reference" */
static void
exec_ref_pic_marking_adaptive_mmco_5(
GstVaapiDecoderH264 *decoder,
GstVaapiPictureH264 *picture,
GstH264RefPicMarking *ref_pic_marking
)
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
h264: move MMCO handlers out of the loop (cosmetics). This change only splits each individual MMCO handler into several functions dedicated for each operation. This is needed to perform further work later on.
2012-10-22 08:50:29 +00:00
decoder: h264: simplify storage of decoded picture into DPB. Factor out process by which the decoded picture with the lowest POC is found, and possibly output. Likewise, the storage and marking of a reference decoded, or non-reference decoded picture, into the DPB could also be simplified as they mostly share the same operations.
2014-05-04 12:49:28 +00:00
dpb_flush(decoder, picture);
h264: fix MMCO-based reference picture marking process. Fix adaptive memory control decoded reference picture marking process implementation for operations 2 to 6, thus also fixing support for long-term reference pictures.
2012-10-22 09:52:13 +00:00
h264: simplify code when MMCO is 5. Remove exit_picture() and exit_picture_poc() since PicOrderCnt(CurrPic) is now updated accordingly to the standard. Besides, MMCO = 5 specific operations are moved up to exec_ref_pic_marking_adaptive_mmco_5().
2012-10-23 08:33:50 +00:00
priv->prev_pic_has_mmco5 = TRUE;
h264: fix MMCO-based reference picture marking process. Fix adaptive memory control decoded reference picture marking process implementation for operations 2 to 6, thus also fixing support for long-term reference pictures.
2012-10-22 09:52:13 +00:00
/* The picture shall be inferred to have had frame_num equal to 0 (7.4.3) */
h264: simplify code when MMCO is 5. Remove exit_picture() and exit_picture_poc() since PicOrderCnt(CurrPic) is now updated accordingly to the standard. Besides, MMCO = 5 specific operations are moved up to exec_ref_pic_marking_adaptive_mmco_5().
2012-10-23 08:33:50 +00:00
priv->frame_num = 0;
priv->frame_num_offset = 0;
h264: fix MMCO-based reference picture marking process. Fix adaptive memory control decoded reference picture marking process implementation for operations 2 to 6, thus also fixing support for long-term reference pictures.
2012-10-22 09:52:13 +00:00
picture->frame_num = 0;
h264: simplify code when MMCO is 5. Remove exit_picture() and exit_picture_poc() since PicOrderCnt(CurrPic) is now updated accordingly to the standard. Besides, MMCO = 5 specific operations are moved up to exec_ref_pic_marking_adaptive_mmco_5().
2012-10-23 08:33:50 +00:00
/* Update TopFieldOrderCnt and BottomFieldOrderCnt (8.2.1) */
h264: add picture structure for reference picture marking process. Introduce new `structure' field to the H.264 specific picture structure so that to simplify the reference picture marking process. That local picture structure is derived from the original picture structure, as defined by the syntax elements field_pic_flag and bottom_field_flag.
2012-11-13 13:04:31 +00:00
if (picture->structure != GST_VAAPI_PICTURE_STRUCTURE_BOTTOM_FIELD)
h264: introduce per-field POC in GstVaapiPictureH264. Use the POC member available in the GstVaapiPicture base class and get rid of the dependency on the local VAPictureH264 TopFieldOrderCnt and BottomFieldOrderCnt. Rather, use a simple field_poc[] array initialized to INT_MAX, so that to simplify picture POC calculation for non frame pictures.
2012-10-31 10:52:03 +00:00
picture->field_poc[TOP_FIELD] -= picture->base.poc;
h264: add picture structure for reference picture marking process. Introduce new `structure' field to the H.264 specific picture structure so that to simplify the reference picture marking process. That local picture structure is derived from the original picture structure, as defined by the syntax elements field_pic_flag and bottom_field_flag.
2012-11-13 13:04:31 +00:00
if (picture->structure != GST_VAAPI_PICTURE_STRUCTURE_TOP_FIELD)
h264: introduce per-field POC in GstVaapiPictureH264. Use the POC member available in the GstVaapiPicture base class and get rid of the dependency on the local VAPictureH264 TopFieldOrderCnt and BottomFieldOrderCnt. Rather, use a simple field_poc[] array initialized to INT_MAX, so that to simplify picture POC calculation for non frame pictures.
2012-10-31 10:52:03 +00:00
picture->field_poc[BOTTOM_FIELD] -= picture->base.poc;
picture->base.poc = 0;
h264: fix MMCO-based reference picture marking process. Fix adaptive memory control decoded reference picture marking process implementation for operations 2 to 6, thus also fixing support for long-term reference pictures.
2012-10-22 09:52:13 +00:00
}
/* 8.2.5.4.6. Assign a long-term frame index to the current picture */
static void
exec_ref_pic_marking_adaptive_mmco_6(
GstVaapiDecoderH264 *decoder,
GstVaapiPictureH264 *picture,
GstH264RefPicMarking *ref_pic_marking
)
{
decoder: h264: fix long-term reference picture marking process. Fix reference picture marking process with memory_management_control_op set to 3 and 6, i.e. assign LongTermFrameIdx to a short-term reference picture, or the current picture. This fixes decoding of FRExt_MMCO4_Sony_B. https://bugs.freedesktop.org/show_bug.cgi?id=64624 https://bugzilla.gnome.org/show_bug.cgi?id=724518 [squashed, edited to use GST_VAAPI_PICTURE_IS_COMPLETE() macro] Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2013-05-24 11:00:54 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
decoder: h264: fix assignment of LongTermFrameIdx. Complement fix committed as e95a42e. The H.264 AVC standard has to say: if the field is part of a reference frame or a complementary reference field pair, and the other field of the same reference frame or complementary reference field pair is also marked as "used for long-term reference", the reference frame or complementary reference field pair is also marked as "used for long-term reference" and assigned LongTermFrameIdx equal to long_term_frame_idx. This fixes decoding of MR9_BT_B in strict mode. https://bugs.freedesktop.org/show_bug.cgi?id=64624 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-05-12 17:23:04 +00:00
GstVaapiPictureH264 *other_field;
decoder: h264: fix long-term reference picture marking process. Fix reference picture marking process with memory_management_control_op set to 3 and 6, i.e. assign LongTermFrameIdx to a short-term reference picture, or the current picture. This fixes decoding of FRExt_MMCO4_Sony_B. https://bugs.freedesktop.org/show_bug.cgi?id=64624 https://bugzilla.gnome.org/show_bug.cgi?id=724518 [squashed, edited to use GST_VAAPI_PICTURE_IS_COMPLETE() macro] Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2013-05-24 11:00:54 +00:00
guint i;
for (i = 0; i < priv->long_ref_count; i++) {
if (priv->long_ref[i]->long_term_frame_idx == ref_pic_marking->long_term_frame_idx)
break;
}
if (i != priv->long_ref_count) {
gst_vaapi_picture_h264_set_reference(priv->long_ref[i], 0, TRUE);
ARRAY_REMOVE_INDEX(priv->long_ref, i);
}
h264: introduce GST_VAAPI_PICTURE_{SHORT,LONG}_TERM_REFERENCE flags. Further get rid of GstVaapiPictureH264-local VAPictureH264.flags for reference bits, thus simplifying the reference picture marking process to only track a single set of reference flags. Also introduce a new long_term_frame_idx member.
2012-10-31 10:45:14 +00:00
picture->long_term_frame_idx = ref_pic_marking->long_term_frame_idx;
h264: split remove_reference_at() into finer units. Split remove_reference_at() into a function that actually removes the specified entry from the short-term or long-term reference picture array, and a function that sets reference flags to the desired value, possibly zero. The latters marks the picture as "unused for reference".
2012-11-13 15:35:30 +00:00
gst_vaapi_picture_h264_set_reference(picture,
decoder: h264: fix long-term reference picture marking process. Fix reference picture marking process with memory_management_control_op set to 3 and 6, i.e. assign LongTermFrameIdx to a short-term reference picture, or the current picture. This fixes decoding of FRExt_MMCO4_Sony_B. https://bugs.freedesktop.org/show_bug.cgi?id=64624 https://bugzilla.gnome.org/show_bug.cgi?id=724518 [squashed, edited to use GST_VAAPI_PICTURE_IS_COMPLETE() macro] Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2013-05-24 11:00:54 +00:00
GST_VAAPI_PICTURE_FLAG_LONG_TERM_REFERENCE,
GST_VAAPI_PICTURE_IS_COMPLETE(picture));
decoder: h264: fix assignment of LongTermFrameIdx. Complement fix committed as e95a42e. The H.264 AVC standard has to say: if the field is part of a reference frame or a complementary reference field pair, and the other field of the same reference frame or complementary reference field pair is also marked as "used for long-term reference", the reference frame or complementary reference field pair is also marked as "used for long-term reference" and assigned LongTermFrameIdx equal to long_term_frame_idx. This fixes decoding of MR9_BT_B in strict mode. https://bugs.freedesktop.org/show_bug.cgi?id=64624 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-05-12 17:23:04 +00:00
/* Assign LongTermFrameIdx to the other field if it was also
marked as "used for long-term reference */
decoder: h264: minor clean-ups. Make init_picture_ref_lists() more consistent with other functions related to the reference marking process by supplying the current picture as argument.
2014-05-02 20:40:16 +00:00
other_field = GST_VAAPI_PICTURE_H264(picture->base.parent_picture);
decoder: h264: fix assignment of LongTermFrameIdx. Complement fix committed as e95a42e. The H.264 AVC standard has to say: if the field is part of a reference frame or a complementary reference field pair, and the other field of the same reference frame or complementary reference field pair is also marked as "used for long-term reference", the reference frame or complementary reference field pair is also marked as "used for long-term reference" and assigned LongTermFrameIdx equal to long_term_frame_idx. This fixes decoding of MR9_BT_B in strict mode. https://bugs.freedesktop.org/show_bug.cgi?id=64624 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-05-12 17:23:04 +00:00
if (other_field && GST_VAAPI_PICTURE_IS_LONG_TERM_REFERENCE(other_field))
other_field->long_term_frame_idx = ref_pic_marking->long_term_frame_idx;
h264: move MMCO handlers out of the loop (cosmetics). This change only splits each individual MMCO handler into several functions dedicated for each operation. This is needed to perform further work later on.
2012-10-22 08:50:29 +00:00
}
h264: execute reference picture marking process (MMCO).
2012-01-23 14:03:07 +00:00
/* 8.2.5.4. Adaptive memory control decoded reference picture marking process */
h264: execute reference picture marking process (sliding window).
2012-01-18 12:38:34 +00:00
static gboolean
exec_ref_pic_marking_adaptive(
GstVaapiDecoderH264 *decoder,
h264: execute reference picture marking process (MMCO).
2012-01-23 14:03:07 +00:00
GstVaapiPictureH264 *picture,
h264: execute reference picture marking process (sliding window).
2012-01-18 12:38:34 +00:00
GstH264DecRefPicMarking *dec_ref_pic_marking
)
{
h264: execute reference picture marking process (MMCO).
2012-01-23 14:03:07 +00:00
guint i;
h264: execute reference picture marking process (sliding window).
2012-01-18 12:38:34 +00:00
GST_DEBUG("reference picture marking process (adaptive memory control)");
h264: move MMCO handlers out of the loop (cosmetics). This change only splits each individual MMCO handler into several functions dedicated for each operation. This is needed to perform further work later on.
2012-10-22 08:50:29 +00:00
typedef void (*exec_ref_pic_marking_adaptive_mmco_func)(
GstVaapiDecoderH264 *decoder,
GstVaapiPictureH264 *picture,
GstH264RefPicMarking *ref_pic_marking
);
static const exec_ref_pic_marking_adaptive_mmco_func mmco_funcs[] = {
NULL,
exec_ref_pic_marking_adaptive_mmco_1,
exec_ref_pic_marking_adaptive_mmco_2,
exec_ref_pic_marking_adaptive_mmco_3,
h264: fix MMCO-based reference picture marking process. Fix adaptive memory control decoded reference picture marking process implementation for operations 2 to 6, thus also fixing support for long-term reference pictures.
2012-10-22 09:52:13 +00:00
exec_ref_pic_marking_adaptive_mmco_4,
h264: move MMCO handlers out of the loop (cosmetics). This change only splits each individual MMCO handler into several functions dedicated for each operation. This is needed to perform further work later on.
2012-10-22 08:50:29 +00:00
exec_ref_pic_marking_adaptive_mmco_5,
h264: fix MMCO-based reference picture marking process. Fix adaptive memory control decoded reference picture marking process implementation for operations 2 to 6, thus also fixing support for long-term reference pictures.
2012-10-22 09:52:13 +00:00
exec_ref_pic_marking_adaptive_mmco_6,
h264: move MMCO handlers out of the loop (cosmetics). This change only splits each individual MMCO handler into several functions dedicated for each operation. This is needed to perform further work later on.
2012-10-22 08:50:29 +00:00
};
h264: execute reference picture marking process (MMCO).
2012-01-23 14:03:07 +00:00
for (i = 0; i < dec_ref_pic_marking->n_ref_pic_marking; i++) {
GstH264RefPicMarking * const ref_pic_marking =
&dec_ref_pic_marking->ref_pic_marking[i];
h264: move MMCO handlers out of the loop (cosmetics). This change only splits each individual MMCO handler into several functions dedicated for each operation. This is needed to perform further work later on.
2012-10-22 08:50:29 +00:00
const guint mmco = ref_pic_marking->memory_management_control_operation;
if (mmco < G_N_ELEMENTS(mmco_funcs) && mmco_funcs[mmco])
mmco_funcs[mmco](decoder, picture, ref_pic_marking);
else {
GST_ERROR("unhandled MMCO %u", mmco);
return FALSE;
h264: execute reference picture marking process (MMCO).
2012-01-23 14:03:07 +00:00
}
}
h264: execute reference picture marking process (sliding window).
2012-01-18 12:38:34 +00:00
return TRUE;
}
/* 8.2.5 - Execute reference picture marking process */
static gboolean
exec_ref_pic_marking(GstVaapiDecoderH264 *decoder, GstVaapiPictureH264 *picture)
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
h264: execute reference picture marking process (sliding window).
2012-01-18 12:38:34 +00:00
h264: simplify code when MMCO is 5. Remove exit_picture() and exit_picture_poc() since PicOrderCnt(CurrPic) is now updated accordingly to the standard. Besides, MMCO = 5 specific operations are moved up to exec_ref_pic_marking_adaptive_mmco_5().
2012-10-23 08:33:50 +00:00
priv->prev_pic_has_mmco5 = FALSE;
h264: add picture structure for reference picture marking process. Introduce new `structure' field to the H.264 specific picture structure so that to simplify the reference picture marking process. That local picture structure is derived from the original picture structure, as defined by the syntax elements field_pic_flag and bottom_field_flag.
2012-11-13 13:04:31 +00:00
priv->prev_pic_structure = picture->structure;
h264: simplify code when MMCO is 5. Remove exit_picture() and exit_picture_poc() since PicOrderCnt(CurrPic) is now updated accordingly to the standard. Besides, MMCO = 5 specific operations are moved up to exec_ref_pic_marking_adaptive_mmco_5().
2012-10-23 08:33:50 +00:00
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
if (GST_VAAPI_PICTURE_IS_INTER_VIEW(picture))
g_ptr_array_add(priv->inter_views, gst_vaapi_picture_ref(picture));
h264: execute reference picture marking process (sliding window).
2012-01-18 12:38:34 +00:00
if (!GST_VAAPI_PICTURE_IS_REFERENCE(picture))
return TRUE;
h264: introduce GST_VAAPI_PICTURE_FLAG_IDR flag.
2012-10-31 10:33:40 +00:00
if (!GST_VAAPI_PICTURE_IS_IDR(picture)) {
h264: execute reference picture marking process (sliding window).
2012-01-18 12:38:34 +00:00
GstH264DecRefPicMarking * const dec_ref_pic_marking =
h264: drop GstVaapiSliceH264 object. Use standard GstVaapiSlice object from now on since we already have parsed and recorded the slice headers (GstH264SliceHdr decode units).
2012-12-07 16:45:03 +00:00
&picture->last_slice_hdr->dec_ref_pic_marking;
h264: execute reference picture marking process (sliding window).
2012-01-18 12:38:34 +00:00
if (dec_ref_pic_marking->adaptive_ref_pic_marking_mode_flag) {
h264: execute reference picture marking process (MMCO).
2012-01-23 14:03:07 +00:00
if (!exec_ref_pic_marking_adaptive(decoder, picture, dec_ref_pic_marking))
h264: execute reference picture marking process (sliding window).
2012-01-18 12:38:34 +00:00
return FALSE;
}
else {
if (!exec_ref_pic_marking_sliding_window(decoder))
return FALSE;
}
}
return TRUE;
}
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
static void
vaapi_init_picture(VAPictureH264 *pic)
{
pic->picture_id = VA_INVALID_ID;
pic->frame_idx = 0;
pic->flags = VA_PICTURE_H264_INVALID;
pic->TopFieldOrderCnt = 0;
pic->BottomFieldOrderCnt = 0;
}
h264: add vaapi_fill_picture() helper. Add vaapi_fill_picture() helper function to convert GstVaapiPictureH264 to VAPictureH264 structure. This is preparatory work to get rid of the local VAPictureH264 member in GstVaapiPictureH264.
2012-10-31 10:07:48 +00:00
static void
h264: fix VAPictureParameterBufferH264.ReferenceFrames[] construction. ... for interlaced streams. The short_ref[] and long_ref[] arrays may contain up to 32 fields but VA ReferenceFrames[] array expects up to 16 reference frames, thus including both fields.
2012-11-14 13:25:34 +00:00
vaapi_fill_picture(VAPictureH264 *pic, GstVaapiPictureH264 *picture,
guint picture_structure)
h264: add vaapi_fill_picture() helper. Add vaapi_fill_picture() helper function to convert GstVaapiPictureH264 to VAPictureH264 structure. This is preparatory work to get rid of the local VAPictureH264 member in GstVaapiPictureH264.
2012-10-31 10:07:48 +00:00
{
h264: fix VAPictureParameterBufferH264.ReferenceFrames[] construction. ... for interlaced streams. The short_ref[] and long_ref[] arrays may contain up to 32 fields but VA ReferenceFrames[] array expects up to 16 reference frames, thus including both fields.
2012-11-14 13:25:34 +00:00
if (!picture_structure)
picture_structure = picture->structure;
h264: add vaapi_fill_picture() helper. Add vaapi_fill_picture() helper function to convert GstVaapiPictureH264 to VAPictureH264 structure. This is preparatory work to get rid of the local VAPictureH264 member in GstVaapiPictureH264.
2012-10-31 10:07:48 +00:00
pic->picture_id = picture->base.surface_id;
pic->flags = 0;
h264: introduce GST_VAAPI_PICTURE_{SHORT,LONG}_TERM_REFERENCE flags. Further get rid of GstVaapiPictureH264-local VAPictureH264.flags for reference bits, thus simplifying the reference picture marking process to only track a single set of reference flags. Also introduce a new long_term_frame_idx member.
2012-10-31 10:45:14 +00:00
if (GST_VAAPI_PICTURE_IS_LONG_TERM_REFERENCE(picture)) {
h264: add vaapi_fill_picture() helper. Add vaapi_fill_picture() helper function to convert GstVaapiPictureH264 to VAPictureH264 structure. This is preparatory work to get rid of the local VAPictureH264 member in GstVaapiPictureH264.
2012-10-31 10:07:48 +00:00
pic->flags |= VA_PICTURE_H264_LONG_TERM_REFERENCE;
h264: introduce GST_VAAPI_PICTURE_{SHORT,LONG}_TERM_REFERENCE flags. Further get rid of GstVaapiPictureH264-local VAPictureH264.flags for reference bits, thus simplifying the reference picture marking process to only track a single set of reference flags. Also introduce a new long_term_frame_idx member.
2012-10-31 10:45:14 +00:00
pic->frame_idx = picture->long_term_frame_idx;
h264: add vaapi_fill_picture() helper. Add vaapi_fill_picture() helper function to convert GstVaapiPictureH264 to VAPictureH264 structure. This is preparatory work to get rid of the local VAPictureH264 member in GstVaapiPictureH264.
2012-10-31 10:07:48 +00:00
}
else {
h264: introduce GST_VAAPI_PICTURE_{SHORT,LONG}_TERM_REFERENCE flags. Further get rid of GstVaapiPictureH264-local VAPictureH264.flags for reference bits, thus simplifying the reference picture marking process to only track a single set of reference flags. Also introduce a new long_term_frame_idx member.
2012-10-31 10:45:14 +00:00
if (GST_VAAPI_PICTURE_IS_SHORT_TERM_REFERENCE(picture))
h264: add vaapi_fill_picture() helper. Add vaapi_fill_picture() helper function to convert GstVaapiPictureH264 to VAPictureH264 structure. This is preparatory work to get rid of the local VAPictureH264 member in GstVaapiPictureH264.
2012-10-31 10:07:48 +00:00
pic->flags |= VA_PICTURE_H264_SHORT_TERM_REFERENCE;
pic->frame_idx = picture->frame_num;
}
h264: fix VAPictureParameterBufferH264.ReferenceFrames[] construction. ... for interlaced streams. The short_ref[] and long_ref[] arrays may contain up to 32 fields but VA ReferenceFrames[] array expects up to 16 reference frames, thus including both fields.
2012-11-14 13:25:34 +00:00
switch (picture_structure) {
h264: fill in GstVaapiPicture structure. ... and get rid of local VAPictureH264.flags fields in GstVaapiPictureH264.
2012-10-31 09:56:15 +00:00
case GST_VAAPI_PICTURE_STRUCTURE_FRAME:
h264: introduce per-field POC in GstVaapiPictureH264. Use the POC member available in the GstVaapiPicture base class and get rid of the dependency on the local VAPictureH264 TopFieldOrderCnt and BottomFieldOrderCnt. Rather, use a simple field_poc[] array initialized to INT_MAX, so that to simplify picture POC calculation for non frame pictures.
2012-10-31 10:52:03 +00:00
pic->TopFieldOrderCnt = picture->field_poc[TOP_FIELD];
pic->BottomFieldOrderCnt = picture->field_poc[BOTTOM_FIELD];
h264: add vaapi_fill_picture() helper. Add vaapi_fill_picture() helper function to convert GstVaapiPictureH264 to VAPictureH264 structure. This is preparatory work to get rid of the local VAPictureH264 member in GstVaapiPictureH264.
2012-10-31 10:07:48 +00:00
break;
h264: fill in GstVaapiPicture structure. ... and get rid of local VAPictureH264.flags fields in GstVaapiPictureH264.
2012-10-31 09:56:15 +00:00
case GST_VAAPI_PICTURE_STRUCTURE_TOP_FIELD:
h264: add vaapi_fill_picture() helper. Add vaapi_fill_picture() helper function to convert GstVaapiPictureH264 to VAPictureH264 structure. This is preparatory work to get rid of the local VAPictureH264 member in GstVaapiPictureH264.
2012-10-31 10:07:48 +00:00
pic->flags |= VA_PICTURE_H264_TOP_FIELD;
h264: introduce per-field POC in GstVaapiPictureH264. Use the POC member available in the GstVaapiPicture base class and get rid of the dependency on the local VAPictureH264 TopFieldOrderCnt and BottomFieldOrderCnt. Rather, use a simple field_poc[] array initialized to INT_MAX, so that to simplify picture POC calculation for non frame pictures.
2012-10-31 10:52:03 +00:00
pic->TopFieldOrderCnt = picture->field_poc[TOP_FIELD];
h264: add vaapi_fill_picture() helper. Add vaapi_fill_picture() helper function to convert GstVaapiPictureH264 to VAPictureH264 structure. This is preparatory work to get rid of the local VAPictureH264 member in GstVaapiPictureH264.
2012-10-31 10:07:48 +00:00
pic->BottomFieldOrderCnt = 0;
break;
h264: fill in GstVaapiPicture structure. ... and get rid of local VAPictureH264.flags fields in GstVaapiPictureH264.
2012-10-31 09:56:15 +00:00
case GST_VAAPI_PICTURE_STRUCTURE_BOTTOM_FIELD:
h264: add vaapi_fill_picture() helper. Add vaapi_fill_picture() helper function to convert GstVaapiPictureH264 to VAPictureH264 structure. This is preparatory work to get rid of the local VAPictureH264 member in GstVaapiPictureH264.
2012-10-31 10:07:48 +00:00
pic->flags |= VA_PICTURE_H264_BOTTOM_FIELD;
h264: introduce per-field POC in GstVaapiPictureH264. Use the POC member available in the GstVaapiPicture base class and get rid of the dependency on the local VAPictureH264 TopFieldOrderCnt and BottomFieldOrderCnt. Rather, use a simple field_poc[] array initialized to INT_MAX, so that to simplify picture POC calculation for non frame pictures.
2012-10-31 10:52:03 +00:00
pic->BottomFieldOrderCnt = picture->field_poc[BOTTOM_FIELD];
h264: add vaapi_fill_picture() helper. Add vaapi_fill_picture() helper function to convert GstVaapiPictureH264 to VAPictureH264 structure. This is preparatory work to get rid of the local VAPictureH264 member in GstVaapiPictureH264.
2012-10-31 10:07:48 +00:00
pic->TopFieldOrderCnt = 0;
break;
}
}
decoder: h264: fix MVC inter-view prediction process. The inter-view reference components and inter-view only reference components that are included in the reference picture lists shall be considered as not being marked as "used for short-term reference" or "used for long-term reference". This means that reference flags should all be removed from VAPictureH264.flags. This fixes decoding of MVCNV-2.264.
2014-06-04 17:10:44 +00:00
static void
vaapi_fill_picture_for_RefPicListX(VAPictureH264 *pic,
GstVaapiPictureH264 *picture)
{
vaapi_fill_picture(pic, picture, 0);
/* H.8.4 - MVC inter prediction and inter-view prediction process */
if (GST_VAAPI_PICTURE_IS_INTER_VIEW(picture)) {
/* The inter-view reference components and inter-view only
reference components that are included in the reference
picture lists are considered as not being marked as "used for
short-term reference" or "used for long-term reference" */
pic->flags &= ~(VA_PICTURE_H264_SHORT_TERM_REFERENCE|
VA_PICTURE_H264_LONG_TERM_REFERENCE);
}
}
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
static gboolean
decoder: h264: fix activation of picture and sequence parameters. At the time the first VCL NAL unit of a primary coded picture is found, and if that NAL unit was parsed to be an SPS or PPS, then the entries in the parser may have been overriden. This means that, when the picture is to be decoded, slice_hdr->pps could point to an invalid (the next) PPS entry. So, one way to solve this problem is to not use the parser PPS and SPS info but rather maintain our own activation chain in the decoder. https://bugzilla.gnome.org/show_bug.cgi?id=724519 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 17:11:03 +00:00
fill_picture(GstVaapiDecoderH264 *decoder, GstVaapiPictureH264 *picture)
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
GstVaapiPicture * const base_picture = &picture->base;
decoder: h264: fix activation of picture and sequence parameters. At the time the first VCL NAL unit of a primary coded picture is found, and if that NAL unit was parsed to be an SPS or PPS, then the entries in the parser may have been overriden. This means that, when the picture is to be decoded, slice_hdr->pps could point to an invalid (the next) PPS entry. So, one way to solve this problem is to not use the parser PPS and SPS info but rather maintain our own activation chain in the decoder. https://bugzilla.gnome.org/show_bug.cgi?id=724519 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 17:11:03 +00:00
GstH264PPS * const pps = get_pps(decoder);
GstH264SPS * const sps = get_sps(decoder);
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
VAPictureParameterBufferH264 * const pic_param = base_picture->param;
guint i, n;
/* Fill in VAPictureParameterBufferH264 */
h264: fix VAPictureParameterBufferH264.ReferenceFrames[] construction. ... for interlaced streams. The short_ref[] and long_ref[] arrays may contain up to 32 fields but VA ReferenceFrames[] array expects up to 16 reference frames, thus including both fields.
2012-11-14 13:25:34 +00:00
vaapi_fill_picture(&pic_param->CurrPic, picture, 0);
for (i = 0, n = 0; i < priv->dpb_count; i++) {
GstVaapiFrameStore * const fs = priv->dpb[i];
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
if ((gst_vaapi_frame_store_has_reference(fs) &&
fs->view_id == picture->base.view_id) ||
decoder: h264: reduce ReferenceFrames entries to the essential set. When decoding Multiview High profile streams with a large number of views, it is not possible to make the VAPictureParameterBufferH264. ReferenceFrames[] array hold the complete DPB, with all possibly active pictures to be used for inter-view prediction in the current access unit. So reduce the scope of the ReferenceFrames[] array to only include the set of reference pictures that are going to be used for decoding the current picture. Basically, this is a union of all RefPicListX[] array, for all slices constituting the decoded picture.
2014-06-03 15:36:38 +00:00
(gst_vaapi_frame_store_has_inter_view(fs) &&
is_inter_view_reference_for_picture(decoder, fs->view_id, picture)))
h264: fix VAPictureParameterBufferH264.ReferenceFrames[] construction. ... for interlaced streams. The short_ref[] and long_ref[] arrays may contain up to 32 fields but VA ReferenceFrames[] array expects up to 16 reference frames, thus including both fields.
2012-11-14 13:25:34 +00:00
vaapi_fill_picture(&pic_param->ReferenceFrames[n++],
fs->buffers[0], fs->structure);
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
if (n >= G_N_ELEMENTS(pic_param->ReferenceFrames))
break;
h264: fix VAPictureParameterBufferH264.ReferenceFrames[] construction. ... for interlaced streams. The short_ref[] and long_ref[] arrays may contain up to 32 fields but VA ReferenceFrames[] array expects up to 16 reference frames, thus including both fields.
2012-11-14 13:25:34 +00:00
}
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
for (; n < G_N_ELEMENTS(pic_param->ReferenceFrames); n++)
vaapi_init_picture(&pic_param->ReferenceFrames[n]);
#define COPY_FIELD(s, f) \
pic_param->f = (s)->f
#define COPY_BFM(a, s, f) \
pic_param->a.bits.f = (s)->f
h264: fix picture size in macroblocks. The picture size signalled by sps->{width,height} is the actual size with cropping applied, not the original size derived from pic_width_in_mbs_minus1 and pic_height_in_map_units_minus1. VA driver expects that original size, uncropped. There is another issue pending: frame cropping information needs to be taken care of.
2012-11-16 17:00:10 +00:00
pic_param->picture_width_in_mbs_minus1 = priv->mb_width - 1;
pic_param->picture_height_in_mbs_minus1 = priv->mb_height - 1;
h264: fix activation order of picture and sequence parameters. Delay ensure_context() until we actually need a VA context for allocating new VA surfaces, and then GstVaapiPictures, but also when a real activation of a new picture parameter set occurs, thus also implying an activation of the related sequence parameter set. The most important thing was to drop the global pps and sps pointers since they may not have matched the currently activated picture parameter or sequence parameter sets at the specified decode point. Anoter positive side-effect is that this cleans up all occurrences of decode_current_picture() to only keep those useful in decode_picture(), before a new picture is allocated, or in decode_sequence_end() when an end-of-stream or end-of-sequence condition occurred.
2012-10-26 14:12:05 +00:00
pic_param->frame_num = priv->frame_num;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
COPY_FIELD(sps, bit_depth_luma_minus8);
COPY_FIELD(sps, bit_depth_chroma_minus8);
COPY_FIELD(sps, num_ref_frames);
COPY_FIELD(pps, num_slice_groups_minus1);
COPY_FIELD(pps, slice_group_map_type);
COPY_FIELD(pps, slice_group_change_rate_minus1);
COPY_FIELD(pps, pic_init_qp_minus26);
COPY_FIELD(pps, pic_init_qs_minus26);
COPY_FIELD(pps, chroma_qp_index_offset);
COPY_FIELD(pps, second_chroma_qp_index_offset);
pic_param->seq_fields.value = 0; /* reset all bits */
pic_param->seq_fields.bits.residual_colour_transform_flag = sps->separate_colour_plane_flag;
pic_param->seq_fields.bits.MinLumaBiPredSize8x8 = sps->level_idc >= 31; /* A.3.3.2 */
COPY_BFM(seq_fields, sps, chroma_format_idc);
COPY_BFM(seq_fields, sps, gaps_in_frame_num_value_allowed_flag);
COPY_BFM(seq_fields, sps, frame_mbs_only_flag);
COPY_BFM(seq_fields, sps, mb_adaptive_frame_field_flag);
COPY_BFM(seq_fields, sps, direct_8x8_inference_flag);
COPY_BFM(seq_fields, sps, log2_max_frame_num_minus4);
COPY_BFM(seq_fields, sps, pic_order_cnt_type);
COPY_BFM(seq_fields, sps, log2_max_pic_order_cnt_lsb_minus4);
COPY_BFM(seq_fields, sps, delta_pic_order_always_zero_flag);
pic_param->pic_fields.value = 0; /* reset all bits */
decoder: h264: fix activation of picture and sequence parameters. At the time the first VCL NAL unit of a primary coded picture is found, and if that NAL unit was parsed to be an SPS or PPS, then the entries in the parser may have been overriden. This means that, when the picture is to be decoded, slice_hdr->pps could point to an invalid (the next) PPS entry. So, one way to solve this problem is to not use the parser PPS and SPS info but rather maintain our own activation chain in the decoder. https://bugzilla.gnome.org/show_bug.cgi?id=724519 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 17:11:03 +00:00
pic_param->pic_fields.bits.field_pic_flag = GST_VAAPI_PICTURE_IS_INTERLACED(picture);
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
pic_param->pic_fields.bits.reference_pic_flag = GST_VAAPI_PICTURE_IS_REFERENCE(picture);
COPY_BFM(pic_fields, pps, entropy_coding_mode_flag);
COPY_BFM(pic_fields, pps, weighted_pred_flag);
COPY_BFM(pic_fields, pps, weighted_bipred_idc);
COPY_BFM(pic_fields, pps, transform_8x8_mode_flag);
COPY_BFM(pic_fields, pps, constrained_intra_pred_flag);
COPY_BFM(pic_fields, pps, pic_order_present_flag);
COPY_BFM(pic_fields, pps, deblocking_filter_control_present_flag);
COPY_BFM(pic_fields, pps, redundant_pic_cnt_present_flag);
return TRUE;
}
h264: fix detection of picture boundaries. Strictly follow the standard (7.4.1.2.4) to detect the first VCL NAL unit of a primary coded picture.
2012-10-24 16:23:09 +00:00
/* Detection of the first VCL NAL unit of a primary coded picture (7.4.1.2.4) */
static gboolean
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
is_new_picture(GstVaapiParserInfoH264 *pi, GstVaapiParserInfoH264 *prev_pi)
h264: fix detection of picture boundaries. Strictly follow the standard (7.4.1.2.4) to detect the first VCL NAL unit of a primary coded picture.
2012-10-24 16:23:09 +00:00
{
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
GstH264SliceHdr * const slice_hdr = &pi->data.slice_hdr;
h264: fix detection of picture boundaries. Strictly follow the standard (7.4.1.2.4) to detect the first VCL NAL unit of a primary coded picture.
2012-10-24 16:23:09 +00:00
GstH264PPS * const pps = slice_hdr->pps;
GstH264SPS * const sps = pps->sequence;
GstH264SliceHdr *prev_slice_hdr;
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
if (!prev_pi)
h264: fix detection of picture boundaries. Strictly follow the standard (7.4.1.2.4) to detect the first VCL NAL unit of a primary coded picture.
2012-10-24 16:23:09 +00:00
return TRUE;
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
prev_slice_hdr = &prev_pi->data.slice_hdr;
h264: fix detection of picture boundaries. Strictly follow the standard (7.4.1.2.4) to detect the first VCL NAL unit of a primary coded picture.
2012-10-24 16:23:09 +00:00
#define CHECK_EXPR(expr, field_name) do { \
if (!(expr)) { \
GST_DEBUG(field_name " differs in value"); \
return TRUE; \
} \
} while (0)
#define CHECK_VALUE(new_slice_hdr, old_slice_hdr, field) \
CHECK_EXPR(((new_slice_hdr)->field == (old_slice_hdr)->field), #field)
decoder: h264: detect the first VCL NAL unit of a picture for MVC. Detect the first VCL NAL unit of a picture for MVC, based on the view_id as per H.7.4.1.2.4. Note that we only need to detect new view components. Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2013-03-13 09:44:38 +00:00
/* view_id differs in value and VOIdx of current slice_hdr is less
than the VOIdx of the prev_slice_hdr */
decoder: h264: compute view ids only once per slice. Optimize lookups of view ids / view order indices by caching the result of the calculatiosn right into the GstVaapiParserInfoH264 struct. This terribly simplifies is_new_access_unit() and find_first_field() functions.
2014-05-22 08:04:46 +00:00
CHECK_VALUE(pi, prev_pi, view_id);
decoder: h264: detect the first VCL NAL unit of a picture for MVC. Detect the first VCL NAL unit of a picture for MVC, based on the view_id as per H.7.4.1.2.4. Note that we only need to detect new view components. Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2013-03-13 09:44:38 +00:00
h264: fix detection of picture boundaries. Strictly follow the standard (7.4.1.2.4) to detect the first VCL NAL unit of a primary coded picture.
2012-10-24 16:23:09 +00:00
/* frame_num differs in value, regardless of inferred values to 0 */
CHECK_VALUE(slice_hdr, prev_slice_hdr, frame_num);
/* pic_parameter_set_id differs in value */
CHECK_VALUE(slice_hdr, prev_slice_hdr, pps);
/* field_pic_flag differs in value */
CHECK_VALUE(slice_hdr, prev_slice_hdr, field_pic_flag);
/* bottom_field_flag is present in both and differs in value */
if (slice_hdr->field_pic_flag && prev_slice_hdr->field_pic_flag)
CHECK_VALUE(slice_hdr, prev_slice_hdr, bottom_field_flag);
/* nal_ref_idc differs in value with one of the nal_ref_idc values is 0 */
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
CHECK_EXPR((pi->nalu.ref_idc != 0) ==
(prev_pi->nalu.ref_idc != 0), "nal_ref_idc");
h264: fix detection of picture boundaries. Strictly follow the standard (7.4.1.2.4) to detect the first VCL NAL unit of a primary coded picture.
2012-10-24 16:23:09 +00:00
/* POC type is 0 for both and either pic_order_cnt_lsb differs in
value or delta_pic_order_cnt_bottom differs in value */
if (sps->pic_order_cnt_type == 0) {
CHECK_VALUE(slice_hdr, prev_slice_hdr, pic_order_cnt_lsb);
if (pps->pic_order_present_flag && !slice_hdr->field_pic_flag)
CHECK_VALUE(slice_hdr, prev_slice_hdr, delta_pic_order_cnt_bottom);
}
/* POC type is 1 for both and either delta_pic_order_cnt[0]
differs in value or delta_pic_order_cnt[1] differs in value */
else if (sps->pic_order_cnt_type == 1) {
CHECK_VALUE(slice_hdr, prev_slice_hdr, delta_pic_order_cnt[0]);
CHECK_VALUE(slice_hdr, prev_slice_hdr, delta_pic_order_cnt[1]);
}
/* IdrPicFlag differs in value */
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
CHECK_VALUE(&pi->nalu, &prev_pi->nalu, idr_pic_flag);
h264: fix detection of picture boundaries. Strictly follow the standard (7.4.1.2.4) to detect the first VCL NAL unit of a primary coded picture.
2012-10-24 16:23:09 +00:00
/* IdrPicFlag is equal to 1 for both and idr_pic_id differs in value */
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
if (pi->nalu.idr_pic_flag)
h264: fix detection of picture boundaries. Strictly follow the standard (7.4.1.2.4) to detect the first VCL NAL unit of a primary coded picture.
2012-10-24 16:23:09 +00:00
CHECK_VALUE(slice_hdr, prev_slice_hdr, idr_pic_id);
#undef CHECK_EXPR
#undef CHECK_VALUE
return FALSE;
}
decoder: h264: fix detection of access unit boundaries. In order to have a stricter conforming implementation, we need to carefully detect access unit boundaries. Additional operations could be necessary to perform at those boundaries.
2014-05-01 17:33:40 +00:00
/* Detection of a new access unit, assuming we are already in presence
of a new picture */
decoder: h264: compute view ids only once per slice. Optimize lookups of view ids / view order indices by caching the result of the calculatiosn right into the GstVaapiParserInfoH264 struct. This terribly simplifies is_new_access_unit() and find_first_field() functions.
2014-05-22 08:04:46 +00:00
static inline gboolean
decoder: h264: fix detection of access unit boundaries. In order to have a stricter conforming implementation, we need to carefully detect access unit boundaries. Additional operations could be necessary to perform at those boundaries.
2014-05-01 17:33:40 +00:00
is_new_access_unit(GstVaapiParserInfoH264 *pi, GstVaapiParserInfoH264 *prev_pi)
{
decoder: h264: compute view ids only once per slice. Optimize lookups of view ids / view order indices by caching the result of the calculatiosn right into the GstVaapiParserInfoH264 struct. This terribly simplifies is_new_access_unit() and find_first_field() functions.
2014-05-22 08:04:46 +00:00
if (!prev_pi || prev_pi->view_id == pi->view_id)
decoder: h264: fix detection of access unit boundaries. In order to have a stricter conforming implementation, we need to carefully detect access unit boundaries. Additional operations could be necessary to perform at those boundaries.
2014-05-01 17:33:40 +00:00
return TRUE;
decoder: h264: compute view ids only once per slice. Optimize lookups of view ids / view order indices by caching the result of the calculatiosn right into the GstVaapiParserInfoH264 struct. This terribly simplifies is_new_access_unit() and find_first_field() functions.
2014-05-22 08:04:46 +00:00
return pi->voc < prev_pi->voc;
decoder: h264: fix detection of access unit boundaries. In order to have a stricter conforming implementation, we need to carefully detect access unit boundaries. Additional operations could be necessary to perform at those boundaries.
2014-05-01 17:33:40 +00:00
}
decoder: h264: add support for MVC interlaced streams. Fix support for MVC Stereo High profile streams with interlaced frames. Also improve the detection logic of the first field.
2014-05-21 15:57:00 +00:00
/* Finds the first field picture corresponding to the supplied picture */
static GstVaapiPictureH264 *
decoder: h264: compute view ids only once per slice. Optimize lookups of view ids / view order indices by caching the result of the calculatiosn right into the GstVaapiParserInfoH264 struct. This terribly simplifies is_new_access_unit() and find_first_field() functions.
2014-05-22 08:04:46 +00:00
find_first_field(GstVaapiDecoderH264 *decoder, GstVaapiParserInfoH264 *pi)
decoder: h264: add support for MVC interlaced streams. Fix support for MVC Stereo High profile streams with interlaced frames. Also improve the detection logic of the first field.
2014-05-21 15:57:00 +00:00
{
GstVaapiDecoderH264Private * const priv = &decoder->priv;
decoder: h264: compute view ids only once per slice. Optimize lookups of view ids / view order indices by caching the result of the calculatiosn right into the GstVaapiParserInfoH264 struct. This terribly simplifies is_new_access_unit() and find_first_field() functions.
2014-05-22 08:04:46 +00:00
GstH264SliceHdr * const slice_hdr = &pi->data.slice_hdr;
decoder: h264: add support for MVC interlaced streams. Fix support for MVC Stereo High profile streams with interlaced frames. Also improve the detection logic of the first field.
2014-05-21 15:57:00 +00:00
GstVaapiFrameStore *fs;
if (!slice_hdr->field_pic_flag)
return NULL;
decoder: h264: compute view ids only once per slice. Optimize lookups of view ids / view order indices by caching the result of the calculatiosn right into the GstVaapiParserInfoH264 struct. This terribly simplifies is_new_access_unit() and find_first_field() functions.
2014-05-22 08:04:46 +00:00
fs = priv->prev_frames[pi->voc];
decoder: h264: add support for MVC interlaced streams. Fix support for MVC Stereo High profile streams with interlaced frames. Also improve the detection logic of the first field.
2014-05-21 15:57:00 +00:00
if (!fs || gst_vaapi_frame_store_has_frame(fs))
return NULL;
if (fs->buffers[0]->frame_num == slice_hdr->frame_num)
return fs->buffers[0];
return NULL;
}
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
static GstVaapiDecoderStatus
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
decode_picture(GstVaapiDecoderH264 *decoder, GstVaapiDecoderUnit *unit)
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
GstVaapiParserInfoH264 * const pi = unit->parsed_info;
GstH264SliceHdr * const slice_hdr = &pi->data.slice_hdr;
decoder: h264: fix activation of picture and sequence parameters. At the time the first VCL NAL unit of a primary coded picture is found, and if that NAL unit was parsed to be an SPS or PPS, then the entries in the parser may have been overriden. This means that, when the picture is to be decoded, slice_hdr->pps could point to an invalid (the next) PPS entry. So, one way to solve this problem is to not use the parser PPS and SPS info but rather maintain our own activation chain in the decoder. https://bugzilla.gnome.org/show_bug.cgi?id=724519 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 17:11:03 +00:00
GstH264PPS * const pps = ensure_pps(decoder, slice_hdr->pps);
GstH264SPS * const sps = ensure_sps(decoder, slice_hdr->pps->sequence);
decoder: h264: add support for MVC interlaced streams. Fix support for MVC Stereo High profile streams with interlaced frames. Also improve the detection logic of the first field.
2014-05-21 15:57:00 +00:00
GstVaapiPictureH264 *picture, *first_field;
h264: implement {start,end}_frame() hooks. Implement GstVaapiDecoder.start_frame() and end_frame() semantics so that to create new VA context earlier and submit VA pictures to the HW for decoding as soon as possible. i.e. don't wait for the next frame to start decoding the previous one.
2012-12-13 09:21:46 +00:00
GstVaapiDecoderStatus status;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
decoder: h264: fix activation of picture and sequence parameters. At the time the first VCL NAL unit of a primary coded picture is found, and if that NAL unit was parsed to be an SPS or PPS, then the entries in the parser may have been overriden. This means that, when the picture is to be decoded, slice_hdr->pps could point to an invalid (the next) PPS entry. So, one way to solve this problem is to not use the parser PPS and SPS info but rather maintain our own activation chain in the decoder. https://bugzilla.gnome.org/show_bug.cgi?id=724519 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 17:11:03 +00:00
g_return_val_if_fail(pps != NULL, GST_VAAPI_DECODER_STATUS_ERROR_UNKNOWN);
g_return_val_if_fail(sps != NULL, GST_VAAPI_DECODER_STATUS_ERROR_UNKNOWN);
decoder: h264: add support for MVC base views. Allow decoding for base views of MVC encoded streams. For now, just skip the slice extension and prefix NAL units, and skip non-base view frames. Signed-off-by: Xiaowei Li <xiaowei.a.li@intel.com> [fixed memory leak, improved check for MVC NAL units] Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2013-02-28 07:59:55 +00:00
/* Only decode base stream for MVC */
switch (sps->profile_idc) {
case GST_H264_PROFILE_MULTIVIEW_HIGH:
case GST_H264_PROFILE_STEREO_HIGH:
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
if (0) {
decoder: h264: add support for MVC base views. Allow decoding for base views of MVC encoded streams. For now, just skip the slice extension and prefix NAL units, and skip non-base view frames. Signed-off-by: Xiaowei Li <xiaowei.a.li@intel.com> [fixed memory leak, improved check for MVC NAL units] Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2013-02-28 07:59:55 +00:00
GST_DEBUG("drop picture from substream");
return GST_VAAPI_DECODER_STATUS_DROP_FRAME;
}
break;
}
h264: fix activation order of picture and sequence parameters. Delay ensure_context() until we actually need a VA context for allocating new VA surfaces, and then GstVaapiPictures, but also when a real activation of a new picture parameter set occurs, thus also implying an activation of the related sequence parameter set. The most important thing was to drop the global pps and sps pointers since they may not have matched the currently activated picture parameter or sequence parameter sets at the specified decode point. Anoter positive side-effect is that this cleans up all occurrences of decode_current_picture() to only keep those useful in decode_picture(), before a new picture is allocated, or in decode_sequence_end() when an end-of-stream or end-of-sequence condition occurred.
2012-10-26 14:12:05 +00:00
status = ensure_context(decoder, sps);
if (status != GST_VAAPI_DECODER_STATUS_SUCCESS)
return status;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
h264: improve robustness when packets are missing. Improve robustness when some expected packets where not received yet or that were not correctly decoded. For example, don't try to decode a picture if there was no valid frame headers parsed so far. https://bugs.freedesktop.org/show_bug.cgi?id=57902
2013-12-10 15:14:27 +00:00
priv->decoder_state = 0;
decoder: h264: compute view ids only once per slice. Optimize lookups of view ids / view order indices by caching the result of the calculatiosn right into the GstVaapiParserInfoH264 struct. This terribly simplifies is_new_access_unit() and find_first_field() functions.
2014-05-22 08:04:46 +00:00
first_field = find_first_field(decoder, pi);
decoder: h264: add support for MVC interlaced streams. Fix support for MVC Stereo High profile streams with interlaced frames. Also improve the detection logic of the first field.
2014-05-21 15:57:00 +00:00
if (first_field) {
h264: add initial support for interlaced streams. Decoded frames are only output when they are complete, i.e. when both fields are decoded. This also means that the "interlaced" caps is not propagated to vaapipostproc or vaapisink elements. Another limitation is that interlaced bitstreams with MMCO are unlikely to work.
2012-11-13 16:14:39 +00:00
/* Re-use current picture where the first field was decoded */
decoder: h264: add support for MVC interlaced streams. Fix support for MVC Stereo High profile streams with interlaced frames. Also improve the detection logic of the first field.
2014-05-21 15:57:00 +00:00
picture = gst_vaapi_picture_h264_new_field(first_field);
h264: add initial support for interlaced streams. Decoded frames are only output when they are complete, i.e. when both fields are decoded. This also means that the "interlaced" caps is not propagated to vaapipostproc or vaapisink elements. Another limitation is that interlaced bitstreams with MMCO are unlikely to work.
2012-11-13 16:14:39 +00:00
if (!picture) {
GST_ERROR("failed to allocate field picture");
return GST_VAAPI_DECODER_STATUS_ERROR_ALLOCATION_FAILED;
}
}
else {
/* Create new picture */
picture = gst_vaapi_picture_h264_new(decoder);
if (!picture) {
GST_ERROR("failed to allocate picture");
return GST_VAAPI_DECODER_STATUS_ERROR_ALLOCATION_FAILED;
}
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
}
h264: add initial support for interlaced streams. Decoded frames are only output when they are complete, i.e. when both fields are decoded. This also means that the "interlaced" caps is not propagated to vaapipostproc or vaapisink elements. Another limitation is that interlaced bitstreams with MMCO are unlikely to work.
2012-11-13 16:14:39 +00:00
gst_vaapi_picture_replace(&priv->current_picture, picture);
gst_vaapi_picture_unref(picture);
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
decoder: h264: add initial support for MVC. https://bugzilla.gnome.org/show_bug.cgi?id=721772
2014-05-02 12:58:45 +00:00
/* Clear inter-view references list if this is the primary coded
picture of the current access unit */
if (pi->flags & GST_VAAPI_DECODER_UNIT_FLAG_AU_START)
g_ptr_array_set_size(priv->inter_views, 0);
h264: add support for video cropping. If the encoded stream has the frame_cropping_flag set, then associate the cropping rectangle to GstVaapiPicture. Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2013-02-15 16:50:26 +00:00
/* Update cropping rectangle */
if (sps->frame_cropping_flag) {
GstVaapiRectangle crop_rect;
crop_rect.x = sps->crop_rect_x;
crop_rect.y = sps->crop_rect_y;
crop_rect.width = sps->crop_rect_width;
crop_rect.height = sps->crop_rect_height;
gst_vaapi_picture_set_crop_rect(&picture->base, &crop_rect);
}
h264: optimize handling of scaling lists. Don't copy scaling lists twice to an intermediate state. Rather, directly use the scaling lists from GstH264PPS since they would match those provided by SPS header, if necessary. i.e. if PPS-specific scaling lists are not available in the bitstream.
2012-10-23 12:50:14 +00:00
status = ensure_quant_matrix(decoder, picture);
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
if (status != GST_VAAPI_DECODER_STATUS_SUCCESS) {
h264: review and report errors accordingly. Use GST_ERROR() to report real errors instead of hiding them into GST_DEBUG().
2012-09-21 14:43:38 +00:00
GST_ERROR("failed to reset quantizer matrix");
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
return status;
}
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
if (!init_picture(decoder, picture, pi))
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
return GST_VAAPI_DECODER_STATUS_ERROR_UNKNOWN;
decoder: h264: fix activation of picture and sequence parameters. At the time the first VCL NAL unit of a primary coded picture is found, and if that NAL unit was parsed to be an SPS or PPS, then the entries in the parser may have been overriden. This means that, when the picture is to be decoded, slice_hdr->pps could point to an invalid (the next) PPS entry. So, one way to solve this problem is to not use the parser PPS and SPS info but rather maintain our own activation chain in the decoder. https://bugzilla.gnome.org/show_bug.cgi?id=724519 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 17:11:03 +00:00
if (!fill_picture(decoder, picture))
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
return GST_VAAPI_DECODER_STATUS_ERROR_UNKNOWN;
h264: improve robustness when packets are missing. Improve robustness when some expected packets where not received yet or that were not correctly decoded. For example, don't try to decode a picture if there was no valid frame headers parsed so far. https://bugs.freedesktop.org/show_bug.cgi?id=57902
2013-12-10 15:14:27 +00:00
decoder: h264: fix robustness patch for bytestream format. Fix parser and decoder state to sync at the right locations. This is because we could reset the parser state, while the decoder state was not copied yet, e.g. when parsing several NAL units from multiple frames whereas the current frame was not decoded yet. This is a regression brought in by commit 6fe5496.
2014-02-04 17:35:28 +00:00
priv->decoder_state = pi->state;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
return GST_VAAPI_DECODER_STATUS_SUCCESS;
}
h264: fix slice_data_bit_offset calculation. Unlike what VA-API documentation defines, the slice_data_bit_offset represents the offset to the first macroblock in the slice data, minus any emulation prevention bytes in the slice_header(). This fix copes with binary-only VA drivers that won't be fixed any time soon. Besides, this aligns with the current FFmpeg behaviour that was based on those proprietary drivers implementing the API incorrectly.
2012-03-02 12:41:16 +00:00
static inline guint
decoder: h264: fix slice data bit offset with MVC NAL units. When MVC slice NAL units (coded slice extension and prefix NAL) are present, the number of NAL header bytes is 3, not 1 as usual. Signed-off-by: Li Xiaowei <xiaowei.a.li@intel.com> Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2013-03-07 03:32:20 +00:00
get_slice_data_bit_offset(GstH264SliceHdr *slice_hdr, guint nal_header_bytes)
h264: fix slice_data_bit_offset calculation. Unlike what VA-API documentation defines, the slice_data_bit_offset represents the offset to the first macroblock in the slice data, minus any emulation prevention bytes in the slice_header(). This fix copes with binary-only VA drivers that won't be fixed any time soon. Besides, this aligns with the current FFmpeg behaviour that was based on those proprietary drivers implementing the API incorrectly.
2012-03-02 12:41:16 +00:00
{
guint epb_count;
epb_count = slice_hdr->n_emulation_prevention_bytes;
decoder: h264: fix slice data bit offset with MVC NAL units. When MVC slice NAL units (coded slice extension and prefix NAL) are present, the number of NAL header bytes is 3, not 1 as usual. Signed-off-by: Li Xiaowei <xiaowei.a.li@intel.com> Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2013-03-07 03:32:20 +00:00
return 8 * nal_header_bytes + slice_hdr->header_size - epb_count * 8;
h264: fix slice_data_bit_offset calculation. Unlike what VA-API documentation defines, the slice_data_bit_offset represents the offset to the first macroblock in the slice data, minus any emulation prevention bytes in the slice_header(). This fix copes with binary-only VA drivers that won't be fixed any time soon. Besides, this aligns with the current FFmpeg behaviour that was based on those proprietary drivers implementing the API incorrectly.
2012-03-02 12:41:16 +00:00
}
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
static gboolean
h264: drop GstVaapiSliceH264 object. Use standard GstVaapiSlice object from now on since we already have parsed and recorded the slice headers (GstH264SliceHdr decode units).
2012-12-07 16:45:03 +00:00
fill_pred_weight_table(GstVaapiDecoderH264 *decoder,
GstVaapiSlice *slice, GstH264SliceHdr *slice_hdr)
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
{
h264: drop GstVaapiSliceH264 object. Use standard GstVaapiSlice object from now on since we already have parsed and recorded the slice headers (GstH264SliceHdr decode units).
2012-12-07 16:45:03 +00:00
VASliceParameterBufferH264 * const slice_param = slice->param;
decoder: h264: fix activation of picture and sequence parameters. At the time the first VCL NAL unit of a primary coded picture is found, and if that NAL unit was parsed to be an SPS or PPS, then the entries in the parser may have been overriden. This means that, when the picture is to be decoded, slice_hdr->pps could point to an invalid (the next) PPS entry. So, one way to solve this problem is to not use the parser PPS and SPS info but rather maintain our own activation chain in the decoder. https://bugzilla.gnome.org/show_bug.cgi?id=724519 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 17:11:03 +00:00
GstH264PPS * const pps = get_pps(decoder);
GstH264SPS * const sps = get_sps(decoder);
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
GstH264PredWeightTable * const w = &slice_hdr->pred_weight_table;
h264: fix pred_weight_table() reconstruction. Only the explicit pred_weight_table(), possibly with the inferred default values, shall be required. e.g. don't fill in the table if weighted_pred_flag is not set for P/SP slices.
2012-01-24 08:20:25 +00:00
guint num_weight_tables = 0;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
gint i, j;
h264: fix pred_weight_table() reconstruction. Only the explicit pred_weight_table(), possibly with the inferred default values, shall be required. e.g. don't fill in the table if weighted_pred_flag is not set for P/SP slices.
2012-01-24 08:20:25 +00:00
if (pps->weighted_pred_flag &&
(GST_H264_IS_P_SLICE(slice_hdr) || GST_H264_IS_SP_SLICE(slice_hdr)))
num_weight_tables = 1;
else if (pps->weighted_bipred_idc == 1 && GST_H264_IS_B_SLICE(slice_hdr))
num_weight_tables = 2;
else
num_weight_tables = 0;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
h264: set {luma,chroma}_log2_weight_denom to 0 if no pred_weight_table(). Force luma_log2_weight_denom and chroma_log2_weight_denom to zero if there is no pred_weight_table() that was parsed. This is a workaround for the VA intel-driver on Ivy Bridge.
2013-02-08 10:56:54 +00:00
slice_param->luma_log2_weight_denom = 0;
slice_param->chroma_log2_weight_denom = 0;
h264: fix pred_weight_table() reconstruction. Only the explicit pred_weight_table(), possibly with the inferred default values, shall be required. e.g. don't fill in the table if weighted_pred_flag is not set for P/SP slices.
2012-01-24 08:20:25 +00:00
slice_param->luma_weight_l0_flag = 0;
slice_param->chroma_weight_l0_flag = 0;
slice_param->luma_weight_l1_flag = 0;
slice_param->chroma_weight_l1_flag = 0;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
h264: fix pred_weight_table() reconstruction. Only the explicit pred_weight_table(), possibly with the inferred default values, shall be required. e.g. don't fill in the table if weighted_pred_flag is not set for P/SP slices.
2012-01-24 08:20:25 +00:00
if (num_weight_tables < 1)
return TRUE;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
h264: set {luma,chroma}_log2_weight_denom to 0 if no pred_weight_table(). Force luma_log2_weight_denom and chroma_log2_weight_denom to zero if there is no pred_weight_table() that was parsed. This is a workaround for the VA intel-driver on Ivy Bridge.
2013-02-08 10:56:54 +00:00
slice_param->luma_log2_weight_denom = w->luma_log2_weight_denom;
slice_param->chroma_log2_weight_denom = w->chroma_log2_weight_denom;
h264: fix pred_weight_table() reconstruction. Only the explicit pred_weight_table(), possibly with the inferred default values, shall be required. e.g. don't fill in the table if weighted_pred_flag is not set for P/SP slices.
2012-01-24 08:20:25 +00:00
slice_param->luma_weight_l0_flag = 1;
for (i = 0; i <= slice_param->num_ref_idx_l0_active_minus1; i++) {
slice_param->luma_weight_l0[i] = w->luma_weight_l0[i];
slice_param->luma_offset_l0[i] = w->luma_offset_l0[i];
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
}
h264: fix pred_weight_table() reconstruction. Only the explicit pred_weight_table(), possibly with the inferred default values, shall be required. e.g. don't fill in the table if weighted_pred_flag is not set for P/SP slices.
2012-01-24 08:20:25 +00:00
slice_param->chroma_weight_l0_flag = sps->chroma_array_type != 0;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
if (slice_param->chroma_weight_l0_flag) {
for (i = 0; i <= slice_param->num_ref_idx_l0_active_minus1; i++) {
for (j = 0; j < 2; j++) {
slice_param->chroma_weight_l0[i][j] = w->chroma_weight_l0[i][j];
slice_param->chroma_offset_l0[i][j] = w->chroma_offset_l0[i][j];
}
}
}
h264: fix pred_weight_table() reconstruction. Only the explicit pred_weight_table(), possibly with the inferred default values, shall be required. e.g. don't fill in the table if weighted_pred_flag is not set for P/SP slices.
2012-01-24 08:20:25 +00:00
if (num_weight_tables < 2)
return TRUE;
slice_param->luma_weight_l1_flag = 1;
for (i = 0; i <= slice_param->num_ref_idx_l1_active_minus1; i++) {
slice_param->luma_weight_l1[i] = w->luma_weight_l1[i];
slice_param->luma_offset_l1[i] = w->luma_offset_l1[i];
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
}
h264: fix pred_weight_table() reconstruction. Only the explicit pred_weight_table(), possibly with the inferred default values, shall be required. e.g. don't fill in the table if weighted_pred_flag is not set for P/SP slices.
2012-01-24 08:20:25 +00:00
slice_param->chroma_weight_l1_flag = sps->chroma_array_type != 0;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
if (slice_param->chroma_weight_l1_flag) {
for (i = 0; i <= slice_param->num_ref_idx_l1_active_minus1; i++) {
for (j = 0; j < 2; j++) {
slice_param->chroma_weight_l1[i][j] = w->chroma_weight_l1[i][j];
slice_param->chroma_offset_l1[i][j] = w->chroma_offset_l1[i][j];
}
}
}
return TRUE;
}
h264: fix pred_weight_table() reconstruction. Only the explicit pred_weight_table(), possibly with the inferred default values, shall be required. e.g. don't fill in the table if weighted_pred_flag is not set for P/SP slices.
2012-01-24 08:20:25 +00:00
static gboolean
h264: drop GstVaapiSliceH264 object. Use standard GstVaapiSlice object from now on since we already have parsed and recorded the slice headers (GstH264SliceHdr decode units).
2012-12-07 16:45:03 +00:00
fill_RefPicList(GstVaapiDecoderH264 *decoder,
GstVaapiSlice *slice, GstH264SliceHdr *slice_hdr)
h264: fix pred_weight_table() reconstruction. Only the explicit pred_weight_table(), possibly with the inferred default values, shall be required. e.g. don't fill in the table if weighted_pred_flag is not set for P/SP slices.
2012-01-24 08:20:25 +00:00
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
h264: drop GstVaapiSliceH264 object. Use standard GstVaapiSlice object from now on since we already have parsed and recorded the slice headers (GstH264SliceHdr decode units).
2012-12-07 16:45:03 +00:00
VASliceParameterBufferH264 * const slice_param = slice->param;
h264: simplify RefPicList reconstruction.
2012-01-27 16:28:50 +00:00
guint i, num_ref_lists = 0;
h264: fix pred_weight_table() reconstruction. Only the explicit pred_weight_table(), possibly with the inferred default values, shall be required. e.g. don't fill in the table if weighted_pred_flag is not set for P/SP slices.
2012-01-24 08:20:25 +00:00
h264: simplify RefPicList reconstruction.
2012-01-27 16:28:50 +00:00
slice_param->num_ref_idx_l0_active_minus1 = 0;
slice_param->num_ref_idx_l1_active_minus1 = 0;
if (GST_H264_IS_B_SLICE(slice_hdr))
num_ref_lists = 2;
else if (GST_H264_IS_I_SLICE(slice_hdr))
num_ref_lists = 0;
else
num_ref_lists = 1;
if (num_ref_lists < 1)
return TRUE;
slice_param->num_ref_idx_l0_active_minus1 =
slice_hdr->num_ref_idx_l0_active_minus1;
h264: fix pred_weight_table() reconstruction. Only the explicit pred_weight_table(), possibly with the inferred default values, shall be required. e.g. don't fill in the table if weighted_pred_flag is not set for P/SP slices.
2012-01-24 08:20:25 +00:00
for (i = 0; i < priv->RefPicList0_count && priv->RefPicList0[i]; i++)
decoder: h264: fix MVC inter-view prediction process. The inter-view reference components and inter-view only reference components that are included in the reference picture lists shall be considered as not being marked as "used for short-term reference" or "used for long-term reference". This means that reference flags should all be removed from VAPictureH264.flags. This fixes decoding of MVCNV-2.264.
2014-06-04 17:10:44 +00:00
vaapi_fill_picture_for_RefPicListX(&slice_param->RefPicList0[i],
priv->RefPicList0[i]);
h264: fix pred_weight_table() reconstruction. Only the explicit pred_weight_table(), possibly with the inferred default values, shall be required. e.g. don't fill in the table if weighted_pred_flag is not set for P/SP slices.
2012-01-24 08:20:25 +00:00
for (; i <= slice_param->num_ref_idx_l0_active_minus1; i++)
vaapi_init_picture(&slice_param->RefPicList0[i]);
h264: simplify RefPicList reconstruction.
2012-01-27 16:28:50 +00:00
if (num_ref_lists < 2)
return TRUE;
slice_param->num_ref_idx_l1_active_minus1 =
slice_hdr->num_ref_idx_l1_active_minus1;
h264: fix pred_weight_table() reconstruction. Only the explicit pred_weight_table(), possibly with the inferred default values, shall be required. e.g. don't fill in the table if weighted_pred_flag is not set for P/SP slices.
2012-01-24 08:20:25 +00:00
for (i = 0; i < priv->RefPicList1_count && priv->RefPicList1[i]; i++)
decoder: h264: fix MVC inter-view prediction process. The inter-view reference components and inter-view only reference components that are included in the reference picture lists shall be considered as not being marked as "used for short-term reference" or "used for long-term reference". This means that reference flags should all be removed from VAPictureH264.flags. This fixes decoding of MVCNV-2.264.
2014-06-04 17:10:44 +00:00
vaapi_fill_picture_for_RefPicListX(&slice_param->RefPicList1[i],
priv->RefPicList1[i]);
h264: fix pred_weight_table() reconstruction. Only the explicit pred_weight_table(), possibly with the inferred default values, shall be required. e.g. don't fill in the table if weighted_pred_flag is not set for P/SP slices.
2012-01-24 08:20:25 +00:00
for (; i <= slice_param->num_ref_idx_l1_active_minus1; i++)
vaapi_init_picture(&slice_param->RefPicList1[i]);
h264: simplify RefPicList reconstruction.
2012-01-27 16:28:50 +00:00
return TRUE;
}
static gboolean
h264: drop GstVaapiSliceH264 object. Use standard GstVaapiSlice object from now on since we already have parsed and recorded the slice headers (GstH264SliceHdr decode units).
2012-12-07 16:45:03 +00:00
fill_slice(GstVaapiDecoderH264 *decoder,
decoder: h264: fix slice data bit offset with MVC NAL units. When MVC slice NAL units (coded slice extension and prefix NAL) are present, the number of NAL header bytes is 3, not 1 as usual. Signed-off-by: Li Xiaowei <xiaowei.a.li@intel.com> Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2013-03-07 03:32:20 +00:00
GstVaapiSlice *slice, GstVaapiParserInfoH264 *pi)
h264: simplify RefPicList reconstruction.
2012-01-27 16:28:50 +00:00
{
h264: drop GstVaapiSliceH264 object. Use standard GstVaapiSlice object from now on since we already have parsed and recorded the slice headers (GstH264SliceHdr decode units).
2012-12-07 16:45:03 +00:00
VASliceParameterBufferH264 * const slice_param = slice->param;
decoder: h264: fix slice data bit offset with MVC NAL units. When MVC slice NAL units (coded slice extension and prefix NAL) are present, the number of NAL header bytes is 3, not 1 as usual. Signed-off-by: Li Xiaowei <xiaowei.a.li@intel.com> Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2013-03-07 03:32:20 +00:00
GstH264SliceHdr * const slice_hdr = &pi->data.slice_hdr;
h264: simplify RefPicList reconstruction.
2012-01-27 16:28:50 +00:00
/* Fill in VASliceParameterBufferH264 */
decoder: h264: fix slice data bit offset with MVC NAL units. When MVC slice NAL units (coded slice extension and prefix NAL) are present, the number of NAL header bytes is 3, not 1 as usual. Signed-off-by: Li Xiaowei <xiaowei.a.li@intel.com> Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2013-03-07 03:32:20 +00:00
slice_param->slice_data_bit_offset =
get_slice_data_bit_offset(slice_hdr, pi->nalu.header_bytes);
h264: simplify RefPicList reconstruction.
2012-01-27 16:28:50 +00:00
slice_param->first_mb_in_slice = slice_hdr->first_mb_in_slice;
slice_param->slice_type = slice_hdr->type % 5;
slice_param->direct_spatial_mv_pred_flag = slice_hdr->direct_spatial_mv_pred_flag;
slice_param->cabac_init_idc = slice_hdr->cabac_init_idc;
slice_param->slice_qp_delta = slice_hdr->slice_qp_delta;
slice_param->disable_deblocking_filter_idc = slice_hdr->disable_deblocking_filter_idc;
slice_param->slice_alpha_c0_offset_div2 = slice_hdr->slice_alpha_c0_offset_div2;
slice_param->slice_beta_offset_div2 = slice_hdr->slice_beta_offset_div2;
h264: fix pred_weight_table() reconstruction. Only the explicit pred_weight_table(), possibly with the inferred default values, shall be required. e.g. don't fill in the table if weighted_pred_flag is not set for P/SP slices.
2012-01-24 08:20:25 +00:00
h264: drop GstVaapiSliceH264 object. Use standard GstVaapiSlice object from now on since we already have parsed and recorded the slice headers (GstH264SliceHdr decode units).
2012-12-07 16:45:03 +00:00
if (!fill_RefPicList(decoder, slice, slice_hdr))
h264: simplify RefPicList reconstruction.
2012-01-27 16:28:50 +00:00
return FALSE;
h264: drop GstVaapiSliceH264 object. Use standard GstVaapiSlice object from now on since we already have parsed and recorded the slice headers (GstH264SliceHdr decode units).
2012-12-07 16:45:03 +00:00
if (!fill_pred_weight_table(decoder, slice, slice_hdr))
h264: fix pred_weight_table() reconstruction. Only the explicit pred_weight_table(), possibly with the inferred default values, shall be required. e.g. don't fill in the table if weighted_pred_flag is not set for P/SP slices.
2012-01-24 08:20:25 +00:00
return FALSE;
return TRUE;
}
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
static GstVaapiDecoderStatus
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
decode_slice(GstVaapiDecoderH264 *decoder, GstVaapiDecoderUnit *unit)
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
GstVaapiParserInfoH264 * const pi = unit->parsed_info;
h264: drop GstVaapiSliceH264 object. Use standard GstVaapiSlice object from now on since we already have parsed and recorded the slice headers (GstH264SliceHdr decode units).
2012-12-07 16:45:03 +00:00
GstVaapiPictureH264 * const picture = priv->current_picture;
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
GstH264SliceHdr * const slice_hdr = &pi->data.slice_hdr;
h264: drop GstVaapiSliceH264 object. Use standard GstVaapiSlice object from now on since we already have parsed and recorded the slice headers (GstH264SliceHdr decode units).
2012-12-07 16:45:03 +00:00
GstVaapiSlice *slice;
decoder: sanitize uses of codec frame input buffer (cosmetics). Alias GST_VAAPI_DECODER_CODEC_FRAME(decoder)->input_buffer to a simple "buffer" variable.
2013-03-21 12:28:05 +00:00
GstBuffer * const buffer =
GST_VAAPI_DECODER_CODEC_FRAME(decoder)->input_buffer;
Add initial support for GStreamer 1.0. This integrates support for GStreamer API >= 1.0 only in the libgstvaapi core decoding library. The changes are kept rather minimal here so that the library retains as little dependency as possible on core GStreamer functionality. Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2012-09-04 12:12:18 +00:00
GstMapInfo map_info;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
GST_DEBUG("slice (%u bytes)", pi->nalu.size);
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
decoder: h264: fix robustness patch for bytestream format. Fix parser and decoder state to sync at the right locations. This is because we could reset the parser state, while the decoder state was not copied yet, e.g. when parsing several NAL units from multiple frames whereas the current frame was not decoded yet. This is a regression brought in by commit 6fe5496.
2014-02-04 17:35:28 +00:00
if (!is_valid_state(pi->state,
h264: improve robustness when packets are missing. Improve robustness when some expected packets where not received yet or that were not correctly decoded. For example, don't try to decode a picture if there was no valid frame headers parsed so far. https://bugs.freedesktop.org/show_bug.cgi?id=57902
2013-12-10 15:14:27 +00:00
GST_H264_VIDEO_STATE_VALID_PICTURE_HEADERS)) {
GST_WARNING("failed to receive enough headers to decode slice");
h264: add codec specific decoder unit. Introduce new GstVaapiDecoderUnitH264 object, which holds the standard NAL unit header (GstH264NalUnit) and additional parsed header info. Besides, we now parse headers as early as in the _parse() function so that to avoid un-necessary creation of sub-buffers in _decode() for NAL units that are not slices. This is a performance win by ~+1.1% only.
2012-12-06 16:25:01 +00:00
return GST_VAAPI_DECODER_STATUS_SUCCESS;
h264: detect new pictures from decode-units. Update is_new_picture() to cope with GstVaapiDecoderUnitH264, instead of assuming frame boundaries when first_mb_in_slice is zero.
2012-12-13 09:47:25 +00:00
}
h264: initialize VA context before allocating the first slice. Fix decode_slice() to ensure a VA context exists prior to creating a new GstVaapiSliceH264, which invokes vaCreateBuffer() with some VA context ID. i.e. the latter was not initialized, thus causing failures on Cedar Trail for example.
2012-12-17 12:42:29 +00:00
decoder: h264: fix activation of picture and sequence parameters. At the time the first VCL NAL unit of a primary coded picture is found, and if that NAL unit was parsed to be an SPS or PPS, then the entries in the parser may have been overriden. This means that, when the picture is to be decoded, slice_hdr->pps could point to an invalid (the next) PPS entry. So, one way to solve this problem is to not use the parser PPS and SPS info but rather maintain our own activation chain in the decoder. https://bugzilla.gnome.org/show_bug.cgi?id=724519 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 17:11:03 +00:00
if (!ensure_pps(decoder, slice_hdr->pps)) {
GST_ERROR("failed to activate PPS");
return GST_VAAPI_DECODER_STATUS_ERROR_UNKNOWN;
}
if (!ensure_sps(decoder, slice_hdr->pps->sequence)) {
GST_ERROR("failed to activate SPS");
return GST_VAAPI_DECODER_STATUS_ERROR_UNKNOWN;
}
Add initial support for GStreamer 1.0. This integrates support for GStreamer API >= 1.0 only in the libgstvaapi core decoding library. The changes are kept rather minimal here so that the library retains as little dependency as possible on core GStreamer functionality. Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2012-09-04 12:12:18 +00:00
if (!gst_buffer_map(buffer, &map_info, GST_MAP_READ)) {
GST_ERROR("failed to map buffer");
return GST_VAAPI_DECODER_STATUS_ERROR_UNKNOWN;
}
decoder: h264: fix detection of access unit boundaries. In order to have a stricter conforming implementation, we need to carefully detect access unit boundaries. Additional operations could be necessary to perform at those boundaries.
2014-05-01 17:33:40 +00:00
/* Check wether this is the first/last slice in the current access unit */
if (pi->flags & GST_VAAPI_DECODER_UNIT_FLAG_AU_START)
GST_VAAPI_PICTURE_FLAG_SET(picture, GST_VAAPI_PICTURE_FLAG_AU_START);
if (pi->flags & GST_VAAPI_DECODER_UNIT_FLAG_AU_END)
GST_VAAPI_PICTURE_FLAG_SET(picture, GST_VAAPI_PICTURE_FLAG_AU_END);
h264: drop GstVaapiSliceH264 object. Use standard GstVaapiSlice object from now on since we already have parsed and recorded the slice headers (GstH264SliceHdr decode units).
2012-12-07 16:45:03 +00:00
slice = GST_VAAPI_SLICE_NEW(H264, decoder,
Add initial support for GStreamer 1.0. This integrates support for GStreamer API >= 1.0 only in the libgstvaapi core decoding library. The changes are kept rather minimal here so that the library retains as little dependency as possible on core GStreamer functionality. Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2012-09-04 12:12:18 +00:00
(map_info.data + unit->offset + pi->nalu.offset), pi->nalu.size);
decoder: fix unpaired GstBuffer map/unmaps. This possibly fixes a few memory leaks along the way.
2013-03-20 10:26:38 +00:00
gst_buffer_unmap(buffer, &map_info);
h264: add codec specific decoder unit. Introduce new GstVaapiDecoderUnitH264 object, which holds the standard NAL unit header (GstH264NalUnit) and additional parsed header info. Besides, we now parse headers as early as in the _parse() function so that to avoid un-necessary creation of sub-buffers in _decode() for NAL units that are not slices. This is a performance win by ~+1.1% only.
2012-12-06 16:25:01 +00:00
if (!slice) {
GST_ERROR("failed to allocate slice");
return GST_VAAPI_DECODER_STATUS_ERROR_ALLOCATION_FAILED;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
}
decoder: h264: fix initialization of RefPicLists for multiple slices. The initialization of reference picture lists (8.2.4.2) applies to all slices. So, the RefPicList0/1 lists need to be constructed prior to each slice submission to the HW decoder. This fixes decoding of video sequences where frames are encoded with multiple slices of different types, e.g. 4 slices in this order I, P, I, and P. More precisely, CABAST3_Sony_E and CABASTBR3_Sony_B. https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-26 18:21:46 +00:00
init_picture_refs(decoder, picture, slice_hdr);
decoder: h264: fix slice data bit offset with MVC NAL units. When MVC slice NAL units (coded slice extension and prefix NAL) are present, the number of NAL header bytes is 3, not 1 as usual. Signed-off-by: Li Xiaowei <xiaowei.a.li@intel.com> Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2013-03-07 03:32:20 +00:00
if (!fill_slice(decoder, slice, pi)) {
h264: drop GstVaapiSliceH264 object. Use standard GstVaapiSlice object from now on since we already have parsed and recorded the slice headers (GstH264SliceHdr decode units).
2012-12-07 16:45:03 +00:00
gst_vaapi_mini_object_unref(GST_VAAPI_MINI_OBJECT(slice));
return GST_VAAPI_DECODER_STATUS_ERROR_UNKNOWN;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
}
h264: drop GstVaapiSliceH264 object. Use standard GstVaapiSlice object from now on since we already have parsed and recorded the slice headers (GstH264SliceHdr decode units).
2012-12-07 16:45:03 +00:00
gst_vaapi_picture_add_slice(GST_VAAPI_PICTURE_CAST(picture), slice);
picture->last_slice_hdr = slice_hdr;
h264: improve robustness when packets are missing. Improve robustness when some expected packets where not received yet or that were not correctly decoded. For example, don't try to decode a picture if there was no valid frame headers parsed so far. https://bugs.freedesktop.org/show_bug.cgi?id=57902
2013-12-10 15:14:27 +00:00
priv->decoder_state |= GST_H264_VIDEO_STATE_GOT_SLICE;
h264: drop GstVaapiSliceH264 object. Use standard GstVaapiSlice object from now on since we already have parsed and recorded the slice headers (GstH264SliceHdr decode units).
2012-12-07 16:45:03 +00:00
return GST_VAAPI_DECODER_STATUS_SUCCESS;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
}
h264: exclusively use GstAdapter, drop sub-buffer hack. Maintaining the sub-buffer is rather suboptimal especially since we were also maintaining a GstAdapter. Now, we only use the GstAdapter thus requiring minor extra parsing when receiving avcC buffers.
2012-09-20 15:58:21 +00:00
static inline gint
scan_for_start_code(GstAdapter *adapter, guint ofs, guint size, guint32 *scp)
{
return (gint)gst_adapter_masked_scan_uint32_peek(adapter,
0xffffff00, 0x00000100,
ofs, size,
scp);
}
h264: add decode_nalu() helper function. Split decode_buffer() into the core infrastructure that determines the NAL units contained in the adapter and the actual function that decodes the NAL unit.
2012-10-10 08:35:20 +00:00
static GstVaapiDecoderStatus
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
decode_unit(GstVaapiDecoderH264 *decoder, GstVaapiDecoderUnit *unit)
h264: add decode_nalu() helper function. Split decode_buffer() into the core infrastructure that determines the NAL units contained in the adapter and the actual function that decodes the NAL unit.
2012-10-10 08:35:20 +00:00
{
decoder: h264: fix robustness patch for bytestream format. Fix parser and decoder state to sync at the right locations. This is because we could reset the parser state, while the decoder state was not copied yet, e.g. when parsing several NAL units from multiple frames whereas the current frame was not decoded yet. This is a regression brought in by commit 6fe5496.
2014-02-04 17:35:28 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
GstVaapiParserInfoH264 * const pi = unit->parsed_info;
h264: add decode_nalu() helper function. Split decode_buffer() into the core infrastructure that determines the NAL units contained in the adapter and the actual function that decodes the NAL unit.
2012-10-10 08:35:20 +00:00
GstVaapiDecoderStatus status;
decoder: h264: fix robustness patch for bytestream format. Fix parser and decoder state to sync at the right locations. This is because we could reset the parser state, while the decoder state was not copied yet, e.g. when parsing several NAL units from multiple frames whereas the current frame was not decoded yet. This is a regression brought in by commit 6fe5496.
2014-02-04 17:35:28 +00:00
priv->decoder_state |= pi->state;
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
switch (pi->nalu.type) {
decoder: h264: fix activation of picture and sequence parameters. At the time the first VCL NAL unit of a primary coded picture is found, and if that NAL unit was parsed to be an SPS or PPS, then the entries in the parser may have been overriden. This means that, when the picture is to be decoded, slice_hdr->pps could point to an invalid (the next) PPS entry. So, one way to solve this problem is to not use the parser PPS and SPS info but rather maintain our own activation chain in the decoder. https://bugzilla.gnome.org/show_bug.cgi?id=724519 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 17:11:03 +00:00
case GST_H264_NAL_SPS:
status = decode_sps(decoder, unit);
break;
decoder: h264: add support for MVC base views. Allow decoding for base views of MVC encoded streams. For now, just skip the slice extension and prefix NAL units, and skip non-base view frames. Signed-off-by: Xiaowei Li <xiaowei.a.li@intel.com> [fixed memory leak, improved check for MVC NAL units] Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2013-02-28 07:59:55 +00:00
case GST_H264_NAL_SUBSET_SPS:
status = decode_subset_sps(decoder, unit);
break;
decoder: h264: fix activation of picture and sequence parameters. At the time the first VCL NAL unit of a primary coded picture is found, and if that NAL unit was parsed to be an SPS or PPS, then the entries in the parser may have been overriden. This means that, when the picture is to be decoded, slice_hdr->pps could point to an invalid (the next) PPS entry. So, one way to solve this problem is to not use the parser PPS and SPS info but rather maintain our own activation chain in the decoder. https://bugzilla.gnome.org/show_bug.cgi?id=724519 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 17:11:03 +00:00
case GST_H264_NAL_PPS:
status = decode_pps(decoder, unit);
break;
decoder: h264: add support for MVC base views. Allow decoding for base views of MVC encoded streams. For now, just skip the slice extension and prefix NAL units, and skip non-base view frames. Signed-off-by: Xiaowei Li <xiaowei.a.li@intel.com> [fixed memory leak, improved check for MVC NAL units] Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2013-02-28 07:59:55 +00:00
case GST_H264_NAL_SLICE_EXT:
h264: add decode_nalu() helper function. Split decode_buffer() into the core infrastructure that determines the NAL units contained in the adapter and the actual function that decodes the NAL unit.
2012-10-10 08:35:20 +00:00
case GST_H264_NAL_SLICE_IDR:
/* fall-through. IDR specifics are handled in init_picture() */
case GST_H264_NAL_SLICE:
h264: add codec specific decoder unit. Introduce new GstVaapiDecoderUnitH264 object, which holds the standard NAL unit header (GstH264NalUnit) and additional parsed header info. Besides, we now parse headers as early as in the _parse() function so that to avoid un-necessary creation of sub-buffers in _decode() for NAL units that are not slices. This is a performance win by ~+1.1% only.
2012-12-06 16:25:01 +00:00
status = decode_slice(decoder, unit);
h264: add decode_nalu() helper function. Split decode_buffer() into the core infrastructure that determines the NAL units contained in the adapter and the actual function that decodes the NAL unit.
2012-10-10 08:35:20 +00:00
break;
case GST_H264_NAL_SEQ_END:
h264: handle end-of-stream NALU. Handle <end-of-stream> NAL unit to actually flush any pending picture from the DPB.
2013-01-17 17:07:03 +00:00
case GST_H264_NAL_STREAM_END:
h264: add decode_nalu() helper function. Split decode_buffer() into the core infrastructure that determines the NAL units contained in the adapter and the actual function that decodes the NAL unit.
2012-10-10 08:35:20 +00:00
status = decode_sequence_end(decoder);
break;
h264: add codec specific decoder unit. Introduce new GstVaapiDecoderUnitH264 object, which holds the standard NAL unit header (GstH264NalUnit) and additional parsed header info. Besides, we now parse headers as early as in the _parse() function so that to avoid un-necessary creation of sub-buffers in _decode() for NAL units that are not slices. This is a performance win by ~+1.1% only.
2012-12-06 16:25:01 +00:00
case GST_H264_NAL_SEI:
decoder: h264: fix detection of top-field-first (TFF) flag. Use the SEI pic_timing() message to track the pic_struct variable when present, or infer it from the regular slice header flags field_pic_flag and bottom_field_flag. This fixes temporal sequence ordering when the output pictures are to be displayed. https://bugzilla.gnome.org/show_bug.cgi?id=739291
2014-11-13 14:00:21 +00:00
status = decode_sei(decoder, unit);
h264: add decode_nalu() helper function. Split decode_buffer() into the core infrastructure that determines the NAL units contained in the adapter and the actual function that decodes the NAL unit.
2012-10-10 08:35:20 +00:00
break;
default:
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
GST_WARNING("unsupported NAL unit type %d", pi->nalu.type);
h264: add decode_nalu() helper function. Split decode_buffer() into the core infrastructure that determines the NAL units contained in the adapter and the actual function that decodes the NAL unit.
2012-10-10 08:35:20 +00:00
status = GST_VAAPI_DECODER_STATUS_ERROR_BITSTREAM_PARSER;
break;
}
return status;
}
h264: handle codec-data. Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2011-11-25 12:37:00 +00:00
static GstVaapiDecoderStatus
decoder: sanitize codec-data decoding. Add a new GstVaapiDecoder::decode_codec_data() hook to actually decode codec-data in the decoder sub-class. Provide a common shared helper function to do the actual work and delegating further to the sub-class.
2013-03-21 13:36:40 +00:00
gst_vaapi_decoder_h264_decode_codec_data(GstVaapiDecoder *base_decoder,
const guchar *buf, guint buf_size)
h264: handle codec-data. Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2011-11-25 12:37:00 +00:00
{
decoder: sanitize codec-data decoding. Add a new GstVaapiDecoder::decode_codec_data() hook to actually decode codec-data in the decoder sub-class. Provide a common shared helper function to do the actual work and delegating further to the sub-class.
2013-03-21 13:36:40 +00:00
GstVaapiDecoderH264 * const decoder =
GST_VAAPI_DECODER_H264_CAST(base_decoder);
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
h264: handle codec-data. Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2011-11-25 12:37:00 +00:00
GstVaapiDecoderStatus status;
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
GstVaapiDecoderUnit unit;
decoder: h264: fix activation of picture and sequence parameters. At the time the first VCL NAL unit of a primary coded picture is found, and if that NAL unit was parsed to be an SPS or PPS, then the entries in the parser may have been overriden. This means that, when the picture is to be decoded, slice_hdr->pps could point to an invalid (the next) PPS entry. So, one way to solve this problem is to not use the parser PPS and SPS info but rather maintain our own activation chain in the decoder. https://bugzilla.gnome.org/show_bug.cgi?id=724519 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 17:11:03 +00:00
GstVaapiParserInfoH264 *pi = NULL;
h264: handle codec-data. Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2011-11-25 12:37:00 +00:00
GstH264ParserResult result;
guint i, ofs, num_sps, num_pps;
decoder: h264: fix activation of picture and sequence parameters. At the time the first VCL NAL unit of a primary coded picture is found, and if that NAL unit was parsed to be an SPS or PPS, then the entries in the parser may have been overriden. This means that, when the picture is to be decoded, slice_hdr->pps could point to an invalid (the next) PPS entry. So, one way to solve this problem is to not use the parser PPS and SPS info but rather maintain our own activation chain in the decoder. https://bugzilla.gnome.org/show_bug.cgi?id=724519 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 17:11:03 +00:00
unit.parsed_info = NULL;
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
h264: handle codec-data. Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2011-11-25 12:37:00 +00:00
if (buf_size < 8)
return GST_VAAPI_DECODER_STATUS_ERROR_NO_DATA;
if (buf[0] != 1) {
h264: review and report errors accordingly. Use GST_ERROR() to report real errors instead of hiding them into GST_DEBUG().
2012-09-21 14:43:38 +00:00
GST_ERROR("failed to decode codec-data, not in avcC format");
h264: handle codec-data. Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2011-11-25 12:37:00 +00:00
return GST_VAAPI_DECODER_STATUS_ERROR_BITSTREAM_PARSER;
}
priv->nal_length_size = (buf[4] & 0x03) + 1;
num_sps = buf[5] & 0x1f;
ofs = 6;
for (i = 0; i < num_sps; i++) {
decoder: h264: fix activation of picture and sequence parameters. At the time the first VCL NAL unit of a primary coded picture is found, and if that NAL unit was parsed to be an SPS or PPS, then the entries in the parser may have been overriden. This means that, when the picture is to be decoded, slice_hdr->pps could point to an invalid (the next) PPS entry. So, one way to solve this problem is to not use the parser PPS and SPS info but rather maintain our own activation chain in the decoder. https://bugzilla.gnome.org/show_bug.cgi?id=724519 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 17:11:03 +00:00
pi = gst_vaapi_parser_info_h264_new();
if (!pi)
return GST_VAAPI_DECODER_STATUS_ERROR_ALLOCATION_FAILED;
unit.parsed_info = pi;
h264: handle codec-data. Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2011-11-25 12:37:00 +00:00
result = gst_h264_parser_identify_nalu_avc(
priv->parser,
buf, ofs, buf_size, 2,
decoder: h264: fix activation of picture and sequence parameters. At the time the first VCL NAL unit of a primary coded picture is found, and if that NAL unit was parsed to be an SPS or PPS, then the entries in the parser may have been overriden. This means that, when the picture is to be decoded, slice_hdr->pps could point to an invalid (the next) PPS entry. So, one way to solve this problem is to not use the parser PPS and SPS info but rather maintain our own activation chain in the decoder. https://bugzilla.gnome.org/show_bug.cgi?id=724519 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 17:11:03 +00:00
&pi->nalu
h264: handle codec-data. Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2011-11-25 12:37:00 +00:00
);
decoder: h264: fix activation of picture and sequence parameters. At the time the first VCL NAL unit of a primary coded picture is found, and if that NAL unit was parsed to be an SPS or PPS, then the entries in the parser may have been overriden. This means that, when the picture is to be decoded, slice_hdr->pps could point to an invalid (the next) PPS entry. So, one way to solve this problem is to not use the parser PPS and SPS info but rather maintain our own activation chain in the decoder. https://bugzilla.gnome.org/show_bug.cgi?id=724519 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 17:11:03 +00:00
if (result != GST_H264_PARSER_OK) {
status = get_status(result);
goto cleanup;
}
h264: handle codec-data. Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2011-11-25 12:37:00 +00:00
h264: add codec specific decoder unit. Introduce new GstVaapiDecoderUnitH264 object, which holds the standard NAL unit header (GstH264NalUnit) and additional parsed header info. Besides, we now parse headers as early as in the _parse() function so that to avoid un-necessary creation of sub-buffers in _decode() for NAL units that are not slices. This is a performance win by ~+1.1% only.
2012-12-06 16:25:01 +00:00
status = parse_sps(decoder, &unit);
h264: handle codec-data. Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2011-11-25 12:37:00 +00:00
if (status != GST_VAAPI_DECODER_STATUS_SUCCESS)
decoder: h264: fix activation of picture and sequence parameters. At the time the first VCL NAL unit of a primary coded picture is found, and if that NAL unit was parsed to be an SPS or PPS, then the entries in the parser may have been overriden. This means that, when the picture is to be decoded, slice_hdr->pps could point to an invalid (the next) PPS entry. So, one way to solve this problem is to not use the parser PPS and SPS info but rather maintain our own activation chain in the decoder. https://bugzilla.gnome.org/show_bug.cgi?id=724519 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 17:11:03 +00:00
goto cleanup;
ofs = pi->nalu.offset + pi->nalu.size;
status = decode_sps(decoder, &unit);
if (status != GST_VAAPI_DECODER_STATUS_SUCCESS)
goto cleanup;
gst_vaapi_parser_info_h264_replace(&pi, NULL);
h264: handle codec-data. Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2011-11-25 12:37:00 +00:00
}
num_pps = buf[ofs];
ofs++;
for (i = 0; i < num_pps; i++) {
decoder: h264: fix activation of picture and sequence parameters. At the time the first VCL NAL unit of a primary coded picture is found, and if that NAL unit was parsed to be an SPS or PPS, then the entries in the parser may have been overriden. This means that, when the picture is to be decoded, slice_hdr->pps could point to an invalid (the next) PPS entry. So, one way to solve this problem is to not use the parser PPS and SPS info but rather maintain our own activation chain in the decoder. https://bugzilla.gnome.org/show_bug.cgi?id=724519 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 17:11:03 +00:00
pi = gst_vaapi_parser_info_h264_new();
if (!pi)
return GST_VAAPI_DECODER_STATUS_ERROR_ALLOCATION_FAILED;
unit.parsed_info = pi;
h264: handle codec-data. Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2011-11-25 12:37:00 +00:00
result = gst_h264_parser_identify_nalu_avc(
priv->parser,
buf, ofs, buf_size, 2,
decoder: h264: fix activation of picture and sequence parameters. At the time the first VCL NAL unit of a primary coded picture is found, and if that NAL unit was parsed to be an SPS or PPS, then the entries in the parser may have been overriden. This means that, when the picture is to be decoded, slice_hdr->pps could point to an invalid (the next) PPS entry. So, one way to solve this problem is to not use the parser PPS and SPS info but rather maintain our own activation chain in the decoder. https://bugzilla.gnome.org/show_bug.cgi?id=724519 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 17:11:03 +00:00
&pi->nalu
h264: handle codec-data. Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2011-11-25 12:37:00 +00:00
);
decoder: h264: fix activation of picture and sequence parameters. At the time the first VCL NAL unit of a primary coded picture is found, and if that NAL unit was parsed to be an SPS or PPS, then the entries in the parser may have been overriden. This means that, when the picture is to be decoded, slice_hdr->pps could point to an invalid (the next) PPS entry. So, one way to solve this problem is to not use the parser PPS and SPS info but rather maintain our own activation chain in the decoder. https://bugzilla.gnome.org/show_bug.cgi?id=724519 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 17:11:03 +00:00
if (result != GST_H264_PARSER_OK) {
status = get_status(result);
goto cleanup;
}
h264: handle codec-data. Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2011-11-25 12:37:00 +00:00
h264: add codec specific decoder unit. Introduce new GstVaapiDecoderUnitH264 object, which holds the standard NAL unit header (GstH264NalUnit) and additional parsed header info. Besides, we now parse headers as early as in the _parse() function so that to avoid un-necessary creation of sub-buffers in _decode() for NAL units that are not slices. This is a performance win by ~+1.1% only.
2012-12-06 16:25:01 +00:00
status = parse_pps(decoder, &unit);
h264: handle codec-data. Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2011-11-25 12:37:00 +00:00
if (status != GST_VAAPI_DECODER_STATUS_SUCCESS)
decoder: h264: fix activation of picture and sequence parameters. At the time the first VCL NAL unit of a primary coded picture is found, and if that NAL unit was parsed to be an SPS or PPS, then the entries in the parser may have been overriden. This means that, when the picture is to be decoded, slice_hdr->pps could point to an invalid (the next) PPS entry. So, one way to solve this problem is to not use the parser PPS and SPS info but rather maintain our own activation chain in the decoder. https://bugzilla.gnome.org/show_bug.cgi?id=724519 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 17:11:03 +00:00
goto cleanup;
ofs = pi->nalu.offset + pi->nalu.size;
status = decode_pps(decoder, &unit);
if (status != GST_VAAPI_DECODER_STATUS_SUCCESS)
goto cleanup;
gst_vaapi_parser_info_h264_replace(&pi, NULL);
h264: handle codec-data. Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2011-11-25 12:37:00 +00:00
}
h264: handle avcC format for decoding buffers.
2012-01-17 09:42:49 +00:00
h264: minor clean-ups. Drop explicit initialization of most fields that are implicitly set to zero. Drop helper macros for casting to GstVaapiPictureH264 or GstVaapiFrameStore. Also remove some useless checks for NULL pointers.
2012-12-13 10:48:06 +00:00
priv->is_avcC = TRUE;
decoder: h264: fix activation of picture and sequence parameters. At the time the first VCL NAL unit of a primary coded picture is found, and if that NAL unit was parsed to be an SPS or PPS, then the entries in the parser may have been overriden. This means that, when the picture is to be decoded, slice_hdr->pps could point to an invalid (the next) PPS entry. So, one way to solve this problem is to not use the parser PPS and SPS info but rather maintain our own activation chain in the decoder. https://bugzilla.gnome.org/show_bug.cgi?id=724519 https://bugzilla.gnome.org/show_bug.cgi?id=724518
2014-04-25 17:11:03 +00:00
status = GST_VAAPI_DECODER_STATUS_SUCCESS;
cleanup:
gst_vaapi_parser_info_h264_replace(&pi, NULL);
return status;
h264: handle codec-data. Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2011-11-25 12:37:00 +00:00
}
h264: initial port to new GstVaapiDecoder API
2012-12-06 13:02:17 +00:00
static GstVaapiDecoderStatus
ensure_decoder(GstVaapiDecoderH264 *decoder)
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
h264: handle codec-data. Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2011-11-25 12:37:00 +00:00
GstVaapiDecoderStatus status;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
if (!priv->is_opened) {
h264: initial port to new GstVaapiDecoder API
2012-12-06 13:02:17 +00:00
priv->is_opened = gst_vaapi_decoder_h264_open(decoder);
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
if (!priv->is_opened)
return GST_VAAPI_DECODER_STATUS_ERROR_UNSUPPORTED_CODEC;
h264: handle codec-data. Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2011-11-25 12:37:00 +00:00
decoder: sanitize codec-data decoding. Add a new GstVaapiDecoder::decode_codec_data() hook to actually decode codec-data in the decoder sub-class. Provide a common shared helper function to do the actual work and delegating further to the sub-class.
2013-03-21 13:36:40 +00:00
status = gst_vaapi_decoder_decode_codec_data(
GST_VAAPI_DECODER_CAST(decoder));
if (status != GST_VAAPI_DECODER_STATUS_SUCCESS)
return status;
h264: initial port to new GstVaapiDecoder API
2012-12-06 13:02:17 +00:00
}
return GST_VAAPI_DECODER_STATUS_SUCCESS;
}
static GstVaapiDecoderStatus
gst_vaapi_decoder_h264_parse(GstVaapiDecoder *base_decoder,
decoder: refactor decoder unit API. Allocate decoder unit earlier in the main parse() function and don't delegate this task to derived classes. The ultimate purpose is to get rid of dynamic allocation of decoder units.
2013-01-07 10:13:07 +00:00
GstAdapter *adapter, gboolean at_eos, GstVaapiDecoderUnit *unit)
h264: initial port to new GstVaapiDecoder API
2012-12-06 13:02:17 +00:00
{
h264: minor clean-ups. Drop explicit initialization of most fields that are implicitly set to zero. Drop helper macros for casting to GstVaapiPictureH264 or GstVaapiFrameStore. Also remove some useless checks for NULL pointers.
2012-12-13 10:48:06 +00:00
GstVaapiDecoderH264 * const decoder =
GST_VAAPI_DECODER_H264_CAST(base_decoder);
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiDecoderH264Private * const priv = &decoder->priv;
h264: optimize scan for the second start code. Optimize scan for the second start code, on the next parse() call so that to avoid scanning again earlier bytes where we didn't find any start code.
2012-12-12 17:33:52 +00:00
GstVaapiParserState * const ps = GST_VAAPI_PARSER_STATE(base_decoder);
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
GstVaapiParserInfoH264 *pi;
h264: initial port to new GstVaapiDecoder API
2012-12-06 13:02:17 +00:00
GstVaapiDecoderStatus status;
GstH264ParserResult result;
guchar *buf;
guint i, size, buf_size, nalu_size, flags;
guint32 start_code;
h264: optimize scan for the second start code. Optimize scan for the second start code, on the next parse() call so that to avoid scanning again earlier bytes where we didn't find any start code.
2012-12-12 17:33:52 +00:00
gint ofs, ofs2;
decoder: h264: handle access unit ("au") optimization. Optimize parsing when buffers are supplied with access unit alignment. This helps determining faster when the end of an access unit is reached.
2014-06-13 13:42:53 +00:00
gboolean at_au_end = FALSE;
h264: initial port to new GstVaapiDecoder API
2012-12-06 13:02:17 +00:00
status = ensure_decoder(decoder);
if (status != GST_VAAPI_DECODER_STATUS_SUCCESS)
return status;
decoder: h264: add support for NALU "alignment" optimization. We can avoid scanning for start codes again if the bitstream is fed in NALU chunks. Currently, we always scan for start codes, and keep track of remaining bits in a GstAdapter, even if, in practice, we are likely receiving one GstBuffer per NAL unit. i.e. h264parse with "nal" alignment. https://bugzilla.gnome.org/show_bug.cgi?id=723284 [use gst_adapter_available_fast() to determine the top buffer size] Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2014-02-06 06:30:10 +00:00
switch (priv->stream_alignment) {
case GST_VAAPI_STREAM_ALIGN_H264_NALU:
decoder: h264: handle access unit ("au") optimization. Optimize parsing when buffers are supplied with access unit alignment. This helps determining faster when the end of an access unit is reached.
2014-06-13 13:42:53 +00:00
case GST_VAAPI_STREAM_ALIGN_H264_AU:
decoder: h264: add support for NALU "alignment" optimization. We can avoid scanning for start codes again if the bitstream is fed in NALU chunks. Currently, we always scan for start codes, and keep track of remaining bits in a GstAdapter, even if, in practice, we are likely receiving one GstBuffer per NAL unit. i.e. h264parse with "nal" alignment. https://bugzilla.gnome.org/show_bug.cgi?id=723284 [use gst_adapter_available_fast() to determine the top buffer size] Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2014-02-06 06:30:10 +00:00
size = gst_adapter_available_fast(adapter);
break;
default:
size = gst_adapter_available(adapter);
break;
}
h264: initial port to new GstVaapiDecoder API
2012-12-06 13:02:17 +00:00
h264: minor clean-ups. Drop explicit initialization of most fields that are implicitly set to zero. Drop helper macros for casting to GstVaapiPictureH264 or GstVaapiFrameStore. Also remove some useless checks for NULL pointers.
2012-12-13 10:48:06 +00:00
if (priv->is_avcC) {
h264: initial port to new GstVaapiDecoder API
2012-12-06 13:02:17 +00:00
if (size < priv->nal_length_size)
return GST_VAAPI_DECODER_STATUS_ERROR_NO_DATA;
buf = (guchar *)&start_code;
g_assert(priv->nal_length_size <= sizeof(start_code));
gst_adapter_copy(adapter, buf, 0, priv->nal_length_size);
nalu_size = 0;
for (i = 0; i < priv->nal_length_size; i++)
nalu_size = (nalu_size << 8) | buf[i];
buf_size = priv->nal_length_size + nalu_size;
if (size < buf_size)
return GST_VAAPI_DECODER_STATUS_ERROR_NO_DATA;
decoder: h264: handle access unit ("au") optimization. Optimize parsing when buffers are supplied with access unit alignment. This helps determining faster when the end of an access unit is reached.
2014-06-13 13:42:53 +00:00
else if (priv->stream_alignment == GST_VAAPI_STREAM_ALIGN_H264_AU)
at_au_end = (buf_size == size);
h264: initial port to new GstVaapiDecoder API
2012-12-06 13:02:17 +00:00
}
else {
if (size < 4)
return GST_VAAPI_DECODER_STATUS_ERROR_NO_DATA;
h264: optimize scan for the second start code. Optimize scan for the second start code, on the next parse() call so that to avoid scanning again earlier bytes where we didn't find any start code.
2012-12-12 17:33:52 +00:00
decoder: h264: add support for NALU "alignment" optimization. We can avoid scanning for start codes again if the bitstream is fed in NALU chunks. Currently, we always scan for start codes, and keep track of remaining bits in a GstAdapter, even if, in practice, we are likely receiving one GstBuffer per NAL unit. i.e. h264parse with "nal" alignment. https://bugzilla.gnome.org/show_bug.cgi?id=723284 [use gst_adapter_available_fast() to determine the top buffer size] Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2014-02-06 06:30:10 +00:00
if (priv->stream_alignment == GST_VAAPI_STREAM_ALIGN_H264_NALU)
buf_size = size;
else {
ofs = scan_for_start_code(adapter, 0, size, NULL);
if (ofs < 0)
return GST_VAAPI_DECODER_STATUS_ERROR_NO_DATA;
h264: optimize scan for the second start code. Optimize scan for the second start code, on the next parse() call so that to avoid scanning again earlier bytes where we didn't find any start code.
2012-12-12 17:33:52 +00:00
decoder: h264: add support for NALU "alignment" optimization. We can avoid scanning for start codes again if the bitstream is fed in NALU chunks. Currently, we always scan for start codes, and keep track of remaining bits in a GstAdapter, even if, in practice, we are likely receiving one GstBuffer per NAL unit. i.e. h264parse with "nal" alignment. https://bugzilla.gnome.org/show_bug.cgi?id=723284 [use gst_adapter_available_fast() to determine the top buffer size] Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2014-02-06 06:30:10 +00:00
if (ofs > 0) {
gst_adapter_flush(adapter, ofs);
size -= ofs;
}
h264: optimize scan for the second start code. Optimize scan for the second start code, on the next parse() call so that to avoid scanning again earlier bytes where we didn't find any start code.
2012-12-12 17:33:52 +00:00
decoder: h264: add support for NALU "alignment" optimization. We can avoid scanning for start codes again if the bitstream is fed in NALU chunks. Currently, we always scan for start codes, and keep track of remaining bits in a GstAdapter, even if, in practice, we are likely receiving one GstBuffer per NAL unit. i.e. h264parse with "nal" alignment. https://bugzilla.gnome.org/show_bug.cgi?id=723284 [use gst_adapter_available_fast() to determine the top buffer size] Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2014-02-06 06:30:10 +00:00
ofs2 = ps->input_offset2 - ofs - 4;
if (ofs2 < 4)
ofs2 = 4;
ofs = G_UNLIKELY(size < ofs2 + 4) ? -1 :
scan_for_start_code(adapter, ofs2, size - ofs2, NULL);
if (ofs < 0) {
// Assume the whole NAL unit is present if end-of-stream
decoder: h264: handle access unit ("au") optimization. Optimize parsing when buffers are supplied with access unit alignment. This helps determining faster when the end of an access unit is reached.
2014-06-13 13:42:53 +00:00
// or stream buffers aligned on access unit boundaries
if (priv->stream_alignment == GST_VAAPI_STREAM_ALIGN_H264_AU)
at_au_end = TRUE;
else if (!at_eos) {
decoder: h264: add support for NALU "alignment" optimization. We can avoid scanning for start codes again if the bitstream is fed in NALU chunks. Currently, we always scan for start codes, and keep track of remaining bits in a GstAdapter, even if, in practice, we are likely receiving one GstBuffer per NAL unit. i.e. h264parse with "nal" alignment. https://bugzilla.gnome.org/show_bug.cgi?id=723284 [use gst_adapter_available_fast() to determine the top buffer size] Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2014-02-06 06:30:10 +00:00
ps->input_offset2 = size;
return GST_VAAPI_DECODER_STATUS_ERROR_NO_DATA;
}
ofs = size;
h264: optimize scan for the second start code. Optimize scan for the second start code, on the next parse() call so that to avoid scanning again earlier bytes where we didn't find any start code.
2012-12-12 17:33:52 +00:00
}
decoder: h264: add support for NALU "alignment" optimization. We can avoid scanning for start codes again if the bitstream is fed in NALU chunks. Currently, we always scan for start codes, and keep track of remaining bits in a GstAdapter, even if, in practice, we are likely receiving one GstBuffer per NAL unit. i.e. h264parse with "nal" alignment. https://bugzilla.gnome.org/show_bug.cgi?id=723284 [use gst_adapter_available_fast() to determine the top buffer size] Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2014-02-06 06:30:10 +00:00
buf_size = ofs;
h264: initial port to new GstVaapiDecoder API
2012-12-06 13:02:17 +00:00
}
}
h264: optimize scan for the second start code. Optimize scan for the second start code, on the next parse() call so that to avoid scanning again earlier bytes where we didn't find any start code.
2012-12-12 17:33:52 +00:00
ps->input_offset2 = 0;
h264: initial port to new GstVaapiDecoder API
2012-12-06 13:02:17 +00:00
Add initial support for GStreamer 1.0. This integrates support for GStreamer API >= 1.0 only in the libgstvaapi core decoding library. The changes are kept rather minimal here so that the library retains as little dependency as possible on core GStreamer functionality. Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2012-09-04 12:12:18 +00:00
buf = (guchar *)gst_adapter_map(adapter, buf_size);
h264: initial port to new GstVaapiDecoder API
2012-12-06 13:02:17 +00:00
if (!buf)
return GST_VAAPI_DECODER_STATUS_ERROR_NO_DATA;
decoder: refactor decoder unit API. Allocate decoder unit earlier in the main parse() function and don't delegate this task to derived classes. The ultimate purpose is to get rid of dynamic allocation of decoder units.
2013-01-07 10:13:07 +00:00
unit->size = buf_size;
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
pi = gst_vaapi_parser_info_h264_new();
if (!pi)
return GST_VAAPI_DECODER_STATUS_ERROR_ALLOCATION_FAILED;
gst_vaapi_decoder_unit_set_parsed_info(unit,
pi, (GDestroyNotify)gst_vaapi_mini_object_unref);
h264: initial port to new GstVaapiDecoder API
2012-12-06 13:02:17 +00:00
h264: minor clean-ups. Drop explicit initialization of most fields that are implicitly set to zero. Drop helper macros for casting to GstVaapiPictureH264 or GstVaapiFrameStore. Also remove some useless checks for NULL pointers.
2012-12-13 10:48:06 +00:00
if (priv->is_avcC)
h264: initial port to new GstVaapiDecoder API
2012-12-06 13:02:17 +00:00
result = gst_h264_parser_identify_nalu_avc(priv->parser,
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
buf, 0, buf_size, priv->nal_length_size, &pi->nalu);
h264: initial port to new GstVaapiDecoder API
2012-12-06 13:02:17 +00:00
else
result = gst_h264_parser_identify_nalu_unchecked(priv->parser,
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
buf, 0, buf_size, &pi->nalu);
h264: initial port to new GstVaapiDecoder API
2012-12-06 13:02:17 +00:00
status = get_status(result);
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
if (status != GST_VAAPI_DECODER_STATUS_SUCCESS)
h264: initial port to new GstVaapiDecoder API
2012-12-06 13:02:17 +00:00
return status;
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
switch (pi->nalu.type) {
h264: add codec specific decoder unit. Introduce new GstVaapiDecoderUnitH264 object, which holds the standard NAL unit header (GstH264NalUnit) and additional parsed header info. Besides, we now parse headers as early as in the _parse() function so that to avoid un-necessary creation of sub-buffers in _decode() for NAL units that are not slices. This is a performance win by ~+1.1% only.
2012-12-06 16:25:01 +00:00
case GST_H264_NAL_SPS:
status = parse_sps(decoder, unit);
break;
decoder: h264: add support for MVC base views. Allow decoding for base views of MVC encoded streams. For now, just skip the slice extension and prefix NAL units, and skip non-base view frames. Signed-off-by: Xiaowei Li <xiaowei.a.li@intel.com> [fixed memory leak, improved check for MVC NAL units] Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2013-02-28 07:59:55 +00:00
case GST_H264_NAL_SUBSET_SPS:
status = parse_subset_sps(decoder, unit);
break;
h264: add codec specific decoder unit. Introduce new GstVaapiDecoderUnitH264 object, which holds the standard NAL unit header (GstH264NalUnit) and additional parsed header info. Besides, we now parse headers as early as in the _parse() function so that to avoid un-necessary creation of sub-buffers in _decode() for NAL units that are not slices. This is a performance win by ~+1.1% only.
2012-12-06 16:25:01 +00:00
case GST_H264_NAL_PPS:
status = parse_pps(decoder, unit);
break;
case GST_H264_NAL_SEI:
status = parse_sei(decoder, unit);
break;
decoder: h264: add support for MVC base views. Allow decoding for base views of MVC encoded streams. For now, just skip the slice extension and prefix NAL units, and skip non-base view frames. Signed-off-by: Xiaowei Li <xiaowei.a.li@intel.com> [fixed memory leak, improved check for MVC NAL units] Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2013-02-28 07:59:55 +00:00
case GST_H264_NAL_SLICE_EXT:
if (!GST_H264_IS_MVC_NALU(&pi->nalu)) {
status = GST_VAAPI_DECODER_STATUS_SUCCESS;
break;
}
/* fall-through */
h264: add codec specific decoder unit. Introduce new GstVaapiDecoderUnitH264 object, which holds the standard NAL unit header (GstH264NalUnit) and additional parsed header info. Besides, we now parse headers as early as in the _parse() function so that to avoid un-necessary creation of sub-buffers in _decode() for NAL units that are not slices. This is a performance win by ~+1.1% only.
2012-12-06 16:25:01 +00:00
case GST_H264_NAL_SLICE_IDR:
case GST_H264_NAL_SLICE:
status = parse_slice(decoder, unit);
break;
default:
status = GST_VAAPI_DECODER_STATUS_SUCCESS;
break;
}
if (status != GST_VAAPI_DECODER_STATUS_SUCCESS)
return status;
h264: initial port to new GstVaapiDecoder API
2012-12-06 13:02:17 +00:00
flags = 0;
decoder: h264: handle access unit ("au") optimization. Optimize parsing when buffers are supplied with access unit alignment. This helps determining faster when the end of an access unit is reached.
2014-06-13 13:42:53 +00:00
if (at_au_end) {
flags |= GST_VAAPI_DECODER_UNIT_FLAG_FRAME_END |
GST_VAAPI_DECODER_UNIT_FLAG_AU_END;
}
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
switch (pi->nalu.type) {
h264: initial port to new GstVaapiDecoder API
2012-12-06 13:02:17 +00:00
case GST_H264_NAL_AU_DELIMITER:
decoder: h264: fix detection of access unit boundaries. In order to have a stricter conforming implementation, we need to carefully detect access unit boundaries. Additional operations could be necessary to perform at those boundaries.
2014-05-01 17:33:40 +00:00
flags |= GST_VAAPI_DECODER_UNIT_FLAG_AU_START;
h264: initial port to new GstVaapiDecoder API
2012-12-06 13:02:17 +00:00
flags |= GST_VAAPI_DECODER_UNIT_FLAG_FRAME_START;
/* fall-through */
case GST_H264_NAL_FILLER_DATA:
flags |= GST_VAAPI_DECODER_UNIT_FLAG_SKIP;
break;
case GST_H264_NAL_STREAM_END:
flags |= GST_VAAPI_DECODER_UNIT_FLAG_STREAM_END;
/* fall-through */
case GST_H264_NAL_SEQ_END:
flags |= GST_VAAPI_DECODER_UNIT_FLAG_FRAME_END;
decoder: h264: fix detection of access unit boundaries. In order to have a stricter conforming implementation, we need to carefully detect access unit boundaries. Additional operations could be necessary to perform at those boundaries.
2014-05-01 17:33:40 +00:00
flags |= GST_VAAPI_DECODER_UNIT_FLAG_AU_END;
h264: initial port to new GstVaapiDecoder API
2012-12-06 13:02:17 +00:00
break;
case GST_H264_NAL_SPS:
decoder: h264: add support for MVC base views. Allow decoding for base views of MVC encoded streams. For now, just skip the slice extension and prefix NAL units, and skip non-base view frames. Signed-off-by: Xiaowei Li <xiaowei.a.li@intel.com> [fixed memory leak, improved check for MVC NAL units] Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2013-02-28 07:59:55 +00:00
case GST_H264_NAL_SUBSET_SPS:
h264: initial port to new GstVaapiDecoder API
2012-12-06 13:02:17 +00:00
case GST_H264_NAL_PPS:
case GST_H264_NAL_SEI:
decoder: h264: fix detection of access unit boundaries. In order to have a stricter conforming implementation, we need to carefully detect access unit boundaries. Additional operations could be necessary to perform at those boundaries.
2014-05-01 17:33:40 +00:00
flags |= GST_VAAPI_DECODER_UNIT_FLAG_AU_START;
h264: initial port to new GstVaapiDecoder API
2012-12-06 13:02:17 +00:00
flags |= GST_VAAPI_DECODER_UNIT_FLAG_FRAME_START;
break;
decoder: h264: add support for MVC base views. Allow decoding for base views of MVC encoded streams. For now, just skip the slice extension and prefix NAL units, and skip non-base view frames. Signed-off-by: Xiaowei Li <xiaowei.a.li@intel.com> [fixed memory leak, improved check for MVC NAL units] Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2013-02-28 07:59:55 +00:00
case GST_H264_NAL_SLICE_EXT:
if (!GST_H264_IS_MVC_NALU(&pi->nalu)) {
flags |= GST_VAAPI_DECODER_UNIT_FLAG_SKIP;
break;
}
/* fall-through */
h264: initial port to new GstVaapiDecoder API
2012-12-06 13:02:17 +00:00
case GST_H264_NAL_SLICE_IDR:
case GST_H264_NAL_SLICE:
flags |= GST_VAAPI_DECODER_UNIT_FLAG_SLICE;
decoder: h264: slightly optimize the process to detect new pictures. Optimize the process to detect new pictures or start of new access units by checking if the previous NAL unit was the end of a picture, or the end of the previous access unit.
2014-06-30 09:06:29 +00:00
if (priv->prev_pi &&
(priv->prev_pi->flags & GST_VAAPI_DECODER_UNIT_FLAG_AU_END)) {
flags |= GST_VAAPI_DECODER_UNIT_FLAG_AU_START |
GST_VAAPI_DECODER_UNIT_FLAG_FRAME_START;
}
else if (is_new_picture(pi, priv->prev_slice_pi)) {
h264: initial port to new GstVaapiDecoder API
2012-12-06 13:02:17 +00:00
flags |= GST_VAAPI_DECODER_UNIT_FLAG_FRAME_START;
decoder: h264: fix detection of access unit boundaries. In order to have a stricter conforming implementation, we need to carefully detect access unit boundaries. Additional operations could be necessary to perform at those boundaries.
2014-05-01 17:33:40 +00:00
if (is_new_access_unit(pi, priv->prev_slice_pi))
flags |= GST_VAAPI_DECODER_UNIT_FLAG_AU_START;
}
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
gst_vaapi_parser_info_h264_replace(&priv->prev_slice_pi, pi);
h264: initial port to new GstVaapiDecoder API
2012-12-06 13:02:17 +00:00
break;
decoder: h264: skip SPS extension and auxiliary slice NALs. When NAL units of type 13 (SPS extension) or type 19 (auxiliary slice) are present in a video, decoders shall perform the (optional) decoding process specified for these NAL units or shall ignore them (7.4.1). Implement option 2 (skip) for now, as alpha composition is not supported yet during the decoding process. This fixes decoding of the primary coded video in alphaconformanceG. https://bugzilla.gnome.org/show_bug.cgi?id=703928 https://bugzilla.gnome.org/show_bug.cgi?id=728869 https://bugzilla.gnome.org/show_bug.cgi?id=724518 [skip NAL units earlier, i.e. at parsing time] Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2013-06-04 07:01:46 +00:00
case GST_H264_NAL_SPS_EXT:
case GST_H264_NAL_SLICE_AUX:
/* skip SPS extension and auxiliary slice for now */
flags |= GST_VAAPI_DECODER_UNIT_FLAG_SKIP;
break;
decoder: h264: add support for MVC base views. Allow decoding for base views of MVC encoded streams. For now, just skip the slice extension and prefix NAL units, and skip non-base view frames. Signed-off-by: Xiaowei Li <xiaowei.a.li@intel.com> [fixed memory leak, improved check for MVC NAL units] Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2013-02-28 07:59:55 +00:00
case GST_H264_NAL_PREFIX_UNIT:
/* skip Prefix NAL units for now */
flags |= GST_VAAPI_DECODER_UNIT_FLAG_SKIP |
decoder: h264: fix detection of access unit boundaries. In order to have a stricter conforming implementation, we need to carefully detect access unit boundaries. Additional operations could be necessary to perform at those boundaries.
2014-05-01 17:33:40 +00:00
GST_VAAPI_DECODER_UNIT_FLAG_AU_START |
decoder: h264: add support for MVC base views. Allow decoding for base views of MVC encoded streams. For now, just skip the slice extension and prefix NAL units, and skip non-base view frames. Signed-off-by: Xiaowei Li <xiaowei.a.li@intel.com> [fixed memory leak, improved check for MVC NAL units] Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2013-02-28 07:59:55 +00:00
GST_VAAPI_DECODER_UNIT_FLAG_FRAME_START;
break;
h264: initial port to new GstVaapiDecoder API
2012-12-06 13:02:17 +00:00
default:
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
if (pi->nalu.type >= 14 && pi->nalu.type <= 18)
decoder: h264: fix detection of access unit boundaries. In order to have a stricter conforming implementation, we need to carefully detect access unit boundaries. Additional operations could be necessary to perform at those boundaries.
2014-05-01 17:33:40 +00:00
flags |= GST_VAAPI_DECODER_UNIT_FLAG_AU_START |
GST_VAAPI_DECODER_UNIT_FLAG_FRAME_START;
h264: initial port to new GstVaapiDecoder API
2012-12-06 13:02:17 +00:00
break;
}
decoder: h264: fix detection of access unit boundaries. In order to have a stricter conforming implementation, we need to carefully detect access unit boundaries. Additional operations could be necessary to perform at those boundaries.
2014-05-01 17:33:40 +00:00
if ((flags & GST_VAAPI_DECODER_UNIT_FLAGS_AU) && priv->prev_slice_pi)
priv->prev_slice_pi->flags |= GST_VAAPI_DECODER_UNIT_FLAG_AU_END;
h264: initial port to new GstVaapiDecoder API
2012-12-06 13:02:17 +00:00
GST_VAAPI_DECODER_UNIT_FLAG_SET(unit, flags);
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
pi->nalu.data = NULL;
decoder: h264: fix robustness patch for bytestream format. Fix parser and decoder state to sync at the right locations. This is because we could reset the parser state, while the decoder state was not copied yet, e.g. when parsing several NAL units from multiple frames whereas the current frame was not decoded yet. This is a regression brought in by commit 6fe5496.
2014-02-04 17:35:28 +00:00
pi->state = priv->parser_state;
decoder: h264: fix detection of access unit boundaries. In order to have a stricter conforming implementation, we need to carefully detect access unit boundaries. Additional operations could be necessary to perform at those boundaries.
2014-05-01 17:33:40 +00:00
pi->flags = flags;
decoder: h264: properly handle Prefix NAL units. Always cache the previous NAL unit so that we could check whether there is a Prefix NAL unit immediately preceding the current slice or IDR NAL unit. In that case, the NAL unit metadata is copied into the current NAL unit. Otherwise, some default values are inferred, tentatively. e.g. view_id shall be set to 0 and inter_view_flag to 1. [infer default values for slice if previous NAL was not a Prefix] Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2013-10-31 11:32:55 +00:00
gst_vaapi_parser_info_h264_replace(&priv->prev_pi, pi);
h264: initial port to new GstVaapiDecoder API
2012-12-06 13:02:17 +00:00
return GST_VAAPI_DECODER_STATUS_SUCCESS;
}
static GstVaapiDecoderStatus
gst_vaapi_decoder_h264_decode(GstVaapiDecoder *base_decoder,
GstVaapiDecoderUnit *unit)
{
h264: minor clean-ups. Drop explicit initialization of most fields that are implicitly set to zero. Drop helper macros for casting to GstVaapiPictureH264 or GstVaapiFrameStore. Also remove some useless checks for NULL pointers.
2012-12-13 10:48:06 +00:00
GstVaapiDecoderH264 * const decoder =
GST_VAAPI_DECODER_H264_CAST(base_decoder);
h264: initial port to new GstVaapiDecoder API
2012-12-06 13:02:17 +00:00
GstVaapiDecoderStatus status;
status = ensure_decoder(decoder);
if (status != GST_VAAPI_DECODER_STATUS_SUCCESS)
return status;
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
return decode_unit(decoder, unit);
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
}
h264: implement {start,end}_frame() hooks. Implement GstVaapiDecoder.start_frame() and end_frame() semantics so that to create new VA context earlier and submit VA pictures to the HW for decoding as soon as possible. i.e. don't wait for the next frame to start decoding the previous one.
2012-12-13 09:21:46 +00:00
static GstVaapiDecoderStatus
gst_vaapi_decoder_h264_start_frame(GstVaapiDecoder *base_decoder,
h264: introduce parser info instead of H.264 specific decoder unit. Use a new GstVaapiParserInfoH264 data structure instead of deriving from GstVaapiDecoderUnit for H.264 specific parser information.
2013-01-07 09:22:54 +00:00
GstVaapiDecoderUnit *unit)
h264: implement {start,end}_frame() hooks. Implement GstVaapiDecoder.start_frame() and end_frame() semantics so that to create new VA context earlier and submit VA pictures to the HW for decoding as soon as possible. i.e. don't wait for the next frame to start decoding the previous one.
2012-12-13 09:21:46 +00:00
{
h264: minor clean-ups. Drop explicit initialization of most fields that are implicitly set to zero. Drop helper macros for casting to GstVaapiPictureH264 or GstVaapiFrameStore. Also remove some useless checks for NULL pointers.
2012-12-13 10:48:06 +00:00
GstVaapiDecoderH264 * const decoder =
GST_VAAPI_DECODER_H264_CAST(base_decoder);
h264: implement {start,end}_frame() hooks. Implement GstVaapiDecoder.start_frame() and end_frame() semantics so that to create new VA context earlier and submit VA pictures to the HW for decoding as soon as possible. i.e. don't wait for the next frame to start decoding the previous one.
2012-12-13 09:21:46 +00:00
return decode_picture(decoder, unit);
}
static GstVaapiDecoderStatus
gst_vaapi_decoder_h264_end_frame(GstVaapiDecoder *base_decoder)
{
h264: minor clean-ups. Drop explicit initialization of most fields that are implicitly set to zero. Drop helper macros for casting to GstVaapiPictureH264 or GstVaapiFrameStore. Also remove some useless checks for NULL pointers.
2012-12-13 10:48:06 +00:00
GstVaapiDecoderH264 * const decoder =
GST_VAAPI_DECODER_H264_CAST(base_decoder);
h264: implement {start,end}_frame() hooks. Implement GstVaapiDecoder.start_frame() and end_frame() semantics so that to create new VA context earlier and submit VA pictures to the HW for decoding as soon as possible. i.e. don't wait for the next frame to start decoding the previous one.
2012-12-13 09:21:46 +00:00
return decode_current_picture(decoder);
}
h264: implement GstVaapiDecoder::flush() as a DPB flush.
2013-01-17 17:35:58 +00:00
static GstVaapiDecoderStatus
gst_vaapi_decoder_h264_flush(GstVaapiDecoder *base_decoder)
{
GstVaapiDecoderH264 * const decoder =
GST_VAAPI_DECODER_H264_CAST(base_decoder);
decoder: h264: simplify storage of decoded picture into DPB. Factor out process by which the decoded picture with the lowest POC is found, and possibly output. Likewise, the storage and marking of a reference decoded, or non-reference decoded picture, into the DPB could also be simplified as they mostly share the same operations.
2014-05-04 12:49:28 +00:00
dpb_flush(decoder, NULL);
h264: implement GstVaapiDecoder::flush() as a DPB flush.
2013-01-17 17:35:58 +00:00
return GST_VAAPI_DECODER_STATUS_SUCCESS;
}
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
static void
gst_vaapi_decoder_h264_class_init(GstVaapiDecoderH264Class *klass)
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
GstVaapiMiniObjectClass * const object_class =
GST_VAAPI_MINI_OBJECT_CLASS(klass);
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
GstVaapiDecoderClass * const decoder_class = GST_VAAPI_DECODER_CLASS(klass);
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
object_class->size = sizeof(GstVaapiDecoderH264);
object_class->finalize = (GDestroyNotify)gst_vaapi_decoder_finalize;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
decoder_class->create = gst_vaapi_decoder_h264_create;
decoder_class->destroy = gst_vaapi_decoder_h264_destroy;
h264: initial port to new GstVaapiDecoder API
2012-12-06 13:02:17 +00:00
decoder_class->parse = gst_vaapi_decoder_h264_parse;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
decoder_class->decode = gst_vaapi_decoder_h264_decode;
h264: implement {start,end}_frame() hooks. Implement GstVaapiDecoder.start_frame() and end_frame() semantics so that to create new VA context earlier and submit VA pictures to the HW for decoding as soon as possible. i.e. don't wait for the next frame to start decoding the previous one.
2012-12-13 09:21:46 +00:00
decoder_class->start_frame = gst_vaapi_decoder_h264_start_frame;
decoder_class->end_frame = gst_vaapi_decoder_h264_end_frame;
h264: implement GstVaapiDecoder::flush() as a DPB flush.
2013-01-17 17:35:58 +00:00
decoder_class->flush = gst_vaapi_decoder_h264_flush;
decoder: sanitize codec-data decoding. Add a new GstVaapiDecoder::decode_codec_data() hook to actually decode codec-data in the decoder sub-class. Provide a common shared helper function to do the actual work and delegating further to the sub-class.
2013-03-21 13:36:40 +00:00
decoder_class->decode_codec_data =
gst_vaapi_decoder_h264_decode_codec_data;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
}
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
static inline const GstVaapiDecoderClass *
gst_vaapi_decoder_h264_class(void)
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
static GstVaapiDecoderH264Class g_class;
static gsize g_class_init = FALSE;
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
if (g_once_init_enter(&g_class_init)) {
gst_vaapi_decoder_h264_class_init(&g_class);
g_once_init_leave(&g_class_init, TRUE);
}
return GST_VAAPI_DECODER_CLASS(&g_class);
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
}
decoder: h264: add support for NALU "alignment" optimization. We can avoid scanning for start codes again if the bitstream is fed in NALU chunks. Currently, we always scan for start codes, and keep track of remaining bits in a GstAdapter, even if, in practice, we are likely receiving one GstBuffer per NAL unit. i.e. h264parse with "nal" alignment. https://bugzilla.gnome.org/show_bug.cgi?id=723284 [use gst_adapter_available_fast() to determine the top buffer size] Signed-off-by: Gwenole Beauchesne <gwenole.beauchesne@intel.com>
2014-02-06 06:30:10 +00:00
/**
* gst_vaapi_decoder_h264_set_alignment:
* @decoder: a #GstVaapiDecoderH264
* @alignment: the #GstVaapiStreamAlignH264
*
* Specifies how stream buffers are aligned / fed, i.e. the boundaries
* of each buffer that is supplied to the decoder. This could be no
* specific alignment, NAL unit boundaries, or access unit boundaries.
*/
void
gst_vaapi_decoder_h264_set_alignment(GstVaapiDecoderH264 *decoder,
GstVaapiStreamAlignH264 alignment)
{
g_return_if_fail(decoder != NULL);
decoder->priv.stream_alignment = alignment;
}
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
/**
* gst_vaapi_decoder_h264_new:
* @display: a #GstVaapiDisplay
* @caps: a #GstCaps holding codec information
*
* Creates a new #GstVaapiDecoder for MPEG-2 decoding. The @caps can
* hold extra information like codec-data and pictured coded size.
*
* Return value: the newly allocated #GstVaapiDecoder object
*/
GstVaapiDecoder *
gst_vaapi_decoder_h264_new(GstVaapiDisplay *display, GstCaps *caps)
{
libs: use GstVaapiMiniObject for video decoders. Port GstVaapiDecoder and GstVaapiDecoder{MPEG2,MPEG4,JPEG,H264,VC1} to GstVaapiMiniObject. Add gst_vaapi_decoder_set_codec_state_changed_func() helper function to let the user add a callback to a function triggered whenever the codec state (e.g. caps) changes.
2013-05-06 12:07:17 +00:00
return gst_vaapi_decoder_new(gst_vaapi_decoder_h264_class(), display, caps);
Add initial H.264 decoder.
2011-08-12 15:43:55 +00:00
}
Reference in a new issue Copy permalink