We forget one case that is the frame and field pictures may be mixed
together. For this case, the dpb is interlaced while the last picture
may be a complete frame. We do not need to cache that complete picture
and should output it directly.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2448>
We need to consider the first field of the last picture when the
last picture can not enter the DPB.
Another change is, when prev field's frame_num is not equal to the
current field's frame_num, we should also return FASLE because it
is also a case of losing some field.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2430>
For interlaced streams, it is also possible that the last frame is
not able to be inserted into DPB when the DPB is full and the last
frame is a non ref. For this case, we need to hold a extra ref for
the first field of the last frame and wait for the complete frame
with both top and bottom fields. For the progressive stream, the
behaviour is unchanged.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2430>
The print_ref_pic_list_b now not only needs to trace the ref_pic_list_b0/1,
but also need to trace the ref_frame_list_0_short_term. We need to pass the
name directly to it rather than an index to refer to ref_pic_list_b0/1.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2425>
In low_latency mode, try to bump the picture as soon as possible
without the frames disorder:
1. We can directly output the continuous non-reference frame.
2. Consider max_num_reorder_frames, which is special useful for
I-P mode.
3. Consider the leading pictures with negative POC.
4 Output small POC pictures when non-reference frame comes.
4. Output the POC increment<=2 pictures. This is not 100% safe,
but in practice this condition can be used.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2373>
The picture->ref field will change from time to time according to decoder's
state and reference sliding window. We need another flag to record whether
the picture is a reference picture when it is created, and this can help
the bumping check.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2373>
Accord to spec, we should not add the current picture into the DPB
when we check whether it needs to bump, so the checks of the IDR and
the "memory_management_control_operation equal to 5" are no needed.
And the spec also says that the DPB only needs to bump when there is
no empty frame buffer left(We handle the IDR cases in other places).
We need to follow that and the max_num_reorder_frames is useless.
We also minus 1 in has_empty_frame_buffer because the current frame
has not been added yet.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2373>
There may be leading frames after the IDR frame, which has negative
POC. This kind of frames are allowed and they will be displayed before
the IDR frame. So the warning should not be triggered for them. Init
the last_output_poc to G_MININT32 can avoid this.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2380>
Some decoding APIs support delayed output or a command for decoding
a frame doesn't need to be sequential to corresponding command for
getting decoded frame. For instance, subclass might be able to
request decoding for multiple frames and then get for one (oldest)
decoded frame or so.
If aforementioned case is supported by specific decoding API,
delayed output might show better throughput performance.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1925>
In case that "pic_order_cnt_type" is equal to zero, ref picture
list for B slice should not include non-existing picture
as per spec 8.2.4.2.3. And, the second field is not needed
for the process of frame picture reference list construction
since it needs to be frame unit, not field picture in that case.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1812>
We were trying to construct reference picture list even for
I slice before this commit. Reference list is required only for
P, SP and B slices. Also, if all existing reference pictures
are gap pictures, we don't need to construct lists.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1812>
Initial support for interlaced stream. Subclass should implement
new_field_picture() vfunc. Otherwise, baseclass will assume that
subclass doesn't support interlaced stream.
Restrictions:
* Reference picture modification process for interlaced stream
is not implemented yet
* PAFF (Picture Adaptive Frame Field) is not properly implemented.
* Field display ordering (e.g., top-field-first) decision should
be enhanced via picture timing SEI parsing
* Gap in field picture should be handled
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1534>
Managing reference picture type by using two variables
(ref and long_term) seems to be redundant and that can be
represented by using a single enum value.
This is to sync this implementation with gstreamer-vaapi so that
make comparison between this and gstreamer-vaapi easier and also
in order to minimize the change required for subclass to be able
to support interlaced.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1534>
This is to avoid having to create a reference chain in decoders from
GstVideoCodecFrame to GstH264PIcture to implementation wrapper.
So this change introduces:
- gst_h264_dpb_remove_outputed (dpb)
- gst_h264_dpb_get_picture(dpb, system_frame_num)
- gst_h264_decoder_get_picture (dec, system_frame_num)
In order to ensure that frames can be looked up during the draining
process, we now first remove all (including reference) frames that
have been outputed but are still in the DPB. Then for each remaining
buffers, we remove it from the DPB to reach reference 1 and output it.
Previously we could take all not outputed outside of the DPB which would
prevent lookup by the base class.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1449>
... when gst_video_decoder_finish_frame() is called.
Some subclasses attach GstBuffer to GstH264Picture as an user data
which will increase reference count of the buffer. It would result
to buffer copy per every frame.
Background here is, GstVideoDecoder wants writable output buffer for
GstMeta handling, and if the output buffer is not writable
(i.e., reference count is not one), the buffer will be copied.
Even if underlying GstMemory wouldn't be copied, buffer copy operation
will introduce extra memory allocation overhead which is not optimal.
By this modification, subclass might be able to receive the last
reference to GstH264Picture
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1449>
All subclasses are retrieving list to get target output frame, which
can be done by baseclass. And pass the ownership of the GstH264Picture
to subclass so that subclass can clear implementation dependent resources
before finishing the frame.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1449>
There are some streams, with HRD, where the the calculated
max_dpb_frames is zero (max_dpb_mbs is less than size mb). In order to
get the dbp size it is required to rely on the VUI parameters if they
are present.
According to the spec Annex E.2.1
**max_dec_frame_buffering** specifies the required size of the HRD
decoded picture buffer (DPB) in units of frame buffers. It is a
requirement of bitstream conformance that the coded video sequence
shall not require a decoded picture buffer with size of more than
Max(1, max_dec_frame_buffering) frame buffers to enable the output of
decoded pictures at the output times specified by dpb_output_delay of
the picture timing SEI messages. The value of max_dec_frame_buffering
shall be greater than or equal to max_num_ref_frames. An upper bound
for the value of max_dec_frame_buffering is specified by the level
limits in clauses A.3.1, A.3.2, G.10.2.1, and H.10.2.
When the max_dec_frame_buffering syntax element is not present, the
value of max_dec_frame_buffering shall be inferred as follows:
– If profile_idc is equal to 44, 86, 100, 110, 122, or 244 and
constraint_set3_flag is equal to 1, the value of
max_dec_frame_buffering shall be inferred to be equal to 0.
– Otherwise (profile_idc is not equal to 44, 86, 100, 110, 122, or 244
or constraint_set3_flag is equal to 0), the value of
max_dec_frame_buffering shall be inferred to be equal to MaxDpbFrames.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1381>
