During the output process, if there are multiple frames in a TU (i.e. multi-spatial
layers case), only one frame with the highest spatial layer id should be selected
according to av1 spec. The highest spatial layer id is obtained from idc value of
the operating point.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2475>
We forget one case that is the frame and field pictures may be mixed
together. For this case, the dpb is interlaced while the last picture
may be a complete frame. We do not need to cache that complete picture
and should output it directly.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2448>
We need to consider the first field of the last picture when the
last picture can not enter the DPB.
Another change is, when prev field's frame_num is not equal to the
current field's frame_num, we should also return FASLE because it
is also a case of losing some field.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2430>
For interlaced streams, it is also possible that the last frame is
not able to be inserted into DPB when the DPB is full and the last
frame is a non ref. For this case, we need to hold a extra ref for
the first field of the last frame and wait for the complete frame
with both top and bottom fields. For the progressive stream, the
behaviour is unchanged.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2430>
The print_ref_pic_list_b now not only needs to trace the ref_pic_list_b0/1,
but also need to trace the ref_frame_list_0_short_term. We need to pass the
name directly to it rather than an index to refer to ref_pic_list_b0/1.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2425>
In low_latency mode, try to bump the picture as soon as possible
without the frames disorder:
1. We can directly output the continuous non-reference frame.
2. Consider max_num_reorder_frames, which is special useful for
I-P mode.
3. Consider the leading pictures with negative POC.
4 Output small POC pictures when non-reference frame comes.
4. Output the POC increment<=2 pictures. This is not 100% safe,
but in practice this condition can be used.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2373>
The picture->ref field will change from time to time according to decoder's
state and reference sliding window. We need another flag to record whether
the picture is a reference picture when it is created, and this can help
the bumping check.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2373>
Accord to spec, we should not add the current picture into the DPB
when we check whether it needs to bump, so the checks of the IDR and
the "memory_management_control_operation equal to 5" are no needed.
And the spec also says that the DPB only needs to bump when there is
no empty frame buffer left(We handle the IDR cases in other places).
We need to follow that and the max_num_reorder_frames is useless.
We also minus 1 in has_empty_frame_buffer because the current frame
has not been added yet.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2373>
When current frame memory_management_control_operation equal to 5, that
means we need to drain the dpb and the current picture act as an IDR frame.
So it should have smaller poc than the later pictures to ensure the output
order.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2373>
We should use the NumPocTotalCurr value stored in decoder, which is a calculated
valid value, rather than use the invalid value in the slice header. Most of the
time, the NumPocTotalCurr is 0 and make the tmp_refs a very short length, and
causes the decoder's wrong result.
By the way, the NumPocTotalCurr is not the correct name specified in H265 spec,
its name should be NumPicTotalCurr. We change it to the correct name.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2414>
There may be leading frames after the IRAP frames, which has negative
POC. This kind of frames are allowed and they will be displayed before
the IRAP frame. So the warning should not be triggered for them. Init
the last_output_poc to G_MININT32 can avoid this.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2380>
There may be leading frames after the IDR frame, which has negative
POC. This kind of frames are allowed and they will be displayed before
the IDR frame. So the warning should not be triggered for them. Init
the last_output_poc to G_MININT32 can avoid this.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2380>
We are dropping RASL (Random Access Skipped Leading picture) which
is associated with an IRAP (Intra Random Access Picture) that has
NoRaslOutputFlag equal to 1, since the RASL picture will not be
outputted and also it should not be used for reference picture.
So, corresponding GstVideoCodecFrame should be released immediately.
Otherwise GstVideoDecoder baseclass will hold the unused frame.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2330>
We always get a warning such as:
h265decoder gsth265decoder.c:1432:gst_h265_decoder_do_output_picture: \
<vah265dec0> Outputting out of order 255 -> 0, likely a broken stream
in H265 decoder.
The problem is caused because we fail to reset the last_output_poc when
we get IDR and BLA. The incoming IDR and BLA frame already bump all the
frames in the DPB, but we forget to reset the last_output_poc, which
make the POC out of order and generate the warning all the time.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2294>
The current DPB check of H265 is not very correct. The current frame
is already in the DPB when we check whether the DPB is full.
For example, the DPB max size is 16 and we have 15 ref frames in the
DPB, so the gst_h265_dpb_delete_unused() cleans no one, and then plus
the current frame, the DPB is 16. This causes an error return, but in
fact, the stream is correct.
We now integrate the DPB full check into the need_bump() function.
We add the correct frame into to DPB and then check whether the picture
num is bigger than max_num_pics of DPB(which means there is no room for
the current picture). If true, we bump the DPB immediately.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2291>
Some decoding APIs support delayed output for performance reasons.
One example would be to request decoding for multiple frames and
then query for the oldest frame in the output queue.
This also increases throughput for transcoding and improves seek
performance when supported by the underlying backend.
Introduce support in the vp8 base class, so that backends that
support render delays can actually implement it.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2150>
Do not carry over segmentation flags from previous frames. The spec
says in 7.2.10 that the feature data carry over from previous frames
if not updated, but the flags do not.
Consider what would happen if a flag B is to depend on a flag A, and
B carries over as set from another frame. Further consider that A is
now not set in this particular frame. This leads to the invalid state
in which flag B is set but flag A isn't.
This might cause the bitstream to be rejected by accelerators down
the line.
Fix it.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2203>
The default implementation was required when superframe parsing
was handled by vp9decoder. For instance, if a superframe consists
of multiple frames with show_existing_frame header, it was vague
that which GstVp9Picture should consume GstVideoCodecFrame.
After 1.18 release, we introduced vp9parse element and
superframe should be handled by upstream vp9parse elemenet now.
So, we don't need to care about the superframe at vp9decoder class
level anymore. Simply, a frame corresponding to show_existing_frame
can be dropped if subclass doesn't implement duplicate_picture().
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2151>
Existing VP9 parser implementation doesn't provide information
required by other stateless decoding APIs (i.e., DXVA and NVDEC),
specifically loop filter and segmentation parameters might not exist
current frame. So parser needs to fill the information by using previously
parsed information.
We can update the gstvp9parser implementation so that it can provide
all information required by stateless decoding APIs with a huge API break,
or adding more ugly struct in it.
Instead doing as such, this commit introduce a new VP9 parser implementation.
What is different from existing one?
* All variables will follow the specification as much as possible:
VP9 Bitstream & Decoding Process Specification - v0.6 31st March 2016
* Parser will fill all the required information for decoding frame
to GstVp9FrameHeader struct. In case of old VP9 parser,
user needs to read additional data from parser's member variables.
* GstVp9StatefulParser object struct is completely completely opaque
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2112>
In the current code, we call frame_unref only when the frame is
outputted. This is OK for normal playback, but when seek happens,
the frames stored in DPB is not outputted and causes some memory
leak.
The correct way is that we should call frame_unref every time we
finish the handle_frame(), which is also the behaviour of H264/H265
decoder.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2014>
* Invoke GstH265DecoderClass::new_sequence() method per interlaced
stream status update so that subclass can update caps.
* Parse picture timing SEI and set buffer flags on GstH265Picture
object. Subclass can refer to it like that of our h264decoder
implementation.
* Remove pointless GstH265PictureField enum
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2008>
Some decoding APIs support delayed output or a command for decoding
a frame doesn't need to be sequential to corresponding command for
getting decoded frame. For instance, subclass might be able to
request decoding for multiple frames and then get for one (oldest)
decoded frame or so.
If aforementioned case is supported by specific decoding API,
delayed output might show better throughput performance.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1925>
