The max_num_reorder_frames number can change the way we bumping the
pictures in the DPB. The smaller it is, the lower latency we will
get. So it is important for live mode streams, but it is not given
in VUI parameters sometimes. We now improve the policy to infer it:
1. Never guess it in the "strict" compliance.
2. For baseline and constrained baseline profiles, which do not have
B frames, set it to 0.
3. For -intra only profiles, set it to 0.
4. Otherwise, not guess it.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2501>
We control the policy of fast bump by the profile and the compliance
property. For baseline and constrained baseline profiles, we can use
more radical bump policy. User can also change the bump policy by
setting the compliance property in run time.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2432>
Some features such as the low-latency DPB bumping and mapping the
baseline profile as the constrained-baseline profile do not conform
to the H264 offical spec. But in practice, they are very useful and
are widely needed. We add this compliance property to control the
behavior of the decoder, make it fit more requirement.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2432>
During the output process, if there are multiple frames in a TU (i.e. multi-spatial
layers case), only one frame with the highest spatial layer id should be selected
according to av1 spec. The highest spatial layer id is obtained from idc value of
the operating point.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2475>
GstWebRTCSCTPTransport is now made into into an abstract base class
that only contains property specifications matching the
RTCSctpTransport interface of the W3C WebRTC specification, see
https://w3c.github.io/webrtc-pc/#rtcsctptransport-interface. This
class is put into the WebRTC library to expose it for applications and
to allow for generation of bindings for non-dynamic languages using
GObject introspection.
The actual implementation is moved to the subclass WebRTCSCTPTransport
located in the WebRTC plugin.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2214>
User can get the required buffer size by using buffer pool config.
Since d3d11 implementation is a candidate for public library in the future,
we need to hide everything from header as much as possible.
Note that the total size of allocated d3d11 texture memory by GPU is not
controllable factor. It depends on hardware specific alignment/padding
requirement. So, GstD3D11 implementation updates actual buffer size
by allocating D3D11 texture, since there's no way to get CPU accessible
memory size without allocating real D3D11 texture.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2482>
We forget one case that is the frame and field pictures may be mixed
together. For this case, the dpb is interlaced while the last picture
may be a complete frame. We do not need to cache that complete picture
and should output it directly.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2448>
We need to consider the first field of the last picture when the
last picture can not enter the DPB.
Another change is, when prev field's frame_num is not equal to the
current field's frame_num, we should also return FASLE because it
is also a case of losing some field.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2430>
For interlaced streams, it is also possible that the last frame is
not able to be inserted into DPB when the DPB is full and the last
frame is a non ref. For this case, we need to hold a extra ref for
the first field of the last frame and wait for the complete frame
with both top and bottom fields. For the progressive stream, the
behaviour is unchanged.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2430>
The print_ref_pic_list_b now not only needs to trace the ref_pic_list_b0/1,
but also need to trace the ref_frame_list_0_short_term. We need to pass the
name directly to it rather than an index to refer to ref_pic_list_b0/1.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2425>
In low_latency mode, try to bump the picture as soon as possible
without the frames disorder:
1. We can directly output the continuous non-reference frame.
2. Consider max_num_reorder_frames, which is special useful for
I-P mode.
3. Consider the leading pictures with negative POC.
4 Output small POC pictures when non-reference frame comes.
4. Output the POC increment<=2 pictures. This is not 100% safe,
but in practice this condition can be used.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2373>
The picture->ref field will change from time to time according to decoder's
state and reference sliding window. We need another flag to record whether
the picture is a reference picture when it is created, and this can help
the bumping check.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2373>
Accord to spec, we should not add the current picture into the DPB
when we check whether it needs to bump, so the checks of the IDR and
the "memory_management_control_operation equal to 5" are no needed.
And the spec also says that the DPB only needs to bump when there is
no empty frame buffer left(We handle the IDR cases in other places).
We need to follow that and the max_num_reorder_frames is useless.
We also minus 1 in has_empty_frame_buffer because the current frame
has not been added yet.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2373>
When current frame memory_management_control_operation equal to 5, that
means we need to drain the dpb and the current picture act as an IDR frame.
So it should have smaller poc than the later pictures to ensure the output
order.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2373>
We should use the NumPocTotalCurr value stored in decoder, which is a calculated
valid value, rather than use the invalid value in the slice header. Most of the
time, the NumPocTotalCurr is 0 and make the tmp_refs a very short length, and
causes the decoder's wrong result.
By the way, the NumPocTotalCurr is not the correct name specified in H265 spec,
its name should be NumPicTotalCurr. We change it to the correct name.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2414>
The spec mandates this field be parsed using unsigned arithmetic. Nevertheless,
av1parser will use -1 apparently as an uninitialized value in
gst_av1_parse_frame_header. This immediately underflows last_frame_idx
though, since its type was defined as guint8. Fix this by converting to gint8.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2401>
failure to pass a display in 'handle' would result in uninitialized value
being returned, which would often segfault later down the road when trying
to initialize gstreamer context with it.
Check the return value of gst_structure_get() to make sure we return valid
data.
Furthermore, the gstglimagesink in gst-plugins-base also has a similar
mechanism but uses 'display' as field name to pass the value; instead of
requiring the application to behave differently depending on what sink
was automatically detected just try to read both values here, with display
being the new default.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2292>
There may be leading frames after the IRAP frames, which has negative
POC. This kind of frames are allowed and they will be displayed before
the IRAP frame. So the warning should not be triggered for them. Init
the last_output_poc to G_MININT32 can avoid this.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2380>
There may be leading frames after the IDR frame, which has negative
POC. This kind of frames are allowed and they will be displayed before
the IDR frame. So the warning should not be triggered for them. Init
the last_output_poc to G_MININT32 can avoid this.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2380>
We are dropping RASL (Random Access Skipped Leading picture) which
is associated with an IRAP (Intra Random Access Picture) that has
NoRaslOutputFlag equal to 1, since the RASL picture will not be
outputted and also it should not be used for reference picture.
So, corresponding GstVideoCodecFrame should be released immediately.
Otherwise GstVideoDecoder baseclass will hold the unused frame.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2330>
The build fails on macos with the following error:
/usr/local/Cellar/opencv/4.5.0_5/include/opencv4/opencv2/core/mat.hpp:2226:15: error: no template named 'initializer_list' in namespace 'std'
Mat_(std::initializer_list<_Tp> values);
fatal error: too many errors emitted, stopping now [-ferror-limit=]
35 warnings and 20 errors generated.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2368>
