Commit graph

4312 commits

Author SHA1 Message Date
Daniel Almeida
b8c826afe3 codecs: gstvp9statefulparser: feature_data should be 0 if feature_enable is 0
The spec says in 6.2.11 that feature_data[i][j] should be zero if
feature_enabled[i][j] is zero. Instead we retained the old value in the parser.
Fix it.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2449>
2021-09-09 18:32:42 +00:00
Marek Vasut
8239ff343f gsth264parser: Fix handling of NALs with emulation byte set
In case a set of NALs with emulation_prevention_three_byte is decoded using
hardware decoder like Hantro G1, wrong struct v4l2_ctrl_h264_decode_params
.dec_ref_pic_marking_bit_size is passed into the kernel, which results in
decoding artifacts. Subtract the number of emulation three bytes from the
.dec_ref_pic_m->bit_size to get the correct bit size and avoid having these
artifacts. Apply the exact same fix to slice->pic_order_cnt_bit_size as well.

The following NALs (7, 8, 6, 5) decode with artifacts,
.dec_ref_pic_marking_bit_size is set to 10 without this patch.
00000000  00 00 00 01 27 4d 00 20  89 8b 60 3c 04 bf 2e 02  |....'M. ..`<....|
00000010  d4 18 04 18 c0 c0 01 77  00 00 5d c1 7b df 05 00  |.......w..].{...|
00000020  00 00 01 28 ee 1f 20 00  00 01 06 05 10 b9 ed b9  |...(.. .........|
00000030  30 5d 21 4b 71 83 71 2c  10 a3 14 bb 29 80 00 00  |0]!Kq.q,....)...|
00000040  01 25 b8 00 05 00 00 03  03 7f fa 78 1e e7 fd fe  |.%.........x....|
                         ^^^^^^^^^^^^--- emulation 3 byte
00000050  b4 62 7a 31 ff 7d 81 fd  26 d8 62 b6 d6 25 46 ae  |.bz1.}..&.b..%F.|

The following NALs (7, 8, 6, 5) decode fine,
.dec_ref_pic_marking_bit_size is set to 2 without this patch.
00000000  00 00 00 01 27 4d 00 20  89 8b 60 3c 04 bf 2e 02  |....'M. ..`<....|
00000010  d4 18 04 18 c0 c0 01 77  00 00 5d c1 7b df 05 00  |.......w..].{...|
00000020  00 00 01 28 ee 1f 20 00  00 01 06 05 10 b9 ed b9  |...(.. .........|
00000030  30 5d 21 4b 71 83 71 2c  10 a3 14 bb 29 80 00 00  |0]!Kq.q,....)...|
00000040  01 25 b8 00 04 c0 00 03  7f fa 78 1e e7 fd fe b4  |.%........x.....|
00000050  62 7a 31 ff 7d 81 fd 26  d8 62 b6 d6 25 46 ae ce  |bz1.}..&.b..%F..|

Fixes: d0d65fa875 ("codecparsers: h264: record dec_ref_pic_marking() size")
Fixes: 0cc7d6f093 ("codecparsers: h264: record pic_order_cnt elements size")
Signed-off-by: Marek Vasut <marex@denx.de>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2517>
2021-09-09 16:49:41 +00:00
Aaron Boxer
15d724e671 gsth264parser: reject memory management control op greater than 6
This prevents assertion from being thrown in
gst_h264_dpb_perform_memory_management_control_operation
if corrupt NAL has a control op greater than 6

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2508>
2021-09-09 15:32:14 +00:00
Mathieu Duponchelle
846cf3b20c vulkan: don't link to XOpenDisplay in documentation
hotdoc doesn't know about that symbol

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2518>
2021-09-08 15:34:09 +00:00
Mengkejiergeli Ba
702e69e841 codecs: av1dec: Fix to output frame with highest spatial layer
During the output process, if there are multiple frames in a TU (i.e. multi-spatial
layers case), only one frame with the highest spatial layer id should be selected
according to av1 spec. The highest spatial layer id is obtained from idc value of
the operating point.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2475>
2021-08-27 15:27:31 +00:00
Brad Hards
dee294809f gsth264parser: fix typo in debug message
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2493>
2021-08-27 17:43:44 +10:00
Johan Sternerup
607ef6db60 webrtc: Split sctptransport into lib and implementation parts
GstWebRTCSCTPTransport is now made into into an abstract base class
that only contains property specifications matching the
RTCSctpTransport interface of the W3C WebRTC specification, see
https://w3c.github.io/webrtc-pc/#rtcsctptransport-interface. This
class is put into the WebRTC library to expose it for applications and
to allow for generation of bindings for non-dynamic languages using
GObject introspection.

The actual implementation is moved to the subclass WebRTCSCTPTransport
located in the WebRTC plugin.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2214>
2021-08-25 13:20:22 +00:00
Seungha Yang
fe4ec03a4b d3d11bufferpool: Hide buffer_size field from header
User can get the required buffer size by using buffer pool config.
Since d3d11 implementation is a candidate for public library in the future,
we need to hide everything from header as much as possible.

Note that the total size of allocated d3d11 texture memory by GPU is not
controllable factor. It depends on hardware specific alignment/padding
requirement. So, GstD3D11 implementation updates actual buffer size
by allocating D3D11 texture, since there's no way to get CPU accessible
memory size without allocating real D3D11 texture.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2482>
2021-08-22 00:46:19 +09:00
He Junyan
fbf6bfd4d8 va: Use GST_CAPS_FEATURE_MEMORY_VA to replace "memory:VAMemory".
"memory:VAMemory" is a commonly used string which notates our VA-kind
memory type. We now used a definition in va lib to replace the simply
string usage.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2466>
2021-08-16 16:25:15 +08:00
He Junyan
70ce2327d0 codecs: h264dec: Output the picture directly if already a frame.
We forget one case that is the frame and field pictures may be mixed
together. For this case, the dpb is interlaced while the last picture
may be a complete frame. We do not need to cache that complete picture
and should output it directly.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2448>
2021-08-12 07:41:28 +00:00
Tim-Philipp Müller
a561b1bd86 Use g_memdup2() where available and add fallback for older GLib versions
g_memdup() is deprecated since GLib 2.68 and we want to avoid
deprecation warnings with recent versions of GLib.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2280>
2021-08-05 20:51:00 +05:30
Seungha Yang
554855f87f d3d11: Disable packed and subsampled YUV formats
Direct3D11 sampler doesn't support them very well, and conversion
outputs usually result in poor visual quality with our shader code.
Should disable support for such formats for now

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2344>
2021-07-29 22:33:30 +09:00
He Junyan
704689ec4a codecs: h264dec: Improve the find_first_field_picture().
We need to consider the first field of the last picture when the
last picture can not enter the DPB.

Another change is, when prev field's frame_num is not equal to the
current field's frame_num, we should also return FASLE because it
is also a case of losing some field.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2430>
2021-07-28 10:54:21 +00:00
He Junyan
f95aa0a374 codecs: h264dec: consider the last field when add picture to DPB.
There are cases that the first field of the last picture is not a
ref but the second field is a ref. We need to add both of them
because the bumping always needs a complete frame in the DPB.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2430>
2021-07-28 10:54:21 +00:00
He Junyan
055ded53e9 codecs: h264dec: Consider the field case when directly output.
For interlaced streams, it is also possible that the last frame is
not able to be inserted into DPB when the DPB is full and the last
frame is a non ref. For this case, we need to hold a extra ref for
the first field of the last frame and wait for the complete frame
with both top and bottom fields. For the progressive stream, the
behaviour is unchanged.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2430>
2021-07-28 10:54:21 +00:00
He Junyan
04974015ec codecs: h264dec: Fix a error print of dpb_add.
When the dpb is interlaced, the max size should be 2*dpb->max_num_frames,
correcting the error print info for that.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2430>
2021-07-28 10:54:21 +00:00
Imanol Fernandez
eea28504fa player: Add static keyword to _config_quark_table
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2439>
2021-07-28 06:05:30 +00:00
He Junyan
f9d7b708e1 codecs: h264decoder: let print_ref_pic_list_b print the correct list name.
The print_ref_pic_list_b now not only needs to trace the ref_pic_list_b0/1,
but also need to trace the ref_frame_list_0_short_term. We need to pass the
name directly to it rather than an index to refer to ref_pic_list_b0/1.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2425>
2021-07-23 19:52:48 +00:00
He Junyan
eef9067619 codecs: h264dec: Fix a typo in construct_ref_field_pic_lists_b.
The array sort of ref_frame_list_0_short_term has some typo. The
typo makes this list not in the POC ascend order and generate wrong
decoding result for interlaced streams.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2425>
2021-07-23 19:52:48 +00:00
He Junyan
130205629f codecs: h264dec: Improve the algorithm for low latency mode.
In low_latency mode, try to bump the picture as soon as possible
without the frames disorder:
1. We can directly output the continuous non-reference frame.
2. Consider max_num_reorder_frames, which is special useful for
   I-P mode.
3. Consider the leading pictures with negative POC.
4  Output small POC pictures when non-reference frame comes.
4. Output the POC increment<=2 pictures. This is not 100% safe,
   but in practice this condition can be used.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2373>
2021-07-21 15:23:17 +00:00
He Junyan
573d3f5ba5 codecs: h264dec: Add help function of dpb_set_max_num_reorder_frames.
The max_num_reorder_frames can be useful for bump check. We store it
in the DPB and no need for the decoder now.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2373>
2021-07-21 15:23:17 +00:00
He Junyan
be223ad316 codecs: h264dec: Add a flag to record whether picture is reference.
The picture->ref field will change from time to time according to decoder's
state and reference sliding window. We need another flag to record whether
the picture is a reference picture when it is created, and this can help
the bumping check.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2373>
2021-07-21 15:23:17 +00:00
He Junyan
b4e3887449 codecs: h264dec: Change the order of dpb_add and dpb_bump.
The current behavior is different from the SPEC. We should check
and bump the DPB or drain the DPB before we insert the current
picture into it. This may cause the output picture disorder.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2373>
2021-07-21 15:23:17 +00:00
He Junyan
f1df120797 codecs: h264dec: Modify the DPB need bump check.
Accord to spec, we should not add the current picture into the DPB
when we check whether it needs to bump, so the checks of the IDR and
the "memory_management_control_operation equal to 5" are no needed.

And the spec also says that the DPB only needs to bump when there is
no empty frame buffer left(We handle the IDR cases in other places).
We need to follow that and the max_num_reorder_frames is useless.

We also minus 1 in has_empty_frame_buffer because the current frame
has not been added yet.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2373>
2021-07-21 15:23:17 +00:00
He Junyan
4411a4e00f codecs: h264dec: Make dpb_has_empty_frame_buffer a codecs API.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2373>
2021-07-21 15:23:17 +00:00
He Junyan
23b15aa5d0 codecs: h264dec: Set picture to a small poc when mem_mgmt_5.
When current frame memory_management_control_operation equal to 5, that
means we need to drain the dpb and the current picture act as an IDR frame.
So it should have smaller poc than the later pictures to ensure the output
order.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2373>
2021-07-21 15:23:17 +00:00
He Junyan
4299596d16 codecs: h265decoder: Fix a typo of NumPocTotalCurr when process ref pic list.
We should use the NumPocTotalCurr value stored in decoder, which is a calculated
valid value, rather than use the invalid value in the slice header. Most of the
time, the NumPocTotalCurr is 0 and make the tmp_refs a very short length, and
causes the decoder's wrong result.
By the way, the NumPocTotalCurr is not the correct name specified in H265 spec,
its name should be NumPicTotalCurr. We change it to the correct name.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2414>
2021-07-16 23:05:18 +08:00
Mads Buvik Sandvei
bc6b68215f codecs: h265decoder: Always free messages while parsing SEI
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2390>
2021-07-14 17:56:27 +00:00
He Junyan
55ccc581b2 codecs: vp9statefulparser: not init segmentation_abs_or_delta_update.
The segmentation_abs_or_delta_update is a stateful var, it should not
be inited every time when parsing the segmentation.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2403>
2021-07-13 15:34:57 +00:00
Daniel Almeida
d0e614b354 codecparsers: av1: fix underflow in last_frame_idx
The spec mandates this field be parsed using unsigned arithmetic. Nevertheless,
av1parser will use -1 apparently as an uninitialized value in
gst_av1_parse_frame_header. This immediately underflows last_frame_idx
though, since its type was defined as guint8. Fix this by converting to gint8.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2401>
2021-07-12 17:49:17 +00:00
Guido Günther
f331082b20 play: Emit correct signal
SIGNAL_MEDIA_INFO_UPDATED should be emitted on media info changes,
not SIGNAL_VIDEO_DIMENSIONS_CHANGED.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2398>
2021-07-12 06:28:41 +00:00
Víctor Manuel Jáquez Leal
c22ae0700b vulkan: Declare missing auto-pointer cleanup functions.
Also removed a couple guards since, given the glib dependency, they
are set.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2396>
2021-07-12 05:08:09 +00:00
Víctor Manuel Jáquez Leal
9c3363a25d vulkan: filter: Use filter variable name for choosing queue.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2396>
2021-07-12 05:08:09 +00:00
Dominique Martinet
ff6442f6f9 gst-libs/gst/wayland: handle display passing better
failure to pass a display in 'handle' would result in uninitialized value
being returned, which would often segfault later down the road when trying
to initialize gstreamer context with it.
Check the return value of gst_structure_get() to make sure we return valid
data.

Furthermore, the gstglimagesink in gst-plugins-base also has a similar
mechanism but uses 'display' as field name to pass the value; instead of
requiring the application to behave differently depending on what sink
was automatically detected just try to read both values here, with display
being the new default.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2292>
2021-07-09 13:15:54 +00:00
He Junyan
078b2699e7 codecs: h265dec: Disable the POC order warning for negative POC.
There may be leading frames after the IRAP frames, which has negative
POC. This kind of frames are allowed and they will be displayed before
the IRAP frame. So the warning should not be triggered for them. Init
the last_output_poc to G_MININT32 can avoid this.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2380>
2021-07-06 14:33:21 +00:00
He Junyan
1fa9f26a53 codecs: h264dec: Disable the POC order warning for negative POC.
There may be leading frames after the IDR frame, which has negative
POC. This kind of frames are allowed and they will be displayed before
the IDR frame. So the warning should not be triggered for them. Init
the last_output_poc to G_MININT32 can avoid this.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2380>
2021-07-06 14:33:21 +00:00
He Junyan
17e8fea158 codecparsers: vp9statefulparser: Fix the gst_vp9_get_qindex clamp issue.
The alternate quantizer is a delta value and should be int type.
We mark it wrongly as uint, that will make CLAMP (data, 0, 255)
always choose 255 rather than 0 if the data < 0.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2369>
2021-07-02 08:44:15 +00:00
He Junyan
f38d944c0f codecparsers: vp9parser: Use macro to define the size of filter_level in Segmentation.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2369>
2021-07-02 08:44:15 +00:00
He Junyan
f941673862 codecparsers: vp9parser: Delete the verbose redefine of MAX_LOOP_FILTER.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2369>
2021-07-02 08:44:15 +00:00
He Junyan
5d96e9e7bd codecs: h265decoder: Consider the conformance window changes when new_sequence().
The change of conformance_window_flag and crop windows size also has impact on the
output resolution and caps. So it deserves a trigger of new_sequence() to notify
the sub class to update caps and pool.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2312>
2021-07-01 14:39:53 +00:00
Seungha Yang
b5b13a6f5c h265decoder: Don't hold reference to GstVideoCodecFrame for dropped picture
We are dropping RASL (Random Access Skipped Leading picture) which
is associated with an IRAP (Intra Random Access Picture) that has
NoRaslOutputFlag equal to 1, since the RASL picture will not be
outputted and also it should not be used for reference picture.
So, corresponding GstVideoCodecFrame should be released immediately.
Otherwise GstVideoDecoder baseclass will hold the unused frame.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2330>
2021-07-01 14:10:30 +00:00
youngsoo.lee
260a82e514 opencv: Fix build error on macOS
The build fails on macos with the following error:
/usr/local/Cellar/opencv/4.5.0_5/include/opencv4/opencv2/core/mat.hpp:2226:15: error: no template named 'initializer_list' in namespace 'std'
      Mat_(std::initializer_list<_Tp> values);
fatal error: too many errors emitted, stopping now [-ferror-limit=]
 35 warnings and 20 errors generated.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2368>
2021-06-30 14:03:22 +09:00
Seungha Yang
c3b26de1f2 av1decoder: Store display resolution for duplicated picture
Target display resolution might be required by subclass implementation

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2365>
2021-06-29 07:12:27 +00:00
Seungha Yang
7a1effc499 av1decoder: Fix debug typo
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2365>
2021-06-29 07:12:27 +00:00
Seungha Yang
6dbb9c705a av1parser: Fix tile size calculation
Remaining size should exclude already read "tile size bits".
And see also "5.11.1. General tile group OBU syntax"

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2365>
2021-06-29 07:12:27 +00:00
Sebastian Dröge
0e559fc2f3 webrtcbin: Sync to the clock per stream and not per bundle
By using the clocksync inside the dtlssrtpenc, all streams inside a
bundled are synchronized together. This will cause problems if their
buffers are not already arriving synchronized: clocksync would wait for
a buffer on one stream and then buffers from the other stream(s) with
lower timestamps would all be sent out too late.

Placing the clocksync before the rtpbin and rtpfunnel synchronizes each
stream individually and they will be send out more smoothly as a result.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2355>
2021-06-28 16:38:33 +00:00
Olivier Crête
ee0124cb36 webrtc: Remove the webrtc-priv.h header from public headers
And this time for real, also import it in a couple more places
inside the webrtc element to make it build.

Fixes #1607

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2359>
2021-06-28 16:06:59 +00:00
Sebastian Dröge
03d3e0fe73 webrtc: Re-add WebRTC object docs to the public headers
So they end up in the generated documentation and the Since markers
appear in the .gir files too.

Also remove wrong "Since: 1.16" markers for some objects that were
available since 1.14.0 already.

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/issues/1609

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2366>
2021-06-28 14:45:37 +00:00
Seungha Yang
7c94b9c4b0 d3d11: Add support for GRAY and more YUV formats
By this commit, following formats will be newly supported by d3d11 elements

* Y444_{8, 12, 16}LE formats:
  Similar to other planar formats. Such Y444 variants are not supported
  by Direct3D11 natively, but we can simply map each plane by
  using R8 and/or R16 texture.
* P012_LE:
  It is not different from P016_LE, but defining P012 and P016 separately
  for more explicit signalling. Note that DXVA uses P016 texture
  for 12bits encoded bitstreams.
* GRAY:
  This format is required for some codecs (e.g., AV1) if monochrome
  is supported
* 4:2:0 planar 12bits (I420_12LE) and 4:2:2 planar 8, 10, 12bits
  formats (Y42B, I422_10LE, and I422_12LE)

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2346>
2021-06-23 15:35:36 +00:00
Olivier Crête
a931e31141 webrtc lib: Make the datachannel struct private
This will prevent any unsafe access.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2241>
2021-06-21 20:53:09 +00:00