gstreamer/gst-libs/gst
Marek Vasut 8239ff343f gsth264parser: Fix handling of NALs with emulation byte set
In case a set of NALs with emulation_prevention_three_byte is decoded using
hardware decoder like Hantro G1, wrong struct v4l2_ctrl_h264_decode_params
.dec_ref_pic_marking_bit_size is passed into the kernel, which results in
decoding artifacts. Subtract the number of emulation three bytes from the
.dec_ref_pic_m->bit_size to get the correct bit size and avoid having these
artifacts. Apply the exact same fix to slice->pic_order_cnt_bit_size as well.

The following NALs (7, 8, 6, 5) decode with artifacts,
.dec_ref_pic_marking_bit_size is set to 10 without this patch.
00000000  00 00 00 01 27 4d 00 20  89 8b 60 3c 04 bf 2e 02  |....'M. ..`<....|
00000010  d4 18 04 18 c0 c0 01 77  00 00 5d c1 7b df 05 00  |.......w..].{...|
00000020  00 00 01 28 ee 1f 20 00  00 01 06 05 10 b9 ed b9  |...(.. .........|
00000030  30 5d 21 4b 71 83 71 2c  10 a3 14 bb 29 80 00 00  |0]!Kq.q,....)...|
00000040  01 25 b8 00 05 00 00 03  03 7f fa 78 1e e7 fd fe  |.%.........x....|
                         ^^^^^^^^^^^^--- emulation 3 byte
00000050  b4 62 7a 31 ff 7d 81 fd  26 d8 62 b6 d6 25 46 ae  |.bz1.}..&.b..%F.|

The following NALs (7, 8, 6, 5) decode fine,
.dec_ref_pic_marking_bit_size is set to 2 without this patch.
00000000  00 00 00 01 27 4d 00 20  89 8b 60 3c 04 bf 2e 02  |....'M. ..`<....|
00000010  d4 18 04 18 c0 c0 01 77  00 00 5d c1 7b df 05 00  |.......w..].{...|
00000020  00 00 01 28 ee 1f 20 00  00 01 06 05 10 b9 ed b9  |...(.. .........|
00000030  30 5d 21 4b 71 83 71 2c  10 a3 14 bb 29 80 00 00  |0]!Kq.q,....)...|
00000040  01 25 b8 00 04 c0 00 03  7f fa 78 1e e7 fd fe b4  |.%........x.....|
00000050  62 7a 31 ff 7d 81 fd 26  d8 62 b6 d6 25 46 ae ce  |bz1.}..&.b..%F..|

Fixes: d0d65fa875 ("codecparsers: h264: record dec_ref_pic_marking() size")
Fixes: 0cc7d6f093 ("codecparsers: h264: record pic_order_cnt elements size")
Signed-off-by: Marek Vasut <marex@denx.de>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/2517>
2021-09-09 16:49:41 +00:00
..
adaptivedemux gst: don't use volatile to mean atomic 2021-03-22 14:34:36 +11:00
audio gst: don't use volatile to mean atomic 2021-03-22 14:34:36 +11:00
basecamerabinsrc basecamerabinsrc: silence g-ir-scanner warnings 2020-07-29 15:08:48 +00:00
codecparsers gsth264parser: Fix handling of NALs with emulation byte set 2021-09-09 16:49:41 +00:00
codecs codecs: av1dec: Fix to output frame with highest spatial layer 2021-08-27 15:27:31 +00:00
d3d11 d3d11bufferpool: Hide buffer_size field from header 2021-08-22 00:46:19 +09:00
insertbin Meson: Use pkg-config generator 2020-10-23 11:14:18 -04:00
interfaces Meson: Use pkg-config generator 2020-10-23 11:14:18 -04:00
isoff Remove autotools build system 2019-10-14 13:54:27 +01:00
mpegts Use g_memdup2() where available and add fallback for older GLib versions 2021-08-05 20:51:00 +05:30
opencv opencv: Fix build error on macOS 2021-06-30 14:03:22 +09:00
play player: Add static keyword to _config_quark_table 2021-07-28 06:05:30 +00:00
player player: Add static keyword to _config_quark_table 2021-07-28 06:05:30 +00:00
sctp gst: don't use volatile to mean atomic 2021-03-22 14:34:36 +11:00
transcoder transcoder: Fix usage of g_error_propagate 2021-06-21 09:18:32 +00:00
uridownloader remove various useless linefeed in logs 2019-12-11 10:51:29 +01:00
va va: Use GST_CAPS_FEATURE_MEMORY_VA to replace "memory:VAMemory". 2021-08-16 16:25:15 +08:00
vulkan vulkan: don't link to XOpenDisplay in documentation 2021-09-08 15:34:09 +00:00
wayland gst-libs/gst/wayland: handle display passing better 2021-07-09 13:15:54 +00:00
webrtc webrtc: Split sctptransport into lib and implementation parts 2021-08-25 13:20:22 +00:00
gettext.h Fix FSF address 2012-11-04 00:09:59 +00:00
glib-compat-private.h Update GLib dependency to 2.40.0 2015-10-02 22:24:19 +03:00
gst-i18n-plugin.h Fix FSF address 2012-11-04 00:09:59 +00:00
meson.build libs: va: Move the VA common logic as a lib. 2021-05-18 12:15:30 +02:00