Commit graph

118 commits

Author SHA1 Message Date
He Junyan
f63a70966e av1parse: Handle the padding OBU correctly
The current av1parse can not find the edge of frame correctly if there
is padding OBUs inside the stream. We now use a flag seen_non_padding to
check whether we see some valid data after a data push. Then the padding
OBUs will be the part of the new frame.
We also refine the code logic to make the code more readable.

Fixes: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4044
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8269>
2025-02-07 16:56:53 +00:00
Carlos Bentzen
bb74aceb5d h266parse: clean up unused APS fields
Since APS is always carried in-band, we don't need to keep the APS
NALs around in the parser anymore.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8359>
2025-02-07 08:25:17 +00:00
Carlos Bentzen
5fad23d26b h266parse: enable vvc1 and vvi1 stream formats
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8359>
2025-02-07 08:25:17 +00:00
Carlos Bentzen
05d4a2022e h266parse: handle packetized frames
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8359>
2025-02-07 08:25:17 +00:00
Carlos Bentzen
551782b32b h266parse: don't prepend APS NALs on IDR frames
Instead, APS NALs can just be pushed an in-band NALs like PH and SEI.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8359>
2025-02-07 08:25:17 +00:00
Carlos Bentzen
c7e9aba025 h266parse: negotiate before handling codec_data NALs
If we find VPS/SPS/PPS in codec_data and call gst_h266_parse_process_nal
with them, we need to have negotiated before in order to correctly
process them with flags like h266parse->transform set or not depending
on the negotiation. This is important because in certain vvc1/vvi1 streams we
may have correct codec_data but faulty parameter sets in the stream and
we would want to push the parameter sets from codec_data first.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8359>
2025-02-07 08:25:17 +00:00
Carlos Bentzen
f4f7e99e05 h266parse: parse codec_data
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8359>
2025-02-07 08:25:17 +00:00
Carlos Bentzen
8607ccdf2c h266parse: update IDR position in more cases
The IDR position should be updated if we're processing an
IDR frame or pushing codec NALs. Not only when picture_header_in_slice_header_flag
is set.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8359>
2025-02-07 08:25:17 +00:00
Carlos Bentzen
7c29249ca5 h266parse: fix typos
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8359>
2025-02-07 08:25:17 +00:00
Carlos Bentzen
fe61b43814 h266parse: implement make_codec_data
implement serialization of codec_data containing VvcDecoderConfigurationRecord
as defined in ISO/IEC 14996-15.

The VPS/SPS/PPS NALs are added to the codec_data. APS NALs could be
optionally included as well but will be pushed in-band instead, because:
1. Logic is easier that way. We'd have to filter out for PREFIX_APS only
   (SUFFIX_APS aren't allowed in codec_data).
2. APS NALs can also be sent for every non-keyframe slice, and often are, so just pushing
   them in-band makes more sense to have less to keep track and avoid possible
   duplicates.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8359>
2025-02-07 08:25:17 +00:00
Brad Hards
9b578abe15 h264parse: add conditional values to AVCConfigurationRecord
This adds the data required in AVCDecoderConfigurationRecord for
higher profile (High variants) configurations - everything in the if(...) {...} part
of ISO/IEC 14496-15:2024 Section 5.3.2.1.2. (or 5.3.3.1.2 in the 2019 version).

Resolves an error flagged by ComplianceWarden when muxing this into ISOBMFF.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8308>
2025-01-22 15:58:21 +11:00
Ruben Gonzalez
c6ed0c33a0 h264parse: drop duplicated call
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8299>
2025-01-13 22:09:10 +01:00
Mathieu Duponchelle
3de86b2b97 docs: port plugins to explicit sources
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8273>
2025-01-13 19:17:13 +01:00
Edward Hervey
21d81d25ec bad: Clearly specify fallthrough in switch/case
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8229>
2025-01-13 12:51:45 +00:00
Ruben Gonzalez
da5a4f7820 videoparsers: Fix indentation
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8281>
2025-01-12 22:51:44 +00:00
Nicolas Dufresne
988f806e7a av1parse: Fix some debug trace and comment typo
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8026>
2024-12-20 01:26:12 +00:00
Nicolas Dufresne
98d40251d8 av1parse: Don't immediatly reset timestamp in presence of TD
When a TD is being processed, it is not always pushed immediatly. Resetting
the time information lead to lost of timestamp in TU to Frame conversion. The
TU would be formed by buffer of [TD][Frame], and the timestamp taken from
the TU buffer was lost then the TD was handled.

The handling of TS should be entirely done by the 3 functions:

- gst_av1_parse_handle_obu_to_obu() (direct input to output)
- gst_av1_parse_handle_to_big_align() Reset DTS on detected TU or TD
- gst_av1_parse_handle_to_small_and_equal_align() PTS on show frame, flat DTS

Fixes: 79312357a6
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8026>
2024-12-20 01:26:12 +00:00
He Junyan
c79f83f4ec av1parse: Fix a typo in the comments about its usage
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5710>
2024-12-20 08:22:31 +08:00
He Junyan
b5e4c8b774 h266parse: Add the new h266parse element
TODO: Need to refer to the new ISO/IEC 14496-15 for vvc1 and vvi1's
codec data

Co-authored-by: Zhong Hongcheng <spartazhc@gmail.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5710>
2024-12-20 08:22:31 +08:00
Sebastian Dröge
9f79d39b1b vp9parse: Add video codec tag to the tag list
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8156>
2024-12-19 23:30:31 +00:00
Sebastian Dröge
f590e815fe av1parse: Add video codec tag to the tag list
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8156>
2024-12-19 23:30:31 +00:00
Aaron Boxer
ea1b043047 h265parse: reset nalparser to NULL after it is freed
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8074>
2024-12-05 13:19:48 +00:00
sachin gadag
3f6020ddb5 h264parse: set nalparser to NULL after it is freed
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/8074>
2024-12-05 13:19:48 +00:00
Guillaume Desmottes
bfda0f4858 h265parse: parse unregistered SEI without user data
Same change as in h264parse.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7931>
2024-12-04 13:56:13 +00:00
Guillaume Desmottes
082a8fcd5e h264parse: parse unregistered SEI without user data
We get loads of warnings when parsing videos from users:

gsth264parser.c:1115:gst_h264_parser_parse_user_data_unregistered: No more remaining payload data to store
gsth264parse.c:646:gst_h264_parse_process_sei:<h264parse0> failed to parse one or more SEI message

Those are raised because of unregistered SEI without user data.

The spec does not explicitly state that unregistered SEI needs to have
data and I suppose the UUID by itself can carry valuable information.
FFmpeg also parses and exposes such SEI so there is no reason for us no
too as well.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7931>
2024-12-04 13:56:13 +00:00
Théo Maillart
a1c92c0cb1 mpegvideoparse: do not set delta unit flag on unknown frame type
When encoding an image to mpeg2 video, with something like:
  gst-launch-1.0 encodebin name=e profile=mpegpsmux:video/mpeg,mpegversion=2,systemstream=false ! \
                 filesink location=sample.mpg filesrc num-buffers=1 blocksize=$(stat -c%s sample.png) \
                 location=sample/dts.png ! pngdec ! e.
The only frame's type is set to an invalid value 0
The consequence is that mpegvideoparse sets the delta unit flag on the buffer because
it is not an I frame, then decodebin3 drops this only frame because the delta
unit flag is set and the decoder receives eos before it was able to receive any
encoded data

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7832>
2024-11-12 09:13:10 +00:00
Jan Alexander Steffens (heftig)
6ddd51c239 h264parse: Fix pic_timing SEI replacement
The calculated position was off. I'm not sure of the exact cause;
possibly because we're in AU-aligned byte-stream mode, which means
`transform` is true.

Replacing the math that calculates the NALU positions with code more
similar to what is already in use for `idr_pos` seems to have fixed it.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7318>
2024-10-31 20:29:37 +00:00
Jan Alexander Steffens (heftig)
04238d3b3c h264parse, h265parse: Support drop frame codes with counting_type 6
Tested with an Ateme Kyrion CM5000, which uses 6 when it drops 4 frames
from the code for 1080p@59.94.

Apply the same change to h265parse, with reference to the spec.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7316>
2024-10-31 19:34:53 +00:00
Julian Bouzas
9accf21e53 h264parse: Wait for SEI before exposing src caps
This makes sure 'lcevc=false' src caps are not set before parsing SEI. It is
needed for decodebin2 to work properly with the LCEVC decoder.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7330>
2024-10-02 20:33:13 +00:00
Olivier Crête
8055b0386f h264parse: Don't fake IDR without at least an i-slice
There was an override to fake an IDR as soon as a SPS/PPS
is encountered, but that's not valid, at least an i-slice is needed.

Amend the visl result, as the output is slightly more correct, not
duplicating frame_num.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7330>
2024-10-02 20:33:13 +00:00
Julian Bouzas
2c514ac2f1 h264parse: attach LCEVC meta to buffers if it is present in SEI
This improves the h264parse element to attach LCEVC enhancement data to buffers
using the new GstLcevcMeta API. This metadata will eventually be used downstream
by LCEVC decoders to enhance the RAW video frame.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7330>
2024-10-02 20:33:13 +00:00
Jan Alexander Steffens (heftig)
5ca52ea026 h264parse, h265parse: Fix time code calculation
We need to multiply for the nuit_field_based_flag before scaling, or
we'll lose precision and end up only adding even timecodes.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7241>
2024-08-26 14:04:13 +00:00
Jordan Yelloz
317c70651f h265parse: Reject FD received before SPS
A previous fix, a275e1e029, is correct but was too
permissive since it treats all un-matched NAL units the same as AU delimiters
even though some other NAL unit types can be encountered in the processing loop.

The problem this can cause is that some hardware decoders experience bad
performance when handling FD units that precede the SPS.

This change restores the original behavior for FDs so that they're ignored until
the SPS is received and it preserves the codec conformance test gains that the
fix has achieved.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/7166>
2024-08-02 13:51:43 +00:00
Nicolas Dufresne
79312357a6 av1parse: Properly transfer TU timestamp
When transforming from unknown alignment to frame or obu, the TU timestamp
was not properly transferred. Fix this by saving the TU DTS as the first
DTS seen within the the TU data, and the PTS as the last PTS seen in that
TU data. Finally, reset the TU timestamp after each TU have completed.

Fixes #1496

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6895>
2024-05-28 23:59:36 +00:00
Nicolas Dufresne
5b1bc0f19f av1parse: Only place a marker on the last frame of a TU
Markers are meant to indicate the buffer that ends a frame, which imply
something can be displayed. The dependent decode only frames should not
have markers. This should also fix last subframe detection.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6895>
2024-05-28 23:59:36 +00:00
Rafael Caricio
2bcbbe0d0a av1parse: No default will trigger warning at compile time
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6778>
2024-05-02 16:31:53 +00:00
Rafael Caricio
27041bf9e9 av1parse: Add max-level and max-tier to caps
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6778>
2024-05-02 16:31:53 +00:00
Rafael Caricio
c9a79b7387 av1parse: Add level and tier to caps
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6778>
2024-05-02 16:31:53 +00:00
Daniel Morin
e57f561a8e Revert "h264parse: Improved AU boundary detection"
This reverts commit 49f200cb54.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6540>
2024-04-07 15:25:23 +00:00
Daniel Morin
694c6e77b1 Revert "h264parse: Remove dead code"
This reverts commit 141cd38715.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6540>
2024-04-07 15:25:22 +00:00
Daniel Morin
ac3272db40 Revert "h264parse: Fix AU collection"
This reverts commit 495390f63a.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6540>
2024-04-07 15:25:22 +00:00
Daniel Morin
64c2c8d542 Revert "h264parse: Remove un-needed check on SPS state"
This reverts commit 73dedf9a51.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6540>
2024-04-07 15:25:22 +00:00
Daniel Morin
6ff1973b5b Revert "h264parse: use AUD to detect first VCL NAL"
This reverts commit 90a3b63eed.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6540>
2024-04-07 15:25:22 +00:00
Daniel Morin
0bc5d97321 Revert "h264parse: correct NAL mode backlog processing"
This reverts commit b2098849dc.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6540>
2024-04-07 15:25:22 +00:00
Edward Hervey
eedfb5e6d7 videoparsers: Demote CC warning message
Another warning message which isn't fatal and therefore should just be a DEBUG
line.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6550>
2024-04-06 06:11:03 +02:00
Sebastian Dröge
121e52886b videoparsers: Don't verbosely warn about CEA_708_PROCESS_EM_DATA_FLAG not being set
And the same for CEA_708_PROCESS_CC_DATA_FLAG. This is not really a
problem and was polluting logs with warnings for every single frame.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/6336>
2024-03-12 21:26:18 +00:00
Jan Alexander Steffens (heftig)
9426eaae6a videoparseutils: Don't double-attach AFD, Bar or unregistered data
We already did this for captions. We also need to do it for the other
meta types to ensure our parsers are idempotent and don't attach
duplicates.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5450>
2024-02-15 13:32:30 +00:00
Jan Alexander Steffens (heftig)
92017b983e videoparseutils: Store multiple user data unregistered messages
A frame can have multiple unregistered messages attached. We need to
store them all.

For: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/3008
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5450>
2024-02-15 13:32:30 +00:00
Jan Alexander Steffens (heftig)
75f61cba80 videoparsers: Clear user_data_unregistered explicitly in _reset_frame
This fits better with the model used by the parser elements. It also
properly resets the data when the parser is reset.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5450>
2024-02-15 13:32:30 +00:00
Jan Alexander Steffens (heftig)
3577fdec51 videoparsers: Clear user data explicitly in _reset_frame
This fits better with the model used by the parser elements. It also
properly resets the data when the parser is reset.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5450>
2024-02-15 13:32:30 +00:00