Commit graph

83 commits

Author SHA1 Message Date
François Laignel
1d00f726a0 qtdemux: opus: set entry as sampled
... otherwise streams with constant size samples defined with a single
`sample_size` for all samples in the `stsz` box fall in the category
`chunks_are_samples` in `qtdemux_stbl_init`, overriding the actual
sample count.

`FOURCC_soun` would set this automatically for `compression_id == 0xfffe`,
however `compression_id` is read from the Audio Sample Entry box at an offset
marked as "pre-defined" in some version of the spec and set to 0 both by
GStreamer and FFmpeg for opus streams.

Considering the stream `sampled` flag is set explicitely by other fourcc
variants, doing so for opus seems consistent.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4903>
2023-06-20 17:15:22 +00:00
François Laignel
fa30504ec2 qtdemux: parse Opus and dOps as qtdemux nodes and add size checks
This allows checking the nodes conformity and dumping parsed values.

Note: Audio Sample Entry version parsing and offset handling is handled as part
of `FOURCC_soun` common processing and in `qtdemux_parse_node`.

Also, only read `stream_count` and `coupled_count` when
`channel_mapping_family` != 0. See:

https://opus-codec.org/docs/opus_in_isobmff.html#4.3.2

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4875>
2023-06-19 14:31:55 +00:00
François Laignel
439717ab65 qtdemux: fix byte order for opus extension and version field type
The "Encapsulation of Opus in ISO Base Media File Format" [1] specifications,
§ 4.3.2 Opus Specific Box, indicates that data must be stored as big-endian.

[1] https://opus-codec.org/docs/opus_in_isobmff.html#4.3.2

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4875>
2023-06-19 14:31:55 +00:00
François Laignel
f3496ea3bf qtmux: fix byte order for opus extension
The "Encapsulation of Opus in ISO Base Media File Format" [1] specifications,
§ 4.3.2 Opus Specific Box, indicates that data must be stored as big-endian.

In `build_opus_extension`, `gst_byte_writer_put*_le ()` variants were used,
causing audio streams conversion to Opus in mp4 to offset samples due to the
PreSkip field incorrect value (29ms early in our test cases).

[1] https://opus-codec.org/docs/opus_in_isobmff.html#4.3.2

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4875>
2023-06-19 14:31:55 +00:00
Xabier Rodriguez Calvar
bdff780fe9 qtdemux: Fix critical message on cenc sample grouping parsing
Inside qtdemux_parse_sbgp there is already a check to ensure the fragment group
properties are not null but it is being hit in some examples and it is better to
directly avoid the critical.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4576>
2023-06-07 11:01:20 +00:00
Jan Alexander Steffens (heftig)
93699123b4 isomp4: Fix (E)AC-3 channel count handling
The muxer used a fixed value of 2 channels because the TR 102 366 spec
says they're to be ignored. However, the demuxer still trusted them,
resulting in bad caps.

Make the muxer fill in the correct channel count anyway (FFmpeg already
does) and make the demuxer ignore the value.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4739>
2023-06-02 19:07:58 +00:00
Sebastian Dröge
d5a0cfc563 qtdemux: Add support for SpeedHQ video codec
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3982>
2023-05-19 07:16:03 +00:00
Sebastian Dröge
99285bb566 qtmux: Fix extraction of CEA608 data from S334-1A packets
The index is already incremented by 3 every iteration so multiplying it
by 3 additionally on each array access is doing it twice and does not
work.

This caused invalid files to be created if there's more than one CEA608
triplet in a buffer, and out of bounds memory reads.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4634>
2023-05-16 11:29:45 +00:00
Mathieu Duponchelle
3d3d2ed447 Revert "qtdemux: fix conditions for end of segment in reverse playback"
This reverts commit 9deb3c27ac.

The test case that was described in the associated MR
(https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/-/merge_requests/262)
remains adequately fixed by a related MR that was merged later
(https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/-/merge_requests/275).

It introduced incorrect logic that broke edit lists as described in
https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/2549

Fixes: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/2549
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4560>
2023-05-11 16:45:37 +00:00
Xabier Rodriguez Calvar
021572de93 qtdemux: emit no-more-pads after pruning old pads
If we don't do that, clients can rely on this signal to see the final pad
topology but it won't be the real one as some of them will disappear after
emitting that signal. This can happen after injecting a different init segment.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4535>
2023-05-03 12:06:00 +00:00
Carlos Rafael Giani
0071c97128 qtdemux: Add audio clipping meta when playing gapless m4a content
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4200>
2023-05-03 08:47:55 +00:00
Carlos Rafael Giani
51ebda4df5 qtdemux: use qtdemux debug category instead of default in qtdemux_tags.c
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4200>
2023-05-03 08:47:55 +00:00
Xabier Rodriguez Calvar
66c15bc753 qtdemux: Fix segfault in cenc sample grouping
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4523>
2023-05-02 11:32:01 +02:00
Sebastian Szczepaniak
277a9f0cef qtdemux: Add support for cenc sample grouping
Co-authored-by: Xabier Rodriguez Calvar <calvaris@igalia.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3551>
2023-04-26 18:51:56 +00:00
Edward Hervey
4c6f41a00a qtdemux: Fix av1C parsing
This is a regression introduced by
https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3882

The av1c codec configuration parsing would always fail due to an off-by-one
error, the content of an atom starting at offset 8 (i.e. the 9th byte) and not
9 (the 10th byte).

Also introduce a break in order to not get stray warnings

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4433>
2023-04-17 09:28:43 +02:00
Piotr Brzeziński
5beef42922 qtdemux: Fix seek adjustment with SNAP_AFTER flag
With GST_SEEK_FLAG_SNAP_AFTER present, the previous version would
adjust seek time based on the keyframe farthest away from desired_time.
This was incorrect, because we always want the *earliest* suitable keyframe
to seek to, not the last one.
With this fix, in case of the SNAP_AFTER, we now look for the closest keyframe
that can be found after desired_time. Behaviour for SNAP_BEFORE should remain
unchanged.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4183>
2023-03-22 13:05:53 +00:00
Enrique Ocaña González
735dac9d2f qtdemux: Fix crash on MSE-style flush
The flowcombiner and active_streams shouldn't be cleared in the
mse-bytestream variant, only in the mss-fragmented one. Otherwise the
soft reset leaves qtdemux in a state where it still believes that it has
streams, but they've been cleared. In that case, a null pointer
dereference happens and the app crashes.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4199>
2023-03-17 15:33:49 +00:00
Tim-Philipp Müller
0fc568c6b1 gst-plugins-good: re-indent with GNU indent 2.2.12
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4182>
2023-03-17 03:18:54 +00:00
Alicia Boya García
c1f4bd5a3f qtdemux: Add MSE-style flush
The abort() method of SourceBuffer in Media Source Extensions is
expected to flush the demuxer and discard the current fragment,
if any. The configuration of tracks, if any, should be preserved.

qtdemux has different behavior for flush events depending on the
context.

This patch activates the intended behaviour only for streams of the
VARIANT_MSE_BYTESTREAM type, conformant to the ISO BMFF Bytestream
specification[1]. This flush behaviour is the same as the one
already in use for adaptivedemux sources.

[1] https://www.w3.org/TR/mse-byte-stream-format-isobmff/

https://bugzilla.gnome.org/show_bug.cgi?id=795424

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4101>
2023-03-02 17:54:41 +00:00
Seungha Yang
1f0528b428 qtmux: Fix assertion on caps update
GstQTMuxPad.configured_caps should be protected since it's
updated from streaming thread and accessed in aggregate thread

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4042>
2023-02-22 19:16:52 +00:00
Hosang Lee
88f16ebd2a qtdemux: compensate wrong data offset for MSS fragments
A data offset with an offset smaller than the moof length is wrong
in smooth streaming streams.

The samples will not be located and eventually playback will
error out. So compensate assuming data is in mdat following moof.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3840>
2023-02-16 00:43:57 +00:00
Vivia Nikolaidou
4e7a5ebb11 qtdemux: Handle moov atom length=0 case by reading until the end
Previously it would fail to demux the file by trying to read G_MAXUINT64
bytes.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3934>
2023-02-11 02:20:39 +00:00
Vivia Nikolaidou
3a9acff978 qtdemux: Fix guint vs gsize type confusion
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3934>
2023-02-11 02:20:39 +00:00
Sebastian Dröge
5486ed24a5 qtmux: Implement writing of av1C version 1 box
Version 0 is ancient and not specified in any documents. Take it
directly from the `codec_data` if presents or otherwise try to construct
a reasonably looking `av1C` box.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3882>
2023-02-09 14:04:06 +00:00
Sebastian Dröge
8593a58916 qtdemux: Drop av1C version 0 parsing and implement version 1 parsing
The av1C box is optional so dropping parsing does not break anything
fundamentally, and there seems to be no historical record how version 0
even looks like while the comments and the parsing disagreed with each
other.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3882>
2023-02-09 14:04:06 +00:00
Enrique Ocaña González
92a4cfe20f qtdemux: Don't emit GstSegment correcting start time when in MSE mode
When using qtdemux in a pipeline that should only work as a pure demuxer (not
for actual playback), qtdemux shouldn't emit new GstSegments to correct
the start time (jump to the future) to ensure that the user experiences no
playback delay. By doing so, it's generating the wrong segments when an append
of data from the past happens. When that happens, downstream elements such as
parsers (eg: aacparse) may clip those buffers laying before the GstSegment and
create problems on the GStreamer client app (eg: WebKit).

Getting buffers clipped out because of the wrong GstSegments started becoming
a problen when this commit was introduced:

ab6e49e9cc audioparsers: add back segment clipping to parsers that have lost it

This clipping makes test DASH shaka 35 from MVT tests[1] to fail in
WebKitGTK/WPE (at least) and can potentially cause a number of other problems
in the WebKit Media Source Extensions (MSE) code.

Note that this new behaviour of not emitting new GstSegments only makes sense
when qtdemux is being used as a pure demuxer and not as part of a regular
pipeline. This is why the variant field has been added. When equal to
VARIANT_MSE_BYTESTREAM, it will make qtdemux behave differently in push mode,
taking decisions that meet the expectations for an MSE-like processing mode.
This kind of tweaks have been done in the past for MSS streams, for instance.
That code has been refactored to use VARIANT_MSS_FRAGMENTED now, instead of
its own dedicated boolean flag.

Co-authored by: Alicia Boya García <ntrrgc@gmail.com>

...who suggested to use "variant: mse-bytestream" in the caps to identify that
mode, as proposed in her unmerged patch:

https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/-/issues/467

[1] https://github.com/rdkcentral/mvt

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3867>
2023-02-06 12:42:49 +00:00
Alicia Boya García
8a6023a38a qtdemux: Use safer clearing functions in dispose()
In theory, `dispose()` functions should be idempotent and should be
prepared not to crash or cause a double-free if an unref done from
inside caused a recursive call to `dispose()` of the same object.

https://developer.gnome.org/gobject/stable/howto-gobject-destruction.html

This patch modifies the `dispose()` method to honor these constraints.

Since the double `dispose()` call won't actually occur in qtdemux (there
is no cycle detection mechanism that could invoke it to work that way),
this is more of a code cleanup than a user-facing problem fix.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3822>
2023-01-28 00:32:57 +00:00
Sebastian Dröge
4c8141a0c3 isomp4: Add stream-format = (string) obu-stream to AV1 caps
Anything else is not allowed in MP4.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3740>
2023-01-19 12:10:40 +02:00
Jan Alexander Steffens (heftig)
211191564e qtdemux: Add basic support for AVC-Intra video
AVC-Intra is a range of H.264-compliant intra-only codecs from
Panasonic. The codes and descriptions have been taken from VLC.

The (encumbered) sample I have here produces byte-stream H.264,
including SPS and PPS and no `avcC` box.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3739>
2023-01-18 10:01:30 +00:00
Patricia Muscalu
d752bf1b46 qtmux: Fix buffer leak in fragment_buffers
When pushing buffers from one of the sink pads fail,
make sure that all buffers added to fragment_buffers on other pads
are freed as well.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3624>
2022-12-22 14:11:10 +00:00
Mathieu Duponchelle
194dcd91e0 qtmux: For video with N/1001 framerates use N as timescale instead of centiframes
This is recommended by various specifications for such framerates, while
for integer framerates we continue using centiframes to allow for some
more accuracy.

Using N means that no rounding error accumulates, eventually leading to
outputting a packet with a different duration.

Some tools such as MediaInfo determine that a stream is variable
framerate if any packet has a different duration than the others, and
there is no reason I can see for not using the full 4 bytes of
resolution that the mp4 timescale offers.

Example problematic pipeline:

```
videotestsrc num-buffers=5001 ! video/x-raw,framerate=60000/1001,width=320,height=240 ! \
videoconvert ! x264enc bitrate=80000 speed-preset=1 tune=zerolatency ! h264parse ! \
video/x-h264,profile=high-10 ! mp4mux ! filesink location="result2.mp4"
```

This results in a media file that MediaInfo detects as variable
framerate because the 5000th packet has duration 99 instead of 100.

With this patch, the timescale is 60000 and all packets have duration
1001.

Related issue for context: https://bugzilla.gnome.org/show_bug.cgi?id=769041

Co-authored-by: Sebastian Dröge <sebastian@centricular.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3049>
2022-12-22 12:31:06 +02:00
Sebastian Dröge
066558cba1 qtdemux: Always use tfdt if available in BYTE segments
This reverts the decision from
  https://bugzilla.gnome.org/show_bug.cgi?id=754230
where it was decided that we rather play safe and only use the `tfdt` if
it is "significantly different" to the sum of sample durations.

As the specification says

    If the time expressed in the track fragment decode time (‘tfdt’) box
    exceeds the sum of the durations of the samples in the preceding
    movie and movie fragments, then the duration of the last sample
    preceding this track fragment is extended such that the sum now
    equals the time given in this box.

we have to use the `tfdt` in general to allow for it to signal gaps in
the stream.

A muxer producing fragments might not yet know the full duration of the
last sample of a previous fragment if the next fragment starts with a
gap, and knowing the actual start of the next fragment would potentially
require to violate latency requirements.

Additionally, the existence of `tfdt` allows to avoid accumulating
rounding errors from summing up the durations.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3586>
2022-12-17 19:26:19 +02:00
Xabier Rodriguez Calvar
87ae60176b qtdemux: Clear protection events when we get new ones
If we keep the old events they can be end up being passed to the app, that could
discard the protection information because it has been seen before.

Drive by improvement: use g_queue_clear_full instead of foreach+clear for
protection events.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3547>
2022-12-14 11:01:23 +01:00
Jacek Skiba
61c17c5665 qtdemux: exit when protection caps are not defined during PIFF parsing
Reproduction testcase (uses PlayReady):
https://developers.canal-plus.com/rx-player/upc/?appTileLocation=[object%20Object]

In test streams we are using PIFF box, but caps did not had
present GST_PROTECTION_SYSTEM_ID_CAPS_FIELD. In consequence, invalid
system_id was returned which caused SIGSEGV crash.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3535>
2022-12-07 18:35:37 +00:00
Matthew Waters
8e355d23a1 qtmux: use trun with multiple entries in more cases
The only case where we definitely need to write a new trun is when the
data_offset value does not match the end of the list of entries.
Needing multiple trun atoms is required when interleaving multiple
streams together.

All other cases can be covered by adding more entries to the existing
trun atom.

Fixes playback of fragemented mp4 in ffplay and chrome.

Using e.g. mp4mux fragment-duration=1000 fragment-mode=dash-or-mss
and
mp4mux fragment-duration=1000 fragment-mode=first-moov-then-finalise

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3426>
2022-11-17 21:04:57 +11:00
Justin Chadwell
fd96fc23c5 qtdemux: use unsigned int types to store result of QT_UINT32
In a few cases throughout qtdemux, the results of QT_UINT32 were being
stored in a signed integer, which could cause subtle bugs in the case of
an integer overflow, even allowing the the result to equal a negative
number!

This patch prevents this by simply storing the results of this function
call properly in an unsigned integer type. Additionally, we fix up the
length checking with stsd parsing to prevent cases of child atoms
exceeding their parent atom sizes.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3344>
2022-11-06 12:00:31 +00:00
Sebastian Dröge
b368a5fcd2 qtmux: Add durations to raw audio buffers from the raw audio adapter in prefill mode
This ensures that a duration can also be calculated and stored for the
last buffer at EOS.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3321>
2022-11-04 19:02:22 +00:00
Sebastian Dröge
7b60e48c8c qtmux: Release object lock before posting an error message
GST_ELEMENT_ERROR() also takes the object lock and this would then
deadlock.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3321>
2022-11-04 19:02:22 +00:00
Edward Hervey
6ffae88a9f qtdemux: Fix cenc-related leaks
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3319>
2022-11-04 17:59:21 +00:00
Matthew Waters
e2081ce31e mp4mux: enable muxing VP9 streams
As specified in https://www.webmproject.org/vp9/mp4/

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3260>
2022-10-28 00:06:07 +00:00
Matthew Waters
5bed545113 qtmux: add support for writing vpcC box for VP9
Increases compatibility for VP9 in .mov in at least VLC.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3260>
2022-10-28 00:06:07 +00:00
Edward Hervey
f2a1769236 qtdemux: Don't stop task when resetting
This is a regression that was introduced in
cca2f555d1 (yes, 9 years ago).

The only place where a demuxer streaming thread should be stopped is when the
sinkpad is deactivated from pull mode (i.e. PAUSED->READY).

Attempting to stop the task in this function would cause this to happen when a
FLUSH_STOP or STREAM_START event is received... which can cause deadlocks.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3109>
2022-10-03 14:41:18 +02:00
Tim-Philipp Müller
02a8f9973b qtdemux: guard against timestamp calculation overflow in gap event loop
Could possibly cause an endless loop.

Fixes #1400.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3084>
2022-09-27 13:07:15 +00:00
Thibault Saunier
6a4425e46a meson: Call pkgconfig.generate in the loop where we declare plugins dependencies
Removing some copy pasted code

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2970>
2022-09-01 21:17:35 +00:00
Mathieu Duponchelle
8756f523d1 playback: add onvif metadata caps to raw caps
+ remove encoding from x-onvif-metadata caps output by qtdemux

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2889>
2022-08-24 12:21:18 +03:00
zhiyuan.liu
ffebd52e46 isoff: Fix earliest pts field parse issue
earliest pts will be covered by first_offset field on version 0 case.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2927>
2022-08-23 10:59:56 +00:00
Jan Schmidt
c2fa0b50ce qtdemux: Avoid crash on reconfiguring.
When reconfiguring a stream that never created
an output pad, don't access a NULL GstPad pointer.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2869>
2022-08-16 19:01:28 +00:00
Sebastian Dröge
a3037eb453 qtdemux: Set parsed=true on ONVIF Timed Metadata caps
Inside MP4 the metadata must be properly parsed into frames and in
order.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2897>
2022-08-16 18:11:53 +00:00
Sebastian Dröge
b0533d1ea0 qtdemux: Add reference timestamp meta with UTC times based on the ONVIF Export File Format CorrectStartTime box to outgoing buffers
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2525>
2022-08-12 16:13:50 +00:00
Edward Hervey
63dcee34fb qtdemux: Don't use invalid values from failed trex parsing
If parsing the fragment default values (`trex` atom) failed, don't try to
compute a bogus sample_description_id value.

Fixes #1369

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2860>
2022-08-11 08:50:34 +02:00