Commit graph

569 commits

Author SHA1 Message Date
Sebastian Dröge
f757fbe0f7 qtdemux: Send instant-rate-change event if requested in the SEEK event
Handle an instant rate change seek immediately by reflecting
it downstream as an instant-rate-change event, and do no
further seek handling.
2020-04-02 05:23:17 +00:00
yychao
7f89085251 qtdemux: Add support for AC4
The caps received from qtdemux for AC-4 content are audio/x-gst-fourcc-ac_4

Based on patch by: Savinderjit Kaur

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/issues/413
2020-03-10 15:28:01 +00:00
Sebastian Dröge
885d330ee6 qtdemux: Try to infer useful header values for raw audio if the sound sample descriptions contain zero values 2020-02-28 13:52:40 +00:00
Sebastian Dröge
9e9af6711d qtdemux: Also use the enda atom for determining endianess of in32, fl32 and fl64 formats
Previously it was only used for in24.
2020-02-28 13:52:40 +00:00
Sebastian Dröge
67be373221 qtdemux: Fix up header information for various fixed-format raw audio formats
Sometimes the headers contain useless, wrong or zero values for e.g. the
sample size with these formats. There's only a single valid value for
them so let's set these instead.
2020-02-28 13:52:40 +00:00
Sebastian Dröge
2c5f6e508c qtdemux: Don't print "unhandled type" warnings for various other raw audio fourccs 2020-02-28 13:52:40 +00:00
Sebastian Dröge
65b30ecce6 qtdemux: Add some more raw audio fourccs to the header instead of duplicating them 2020-02-28 13:52:40 +00:00
Seungha Yang
f286f30640 qtdemux: Parse VP Codec Configuration Box
The VP Codec Configuration Box (vpcC) contains vp9 profile and
colorimetry information. Especially the profile information might
be useful for downstream to select capable decoder element.
2020-02-19 23:18:51 +09:00
Sebastian Dröge
9593a3679e qtdemux: Merge sample tables for raw audio streams with one container sample per audio sample
Instead of having chunks with one sample per raw audio sample, have
chunks with a single sample that contains lots of raw audio samples. If
necessary these are still split again later when reading the stream.

With this we are allocating a lot less memory for the parsed sample
tables and can play files that previously triggered our limit of 200MB
for the sample table. For example, one file here would previously
allocate 3.5GB for the sample table and now only allocates 70KB.
2020-02-14 08:48:01 +00:00
Sebastian Dröge
be1c97d3c9 qtdemux: Add a minimum buffer size for raw audio to not output one buffer per frame
Outputting 48000 buffers per second is not a good idea performance-wise.
If a container sample is less than 1024 raw audio frames, combine
multiple samples to get at least 1024 raw audio samples as long as
they're stored contiguous in the file.

For the other direction, if a container sample contains more than 4096
samples there is already code for splitting them up.

Fixes https://bugzilla.gnome.org/show_bug.cgi?id=692750
2020-02-14 08:48:01 +00:00
Sebastian Dröge
5877d945a4 qtdemux: Always prefer information from v1/v2 sound sample description over sample description entry
ffmpeg is doing the same and various files in the wild have bogus
information in the sample description if the same information is also
duplicated afterwards in the v1/v2 sound sample desription.

Previously we only did this for non-raw audio due to
  https://bugzilla.gnome.org/show_bug.cgi?id=374914
but this specific file is already worked around differently. It still
works after this change.

Also remove ad-hoc GST_READ_DOUBLE_BE re-implementation and move the
switch for legacy audio formats after reading all the sample
descriptions as we want to override the values from there.
2020-01-27 14:14:50 +02:00
Alicia Boya García
8dd42666e3 qtdemux: Fix race on pad reconnection
Elements emitting frames through several srcpads should use a
flow combiner to aggregate the chain returns and therefore only return
GST_FLOW_NOT_LINKED to upstream when all the downstream pads have
received GST_FLOW_NOT_LINKED.

In addition to that, in order to handle pads being relinked downstream,
the flow combiner should be reset in response to RECONFIGURE events.
This ensures that a both srcpads process a chain operation before a
GST_FLOW_NOT_LINKED can be propagated upstream (which would usually stop
the pipeline).

Otherwise, in a configuration with two srcpads, only one linked at a
time, after the relink the element could chain data through the now
unlinked pad and the flow combiner would resolve as GST_FLOW_NOT_LINKED
(stopping the pipeline) just because the now linked pad has not been
chained yet to update the flow combiner.

This patch adds handling of RECONFIGURE events to qtdemux. Also, since
this event handling causes the flow combiner to be used from a thread
other than the qtdemux streaming thread, usages of the flow combiner
has been guarded by the object lock.
2020-01-09 18:43:02 +00:00
Mathieu Duponchelle
625eb00c06 qtdemux: send GAP events for lagging audio and video streams too
The logic is taken straight from matroskademux, see
77403d0afe
2019-12-11 19:59:13 +00:00
Edward Hervey
8e1c224fbc good: Avoid usage of deprecated API
GTimeval and related functions are now deprecated in glib.
Replacement APIs have been present since 2.26
2019-10-16 07:46:58 +00:00
Aaron Boxer
46989dca96 documentation: fix a number of typos 2019-10-05 22:38:11 +00:00
Thibault Saunier
a55576d1fd qtdemux: Specify REDIRECT information in error message
There are in the wild (mp4) streams that basically contain no tracks
but do have a redirect info[0], in which case, we won't be able
to expose any pad (there are no tracks) so we can't post anything but
an error on the bus, as:

- it can't send EOS downstream, it has no pad,
- posting an EOS message will be useless as PAUSED state can't be
  reached and there is no sink in the pipeline meaning GstBin will
  simply ignore it

The approach here is to to add details to the ERROR message with a
`redirect-location` field which elements like playbin handle and use right
away.

[0]: http://movietrailers.apple.com/movies/paramount/terminator-dark-fate/terminator-dark-fate-trailer-2_480p.mov
2019-09-30 12:15:43 -03:00
Matthew Waters
5ffd733317 build: fix werror build with newer gcc
In file included from ../../../../dist/linux_x86_64/include/gstreamer-1.0/gst/gst.h:55,
                 from ../../../../dist/linux_x86_64/include/gstreamer-1.0/gst/tag/tag.h:25,
                 from ../gst/isomp4/qtdemux.c:56:
In function ‘qtdemux_inspect_transformation_matrix’,
    inlined from ‘qtdemux_parse_trak’ at ../gst/isomp4/qtdemux.c:10676:5,
    inlined from ‘qtdemux_parse_tree’ at ../gst/isomp4/qtdemux.c:14210:5:
../../../../dist/linux_x86_64/include/gstreamer-1.0/gst/gstinfo.h:645:5: error: ‘%s’ directive argument is null [-Werror=format-overflow=]
  645 |     gst_debug_log ((cat), (level), __FILE__, GST_FUNCTION, __LINE__, \
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  646 |         (GObject *) (object), __VA_ARGS__);    \
      |         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../../../dist/linux_x86_64/include/gstreamer-1.0/gst/gstinfo.h:1062:35: note: in expansion of macro ‘GST_CAT_LEVEL_LOG’
 1062 | #define GST_DEBUG_OBJECT(obj,...) GST_CAT_LEVEL_LOG (GST_CAT_DEFAULT, GST_LEVEL_DEBUG,   obj,  __VA_ARGS__)
      |                                   ^~~~~~~~~~~~~~~~~
../gst/isomp4/qtdemux.c:10294:5: note: in expansion of macro ‘GST_DEBUG_OBJECT’
10294 |     GST_DEBUG_OBJECT (qtdemux, "Transformation matrix rotation %s",
      |     ^~~~~~~~~~~~~~~~
../gst/isomp4/qtdemux.c: In function ‘qtdemux_parse_tree’:
../gst/isomp4/qtdemux.c:10294:64: note: format string is defined here
10294 |     GST_DEBUG_OBJECT (qtdemux, "Transformation matrix rotation %s",
      |                                                                ^~
2019-09-23 18:46:16 +10:00
Sebastian Dröge
5d4a46aa63 qtdemux: Use the new helper functions for mapping the colr atom values to colorimetry 2019-09-18 18:29:27 +03:00
luke.lin
d6ae59c32d qtdemux: enlarge the maximal atom size
For 8K content, frame size is over 25MB, and cause the negotiation failure.
Enlarge the limitation of QTDEMUX_MAX_ATOM_SIZE to 32MB.
2019-08-07 02:46:20 +00:00
Seungha Yang
4146dc905d qtdemux: Use empty-array safe way to cleanup GPtrArray
Fix assertion fail
GLib-CRITICAL **: g_ptr_array_remove_range: assertion 'index_ < rarray->len' failed
2019-08-02 12:32:59 +09:00
Mathieu Duponchelle
4830bbe6ca qtdemux: fix reverse playback EOS conditions
In reverse playback, we don't want to rely on the position of the current
keyframe to decide a stream is EOS: the last GOP we push will start with
a keyframe, which position is likely to be outside of the segment.

Instead, let the normal seek_to_previous_keyframe mechanism do its job,
it works just fine.
2019-07-26 02:42:11 +00:00
Mathieu Duponchelle
104f459258 qtdemux: fix key unit seek corner case
If a key unit seek is performed with a time position that matches
the offset of a keyframe, but not its actual PTS, we need to
adjust the segment nevertheless.

For example consider the following case:

* stream starts with a keyframe at 0 nanosecond, lasting 40 milliseconds
* user does a key unit seek at 20 milliseconds
* we don't adjust the segment as the time position is "over" a keyframe
* we push a segment that starts at 20 milliseconds
* we push a buffer with PTS == 0
* an element downstream (eg rtponviftimestamp) tries to calculate the
  stream time of the buffer, fails to do so and drops it
2019-07-26 01:50:47 +00:00
Mathieu Duponchelle
5fde140e6e qtdemux: implement support for trickmode interval
When the seek event contains a (newly-added) trickmode interval,
and TRICKMODE_KEY_UNITS was requested, only let through keyframes
separated with the required interval
2019-07-18 17:54:43 +02:00
Mathieu Duponchelle
9deb3c27ac qtdemux: fix conditions for end of segment in reverse playback
The time_position field of the stream is offset by the media_start
of its QtDemuxSegment compared to the start of the GstSegment of
the demuxer, take it into account when making comparisons.
2019-07-09 21:21:20 +00:00
Mart Raudsepp
ade531183f qtdemux: Provide a 30 frames lead-in for MP3
mpegaudioparse suggests MP3 needs 10 or 30 frames of lead-in (depending on
mpegaudioversion, which we don't know here), thus provide at least 30 frames
lead-in for such cases as a followup to commit cbfa4531ee.
2019-07-02 20:50:21 +00:00
Mathieu Duponchelle
f4f11530c2 qtdemux: do_seek can never be called with a NULL event 2019-07-02 13:39:55 +02:00
Mathieu Duponchelle
83704e32e6 qtdemux: only adjust segment time when adjusting segment start
We ended up setting segment.time to segment.position when doing
reverse playback, which is obviously wrong.
2019-07-02 13:39:55 +02:00
Mart Raudsepp
cbfa4531ee qtdemux: Provide a 2 frames lead-in for audio decoders
AAC and various other audio codecs need a couple frames of lead-in to
decode it properly. The parser elements like aacparse take care of it
via gst_base_parse_set_frame_rate, but when inside a container, the
demuxer is doing the seek segment handling and never gives lead-in
data downstream.
Handle this similar to going back to a keyframe with video, in the
same place. Without a lead-in, the start of the segment is silence,
when it shouldn't, which becomes especially evident in NLE use cases.
2019-06-05 23:13:33 +03:00
Mart Raudsepp
9b348e755c qtdemux: remove indent exception and reindent
As the indent disabling isn't playing along for a following fix,
remove the indent disabling and reindent in a way that doesn't
look too stupid.
2019-06-05 23:11:13 +03:00
Thibault Saunier
0a6a62aa76 docs: Port all docstring to gtk-doc markdown 2019-05-13 10:24:40 -04:00
Seungha Yang
63bb1e3a4d qtdemux: Don't pass zero to denominator for framerate
Need to respect return of gst_video_guess_framerate() to ensure
non-zero denominator.

This patch is to fix below error with an abnormal (but has valid frame) file.
(gst-play-1.0:17940): GStreamer-CRITICAL **: passed '0' as denominator for `GstFraction'
2019-03-19 12:35:08 +09:00
Charlie Turner
39d32b2394 qtdemux: Find mp4a esds atoms in protected streams sample description tables.
This problem was found in Test. 2 of the YouTube 2018 EME
tests[1]. The code was accidentally not finding an mp4a's esds atom in
the sample description table when the stream was encrypted. It assumed
that if the stream is protected, then only an enca atom will be found
here. What happens with YouTube is they often provide protected
content with a few seconds of clear content, and then switch to the
encrypted stream.

The failure case here was an incorrect codec_data field being sent
into aacparse. The advertisement of stereo audio @ 44.1kHz for the
mp4a (unprotected) stream was incorrect. As usual, the esds contained
the real values here which were mono at 22050 Hz.

Here's what the MP4 tree looks like for these types of files,
demonstrating why the code was making a wrong assumption (or maybe
YouTube is being unusual),

[ftyp] size=8+16
...
[moov] size=8+1571
...
  [trak] size=8+559
...
          [stsd] size=12+234
            entry-count = 2
            [enca] size=8+147
              channel_count = 2
              sample_size = 16
              sample_rate = 44100
              [esds] size=12+27
                ...
            ...
            [mp4a] size=8+67
              channel_count = 2
              sample_size = 16
              sample_rate = 44100
              [esds] size=12+27
                ...

In addition to fixing this, the checks for esds atoms in mp4a and mp4v
have been made symmetrical. While I haven't seen a test case for video
with the same problem, it seemed better to make the same checks. This
also fixes a crash reported from another user[2], they also noted the
asymmetry with mp4v and mp4a.

[1] https://yt-dash-mse-test.commondatastorage.googleapis.com/unit-tests/2018.html?test_type=encryptedmedia-test
[2] https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/issues/398
2019-03-15 12:41:33 +00:00
Edward Hervey
f5f1de54d2 qtdemux: Remove trailing '\n' in debug 2019-02-05 11:01:21 +01:00
Sebastian Dröge
ec931601a6 qtdemux: Split CEA608 buffers correctly so that each output buffer represents a single frame 2019-01-02 10:29:46 +00:00
Sebastian Dröge
aa65ea85f9 qtdemux: Refactor buffer pushing into its own function 2019-01-02 10:29:46 +00:00
Sebastian Dröge
d471be4f3a qtdemux: Extract CEA608 framerate from the (first) video stream
EA608 closed caption tracks are a bit special in that each sample
can contain CCs for multiple frames, and CCs can be omitted and have to
be inferred from the duration of the sample then.

As such we take the framerate from the (first) video track here for
CEA608 as there must be one CC byte pair for every video frame
according to the spec.

For CEA708 all is fine and there is one sample per frame.
2019-01-02 10:29:46 +00:00
Philippe Normand
ce96d6dcd4 qtdemux: Offset correction for track language code parsing
The duration field being a uint64, is stored in 8 bytes, not 4. So the offset of
the following field, language code, needs to be updated accordingly so that the
parsed language code is not garbage.
2018-12-22 20:05:34 +01:00
Sebastian Dröge
4f7ef56c53 isomp4: Replace GST_VIDEO_CAPTION_TYPE_CEA608_IN_CEA708_RAW with CEA608_S334_1A
For the demuxer we have to select line offset 0 for the time being as
this information is not passed over MOV.
2018-12-15 21:31:20 +00:00
Sebastian Dröge
c50be8f146 qtdemux: Put framerate into the closedcaption caps if it can be calculated from the stream
Using the same calculation used for video streams.
2018-12-06 16:05:50 +00:00
Alicia Boya García
38b553dda7 qtdemux: set need_segment after a second moov
stream.segment should be updated with the values of the current edit
list, also when a new `moov` is received. Unfortunately this was not
being the case because of an early return.

As a consequence of this bugs, no end of movie clipping was being
performed on the new moov and no segment event was being emitted.

When performing stream switching (e.g. in MSE) the new moov may have a
different edit list. This is often the case when switching between
baseline H.264 (which lacks B-frames) and more demanding profiles. For
this reason it's important to emit a new segment in order to be able
to get matching stream times.
2018-11-30 20:44:57 +00:00
Alicia Boya García
26cc201c8a qtdemux: Initialize QtDemuxStream.segment in its constructor
This patch moves the initialization of QtDemuxStream.segment from
gst_qtdemux_add_stream() to _create_stream(). This ensures the segment
is always initialized when the stream is created.

Otherwise the segment format is left as GST_FORMAT_UNDEFINED in the case
were a track is reparsed and qtdemux_reuse_and_configure_stream() is
called instead of gst_qtdemux_add_stream(). (See
qtdemux_expose_streams() in the non streams-aware case.)
2018-11-30 20:44:57 +00:00
Matthew Waters
8a7074f748 isomp4: add preliminary support for the bitrate query
Return the upstream size over the duration as a first estimate.

https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/issues/60
2018-11-07 15:07:18 +00:00
Seungha Yang
5d542030db qtdemux: Ignore corrupted CTTS box
If ctts (CompositionOffsetBox) has larger sample_offset
(offset between PTS and DTS) than (2 * duration) of the stream,
assume the ctts box to be corrupted and ignore the box.

https://bugzilla.gnome.org/show_bug.cgi?id=797262
2018-11-01 16:03:12 +02:00
Seungha Yang
7bce030be3 qtdemux: Fix build with GLib versions < 2.54
g_ptr_array_find_with_equal_func was introduced in glib 2.54
which is a higher version than our minimum required one.

https://bugzilla.gnome.org/show_bug.cgi?id=797239
2018-10-20 12:38:32 +01:00
Seungha Yang
05bd25ea35 qtdemux: Don't switch active streams and old streams ...
... before the old streams is not exposed yet for MSS stream.

In case of DASH, newly configured streams will be exposed
whenever demux got moov without delay.
Meanwhile, since there is no moov box in MSS stream,
the caps will act like moov. Then, there is delay for exposing new pads
until demux got the first moof.

So, following scenario is possible only for MSS but not for DASH,
STREAM-START -> CAPS -> (configure stream but NOT EXPOSED YET)
-> STREAM-START-> CAPS (configure stream again).

In above scenario, we can reuse old stream without any stream reconfigure.

https://bugzilla.gnome.org/show_bug.cgi?id=797239
2018-10-19 14:44:43 +02:00
Seungha Yang
b2876ad8a4 qtdemux: Use GPtrArray to store QtDemuxStream structure
GPtrArray has less overhead than linked list and the length also
can be auto updated by using it.

https://bugzilla.gnome.org/show_bug.cgi?id=797239
2018-10-19 14:44:43 +02:00
Seungha Yang
1600323119 qtdemux: Make QtDemuxStream refcounted structure
This a prework for porting GPtrArray.
Refcounting will help the use of g_ptr_array_new_with_free_func()
with QtDemuxStream structure

https://bugzilla.gnome.org/show_bug.cgi?id=797239
2018-10-19 14:44:43 +02:00
Seungha Yang
72123e3da3 qtdemux: Make function foreach method friendly
https://bugzilla.gnome.org/show_bug.cgi?id=797239
2018-10-19 14:44:43 +02:00
Olivier Crête
20d5f92b28 qtdemux: Only set width/height in caps if they're non-0
If they are not valid, then let a downstream parser complete them.

https://bugzilla.gnome.org/show_bug.cgi?id=796878
2018-10-19 14:19:27 +02:00
Philippe Normand
56669205eb qtdemux: Avoid warning when reporting about decryptors
https://bugzilla.gnome.org/show_bug.cgi?id=796652
2018-10-17 15:51:32 +01:00