Commit graph

839 commits

Author SHA1 Message Date
Mathieu Duponchelle
104f459258 qtdemux: fix key unit seek corner case
If a key unit seek is performed with a time position that matches
the offset of a keyframe, but not its actual PTS, we need to
adjust the segment nevertheless.

For example consider the following case:

* stream starts with a keyframe at 0 nanosecond, lasting 40 milliseconds
* user does a key unit seek at 20 milliseconds
* we don't adjust the segment as the time position is "over" a keyframe
* we push a segment that starts at 20 milliseconds
* we push a buffer with PTS == 0
* an element downstream (eg rtponviftimestamp) tries to calculate the
  stream time of the buffer, fails to do so and drops it
2019-07-26 01:50:47 +00:00
Mathieu Duponchelle
5fde140e6e qtdemux: implement support for trickmode interval
When the seek event contains a (newly-added) trickmode interval,
and TRICKMODE_KEY_UNITS was requested, only let through keyframes
separated with the required interval
2019-07-18 17:54:43 +02:00
Mathieu Duponchelle
9deb3c27ac qtdemux: fix conditions for end of segment in reverse playback
The time_position field of the stream is offset by the media_start
of its QtDemuxSegment compared to the start of the GstSegment of
the demuxer, take it into account when making comparisons.
2019-07-09 21:21:20 +00:00
Mart Raudsepp
ade531183f qtdemux: Provide a 30 frames lead-in for MP3
mpegaudioparse suggests MP3 needs 10 or 30 frames of lead-in (depending on
mpegaudioversion, which we don't know here), thus provide at least 30 frames
lead-in for such cases as a followup to commit cbfa4531ee.
2019-07-02 20:50:21 +00:00
Mathieu Duponchelle
f4f11530c2 qtdemux: do_seek can never be called with a NULL event 2019-07-02 13:39:55 +02:00
Mathieu Duponchelle
83704e32e6 qtdemux: only adjust segment time when adjusting segment start
We ended up setting segment.time to segment.position when doing
reverse playback, which is obviously wrong.
2019-07-02 13:39:55 +02:00
Mart Raudsepp
cbfa4531ee qtdemux: Provide a 2 frames lead-in for audio decoders
AAC and various other audio codecs need a couple frames of lead-in to
decode it properly. The parser elements like aacparse take care of it
via gst_base_parse_set_frame_rate, but when inside a container, the
demuxer is doing the seek segment handling and never gives lead-in
data downstream.
Handle this similar to going back to a keyframe with video, in the
same place. Without a lead-in, the start of the segment is silence,
when it shouldn't, which becomes especially evident in NLE use cases.
2019-06-05 23:13:33 +03:00
Mart Raudsepp
9b348e755c qtdemux: remove indent exception and reindent
As the indent disabling isn't playing along for a following fix,
remove the indent disabling and reindent in a way that doesn't
look too stupid.
2019-06-05 23:11:13 +03:00
Sebastian Dröge
2bed2687bb qtmux: Use size of first closed caption buffer in prefill mode
It must be accurate for all samples to work in Final Cut properly, so
the best we can do is to assume that all samples are the same as the
first. Bigger samples are truncated, smaller samples are padded.
2019-06-03 12:46:34 +03:00
Mathieu Duponchelle
d704790519 doc: fix element section documentations
Element sections were not rendered anymore after the hotdoc
port, fixing this revealed a few incorrect links.
2019-05-25 16:57:31 +02:00
Thibault Saunier
af01988534 doc: Port documentation to hotdoc 2019-05-13 11:34:56 -04:00
Thibault Saunier
232e3682ea Mark some properties as DOC_SHOW_DEFAULT 2019-05-13 10:24:40 -04:00
Thibault Saunier
0a6a62aa76 docs: Port all docstring to gtk-doc markdown 2019-05-13 10:24:40 -04:00
Seungha Yang
63bb1e3a4d qtdemux: Don't pass zero to denominator for framerate
Need to respect return of gst_video_guess_framerate() to ensure
non-zero denominator.

This patch is to fix below error with an abnormal (but has valid frame) file.
(gst-play-1.0:17940): GStreamer-CRITICAL **: passed '0' as denominator for `GstFraction'
2019-03-19 12:35:08 +09:00
Charlie Turner
39d32b2394 qtdemux: Find mp4a esds atoms in protected streams sample description tables.
This problem was found in Test. 2 of the YouTube 2018 EME
tests[1]. The code was accidentally not finding an mp4a's esds atom in
the sample description table when the stream was encrypted. It assumed
that if the stream is protected, then only an enca atom will be found
here. What happens with YouTube is they often provide protected
content with a few seconds of clear content, and then switch to the
encrypted stream.

The failure case here was an incorrect codec_data field being sent
into aacparse. The advertisement of stereo audio @ 44.1kHz for the
mp4a (unprotected) stream was incorrect. As usual, the esds contained
the real values here which were mono at 22050 Hz.

Here's what the MP4 tree looks like for these types of files,
demonstrating why the code was making a wrong assumption (or maybe
YouTube is being unusual),

[ftyp] size=8+16
...
[moov] size=8+1571
...
  [trak] size=8+559
...
          [stsd] size=12+234
            entry-count = 2
            [enca] size=8+147
              channel_count = 2
              sample_size = 16
              sample_rate = 44100
              [esds] size=12+27
                ...
            ...
            [mp4a] size=8+67
              channel_count = 2
              sample_size = 16
              sample_rate = 44100
              [esds] size=12+27
                ...

In addition to fixing this, the checks for esds atoms in mp4a and mp4v
have been made symmetrical. While I haven't seen a test case for video
with the same problem, it seemed better to make the same checks. This
also fixes a crash reported from another user[2], they also noted the
asymmetry with mp4v and mp4a.

[1] https://yt-dash-mse-test.commondatastorage.googleapis.com/unit-tests/2018.html?test_type=encryptedmedia-test
[2] https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/issues/398
2019-03-15 12:41:33 +00:00
Vivia Nikolaidou
92272b5e5c qtmux: Only write timecode trak for video
Recent changes in ccextractor were attaching timecode meta to the closed
caption track. We shouldn't write timecode information for the closed
caption trak.
2019-02-08 14:13:46 +02:00
Edward Hervey
f5f1de54d2 qtdemux: Remove trailing '\n' in debug 2019-02-05 11:01:21 +01:00
Sebastian Dröge
ec931601a6 qtdemux: Split CEA608 buffers correctly so that each output buffer represents a single frame 2019-01-02 10:29:46 +00:00
Sebastian Dröge
aa65ea85f9 qtdemux: Refactor buffer pushing into its own function 2019-01-02 10:29:46 +00:00
Sebastian Dröge
d471be4f3a qtdemux: Extract CEA608 framerate from the (first) video stream
EA608 closed caption tracks are a bit special in that each sample
can contain CCs for multiple frames, and CCs can be omitted and have to
be inferred from the duration of the sample then.

As such we take the framerate from the (first) video track here for
CEA608 as there must be one CC byte pair for every video frame
according to the spec.

For CEA708 all is fine and there is one sample per frame.
2019-01-02 10:29:46 +00:00
Philippe Normand
ce96d6dcd4 qtdemux: Offset correction for track language code parsing
The duration field being a uint64, is stored in 8 bytes, not 4. So the offset of
the following field, language code, needs to be updated accordingly so that the
parsed language code is not garbage.
2018-12-22 20:05:34 +01:00
Sebastian Dröge
4f7ef56c53 isomp4: Replace GST_VIDEO_CAPTION_TYPE_CEA608_IN_CEA708_RAW with CEA608_S334_1A
For the demuxer we have to select line offset 0 for the time being as
this information is not passed over MOV.
2018-12-15 21:31:20 +00:00
Sebastian Dröge
c50be8f146 qtdemux: Put framerate into the closedcaption caps if it can be calculated from the stream
Using the same calculation used for video streams.
2018-12-06 16:05:50 +00:00
Sebastian Dröge
830e7dc14b qtmux: Set timescale of closedcaption tracks to the one of the main video track 2018-12-06 16:05:50 +00:00
Alicia Boya García
38b553dda7 qtdemux: set need_segment after a second moov
stream.segment should be updated with the values of the current edit
list, also when a new `moov` is received. Unfortunately this was not
being the case because of an early return.

As a consequence of this bugs, no end of movie clipping was being
performed on the new moov and no segment event was being emitted.

When performing stream switching (e.g. in MSE) the new moov may have a
different edit list. This is often the case when switching between
baseline H.264 (which lacks B-frames) and more demanding profiles. For
this reason it's important to emit a new segment in order to be able
to get matching stream times.
2018-11-30 20:44:57 +00:00
Alicia Boya García
26cc201c8a qtdemux: Initialize QtDemuxStream.segment in its constructor
This patch moves the initialization of QtDemuxStream.segment from
gst_qtdemux_add_stream() to _create_stream(). This ensures the segment
is always initialized when the stream is created.

Otherwise the segment format is left as GST_FORMAT_UNDEFINED in the case
were a track is reparsed and qtdemux_reuse_and_configure_stream() is
called instead of gst_qtdemux_add_stream(). (See
qtdemux_expose_streams() in the non streams-aware case.)
2018-11-30 20:44:57 +00:00
Matthew Waters
8a7074f748 isomp4: add preliminary support for the bitrate query
Return the upstream size over the duration as a first estimate.

https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/issues/60
2018-11-07 15:07:18 +00:00
Seungha Yang
5d542030db qtdemux: Ignore corrupted CTTS box
If ctts (CompositionOffsetBox) has larger sample_offset
(offset between PTS and DTS) than (2 * duration) of the stream,
assume the ctts box to be corrupted and ignore the box.

https://bugzilla.gnome.org/show_bug.cgi?id=797262
2018-11-01 16:03:12 +02:00
Alicia Boya García
5fcb7f715a qtmux: round to nearest when computing mehd and tkhd duration
This fixes a bug where in some files mehd.fragment_duration is one unit
less than the actual duration of the fragmented movie, as explained below:

mehd.fragment_duration is computed by scaling the end timestamp of
the last frame of the movie in (in nanoseconds) by the movie timescale.

In some situations, the end timestamp is innacurate due to lossy conversion to
fixed point required by GstBuffer upstream.

Take for instance a movie with 3 frames at exactly 3 fps.

$ gst-launch-1.0 -v videotestsrc num-buffers=3 \
  ! video/x-raw, framerate="(fraction)3/1" \
  ! x264enc \
  ! fakesink silent=false

dts: 999:59:59.333333334,  pts: 1000:00:00.000000000, duration: 0:00:00.333333333
dts: 999:59:59.666666667,  pts: 1000:00:00.666666666, duration: 0:00:00.333333334
dts: 1000:00:00.000000000, pts: 1000:00:00.333333333, duration: 0:00:00.333333333

The end timestamp is calculated by qtmux in this way:

end timestamp = last frame DTS + last frame DUR - first frame DTS =
  = 1000:00:00.000000000 + 0:00:00.333333333 - 999:59:59.333333334 =
  = 0:00:00.999999999

qtmux needs to round this timestamp to the declared movie timescale, which can
ameliorate this distortion, but it's important that round-neareast is used;
otherwise it would backfire badly.

Take for example a movie with a timescale of 30 units/s.

0.999999999 s * 30 units/s = 29.999999970 units

A round-floor (as it was done before this patch) would set fragment_duration to
29 units, amplifying the original distorsion from 1 nanosecond up to 33
milliseconds less than the correct value. The greatest distortion would occur
in the case where timescale = framerate, where an entire frame duration would
be subtracted.

Also, rounding is added to tkhd duration computation too, which
potentially has the same problem.

https://bugzilla.gnome.org/show_bug.cgi?id=793959
2018-10-27 13:12:56 +01:00
Sebastian Dröge
01a2119ad0 qtmux: Add property for providing a threshold after which we create an edit list for gaps at the start
https://bugzilla.gnome.org/show_bug.cgi?id=797290
2018-10-22 12:29:23 +01:00
Sebastian Dröge
324f8c7f3c qtmux: Correctly set tkhd width/height to the display size
It was previously set to the display aspect ratio, e.g. 4x3, 16x9, etc.
but should be set to the display size.

This is a regression from e655d47dfc
(1.5.1) and was correct before that.

https://bugzilla.gnome.org/show_bug.cgi?id=797318
2018-10-22 12:23:05 +01:00
Seungha Yang
7bce030be3 qtdemux: Fix build with GLib versions < 2.54
g_ptr_array_find_with_equal_func was introduced in glib 2.54
which is a higher version than our minimum required one.

https://bugzilla.gnome.org/show_bug.cgi?id=797239
2018-10-20 12:38:32 +01:00
Seungha Yang
05bd25ea35 qtdemux: Don't switch active streams and old streams ...
... before the old streams is not exposed yet for MSS stream.

In case of DASH, newly configured streams will be exposed
whenever demux got moov without delay.
Meanwhile, since there is no moov box in MSS stream,
the caps will act like moov. Then, there is delay for exposing new pads
until demux got the first moof.

So, following scenario is possible only for MSS but not for DASH,
STREAM-START -> CAPS -> (configure stream but NOT EXPOSED YET)
-> STREAM-START-> CAPS (configure stream again).

In above scenario, we can reuse old stream without any stream reconfigure.

https://bugzilla.gnome.org/show_bug.cgi?id=797239
2018-10-19 14:44:43 +02:00
Seungha Yang
b2876ad8a4 qtdemux: Use GPtrArray to store QtDemuxStream structure
GPtrArray has less overhead than linked list and the length also
can be auto updated by using it.

https://bugzilla.gnome.org/show_bug.cgi?id=797239
2018-10-19 14:44:43 +02:00
Seungha Yang
1600323119 qtdemux: Make QtDemuxStream refcounted structure
This a prework for porting GPtrArray.
Refcounting will help the use of g_ptr_array_new_with_free_func()
with QtDemuxStream structure

https://bugzilla.gnome.org/show_bug.cgi?id=797239
2018-10-19 14:44:43 +02:00
Seungha Yang
72123e3da3 qtdemux: Make function foreach method friendly
https://bugzilla.gnome.org/show_bug.cgi?id=797239
2018-10-19 14:44:43 +02:00
Olivier Crête
20d5f92b28 qtdemux: Only set width/height in caps if they're non-0
If they are not valid, then let a downstream parser complete them.

https://bugzilla.gnome.org/show_bug.cgi?id=796878
2018-10-19 14:19:27 +02:00
Philippe Normand
56669205eb qtdemux: Avoid warning when reporting about decryptors
https://bugzilla.gnome.org/show_bug.cgi?id=796652
2018-10-17 15:51:32 +01:00
Vivia Nikolaidou
09904e59df qtmux: Allow up to 1% of frame rate for lateness
https://bugzilla.gnome.org/show_bug.cgi?id=797290
2018-10-16 16:05:46 +03:00
Philippe Normand
babf4210f0 qtdemux: PIFF track encryption box support
The PIFF track encryption box is a UUID box containing the default encryption
values that should be used for PIFF sample encryption.

https://bugzilla.gnome.org/show_bug.cgi?id=796647
2018-09-25 09:53:31 +01:00
Alicia Boya García
bc0ea0dbbb qtdemux: turn impossible condition into an assert
qtdemux_update_streams() is only ever called after checking
`qtdemux->streams_aware` is TRUE. There is no need to check for that
condition again.

`qtdemux->streams_aware` is only modified when the demuxer is
hard-resetted, which is mutually exclusive with demuxing, so it cannot
be modified during the call.

https://bugzilla.gnome.org/show_bug.cgi?id=797191
2018-09-24 08:33:02 +01:00
Sebastian Dröge
c6e07a6eee qtmux: Set Closed Caption track width/height to that of the first video track
Otherwise software like Premiere or Final Cut Pro won't like our files.

https://bugzilla.gnome.org/show_bug.cgi?id=797111
2018-09-20 11:35:31 +03:00
Sebastian Dröge
54a5b3f845 qtmux: Initialize caption track language code to 0 instead of "und"
Without this, Final Cut considers it "non-standard" and 0 (english) is a
good default for closed captions.

https://bugzilla.gnome.org/show_bug.cgi?id=797111
2018-09-18 17:32:46 +03:00
Vivia Nikolaidou
94f8603411 qtmux: Allow up to 1 trak timescale unit of lateness in prefill mode
For 59.94 FPS, it's common to set 60000 as timescale. For that
timescale, if the audio is late by as little as 0:00:00.000016666
(definitely less than one audio sample), lateness gets rounded to 1.
Added a safeguard that allows lateness up to 1 sample with the specific
trak's timescale, to make sure that values less than e.g. one audio
sample won't break the prefill mode. What will happen in this case is
that the audio will get squeezed back to the video's timestamp, which in
practice means that the audio will be 0.000016666 seconds early (with
the patch).

https://bugzilla.gnome.org/show_bug.cgi?id=797133
2018-09-13 13:51:18 +03:00
Sebastian Dröge
d9b52d1f5e qtmux: Use existing helper function to create "und" language code 2018-09-06 20:07:57 +03:00
Alicia Boya García
794dcfbfee qtdemux: Keep sample data from the current fragment only (push mode)
This patch clears the sample table whenever the demuxing of a new
fragment begins. This avoids increasing memory usage for long videos.
This behavior was already present when upstream_format_is_time; this
patch extends it to all push mode operation (e.g. Media Source
Extensions).

https://bugzilla.gnome.org/show_bug.cgi?id=796899
2018-09-03 12:38:17 -03:00
Mathieu Duponchelle
bfcddb7125 mp4 robust muxing: improve documentation and logging 2018-08-16 19:43:50 +02:00
Sebastian Dröge
5e4f215bfb qtdemux: Handle closed captions as subtitle streams 2018-08-02 16:13:26 +03:00
Sebastian Dröge
7a354d0507 qtmux: The sample size we have to reserve is 256+8 bytes for the header for CDP packets 2018-08-02 13:40:09 +03:00
Sebastian Dröge
8a3075fbae qtmux: Properly allocate 256 bytes per CDP packet in prefill mode
Instead of allowing 256 but only pre-allocating 100.
2018-08-02 12:27:45 +03:00