Commit graph

821 commits

Author SHA1 Message Date
Sebastian Dröge
aa65ea85f9 qtdemux: Refactor buffer pushing into its own function 2019-01-02 10:29:46 +00:00
Sebastian Dröge
d471be4f3a qtdemux: Extract CEA608 framerate from the (first) video stream
EA608 closed caption tracks are a bit special in that each sample
can contain CCs for multiple frames, and CCs can be omitted and have to
be inferred from the duration of the sample then.

As such we take the framerate from the (first) video track here for
CEA608 as there must be one CC byte pair for every video frame
according to the spec.

For CEA708 all is fine and there is one sample per frame.
2019-01-02 10:29:46 +00:00
Philippe Normand
ce96d6dcd4 qtdemux: Offset correction for track language code parsing
The duration field being a uint64, is stored in 8 bytes, not 4. So the offset of
the following field, language code, needs to be updated accordingly so that the
parsed language code is not garbage.
2018-12-22 20:05:34 +01:00
Sebastian Dröge
4f7ef56c53 isomp4: Replace GST_VIDEO_CAPTION_TYPE_CEA608_IN_CEA708_RAW with CEA608_S334_1A
For the demuxer we have to select line offset 0 for the time being as
this information is not passed over MOV.
2018-12-15 21:31:20 +00:00
Sebastian Dröge
c50be8f146 qtdemux: Put framerate into the closedcaption caps if it can be calculated from the stream
Using the same calculation used for video streams.
2018-12-06 16:05:50 +00:00
Sebastian Dröge
830e7dc14b qtmux: Set timescale of closedcaption tracks to the one of the main video track 2018-12-06 16:05:50 +00:00
Alicia Boya García
38b553dda7 qtdemux: set need_segment after a second moov
stream.segment should be updated with the values of the current edit
list, also when a new `moov` is received. Unfortunately this was not
being the case because of an early return.

As a consequence of this bugs, no end of movie clipping was being
performed on the new moov and no segment event was being emitted.

When performing stream switching (e.g. in MSE) the new moov may have a
different edit list. This is often the case when switching between
baseline H.264 (which lacks B-frames) and more demanding profiles. For
this reason it's important to emit a new segment in order to be able
to get matching stream times.
2018-11-30 20:44:57 +00:00
Alicia Boya García
26cc201c8a qtdemux: Initialize QtDemuxStream.segment in its constructor
This patch moves the initialization of QtDemuxStream.segment from
gst_qtdemux_add_stream() to _create_stream(). This ensures the segment
is always initialized when the stream is created.

Otherwise the segment format is left as GST_FORMAT_UNDEFINED in the case
were a track is reparsed and qtdemux_reuse_and_configure_stream() is
called instead of gst_qtdemux_add_stream(). (See
qtdemux_expose_streams() in the non streams-aware case.)
2018-11-30 20:44:57 +00:00
Matthew Waters
8a7074f748 isomp4: add preliminary support for the bitrate query
Return the upstream size over the duration as a first estimate.

https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/issues/60
2018-11-07 15:07:18 +00:00
Seungha Yang
5d542030db qtdemux: Ignore corrupted CTTS box
If ctts (CompositionOffsetBox) has larger sample_offset
(offset between PTS and DTS) than (2 * duration) of the stream,
assume the ctts box to be corrupted and ignore the box.

https://bugzilla.gnome.org/show_bug.cgi?id=797262
2018-11-01 16:03:12 +02:00
Alicia Boya García
5fcb7f715a qtmux: round to nearest when computing mehd and tkhd duration
This fixes a bug where in some files mehd.fragment_duration is one unit
less than the actual duration of the fragmented movie, as explained below:

mehd.fragment_duration is computed by scaling the end timestamp of
the last frame of the movie in (in nanoseconds) by the movie timescale.

In some situations, the end timestamp is innacurate due to lossy conversion to
fixed point required by GstBuffer upstream.

Take for instance a movie with 3 frames at exactly 3 fps.

$ gst-launch-1.0 -v videotestsrc num-buffers=3 \
  ! video/x-raw, framerate="(fraction)3/1" \
  ! x264enc \
  ! fakesink silent=false

dts: 999:59:59.333333334,  pts: 1000:00:00.000000000, duration: 0:00:00.333333333
dts: 999:59:59.666666667,  pts: 1000:00:00.666666666, duration: 0:00:00.333333334
dts: 1000:00:00.000000000, pts: 1000:00:00.333333333, duration: 0:00:00.333333333

The end timestamp is calculated by qtmux in this way:

end timestamp = last frame DTS + last frame DUR - first frame DTS =
  = 1000:00:00.000000000 + 0:00:00.333333333 - 999:59:59.333333334 =
  = 0:00:00.999999999

qtmux needs to round this timestamp to the declared movie timescale, which can
ameliorate this distortion, but it's important that round-neareast is used;
otherwise it would backfire badly.

Take for example a movie with a timescale of 30 units/s.

0.999999999 s * 30 units/s = 29.999999970 units

A round-floor (as it was done before this patch) would set fragment_duration to
29 units, amplifying the original distorsion from 1 nanosecond up to 33
milliseconds less than the correct value. The greatest distortion would occur
in the case where timescale = framerate, where an entire frame duration would
be subtracted.

Also, rounding is added to tkhd duration computation too, which
potentially has the same problem.

https://bugzilla.gnome.org/show_bug.cgi?id=793959
2018-10-27 13:12:56 +01:00
Sebastian Dröge
01a2119ad0 qtmux: Add property for providing a threshold after which we create an edit list for gaps at the start
https://bugzilla.gnome.org/show_bug.cgi?id=797290
2018-10-22 12:29:23 +01:00
Sebastian Dröge
324f8c7f3c qtmux: Correctly set tkhd width/height to the display size
It was previously set to the display aspect ratio, e.g. 4x3, 16x9, etc.
but should be set to the display size.

This is a regression from e655d47dfc
(1.5.1) and was correct before that.

https://bugzilla.gnome.org/show_bug.cgi?id=797318
2018-10-22 12:23:05 +01:00
Seungha Yang
7bce030be3 qtdemux: Fix build with GLib versions < 2.54
g_ptr_array_find_with_equal_func was introduced in glib 2.54
which is a higher version than our minimum required one.

https://bugzilla.gnome.org/show_bug.cgi?id=797239
2018-10-20 12:38:32 +01:00
Seungha Yang
05bd25ea35 qtdemux: Don't switch active streams and old streams ...
... before the old streams is not exposed yet for MSS stream.

In case of DASH, newly configured streams will be exposed
whenever demux got moov without delay.
Meanwhile, since there is no moov box in MSS stream,
the caps will act like moov. Then, there is delay for exposing new pads
until demux got the first moof.

So, following scenario is possible only for MSS but not for DASH,
STREAM-START -> CAPS -> (configure stream but NOT EXPOSED YET)
-> STREAM-START-> CAPS (configure stream again).

In above scenario, we can reuse old stream without any stream reconfigure.

https://bugzilla.gnome.org/show_bug.cgi?id=797239
2018-10-19 14:44:43 +02:00
Seungha Yang
b2876ad8a4 qtdemux: Use GPtrArray to store QtDemuxStream structure
GPtrArray has less overhead than linked list and the length also
can be auto updated by using it.

https://bugzilla.gnome.org/show_bug.cgi?id=797239
2018-10-19 14:44:43 +02:00
Seungha Yang
1600323119 qtdemux: Make QtDemuxStream refcounted structure
This a prework for porting GPtrArray.
Refcounting will help the use of g_ptr_array_new_with_free_func()
with QtDemuxStream structure

https://bugzilla.gnome.org/show_bug.cgi?id=797239
2018-10-19 14:44:43 +02:00
Seungha Yang
72123e3da3 qtdemux: Make function foreach method friendly
https://bugzilla.gnome.org/show_bug.cgi?id=797239
2018-10-19 14:44:43 +02:00
Olivier Crête
20d5f92b28 qtdemux: Only set width/height in caps if they're non-0
If they are not valid, then let a downstream parser complete them.

https://bugzilla.gnome.org/show_bug.cgi?id=796878
2018-10-19 14:19:27 +02:00
Philippe Normand
56669205eb qtdemux: Avoid warning when reporting about decryptors
https://bugzilla.gnome.org/show_bug.cgi?id=796652
2018-10-17 15:51:32 +01:00
Vivia Nikolaidou
09904e59df qtmux: Allow up to 1% of frame rate for lateness
https://bugzilla.gnome.org/show_bug.cgi?id=797290
2018-10-16 16:05:46 +03:00
Philippe Normand
babf4210f0 qtdemux: PIFF track encryption box support
The PIFF track encryption box is a UUID box containing the default encryption
values that should be used for PIFF sample encryption.

https://bugzilla.gnome.org/show_bug.cgi?id=796647
2018-09-25 09:53:31 +01:00
Alicia Boya García
bc0ea0dbbb qtdemux: turn impossible condition into an assert
qtdemux_update_streams() is only ever called after checking
`qtdemux->streams_aware` is TRUE. There is no need to check for that
condition again.

`qtdemux->streams_aware` is only modified when the demuxer is
hard-resetted, which is mutually exclusive with demuxing, so it cannot
be modified during the call.

https://bugzilla.gnome.org/show_bug.cgi?id=797191
2018-09-24 08:33:02 +01:00
Sebastian Dröge
c6e07a6eee qtmux: Set Closed Caption track width/height to that of the first video track
Otherwise software like Premiere or Final Cut Pro won't like our files.

https://bugzilla.gnome.org/show_bug.cgi?id=797111
2018-09-20 11:35:31 +03:00
Sebastian Dröge
54a5b3f845 qtmux: Initialize caption track language code to 0 instead of "und"
Without this, Final Cut considers it "non-standard" and 0 (english) is a
good default for closed captions.

https://bugzilla.gnome.org/show_bug.cgi?id=797111
2018-09-18 17:32:46 +03:00
Vivia Nikolaidou
94f8603411 qtmux: Allow up to 1 trak timescale unit of lateness in prefill mode
For 59.94 FPS, it's common to set 60000 as timescale. For that
timescale, if the audio is late by as little as 0:00:00.000016666
(definitely less than one audio sample), lateness gets rounded to 1.
Added a safeguard that allows lateness up to 1 sample with the specific
trak's timescale, to make sure that values less than e.g. one audio
sample won't break the prefill mode. What will happen in this case is
that the audio will get squeezed back to the video's timestamp, which in
practice means that the audio will be 0.000016666 seconds early (with
the patch).

https://bugzilla.gnome.org/show_bug.cgi?id=797133
2018-09-13 13:51:18 +03:00
Sebastian Dröge
d9b52d1f5e qtmux: Use existing helper function to create "und" language code 2018-09-06 20:07:57 +03:00
Alicia Boya García
794dcfbfee qtdemux: Keep sample data from the current fragment only (push mode)
This patch clears the sample table whenever the demuxing of a new
fragment begins. This avoids increasing memory usage for long videos.
This behavior was already present when upstream_format_is_time; this
patch extends it to all push mode operation (e.g. Media Source
Extensions).

https://bugzilla.gnome.org/show_bug.cgi?id=796899
2018-09-03 12:38:17 -03:00
Mathieu Duponchelle
bfcddb7125 mp4 robust muxing: improve documentation and logging 2018-08-16 19:43:50 +02:00
Sebastian Dröge
5e4f215bfb qtdemux: Handle closed captions as subtitle streams 2018-08-02 16:13:26 +03:00
Sebastian Dröge
7a354d0507 qtmux: The sample size we have to reserve is 256+8 bytes for the header for CDP packets 2018-08-02 13:40:09 +03:00
Sebastian Dröge
8a3075fbae qtmux: Properly allocate 256 bytes per CDP packet in prefill mode
Instead of allowing 256 but only pre-allocating 100.
2018-08-02 12:27:45 +03:00
Sebastian Dröge
bbfd76e94e Revert "qtmux: Allow for CDP packets up to 320 bytes"
This reverts commit 5eed1d49bd.

255 is actually the maximum, there's a bug if more is arriving.
2018-08-02 12:27:17 +03:00
Sebastian Dröge
5eed1d49bd qtmux: Allow for CDP packets up to 320 bytes
Apparently they can be bigger than 256 bytes sometimes.
2018-08-01 16:51:11 +03:00
Sebastian Dröge
7679412b69 qtdemux: Don't assert in prefill mode if a track has no samples at all
Just write it with a duration of 0, no samples, etc.
2018-07-31 12:47:47 +03:00
Sebastian Dröge
6ce2a5f7bf qtdemux: Don't assert if a file does not have any active streams
** (gst-play-1.0:9113): CRITICAL **: 12:31:54.360: qtdemux_is_streams_update: assertion 'qtdemux->active_streams != NULL' failed
2018-07-31 12:33:54 +03:00
Sebastian Dröge
e58dceec3a qtmux: Reserve 256 bytes for CDP packets in pre-fill mode
92 is sometimes too small and compared to the wasted space for other
codecs 256 bytes is small (and should be the maximum CDP packet size)
2018-07-30 13:35:02 +03:00
Olivier Crête
6c478453a8 qtmux: Implement muxing of AV1 into MP4 files
According to
https://aomediacodec.github.io/av1-isobmff/
2018-07-26 14:14:53 -04:00
Olivier Crête
e07ce9afb4 qtdemux: Recognize more AV1 atoms 2018-07-26 14:14:53 -04:00
Olivier Crête
43ee4055c3 qtdemux: Extract AV1 codec_data and put it in the caps
Also extract the presentation-delay and put it in the caps.
2018-07-26 14:14:53 -04:00
Olivier Crête
b6e8c6323e qtdemux: Add initial support for AV1 demuxing
Following the spec at
https://aomediacodec.github.io/av1-isobmff/
2018-07-26 14:14:53 -04:00
Edward Hervey
9be72b585d qtdemux: Store and propagate SEGMENT sequence numbers
* When receiving a segment in TIME, use that seqnum
* Only reset the stored sequence number when doing HARD reset
  (and not when we get a FLUSH event from upstream)
2018-07-06 15:21:33 +02:00
Alicia Boya García
025a430d08 qtdemux: rework segment event pushing, again
This patch aims at fixing the recent regressions in the adaptive test
suite.

All segment pushing in push mode is now done with
gst_qtdemux_check_send_pending_segment(), which is idempotent and
handles both edit lists cases and cases where the upstream TIME segments
have to be sent directly.

Fragmented files that start with a non-zero tfdt are also taken into
account, but their handling has been vastly simplified: now they are
handled as implicit default seeks so there is no need to extend the
GstSegment formulas as was being done before.

qtdemux->segment.duration is no longer modified when
upstream_format_is_time, respecting in this way the durations provided
by dashdemux and fixing bugs in reverse playback tests where mangled
durations appeared in the emitted segments.

https://bugzilla.gnome.org/show_bug.cgi?id=752603
2018-06-29 09:19:31 -04:00
Alicia Boya García
2c394304b7 qtdemux: Don't send EOS during upstream reverse playback
Upstream driving elements such as dashdemux often do reverse playback by
feeding qtdemux with the fragments containing the requested playback
range in reverse order.

But the requested playback range stop may be somewhere in the
middle of a fragment. In that case, a naive pts >= segment.stop
condition may declare end of segment prematurely when demuxing this
first fragment.

This used not to happen because there were places in moov parsing where
segment.stop was overwritten to GST_CLOCK_TIME_NONE even if
upstream_format_is_time -- resulting in this case in a segment with rate
< 0 and stop == -1 and hence not triggering the EOS check, but that was
likely an accident.

This patch modifies the EOS check to take this case into account, not
sending EOS when upstream_format_is_time if rate < 0.

This fixes adaptive.dash.playback.seek_end_live.DASHIF_livestream_testpic_2s

https://bugzilla.gnome.org/show_bug.cgi?id=752603
2018-06-29 09:19:31 -04:00
Thibault Saunier
1840159d0b qtdemux: Do not set INVALID seqnum on events 2018-06-06 11:44:33 -04:00
Seungha Yang
6b29a4507e qtdemux: Forward upstream time-format segment without mapping
Sample table based segment event (genereted by qtdemux) could break
presentation timeline. For example, qtdemux should not modify upstream
time format segment (e.g., adaptivedemux use case)

https://bugzilla.gnome.org/show_bug.cgi?id=796480
2018-06-06 10:54:55 -04:00
Thibault Saunier
0fbde2a079 qtdemux: Clarify field name about stream-encryption-system
This field is actually only informatory and the user can potentially
choose something else. EME tests in WebKit testsuite actually doesn't
take it into and force another encryption system to be used, and expects
to be given the occasion to do so.

This basically also reverts 3e063703b3.
2018-05-28 16:52:10 -04:00
Thiago Santos
ae7b531902 qtdemux: mark segment as sent after pushing when moov is received
Otherwise we would try to send it a second time if the same moov is
received or in any other situation that might trigger segment sending.

https://bugzilla.gnome.org/show_bug.cgi?id=752603
2018-05-28 11:26:36 -07:00
Thiago Santos
364a921d3f qtdemux: do not update segment.stop is it is not a valid time
Otherwise it overflows and starts having a meaningful and wrong value.

https://bugzilla.gnome.org/show_bug.cgi?id=752603
2018-05-25 08:34:29 -07:00
Thiago Santos
9c514f9c7c qtdemux: offset edts segments by the min timestamp of the stream
Otherwise if the stream is starting at timestamp=X it would wait
'X' to start playing.

https://bugzilla.gnome.org/show_bug.cgi?id=752603
2018-05-25 08:34:29 -07:00