Commit graph

223 commits

Author SHA1 Message Date
Sebastian Dröge
830e7dc14b qtmux: Set timescale of closedcaption tracks to the one of the main video track 2018-12-06 16:05:50 +00:00
Alicia Boya García
5fcb7f715a qtmux: round to nearest when computing mehd and tkhd duration
This fixes a bug where in some files mehd.fragment_duration is one unit
less than the actual duration of the fragmented movie, as explained below:

mehd.fragment_duration is computed by scaling the end timestamp of
the last frame of the movie in (in nanoseconds) by the movie timescale.

In some situations, the end timestamp is innacurate due to lossy conversion to
fixed point required by GstBuffer upstream.

Take for instance a movie with 3 frames at exactly 3 fps.

$ gst-launch-1.0 -v videotestsrc num-buffers=3 \
  ! video/x-raw, framerate="(fraction)3/1" \
  ! x264enc \
  ! fakesink silent=false

dts: 999:59:59.333333334,  pts: 1000:00:00.000000000, duration: 0:00:00.333333333
dts: 999:59:59.666666667,  pts: 1000:00:00.666666666, duration: 0:00:00.333333334
dts: 1000:00:00.000000000, pts: 1000:00:00.333333333, duration: 0:00:00.333333333

The end timestamp is calculated by qtmux in this way:

end timestamp = last frame DTS + last frame DUR - first frame DTS =
  = 1000:00:00.000000000 + 0:00:00.333333333 - 999:59:59.333333334 =
  = 0:00:00.999999999

qtmux needs to round this timestamp to the declared movie timescale, which can
ameliorate this distortion, but it's important that round-neareast is used;
otherwise it would backfire badly.

Take for example a movie with a timescale of 30 units/s.

0.999999999 s * 30 units/s = 29.999999970 units

A round-floor (as it was done before this patch) would set fragment_duration to
29 units, amplifying the original distorsion from 1 nanosecond up to 33
milliseconds less than the correct value. The greatest distortion would occur
in the case where timescale = framerate, where an entire frame duration would
be subtracted.

Also, rounding is added to tkhd duration computation too, which
potentially has the same problem.

https://bugzilla.gnome.org/show_bug.cgi?id=793959
2018-10-27 13:12:56 +01:00
Sebastian Dröge
01a2119ad0 qtmux: Add property for providing a threshold after which we create an edit list for gaps at the start
https://bugzilla.gnome.org/show_bug.cgi?id=797290
2018-10-22 12:29:23 +01:00
Vivia Nikolaidou
09904e59df qtmux: Allow up to 1% of frame rate for lateness
https://bugzilla.gnome.org/show_bug.cgi?id=797290
2018-10-16 16:05:46 +03:00
Sebastian Dröge
c6e07a6eee qtmux: Set Closed Caption track width/height to that of the first video track
Otherwise software like Premiere or Final Cut Pro won't like our files.

https://bugzilla.gnome.org/show_bug.cgi?id=797111
2018-09-20 11:35:31 +03:00
Sebastian Dröge
54a5b3f845 qtmux: Initialize caption track language code to 0 instead of "und"
Without this, Final Cut considers it "non-standard" and 0 (english) is a
good default for closed captions.

https://bugzilla.gnome.org/show_bug.cgi?id=797111
2018-09-18 17:32:46 +03:00
Vivia Nikolaidou
94f8603411 qtmux: Allow up to 1 trak timescale unit of lateness in prefill mode
For 59.94 FPS, it's common to set 60000 as timescale. For that
timescale, if the audio is late by as little as 0:00:00.000016666
(definitely less than one audio sample), lateness gets rounded to 1.
Added a safeguard that allows lateness up to 1 sample with the specific
trak's timescale, to make sure that values less than e.g. one audio
sample won't break the prefill mode. What will happen in this case is
that the audio will get squeezed back to the video's timestamp, which in
practice means that the audio will be 0.000016666 seconds early (with
the patch).

https://bugzilla.gnome.org/show_bug.cgi?id=797133
2018-09-13 13:51:18 +03:00
Sebastian Dröge
d9b52d1f5e qtmux: Use existing helper function to create "und" language code 2018-09-06 20:07:57 +03:00
Mathieu Duponchelle
bfcddb7125 mp4 robust muxing: improve documentation and logging 2018-08-16 19:43:50 +02:00
Sebastian Dröge
7a354d0507 qtmux: The sample size we have to reserve is 256+8 bytes for the header for CDP packets 2018-08-02 13:40:09 +03:00
Sebastian Dröge
8a3075fbae qtmux: Properly allocate 256 bytes per CDP packet in prefill mode
Instead of allowing 256 but only pre-allocating 100.
2018-08-02 12:27:45 +03:00
Sebastian Dröge
bbfd76e94e Revert "qtmux: Allow for CDP packets up to 320 bytes"
This reverts commit 5eed1d49bd.

255 is actually the maximum, there's a bug if more is arriving.
2018-08-02 12:27:17 +03:00
Sebastian Dröge
5eed1d49bd qtmux: Allow for CDP packets up to 320 bytes
Apparently they can be bigger than 256 bytes sometimes.
2018-08-01 16:51:11 +03:00
Sebastian Dröge
7679412b69 qtdemux: Don't assert in prefill mode if a track has no samples at all
Just write it with a duration of 0, no samples, etc.
2018-07-31 12:47:47 +03:00
Sebastian Dröge
e58dceec3a qtmux: Reserve 256 bytes for CDP packets in pre-fill mode
92 is sometimes too small and compared to the wasted space for other
codecs 256 bytes is small (and should be the maximum CDP packet size)
2018-07-30 13:35:02 +03:00
Olivier Crête
6c478453a8 qtmux: Implement muxing of AV1 into MP4 files
According to
https://aomediacodec.github.io/av1-isobmff/
2018-07-26 14:14:53 -04:00
Sebastian Dröge
6e7b0d6061 qtmux: Print expected/actual values in debug log on mismatch in prefill mode
This helps debugging a lot.
2018-05-14 21:06:55 +03:00
Edward Hervey
90244da780 qtmux: Read caption from input buffer
And not from unallocated output buffer GstMapInfo

CID #1435131
2018-04-25 10:37:40 +02:00
Edward Hervey
7378f1b4fd isomp4: qtmux: Add Closed Caption support
Supports CEA 608 and CEA 708 CC streams

Also supports usage in "Robust Prefill" mode if the incoming caption
stream is constant (i.e. there is one incoming CC buffer for each
video frame).

https://bugzilla.gnome.org/show_bug.cgi?id=606643
2018-04-23 15:58:10 +02:00
Mathieu Duponchelle
ee99cd7037 qtmux: Fix leak
gst_qt_mux_can_renegotiate () gets called everywhere following
that pattern:

return gst_qt_mux_can_renegotiate (ref(self));

This means the reference must be released both in the success
and failure cases, it was only done in the success case.
2018-04-13 22:49:43 +02:00
Edward Hervey
7229bfd068 qtmux: Add comments and doc about prefill mode 2018-04-09 15:25:50 +02:00
Edward Hervey
cc0c27850a qtmux: Refactor pad re-negotiation code
It was similar for all pads

https://bugzilla.gnome.org/show_bug.cgi?id=606643
2018-04-09 15:25:50 +02:00
Sebastian Dröge
e7177059e9 qtmux: Add support for muxing svmi atom for stereoscopic video information
https://bugzilla.gnome.org/show_bug.cgi?id=793120
2018-02-09 11:05:02 +02:00
Justin Kim
ad8a6cb639 qtmux: send stream warning when refusing video caps
If codec_data is changed, the stream is no longer valid.
Rather than keeping running when refusing new caps,
this patch send a warning  to the bus.

Also fix up splitmuxsink to ignore this warning while changing caps.

https://bugzilla.gnome.org/show_bug.cgi?id=790000
2018-02-01 10:12:46 +00:00
Vivia Nikolaidou
379059b1c7 qtmux: Make sure timecode uses the same timescale as video
Don't blindly derive it from the frame rate, but try to get the per-pad
configured timescale first (if it exists)

https://bugzilla.gnome.org/show_bug.cgi?id=792680
2018-01-23 16:40:36 +02:00
Sebastian Dröge
c71cd08d0e qtmux: Allow configuring trak timescale per pad/trak
It generally makes not much sense to configure it for all pads/traks at
once as this value is usually different for each of them. As such, add a
new property on the pads in addition to the existing property on the
whole muxer.

https://bugzilla.gnome.org/show_bug.cgi?id=792649
2018-01-23 16:40:23 +02:00
Jan Schmidt
a260eb80fb qtmux: Always update reserved-duration-remaining
If a reserved-max-duration is set, we should always track
and update the reserved-duration-remaining estimate, even
if we're not sending periodic moov updates downstream for
full robust muxing.
2017-11-25 00:56:11 +11:00
Seungha Yang
8aa5c59b5f isomp4: Add official fourcc for VP8 codec
fourcc for VP8 codec is "vp08" defined by spec. To follow it,
add it to demux and change legacy VP8 fourcc "VP80" to "vp08" in mux.
Also, enable sync table in case of VP8 codec.
See also https://www.webmproject.org/vp9/mp4/

https://bugzilla.gnome.org/show_bug.cgi?id=790026
2017-11-13 10:08:14 -05:00
Seungha Yang
93f19b19d2 isomp4: Add support VP9 codec
Add fourcc for VP9 codec and support it by qtdemux and qtmux
See also https://www.webmproject.org/vp9/mp4/

https://bugzilla.gnome.org/show_bug.cgi?id=790026
2017-11-13 10:08:14 -05:00
Jan Schmidt
d191584531 qtmux: Allow MPEG layer 1/2, AC3 and Opus in qtmux
qtmux is supposed to be the muxer that allows all formats,
with others (mp4mux and friends) being profile-restricted.
2017-09-06 23:55:38 +10:00
Mark Nauwelaerts
f9f1a03653 qtmux: robustify time tracking for sparse subtitle stream 2017-07-01 18:59:14 +02:00
Mark Nauwelaerts
d1d149ce82 qtmux: correctly track chunk size of subtitle stream
... thereby ensuring correct chunk offset tracking for all streams.
2017-07-01 18:59:07 +02:00
Mark Nauwelaerts
949902f1e2 qtmux: correctly calculate overall first_ts to ensure stream sync
... by minding and compensating for the dts_adjustment that may have
been introduced in the PTS timeline.
2017-06-24 17:36:54 +02:00
Tim-Philipp Müller
3a0fe9c2f5 qtmux: add support for muxing PNG
Demuxer already supported it.
2017-06-15 23:31:24 +01:00
Vivia Nikolaidou
5f7fe63fea qtmux: Do not check timecode data for mp4 container
Timecode trak is only supported for mov right now, not for mp4. That
code would otherwise create an invalid trak if the muxed video contained
timecode metadata.

https://bugzilla.gnome.org/show_bug.cgi?id=782684
2017-05-16 15:16:58 +03:00
Sebastian Dröge
347e814074 qtmux: When accepting renegotiation, just return TRUE and change nothing
We only accept new caps if they are basically the same. We don't want to
reset anything as if the caps are new, otherwise various state could get
out of sync with the current run.
2017-05-11 20:01:15 +02:00
Sebastian Dröge
10f4693163 qtmux: In prefill mode, only pad buffers with > 0 sized memories as needed
Adding a 0-byte memory has not much effect.

Also add some debug output.
2017-05-11 19:21:22 +02:00
Sebastian Dröge
63492ee141 qtmux: Lateness is in QT timescale, diff in GstClockTime
Print the right one in debug output to get meaningful numbers.
2017-05-10 15:58:41 +02:00
Sebastian Dröge
163b62fc25 qtmux: Error out if a gap edit list has to be written in prefill mode
We don't have any space reserved for this in the moov and the
pre-finalized moov would have broken A/V synchronization. Error out here
now
2017-05-10 14:31:40 +02:00
Sebastian Dröge
c0f505e80e qtmux: Calculate with reserved moov size instead of last moov size
We have some padding added after the initial moov, so a bigger updated
moov can be handled to some degree and is expected. Previously we just
ignored the padding and errored out in cases when the padding would've
just been enough.
2017-05-10 11:42:09 +02:00
Sebastian Dröge
419f53259f qtmux: Error out directly if sending filler data results in a flow error
CID 1405994
2017-05-10 11:12:23 +02:00
Sebastian Dröge
3fcfe774e6 qtmux: In prefill mode, handle the case when only the first chunk was ever used 2017-05-09 16:02:43 +02:00
Sebastian Dröge
55f949676e qtmux: Add new prefill recording mode
This sets up a moov with the correct sample positions beforehand and
only works with constant framerate, I-frame only streams.

Currently only support for ProRes and raw audio is implemented but
adding new codecs is just a matter of defining appropriate maximum frame
sizes.

https://bugzilla.gnome.org/show_bug.cgi?id=781447
2017-05-09 14:17:04 +02:00
Sebastian Dröge
f0163a016c qtmux: Error out on discontinuities/gaps when muxing raw audio
When muxing raw audio, we have no way of storing timestamps but are just
storing a continuous stream of audio samples. If the difference between
the expected and the real timestamp becomes to big, we should error out
instead of silently creating files with wrong A/V sync.

https://bugzilla.gnome.org/show_bug.cgi?id=780679
2017-05-09 14:16:51 +02:00
Sebastian Dröge
e51c08b0a2 qtmux: Fix timescale of timecode tracks
They should have ideally the same timescale of the video track, which we
can't guarantee here as in theory timecode configuration and video
framerate could be different. However we should set a correct timescale
based on the framerate given in the timecode configuration, and not just
use the framerate numerator.
2017-04-14 13:38:53 +03:00
Edward Hervey
7077425c3d qtmux: Don't forget to update pad->last_buf
buf is the current pad->last_buf value. If ever it gets copied/unreffed,
we need to make sure to write back the new  pointer to the last_buf
variable.

Fixes using wrong pointer values in the case of decrasing DTS value
2017-04-12 13:26:59 +02:00
Sebastian Dröge
d6b145df5f qtmux: Fix last_dts tracking for raw audio and similar formats
Accumulate the durations directly and don't scale yet another time by
the number of samples.
2017-04-10 15:55:54 +03:00
Sebastian Dröge
6f3737f0c5 qtmux: Simplify buffer refcounting in add_buffer() and remove unneeded NULL checks 2017-04-04 15:24:45 +03:00
Sebastian Dröge
6b889bea69 qtmux: Select the best pad based on the cached last_buf if any
last_buf is the one we're going to write next, not buf. As such we
should check timestamps against that one if there is one to select the
earliest pad.

Also remember the currently selected pad in the very beginning when
storing the first last_buf.

This both solves some edge cases where not the correct next pad was
selected corresponding to the target interleave.
2017-04-04 15:24:45 +03:00
Sebastian Dröge
eb307baff4 qtmux: Error out immediately if a timecode is to be written but downstream return not-OK 2017-04-04 15:24:45 +03:00