The MPEG-A format provides an extension to the ISO base media
file format to store stereoscopic content encoded with different
codecs like H.264 and MPEG-4:2. The stereo video media information(svmi)
atom declares the presence and storage method for the video.
Stereo video information for MPEG-A can also be supplied through
the 'stvi' atom (ref: ISO/IEC_14496-12, ISO/IEC_23000-11), which
is not implemented in this patch.
Also missing is support for stereo video encoded as separate video tracks
for now.
Based on a patch by Sreerenj Balachandran <sreerenj.balachandran@intel.com>
https://bugzilla.gnome.org/show_bug.cgi?id=611157
When performing seek, segment->start is being updated with desired_offset,
but in case of reverse playback segment->start should be 0 and
segment->stop should be updated with desired offset.
https://bugzilla.gnome.org/show_bug.cgi?id=750675
Only update the moov header into the caps if it's the finalised
moov at EOS time. Avoids posting a bogus moov at startup and
repeated updates in robust-recording mode
Implement a robust recording mode, where the output
file is always in a playable state, seeking and rewriting
the moov header at a configurable interval. Rewriting
moov is done using reserved space at the start of
the file, and a ping-pong strategy where the moov
is replaced atomically so it's never invalid.
Track when tags have actually changed, and don't write them into
the moov unless they've changed. Clear any existing tags when
re-writing them, so we can do progressive moov updating in robust
recording mode.
Write placeholder mdat as a free atom plus a 32-bit mdat
with '0' size, which means "rest of the file" in the spec.
Re-write it later to a full 64-bit extended size atom if needed.
Correctly update any edit lists each time the moov is recalculated,
updating existing table entries if they already exist instead of just
adding new ones.
qtdemux creates a samples array and gets the timestamps for buffers by
accumulating their durations. When doing reverse playback of fragments,
accumulating samples will lead to wrong timestamps as the timestamps
should go decreasing from fragment to fragment and the accumulation
will produce wrong results.
In this case, when receiving a discont for fragmented reverse playback,
the previous samples information should be flushed before new data
is processed.
The gst-launch script for example launch line to test qtdemux is
missing a queue before the decodebins, otherwise the gst-launch-1.0
command won't work.
https://bugzilla.gnome.org/show_bug.cgi?id=749054
When doing key uint seek, qtdemux calls gst_qtdemux_adjust_seek
to get proper offset. And then this offset is set to
segment.position and segment.time in gst_qtdemux_perform_seek but
segment.start is not updated.
After that, application sends segment query,
qtdemux sets start and stop to query using gst_segment_to_stream_time. Due
to the wrong value in segment.start, the stop position is smaller than
it should.
https://bugzilla.gnome.org/show_bug.cgi?id=746822
We always write the CTTS in qtmux. Ideally we only want to do that
for streams that need DTS, it should be present on the track information
rather than be decided based on each buffer
As qt uses durations, it doesn't matter, only the difference
between consecutive buffers is important. Also, collectpads
already replaces PTS/DTS with the running times for them.
Instead of checking various state variables around the muxer,
track the current muxing mode in a single 'mux_mode' enum.
Add some implementation notes about the different mux modes
gst_segment_do_seek() does that for us already, and doing it twice
will break non-flushing seeks in interesting ways. Leftover from 1.0
porting.
Also copy over segment offset and applied_rate, just in case.
When not in fast-start or fragmented mode, we need to be able
to rewrite the size of the mdat atom, or else the output just
won't be playable - the mdat placeholder with size == 0 will
cover the rest of the file, including any moov atom we write out.
https://bugzilla.gnome.org/show_bug.cgi?id=708808
Unlike many other seek flags, the KEY_UNIT seek
flag is not copied over into the GstSegment,
since it's only relevant for the seek itself,
so we need to pass it explicitly to the seek
handler here.
https://bugzilla.gnome.org/show_bug.cgi?id=745339
We need different symbol names, because these symbols are also present
in the fragmented plugin ... which will cause conflicts when doing
static linking
Using the sparse streams can make the push-based seeking return
too far in the stream. It also can lead to issues as the
sparse streams will be ignored when restarting playback and,
if the sparse stream is the one that has the earliest sample,
it will confuse qtdemux's offsets as one stream will have
an earlier offset than the demuxer's one which might lead to
early EOS.
https://bugzilla.gnome.org/show_bug.cgi?id=742661
Parse the 'sidx' atom and update the total duration according to the
parser result. The isoff parser code is imported from
gst-plugins-bad's dashdemux and a gst_isoff_sidx_parser_add_data()
function was factored out of the gst_isoff_sidx_parser_add_buffer()
function.
https://bugzilla.gnome.org/show_bug.cgi?id=743578
Keep global and stream tags separately and parse the udta node
that can be found under the trak atom. The udta will contain
stream specific tags and will be pushed as such
https://bugzilla.gnome.org/show_bug.cgi?id=692473
Tags received via events, when marked as stream tags, will
be stored on that stream's trak atom instead of being stored
in the main tags atom. This allows the resulting file to have
global and stream tags stored.
https://bugzilla.gnome.org/show_bug.cgi?id=692473
Refactor the functions that were bound to the 'moov' atom to
directly pass the desired 'udta' that should receive the tags.
This allows the tags to be written to 'udta' at the 'moov' or
the 'trak' level, creating tags that are for the container or
for a stream only.
https://bugzilla.gnome.org/show_bug.cgi?id=692473
For fragmented streams with extra data at the end of the mdat
qtdemux was not dropping those bytes and would try to use
that extra data as the beginning of a new atom, causing the
stream to fail.
https://bugzilla.gnome.org/show_bug.cgi?id=743407
Actually copy the codec data instead of copying nothing
and then bombing out because there's no data.
Fixes: gst-launch-1.0 audiotestsrc ! avenc_alac ! qtmux ! fakesink
https://bugzilla.gnome.org/show_bug.cgi?id=741783
When dealing with fragmented files, we will get more accurate duration
information via the mfra and moof atoms.
In order for playback to not stop at the initial duration (from the
moov atom), we need to check and update the various duration variables
when we find more information.
Fixes playback of fragmented files in pull mode
When seeking or finding the previous keyframe, do
comparisons against targets and segments using composition time
to correctly decide which sample times match.
Currently during header parsing, we scan through the entire file
and skip every moof+mdat chunk for fragmented mp4s, which makes
start-up incredibly slow. Instead, just stop at the first moof
chunk when have a moov, and start exposing the streams, so we
can go and start handling the moofs for real.
Empty segments in an edit list have a media_start time of -1,
as they don't actually play any media. Allow for that when
aligning to the reference stream in reverse play.
DTS delta is used to calculate sample duration. If buffer has missing DTS, we take either segment start or previous buffer end time, whichever is later.
This must only be done for non sparse streams, sparse streams can have gaps between buffers (which is handled later by adding extra empty buffer with duration that fills the gap)
https://bugzilla.gnome.org/show_bug.cgi?id=737095
The old default timescale of 1 millisecond produces irrational
numbers for a lot of framerate/audio-packet-duration multiples.
1/1800 is a nicer number, as it tends to produce better fractions
and therefore slightly higher accuracy overall
Change the way the output framerate is calculated
to ignore the first sample (which is sometimes truncated
in my testing) and use the new gst_video_guess_framerate()
function to recognise common standard framerates better.
Remove the code that was sorting the first 20 sample
durations and then ignoring the result.
This makes sense in DASH reverse playback, where the upstream dashdemux
will download DASH segments in reverse order, but push their buffers
forward to qtdemux and mark each segment start as DISCONT. This needs
to be forwarded downstream to the parser/decoder, otherwise it won't work.
https://bugzilla.gnome.org/show_bug.cgi?id=734443
When writing out a trak with an edit list, make sure the
overall file duration is also updated to reflect the
lengthening of the stream.
Add some more debug to qtdemux to warn about streams that
are longer than the file and get truncated.
Handle the transformation matrix cases where there are only simple rotations
(90, 180 or 270 degrees) and use a tag for those cases. This is a common scenario
when recording with mobile devices
https://bugzilla.gnome.org/show_bug.cgi?id=679522
They are very confusing for people, and more often than not
also just not very accurate. Seeing 'last reviewed: 2005' in
your docs is not very confidence-inspiring. Let's just remove
those comments.
It was used in the past in 0.10 when there was no explicit DTS
field in buffers, now we have it in 1.x series and we can
check it directly with GST_BUFFER_DTS_IS_VALID
Do not try to use subsequent buffer timestamps to calculate
sparse streams durations because the stream is sparse and
the buffers might not be 'time adjacent'. So rely on the
duration and give the option to the pad to provide
custom 'empty' buffers to represent the gaps in the
stream, this can vary on how the data is represented.
Right now, the only sparse stream supported is tx3g subtitles.
Make sure empty segments are used and pushed with a gap event
to represent its data (or lack of it)
Each QtSegment is mapped into a GstSegment with the corresponding
media range. For empty QtSegments a gap event is pushed instead
of GstBuffers and it advances to the next QtSegment.
To make this work with seeks, need to keep track of the starting
'base' to make sure it remains consistently increasing when
pushing new segment events.
For example: if a seek makes qtdemux start from 5s, the first
segment will have a base=0. When the next segment is activated,
its base time will be QtSegment.time - qtdemux.segment_base so
that it doesn't include the first 5s that weren't played and
shouldn't be accounted on the running time
This purposedly will remove the fix made for
https://bugzilla.gnome.org/show_bug.cgi?id=700264, at this
point it was decided to respect the gaps, even if they cause
a delay on playback, because that's the way the file was crafted.
https://bugzilla.gnome.org/show_bug.cgi?id=345830
Mov spec says it uses a pascal style string, while isomedia uses
a null terminated one. Store the current atoms flavor into the HDLR
to be able to generate the correct output.
https://bugzilla.gnome.org/show_bug.cgi?id=705982
Make it clear what should be handled purely by mss mode:
1) Expose the streams on the first moof as there are no moov atoms
2) Properly cleanup streams on flushes
Add a note about the meaning of upstream_newsegment and mss_mode
for future reference.
Make all other special fragment handling shared for both dash
and mss streams.
In a fragmented scenario, qtdemux is operating in push mode
and it gets a fragmented buffer. While processing its data
downstream gets unlinked (or a input-selector changes its
active pad and returns not-linked). Qtdemux stops processing
this fragment and returns not-linked upstream, leaving the
remaining data in its adapter.
When it gets an EOS it should make sure that all the data it
had received is pushed before pushing EOS.
Some buffers can have multiple moov atoms inside and the strategy
of using the gst_adapter_prev_pts timestamp to get the base timestamp
for the media of the fragment would fail as it would reuse the same
base timestamp for all moofs in the buffer instead of accumulating
the durations for all of them.
Heres a better explanation of the issue:
qtdemux receives a buffer where PTS(buf) = X
buf -> moofA | moofB | moofC
The problem was that PTS(buf) was used as the base timestamp for
all 3 moofs, causing all buffers to be X based. In this case we want
only moofA to be X based as it is what the PTS on buf means, and the
other moofB and moofC just use the accumulated timestamp from the
previous moofs durations.
To solve this, this patch uses gst_adapter_prev_pts distance
result, this allows qtdemux to calculate if it should use the
resulting pts or just accumulate the samples as it can identify
if the moofs belong to the same upstream buffer or not.
https://bugzilla.gnome.org/show_bug.cgi?id=719783
In SmoothStreaming fragmented scenario, the timestamps are calculated
starting from the fragment buffer timestamp. When there is a not-linked
return from downstream, qtdemux will return upstream and will keep the
non-pushed data into its adapter.
On a new fragment buffer pushed to qtdemux, the new buffer timestamp
would overwrite the previous one that should be used on the still
to be pushed buffers. Because of this, this patch will also
update the fragment_start timestamp from the adapter last pts
to make sure the moof and timestamps are in sync and will result
in correct timestamps for all fragments.
In the scenario of "mdat | moov (with fragmented artifacts)" qtdemux
could read the moov again after the mdat because it was considering the
media as a fragmented one.
To avoid this loop this patch makes it store
the last processed moov_offset to avoid parsing it again.
And it also checks if there are any samples to play before
resturning to the mdat, so that it knows there is new data to be played.
https://bugzilla.gnome.org/show_bug.cgi?id=691570
When parsing a trak only free streams on failures if those streams
were created locally. They could have been created from a previous
fragment, in this case we they have valid info from the other fragment.
Including pads.
https://bugzilla.gnome.org/show_bug.cgi?id=691570
As for text subtitles and as suggested in #712643, throw
away the 2 byte terminator packets that some encoders insert.
This will make things better when remuxing and causes generation
of gap events.
Clean up the handling of mp4s streams. Use the generic esds
descriptor function to extract the palette, instead of hard coding
a wrong magic offset.
Add some more size safety checks when parsing ES descriptors, and
replace magic numbers with the descriptive constants that are already
defined.
Enhance dump output for stsd atoms.
Streams from both bug 712643 and historic bug 568278 now both work
correctly.
Fixes: #712643
Assume a file with atoms in the following order: moov, mdat, moof,
mdat, moof ...
The first moov usually doesn't contain any sample entries atoms (or
they are all set to 0 length), because the real samples are signaled
at the moofs. In push mode, qtdemux parses the moov and then finds the mdat,
but then it has 0 entries and assumes it is EOS.
This patch makes it continue parsing in case it is a fragmented file so that
it might find the moofs and play the media.
https://bugzilla.gnome.org/show_bug.cgi?id=710623
In push mode, when qtdemux can't use a seek to skip the mdat buffer it has
to buffer it for later use.
The issue is that after parsing the next moov/moof, there might be some
trailing bytes from the next atom in the file. This data was being discarded
along with the already parsed moov/moof and playback would fail to continue
after the contents of this moov/moof are played.
This is particularly bad on fragmented files that have the mdat before the
corresponding moof. So you'd get:
mdat|moof|mdat|moof ...
When a moof was received, it usually came with some extra bytes that would
belong to the next mdat (because upstream doesn't care about atoms alignment).
So those bytes were being discarded and playback would fail.
This patch makes qtdemux store those extra bytes to reuse them later after the
mdat is emptied.
https://bugzilla.gnome.org/show_bug.cgi?id=710623
Doing so would be a regression over 1.0 and breaks the unit test.
However the result will be most likely unusable, so let's post
a warning message on the bus.
Use g_date_time seconds manipulation to allow to cover the quicktime
spec for creation_time. It uses seconds since 1904.
Both paths could be done using the generic approach of seconds since
1904 with GDateTime handling, but the first path using seconds from
1970 should be more commonly found and avoids a few objects creation and
ref/unref, so keep it there for performance.
Additionally, the code for handling seconds since 1970 changed from >
to >= because having 0 seconds since 1970 is also a valid case for that
path to handle.
https://bugzilla.gnome.org/show_bug.cgi?id=707975
The streamable property only make sense for fragmented formats.
For regular MP4, when downstream is not seekable we can't rewrite
the headers, so qtmux can only work with fast-start=TRUE, where
the headers are written finishing the file.
For fragmented MP4, when streamable is not seekable and the streamable
property is FALSE, we must enforce streamable=TRUE warning the user
about this change
https://bugzilla.gnome.org/show_bug.cgi?id=707242
The most common use case for fragmented MP4 (Dash and Smooth Streaming)
is producing streamable content (even for VOD). streamable=FALSE would only
be used to generate fragmented MP4 with and index of MOOF's that could
be reproduced without a playlist/manifest
https://bugzilla.gnome.org/show_bug.cgi?id=707242
Check for GST_SEEK_TYPE_NONE for stop poistion and only update
the stop time if it is requested. Otherwise just maintain whatever
was stored at the segment
https://bugzilla.gnome.org/show_bug.cgi?id=707530
When the segment has a defined stop position, qtdemux should check
when streams reach this position and mark those as EOS. When all
streams are EOS it will return GST_FLOW_EOS to upstream to allow
the pipeline to finish instead of continuously consume buffers
from upstream that are not useful for the segment.
https://bugzilla.gnome.org/show_bug.cgi?id=707530
When handling seeks in push mode, qtdemux converts the seek to bytes
and pushes upstream. It needs to keep track of the seek and the
subsequent segment to be able to map them back to the requested
seek time and properly preserve the segment stop of the seek.
This is done by using the start offset in bytes of the seek,
that should be the same of the segment from upstream. And this
is also backwards compatible with what qtdemux already was using.
https://bugzilla.gnome.org/show_bug.cgi?id=707530
Amendment 2 of ISO/IEC 14496-15 (AVC file format) is defining a new
structure for fragmented MP4 called "avc3". The principal difference
between AVC1 and AVC3 is the location of the codec initialisation
data (e.g. SPS, PPS). In AVC1 this data is placed in the initial
MOOV box (moov.trak.mdia.minf.stbl.stsd.avc1) but in AVC3 this data
goes in the first sample of every fragment (i.e. the first sample in
each mdat box). The principal reason for avc3 is to make it easier
for client implementations, because it removes the requirement to
insert the SPS+PPS in to the decoder pipeline every time there is a
representation change.
This commit adds support for the "avc3" atom, which is almost identical
to the "avc1" atom, except it does not contain any SPS or PPS data.
https://bugzilla.gnome.org/show_bug.cgi?id=702004
This should make decoders able to precisely push buffers until the stop
time in case they need the next keyframe to do it.
Also, according to gst_segment_clip, it should only push a buffer that
the starting ts is strictly smaller than the segment stop, so we change
the min < comparison for <=
Use the same seqnum on messages and events for derived events.
Fixed for flushes / stream-start / segment after a seek, and segment
after a segment.
Fixes#676242
cca2f555d1 introduces a regression, where the demux segment is not
reset on flush stop, so the next upstream segment event will calculate
an invalid base time on the new segment to be sent downstream.
https://bugzilla.gnome.org/show_bug.cgi?id=704255
This can only reliably work if demuxers have a
separate streaming thread per srcpad. This should be
done in a demuxer base class, which integrates parts
of multiqueue
https://bugzilla.gnome.org/show_bug.cgi?id=701856
bug #700505
Following a representation change that causes a resolution change,
the video decoder fails to decode correctly. Dashdemux detects the
representation change and pushes a new caps event and an
initialization segment (a new moov atom) to the downstream qtdemux,
but it doesn't handle this new moov yet, it will only parse the
first one it receives.
This commit changes qtdemux to accept a new moov in a dash bitstream
switching scenario.
In case qtdemux is handling a mss stream, do not mark the stream to wait
for EOS after a segment. Even if it seems to be the last one according to
the current streams information.
MSS handling is different here because there is another demuxer driving
the pipeline
The samplerate field in the STSD atom is not right for some ALAC files
(usually when audio is 96kHz/24bits), so the audio caps must be
extracted from the codec data.
https://bugzilla.gnome.org/show_bug.cgi?id=700382
Whenever the demuxer has a new caps on a stream, it should set the
new_caps variable to true and a new caps event will be pushed before
the next buffer
qtdemux takes its buffers from a GstAdapter. Those buffers are created
from the larger buffer that it obtained from upstream and they carry
the same flags, including DISCONT if it is set. In these cases, all
buffers that qtdemux is going to push would be marked as DISCONT.
This scenario can make parsers/decoders flush on every buffer leading
to no decoding at all hapenning. This patch prevents this by unsetting
the flag if it shouldn't be set.
* Explicitly init variables for fragmented formats at init
* Do not use GstClockTime type if the variable isn't a timestamp
* Fix a style/readability issue at an if block
* Group 2 mss mode conditional blocks together to improve readability
Conflicts:
gst/isomp4/qtdemux.c
This can confuse downstream when they get a byte segment after receiving
the natural time segment from qtdemux that it sends when starting to
push buffers. This is specially the case with parsers that try to
convert the position from byte to time format and might miss the
correct position for playback to start.
Reset different variables on state changes to ready and when
handling a flush-stop. For handling flush stops we should check
if there is an upstream adaptive demuxer driving the pipeline as this
means that qtdemux will get a new moov atom. For 'standard' isomedia
streams this isn't true and qtdemux should keep the previous moov
information around.
Conflicts:
gst/isomp4/qtdemux.c
Whenever dashdemux switches bitrates it sends a new moov with the
new stream configuration. qtdemux should now handle this by splitting
the exposing and configuration of streams into separate functions. When
the stream is new it is configured and exposed, when it is a new bitrate
of an existing stream it is only reconfigured.
Conflicts:
gst/isomp4/qtdemux.c
smoothstreaming streams should be handled as a special kind of
fragmented isomedia. In MSS the fragments will not contain a
'moov' atom with the media descriptions, this has to be extracted
from the caps.
Additionally, there should be another demuxer upstream that is likely
going to be the one to answer/act on queries and events, so qtdemux has
to forward those upstream.
Don't unmap short MOOV atom buffer twice, which happened
in the case where we don't fix up the MOOV atom.
Fixes crashes when thumbnailing partial mp4 file where
the MOOV atom is still incomplete.
https://bugzilla.gnome.org/show_bug.cgi?id=694010
Fixes abort in push mode where the source is not seekable and the
size of the file is not available, as with
cat foo.mp4 | gst-launch-1.0 playbin uri=fd://0
Less noticable with releases, since we disable all
g_assert() there.
https://bugzilla.gnome.org/show_bug.cgi?id=686008
When playing an mp4 file with the MOOV atom at the end of the file, playback
fails with the error message "no 'moov' atom within the first 10 MB". This is
due to a mistake in the upstream_size typing, making the seek to the end of
file never happening.
https://bugzilla.gnome.org/show_bug.cgi?id=684972
Don't reset the segment because we need the values for accumulation. the segment
is reset at start and after a flushing seek. Fixes some problems with files with
quicktime segments.
MPEG DASH has defined a set of new boxes to specify duration, indexes and
offsets of ISOBMFF fragments.
The Track Fragment Base Media Decode Time (tfdt) Box can in particular be
included inside a traf box to specify the absolute decode time, measured on the
media timeline, of the first sample in decode order in the track fragment.
This information can be used by the isomp4 demux to find out the current position of
an MP4 fragment in the timeline.
This patch adds code to isomp4 to:
- parse the tfdt box
- adjust the time/position member of the new segment sent when playback starts
Fixes https://bugzilla.gnome.org/show_bug.cgi?id=677535
After pushing the buffer into the adapter, we should not let the baseclass push
it out anymore. This error was introduced while porting to 0.11.
See https://bugzilla.gnome.org/show_bug.cgi?id=680540
When it receives a seek in push mode, the qtdemux should first try to push the event upstream, and only if upstream fails fall back to
its own seek logic.
This fixes a not-negotiated error at least on mov files with
twos audio with two channels and video dvcp. As playbin and gst-launch
sample coming from the qtdemux.c file uses audioconvert and the latter
require format interleaved.
https://bugzilla.gnome.org/show_bug.cgi?id=675326
We do an ALLOCATION query to find out an allocator and parameters on the
srcpads. This way decoders (and sinks) can specify the memory and parameters
they want us to write into.
Set PTS and DTS on output buffers instead of just the PTS. In streaming cases
you want to synchronized encoded data based on the DTS because that is
monotonically increasing.
If downstream didn't answer our SEEKING query and told us
it's seekable, default to streaming=true. We couldn't do
this in 0.10 for backwards compatibility reasons, but we
can in 0.11. Play it safe.
Ignore the "streamable" property setting and create streamable
output if downstream is known not to be seekable (as queried
via a SEEKABLE query).
Fixes pipelines like qtmux ! appsink possibly creating seemingly
corrupted output if streamable has not been set to true.
Conflicts:
ext/pulse/pulseaudiosink.c
ext/pulse/pulsesrc.c
gst/audioparsers/gstaacparse.c
gst/audioparsers/gstamrparse.c
gst/audioparsers/gstdcaparse.c
gst/audioparsers/gstflacparse.c
gst/effectv/gstradioac.c
gst/effectv/gstradioac.h
gst/effectv/gstripple.c
Some possible FIXMEs remaining in the audio parser getcaps functions.
Subtract the first timestamp of a stream from all input buffers to
get 0-based timestamps for creating a sane ctts table. Without this
patch the ctts could have larger values than needed, causing the
playback to have a delay at startup.
As the first timestamp is only found after a few buffers are queued
(due to possible reordered buffers), once we find the first timestamp
we subtract it from all buffers on the queue, from that point on,
all buffers have their timestamps subtract when they are collected.
https://bugzilla.gnome.org/show_bug.cgi?id=658659
