It was used in the past in 0.10 when there was no explicit DTS
field in buffers, now we have it in 1.x series and we can
check it directly with GST_BUFFER_DTS_IS_VALID
Do not try to use subsequent buffer timestamps to calculate
sparse streams durations because the stream is sparse and
the buffers might not be 'time adjacent'. So rely on the
duration and give the option to the pad to provide
custom 'empty' buffers to represent the gaps in the
stream, this can vary on how the data is represented.
Right now, the only sparse stream supported is tx3g subtitles.
This reverts commit 9f7b1128b1.
This should be handled automatically be rtspsrc if the AVPF profile
is used, and manual enabling of it can be done with the new-manager
signal.
If the expected packet (do_next_seqnum is TRUE) is the one we requested
for retranmission earlier, do the logic to update the retransmission
statistics as well before setting up the timers for the next expected
packet.
Also reset the retransmission counter if the timer is reused for another
seqnum.
Add an accumulator that stops the signal emission as soon as a caps has
been retrieved. Otherwise the default handler would continue emitting
the signal and possibly overwrite the result with NULL again.
Uses information gathered during EBML parsing to
forge a more suitable set of caps instead of blindly
assuming everything is video/x-matroska.
For consistency, stream type reset was added to
matroska-demux too.
https://bugzilla.gnome.org/show_bug.cgi?id=722311
To do that, enqueue the EOS event to be sent from the srcpad task
thread and flush the queue right afterwards, so that no more rtx
buffers can be sent, even if there are more requests coming in.
Fixes https://bugzilla.gnome.org/show_bug.cgi?id=722370
The reason behind this is to minimize the retransmission delay.
Previously, when a NACK was received, rtprtxsend would put a
retransmission packet in a queue and it would send it from chain(),
i.e. only after a new buffer would arrive.
This unfortunately was causing big delays, in the order of 60-100 ms,
which can be critical for the receiver side.
By having a separate GstTask for pushing buffers out of rtxsend,
we can push buffers out right after receiving the event, without
waiting for chain() to get called.
Instead do it like all other demuxers and let parsers and decoders
handle that. The keyframe information inside the container might
be completely wrong like in the sample file of the bug report,
and if it is correct and we push no keyframes, then the parsers
and decoders will handle that properly anyway.
https://bugzilla.gnome.org/show_bug.cgi?id=682276
Make sure empty segments are used and pushed with a gap event
to represent its data (or lack of it)
Each QtSegment is mapped into a GstSegment with the corresponding
media range. For empty QtSegments a gap event is pushed instead
of GstBuffers and it advances to the next QtSegment.
To make this work with seeks, need to keep track of the starting
'base' to make sure it remains consistently increasing when
pushing new segment events.
For example: if a seek makes qtdemux start from 5s, the first
segment will have a base=0. When the next segment is activated,
its base time will be QtSegment.time - qtdemux.segment_base so
that it doesn't include the first 5s that weren't played and
shouldn't be accounted on the running time
This purposedly will remove the fix made for
https://bugzilla.gnome.org/show_bug.cgi?id=700264, at this
point it was decided to respect the gaps, even if they cause
a delay on playback, because that's the way the file was crafted.
https://bugzilla.gnome.org/show_bug.cgi?id=345830
1) pt can be lower than 96
2) there is no point in checking that because rtprtxsend will not
even store buffers for payload types that it doesn't know about,
so this case will never be reached
This patch moves the creation of rtx packets to be done early,
in the src_event() function, when they are requested. The purpose
is to run gst_rtp_rtx_buffer_new() with the object locked to
protect internal data, because if it is done at the pushing stage,
we would have to lock and unlock multiple times in a row while we
are pushing the rtx buffers.
Previously there was no locking at all, which was terribly wrong.
According to ISO/IEC 13818-7, "channel_config" field in ADTS header
may have value of 0, as in the case of frame with PCE.
gst_aac_parse_detect_streams() returned FALSE for those frames
and discarded them.
The need for rewriting apparently is obsolete 0.10 leftover.
We now have caps for subtitles when we create the headers,
so we always write the correct data in the first place.
This avoids issues with writing dummy data first, then having
to come back and write correct data later. Doing so prevents
the muxed stream from being actually streamable.
https://bugzilla.gnome.org/show_bug.cgi?id=712134
Mov spec says it uses a pascal style string, while isomedia uses
a null terminated one. Store the current atoms flavor into the HDLR
to be able to generate the correct output.
https://bugzilla.gnome.org/show_bug.cgi?id=705982
This reverts commit b3aa8755fe.
We are already using the running-time because they were placed on the
buffers with gst_collect_pads_clip_running_time(). Arguably it would be
better to not modify the incomming buffers but collectpads seems to want
to use absolute timestamps from the buffers for finding the best buffer
(this can be changed with a custom compare function..).
This property allows you to specify the amount of buffers
to keep in the retransmission queue expressed as time (ms)
instead of buffer count (which is the max_size_buffers property).
The purpose of the sender RTX object is to keep a history
of RTP packets up to a configurable limit (in time). It will
listen for custom retransmission events from downstream. When
it receives a request for retransmission, it will look up the
requested seqnum in its list of stored packets. If the packet
is available, it will create a RTX packet according to RFC 4588
and send this as an auxiliary stream.
The receiver will listen to the custom retransmission events
from the downstream jitterbuffer and will remember the SSRC1
of the stream and seqnum that was requested. When it sees a
packet with one of the stored seqnum, it associates the SSRC2
of the stream with the SSRC1 of the master stream. From then
on it knows that SSRC2 is the retransmission stream of SSRC1.
This algorithm is stated in RFC 4588. For this algorithm to
work, RFC4588 also states that no two pending retransmission
requests can exist for the same seqnum and different SSRCs or
else it would be impossible to associate the retransmission with
the original requester SSRC.
When the RTX receiver has associated the retransmission packets,
it can depayload and forward them to the source pad of the element.
RTX is SSRC-multiplexed
Fixes https://bugzilla.gnome.org/show_bug.cgi?id=711084
AUX elements are elements that can be inserted into the rtpbin
pipeline right before or after 1 or more session elements.
The AUX elements are essential for implementing functionality such
as error correction (FEC) and retransmission (RTX).
Fixes https://bugzilla.gnome.org/show_bug.cgi?id=711087
Keep track of elements that are added to multiple sessions and make sure
we only add them to the rtpbin once and that we clean them when no
session refers to them anymore.
When a collision is found on the internal ssrc, we have to change it.
Ideally, we want also the payloader upstream to follow this change and use
the new internal ssrc. Ideally we want this condition to be always met:
if there is one payloader sending on this session, its ssrc should match the
internal ssrc.
* gst/rtpmanager/gstrtpbin.[ch]: four new action signals have been
added (request-rtp-encoder, request-rtp-decoder, request-rtcp-encoder
and request-rtcp-decoder). The user will be able to provide encoders
or decoders dynamically. The encoders must follow the srtpenc API and
the decoders the srtpdec API. Having separate signals for RTP and RTCP
allows the user to use different encoders/decoders or provide the same
one (e.g. that would be the case for srtpenc).
Also, rtpbin now allows application/x-srtp in its pads.
https://bugzilla.gnome.org/show_bug.cgi?id=719938
Use the round-trip-time and average jitter to dynamically calculate the
retransmission interval and expected packet arrival time.
Based on patches from Torrie Fischer <torrie.fischer@collabora.co.uk>
Fixes https://bugzilla.gnome.org/show_bug.cgi?id=711412
Don't use the current time calculated from the tmieout loop for when we
last scheduled the NACK because it might be unscheduled because of a max
packet misorder and then we don't accurately calculate the current time.
Instead, take the current element running time using the clock.
Make it clear what should be handled purely by mss mode:
1) Expose the streams on the first moof as there are no moov atoms
2) Properly cleanup streams on flushes
Add a note about the meaning of upstream_newsegment and mss_mode
for future reference.
Make all other special fragment handling shared for both dash
and mss streams.
In a fragmented scenario, qtdemux is operating in push mode
and it gets a fragmented buffer. While processing its data
downstream gets unlinked (or a input-selector changes its
active pad and returns not-linked). Qtdemux stops processing
this fragment and returns not-linked upstream, leaving the
remaining data in its adapter.
When it gets an EOS it should make sure that all the data it
had received is pushed before pushing EOS.
According to RFC 5104 section 4.3.1.2, RTCP PSFB FIR message SHALL
have a media_ssrc field set to 0. The actual media ssrc is in the FCI.
So in that case, we ignore the retained feedback and just let it through
to the rtp_session_process_fir() function which will check for the actual
SSRC inside the FCI.
Fixes a regression introduced by commit 57c27ec3
Previously, when the session had multiple internal sender SSRCs, it would
issue SR reports with RB blocks only on the first RTCP timeout and afterwards
SR reports would be sent empty. This was because the "generation" number
in RTPSource would increase more than once during the same cycle and afterwards
it would always be greater than the session's generation, which would cause
it to be skipped from being included in RBs.
This commit fixes this problem by:
1) Increasing the RTPSource generation only at the end of each cycle,
which essentially fixes the problem but only when the internal senders
are less than GST_RTCP_MAX_RB_COUNT.
2) Keeping for each RTPSource a set of SSRCs which stores which SSRC's
SR the given RTPSource has been reported in, which also fixes the problem
when the internal senders are more than GST_RTCP_MAX_RB_COUNT. This is
necessary because of the fact that any RTPSource is marked as reported
in itself's SR and makes it impossible to know if it has been reported
in other SRs too or not, and which.
Keep an extra stats structure for scheduling the BYE packets. When we
decide to schedule BYE, make a copy of the current stats into the
bye_stats. Then while we schedule the BYE, update and use only the
bye_stats. When we finished scheduling the BYE packet, we use the
regular stats again.
When we are scheduling BYE packets, ignore all RTCP for the sources that
are scheduling a BYE packet. Other sources that are not scheduling BYE
should continue receiving RTCP packets as usual.
Some buffers can have multiple moov atoms inside and the strategy
of using the gst_adapter_prev_pts timestamp to get the base timestamp
for the media of the fragment would fail as it would reuse the same
base timestamp for all moofs in the buffer instead of accumulating
the durations for all of them.
Heres a better explanation of the issue:
qtdemux receives a buffer where PTS(buf) = X
buf -> moofA | moofB | moofC
The problem was that PTS(buf) was used as the base timestamp for
all 3 moofs, causing all buffers to be X based. In this case we want
only moofA to be X based as it is what the PTS on buf means, and the
other moofB and moofC just use the accumulated timestamp from the
previous moofs durations.
To solve this, this patch uses gst_adapter_prev_pts distance
result, this allows qtdemux to calculate if it should use the
resulting pts or just accumulate the samples as it can identify
if the moofs belong to the same upstream buffer or not.
https://bugzilla.gnome.org/show_bug.cgi?id=719783
The parser can accept input that is not completely specified. Use the
ACCEPT_INTERSECT flag on the sinkpad to tweak the acceptcaps function to
check for intersection only. This allows us to proxy downstream
constraints while still allowing non-subset caps as input.
We can then also remove the appended template caps workaround.
Make a unit-test to check the new feature.
This reverts commit 26040ee38c
Fixes https://bugzilla.gnome.org/show_bug.cgi?id=705024
In SmoothStreaming fragmented scenario, the timestamps are calculated
starting from the fragment buffer timestamp. When there is a not-linked
return from downstream, qtdemux will return upstream and will keep the
non-pushed data into its adapter.
On a new fragment buffer pushed to qtdemux, the new buffer timestamp
would overwrite the previous one that should be used on the still
to be pushed buffers. Because of this, this patch will also
update the fragment_start timestamp from the adapter last pts
to make sure the moof and timestamps are in sync and will result
in correct timestamps for all fragments.
In the scenario of "mdat | moov (with fragmented artifacts)" qtdemux
could read the moov again after the mdat because it was considering the
media as a fragmented one.
To avoid this loop this patch makes it store
the last processed moov_offset to avoid parsing it again.
And it also checks if there are any samples to play before
resturning to the mdat, so that it knows there is new data to be played.
https://bugzilla.gnome.org/show_bug.cgi?id=691570
When parsing a trak only free streams on failures if those streams
were created locally. They could have been created from a previous
fragment, in this case we they have valid info from the other fragment.
Including pads.
https://bugzilla.gnome.org/show_bug.cgi?id=691570
Don't reset the expected output seqnum when clearing the pt map because this
could stall the jitterbuffer forever.
Add a unit test for this.
Fixes https://bugzilla.gnome.org/show_bug.cgi?id=709800
As for text subtitles and as suggested in #712643, throw
away the 2 byte terminator packets that some encoders insert.
This will make things better when remuxing and causes generation
of gap events.
Otherwise there were race conditions where we would send tags
on a flushing srcpad.
We have a test for that in GES, but this should be tested
systematically with harness in the future as I believe it
is useful for exactly that kind of cases.
https://bugzilla.gnome.org/show_bug.cgi?id=708165
Clean up the handling of mp4s streams. Use the generic esds
descriptor function to extract the palette, instead of hard coding
a wrong magic offset.
Add some more size safety checks when parsing ES descriptors, and
replace magic numbers with the descriptive constants that are already
defined.
Enhance dump output for stsd atoms.
Streams from both bug 712643 and historic bug 568278 now both work
correctly.
Fixes: #712643
Remove bogus reconfigure event on collision, we don't want to send the event on
the receiving RTP pad and the collision event is now handling this
case.
See https://bugzilla.gnome.org/show_bug.cgi?id=711560
The problem here was that the jitterbuffer lock was unlocked to push
the event, but that caused another thread to remove the timer currently
being processed, probably because the amount of rtx events
(and therefore timers) was getting too high. The solution is to
unlock and push the event only after timer processing has finished.
fixes https://bugzilla.gnome.org/show_bug.cgi?id=711131
Restore the behavior of the element to the state before commit
db29522a43. A non-empty header is
generated and when the EOS event is received the header is generated
again, this time with the correct size.
https://bugzilla.gnome.org/show_bug.cgi?id=711699
An internal sender in a session is also a receiver of its own packets so update
the receiver stats. Other senders in the session will use this info to generate
correct RB blocks in their SR reports.
Assume a file with atoms in the following order: moov, mdat, moof,
mdat, moof ...
The first moov usually doesn't contain any sample entries atoms (or
they are all set to 0 length), because the real samples are signaled
at the moofs. In push mode, qtdemux parses the moov and then finds the mdat,
but then it has 0 entries and assumes it is EOS.
This patch makes it continue parsing in case it is a fragmented file so that
it might find the moofs and play the media.
https://bugzilla.gnome.org/show_bug.cgi?id=710623
In push mode, when qtdemux can't use a seek to skip the mdat buffer it has
to buffer it for later use.
The issue is that after parsing the next moov/moof, there might be some
trailing bytes from the next atom in the file. This data was being discarded
along with the already parsed moov/moof and playback would fail to continue
after the contents of this moov/moof are played.
This is particularly bad on fragmented files that have the mdat before the
corresponding moof. So you'd get:
mdat|moof|mdat|moof ...
When a moof was received, it usually came with some extra bytes that would
belong to the next mdat (because upstream doesn't care about atoms alignment).
So those bytes were being discarded and playback would fail.
This patch makes qtdemux store those extra bytes to reuse them later after the
mdat is emptied.
https://bugzilla.gnome.org/show_bug.cgi?id=710623
Add a new timestamp mode that assumes the local and remote clock are
synchronized. It takes the first timestamp as a base time and then uses the RTP
timestamps for the output PTS.
matroska-demux.c: In function 'gst_matroska_demux_add_stream':
matroska-demux.c:1379:7: error: format '%u' expects argument of type 'unsigned int', but argument 4 has type 'guint64' [-Werror=format=]
"%03u", context->uid);
^
WebM has a couple of specific requirements we need to handle.
Idea is to set this flag once and just rely on mux->is_webm
at run time instead of repeatedly figuring this out from
GST_MATROSKA_DOCTYPE_WEBM (which requires a strcmp()).
WebM spec states SegmentUID is Unsupported. Files produced
with gstreamer without this change will spit an error like
this when passed to mkvalidator:
ERR201: Invalid 'SegmentUID' for profile 'webm' in Info at 192
When flush-stop arrives before we process the result of the _push() in the
loop function, we might pause even though we are not flushing anymore. Fix this
race by waiting for the srcpad loop function to completely pause after doing the
flush-start.
If we were not waiting for the missing seqnum when we insert the lost packet
event in the jitterbuffer, we end up not updating the next_seqnum and wait
forever for the lost packets to arrive. Instead, keep track of the amount of
packets contained by the jitterbuffer item and update the next expected
seqnum only after pushing the buffer/event. This makes sure we correctly handle
GAPS in the sequence numbers.
Doing so would be a regression over 1.0 and breaks the unit test.
However the result will be most likely unusable, so let's post
a warning message on the bus.
Use g_date_time seconds manipulation to allow to cover the quicktime
spec for creation_time. It uses seconds since 1904.
Both paths could be done using the generic approach of seconds since
1904 with GDateTime handling, but the first path using seconds from
1970 should be more commonly found and avoids a few objects creation and
ref/unref, so keep it there for performance.
Additionally, the code for handling seconds since 1970 changed from >
to >= because having 0 seconds since 1970 is also a valid case for that
path to handle.
https://bugzilla.gnome.org/show_bug.cgi?id=707975
Always prepare a lost event in the jitterbuffer, it is to wake up and make the
pushing thread continue. We drop the event when we are not supposed to push lost
events downstream.
Schedule the lost event by placing it inside the jitterbuffer with the seqnum
that was lost so that the pushing thread can interleave and push it properly.
Make the jitterbuffer operate on a structure containing all the packet
information. This avoids mapping the buffer multiple times just to get the RTP
information. It will also make it possible to store other miniobjects such as
events later.
Improve the order of the timeout events, if there are timers with the same
timeout, we want to trigger the lowest seqnum first. For this we need to loop
over the complete array of timers to find the best one before triggering the
timeout.
First send the lost event, then update the next_seqnum counter and then
send the signal to the pushing thread that it can retry to push a buffer. This
avoids pushing out buffers before the lost event is pushed.
There is no need to unschedule the timer in flush-start, flush-stop will remove
the timers and unschedule.
Unschedule the current timer before attempting to join the timer thread.
Use the more correct POFFSET macro to get the offset of a component in its
plane. The offset macro gives the offset of the component relative to the start
of the frame.
clang does not want or need a clobber list for emms:
error: clobbers must be last on the x87 stack
Patch taken from the FreeBSD ports, provided by
Dan McGregor <dan.mcgregor@usask.ca>
The streamable property only make sense for fragmented formats.
For regular MP4, when downstream is not seekable we can't rewrite
the headers, so qtmux can only work with fast-start=TRUE, where
the headers are written finishing the file.
For fragmented MP4, when streamable is not seekable and the streamable
property is FALSE, we must enforce streamable=TRUE warning the user
about this change
https://bugzilla.gnome.org/show_bug.cgi?id=707242
The most common use case for fragmented MP4 (Dash and Smooth Streaming)
is producing streamable content (even for VOD). streamable=FALSE would only
be used to generate fragmented MP4 with and index of MOOF's that could
be reproduced without a playlist/manifest
https://bugzilla.gnome.org/show_bug.cgi?id=707242
Don't assume planar formats have just one memory block with the data but use the
macros to access the right memory block where a component can be found.
Keep a separate delay in the timer so that we still know the original timestamp
of the packet that this timer refers to. We can then place the correct
running-time in the Retransmission event.
Instead of pushing the lost event from the chain function, schedule a timeout
that will push the lost event from the timer thread. This avoid blocking the
upstream thread while we push and sync the event.
When we have a large number of missing packets, generate one lost event for all
the packets that have no chance of being pushed out in time.
Fix and activate unit test for large gaps.
Keep track of the current time in the timeout loop.
Loop over all timers and trigger all the expired ones, we can do this in the
same loop that selects the new best timer.
Also update the timers when retransmission is disabled. We need to
do this because when we added LOST timers when we detected missing packets and
we need to remove those timers when the packet finally arrives.
Check for GST_SEEK_TYPE_NONE for stop poistion and only update
the stop time if it is requested. Otherwise just maintain whatever
was stored at the segment
https://bugzilla.gnome.org/show_bug.cgi?id=707530
We keep the DTS and PTS in running-time inside the jitterbuffer. Make sure to
transform it back to a buffer timestamp before pushing out the buffer.
Fixes https://bugzilla.gnome.org/show_bug.cgi?id=707931
When the segment has a defined stop position, qtdemux should check
when streams reach this position and mark those as EOS. When all
streams are EOS it will return GST_FLOW_EOS to upstream to allow
the pipeline to finish instead of continuously consume buffers
from upstream that are not useful for the segment.
https://bugzilla.gnome.org/show_bug.cgi?id=707530
When handling seeks in push mode, qtdemux converts the seek to bytes
and pushes upstream. It needs to keep track of the seek and the
subsequent segment to be able to map them back to the requested
seek time and properly preserve the segment stop of the seek.
This is done by using the start offset in bytes of the seek,
that should be the same of the segment from upstream. And this
is also backwards compatible with what qtdemux already was using.
https://bugzilla.gnome.org/show_bug.cgi?id=707530
Amendment 2 of ISO/IEC 14496-15 (AVC file format) is defining a new
structure for fragmented MP4 called "avc3". The principal difference
between AVC1 and AVC3 is the location of the codec initialisation
data (e.g. SPS, PPS). In AVC1 this data is placed in the initial
MOOV box (moov.trak.mdia.minf.stbl.stsd.avc1) but in AVC3 this data
goes in the first sample of every fragment (i.e. the first sample in
each mdat box). The principal reason for avc3 is to make it easier
for client implementations, because it removes the requirement to
insert the SPS+PPS in to the decoder pipeline every time there is a
representation change.
This commit adds support for the "avc3" atom, which is almost identical
to the "avc1" atom, except it does not contain any SPS or PPS data.
https://bugzilla.gnome.org/show_bug.cgi?id=702004
On Windows it's not possible to bind to a multicast address
but the OS will make sure to filter out all packets that
arrive not for the multicast address the socket joined.
On Linux and others it is necessary to bind to a multicast
address to let the OS filter out all packets that are received
on the same port but for different addresses than the multicast
address
And deprecate the multicast-group property and replace it with the
address property.
https://bugzilla.gnome.org/show_bug.cgi?id=707042
RTP buffer allocation should not be done with padding for the specific MPEG2
header as the padding is done at the end of the buffer and the last byte is
the size of the padding.
https://bugzilla.gnome.org/show_bug.cgi?id=706970
This should make decoders able to precisely push buffers until the stop
time in case they need the next keyframe to do it.
Also, according to gst_segment_clip, it should only push a buffer that
the starting ts is strictly smaller than the segment stop, so we change
the min < comparison for <=
Don't update the next RTCP check time in all cases but only when we
reconsidered. This avoids delaying sending a full RTCP packet when we
are doing early feedback.
We can't use the clock to time our config-interval because we are not
live (or there might not be a clock or the clock might not be running).
Instead just simply take the timestamp diff.
This is to make sure tags are cleared on the client if the
stream-start was previously lost, otherwise, the client may end
up with a merged taglist of multiple songs
This is useful in case the packet containing the inlined caps was lost
or if new client joins an already running RTP stream and they missed
the previous tag events.
This also makes the payloader keep a list of merged tags so the retransmitted
tag event contains all previously received. A STREAM_START event will
flush the list of tags.
This is necessary to fix event/caps sending. If we send a STREAM_START
packet, it will cause an error because the stream didn't receive its
caps and new-segment events, so we must wait for the first buffer before
sending the stream-start event buffer. However, the caps will be sent
at the same time and so the 'inline caps' will be set for the event.
We need to be able to payload individual packets (data, caps or events)
and only send them when we call flush.
Restructure handling of incomming packet and the gap with the expected seqnum
and register all timers from the _chain function.
Convert a timer to a LOST packet timer when the max amount of retransmission
requests has been reached.
This patch will prepend ADTS headers to raw AAC audio frames, allowing
upstream elements to link to decoders that only support AAC in ADTS format.
Note that no error correction bits are added to ADTS frames in this code.
https://bugzilla.gnome.org/show_bug.cgi?id=615740
In some cases the src caps determined by goom weren't writable, causing
a bunch of assertion failures and failed caps. Fixed by always
explicitely making the caps writable
https://bugzilla.gnome.org/show_bug.cgi?id=705475
If we change the seqnum of an existing timer and we were waiting for
that timer, unschedule it. If we change the timeout of an existing timer and we
were waiting on it, only unschedule when the new time is smaller.
Make the jitterbuffer schedule the timeouts based on the DTS instead
of the PTS. This makes it all smoother with reordered frames and gives
the decoder time to reorder the frames in time.
