The reason behind this is to minimize the retransmission delay.
Previously, when a NACK was received, rtprtxsend would put a
retransmission packet in a queue and it would send it from chain(),
i.e. only after a new buffer would arrive.
This unfortunately was causing big delays, in the order of 60-100 ms,
which can be critical for the receiver side.
By having a separate GstTask for pushing buffers out of rtxsend,
we can push buffers out right after receiving the event, without
waiting for chain() to get called.
1) pt can be lower than 96
2) there is no point in checking that because rtprtxsend will not
even store buffers for payload types that it doesn't know about,
so this case will never be reached
This patch moves the creation of rtx packets to be done early,
in the src_event() function, when they are requested. The purpose
is to run gst_rtp_rtx_buffer_new() with the object locked to
protect internal data, because if it is done at the pushing stage,
we would have to lock and unlock multiple times in a row while we
are pushing the rtx buffers.
Previously there was no locking at all, which was terribly wrong.
This property allows you to specify the amount of buffers
to keep in the retransmission queue expressed as time (ms)
instead of buffer count (which is the max_size_buffers property).
The purpose of the sender RTX object is to keep a history
of RTP packets up to a configurable limit (in time). It will
listen for custom retransmission events from downstream. When
it receives a request for retransmission, it will look up the
requested seqnum in its list of stored packets. If the packet
is available, it will create a RTX packet according to RFC 4588
and send this as an auxiliary stream.
The receiver will listen to the custom retransmission events
from the downstream jitterbuffer and will remember the SSRC1
of the stream and seqnum that was requested. When it sees a
packet with one of the stored seqnum, it associates the SSRC2
of the stream with the SSRC1 of the master stream. From then
on it knows that SSRC2 is the retransmission stream of SSRC1.
This algorithm is stated in RFC 4588. For this algorithm to
work, RFC4588 also states that no two pending retransmission
requests can exist for the same seqnum and different SSRCs or
else it would be impossible to associate the retransmission with
the original requester SSRC.
When the RTX receiver has associated the retransmission packets,
it can depayload and forward them to the source pad of the element.
RTX is SSRC-multiplexed
Fixes https://bugzilla.gnome.org/show_bug.cgi?id=711084
AUX elements are elements that can be inserted into the rtpbin
pipeline right before or after 1 or more session elements.
The AUX elements are essential for implementing functionality such
as error correction (FEC) and retransmission (RTX).
Fixes https://bugzilla.gnome.org/show_bug.cgi?id=711087
Keep track of elements that are added to multiple sessions and make sure
we only add them to the rtpbin once and that we clean them when no
session refers to them anymore.
When a collision is found on the internal ssrc, we have to change it.
Ideally, we want also the payloader upstream to follow this change and use
the new internal ssrc. Ideally we want this condition to be always met:
if there is one payloader sending on this session, its ssrc should match the
internal ssrc.
* gst/rtpmanager/gstrtpbin.[ch]: four new action signals have been
added (request-rtp-encoder, request-rtp-decoder, request-rtcp-encoder
and request-rtcp-decoder). The user will be able to provide encoders
or decoders dynamically. The encoders must follow the srtpenc API and
the decoders the srtpdec API. Having separate signals for RTP and RTCP
allows the user to use different encoders/decoders or provide the same
one (e.g. that would be the case for srtpenc).
Also, rtpbin now allows application/x-srtp in its pads.
https://bugzilla.gnome.org/show_bug.cgi?id=719938
Use the round-trip-time and average jitter to dynamically calculate the
retransmission interval and expected packet arrival time.
Based on patches from Torrie Fischer <torrie.fischer@collabora.co.uk>
Fixes https://bugzilla.gnome.org/show_bug.cgi?id=711412
Don't use the current time calculated from the tmieout loop for when we
last scheduled the NACK because it might be unscheduled because of a max
packet misorder and then we don't accurately calculate the current time.
Instead, take the current element running time using the clock.
According to RFC 5104 section 4.3.1.2, RTCP PSFB FIR message SHALL
have a media_ssrc field set to 0. The actual media ssrc is in the FCI.
So in that case, we ignore the retained feedback and just let it through
to the rtp_session_process_fir() function which will check for the actual
SSRC inside the FCI.
Fixes a regression introduced by commit 57c27ec3
Previously, when the session had multiple internal sender SSRCs, it would
issue SR reports with RB blocks only on the first RTCP timeout and afterwards
SR reports would be sent empty. This was because the "generation" number
in RTPSource would increase more than once during the same cycle and afterwards
it would always be greater than the session's generation, which would cause
it to be skipped from being included in RBs.
This commit fixes this problem by:
1) Increasing the RTPSource generation only at the end of each cycle,
which essentially fixes the problem but only when the internal senders
are less than GST_RTCP_MAX_RB_COUNT.
2) Keeping for each RTPSource a set of SSRCs which stores which SSRC's
SR the given RTPSource has been reported in, which also fixes the problem
when the internal senders are more than GST_RTCP_MAX_RB_COUNT. This is
necessary because of the fact that any RTPSource is marked as reported
in itself's SR and makes it impossible to know if it has been reported
in other SRs too or not, and which.
Keep an extra stats structure for scheduling the BYE packets. When we
decide to schedule BYE, make a copy of the current stats into the
bye_stats. Then while we schedule the BYE, update and use only the
bye_stats. When we finished scheduling the BYE packet, we use the
regular stats again.
When we are scheduling BYE packets, ignore all RTCP for the sources that
are scheduling a BYE packet. Other sources that are not scheduling BYE
should continue receiving RTCP packets as usual.
Don't reset the expected output seqnum when clearing the pt map because this
could stall the jitterbuffer forever.
Add a unit test for this.
Fixes https://bugzilla.gnome.org/show_bug.cgi?id=709800
Remove bogus reconfigure event on collision, we don't want to send the event on
the receiving RTP pad and the collision event is now handling this
case.
See https://bugzilla.gnome.org/show_bug.cgi?id=711560
The problem here was that the jitterbuffer lock was unlocked to push
the event, but that caused another thread to remove the timer currently
being processed, probably because the amount of rtx events
(and therefore timers) was getting too high. The solution is to
unlock and push the event only after timer processing has finished.
fixes https://bugzilla.gnome.org/show_bug.cgi?id=711131
An internal sender in a session is also a receiver of its own packets so update
the receiver stats. Other senders in the session will use this info to generate
correct RB blocks in their SR reports.
Add a new timestamp mode that assumes the local and remote clock are
synchronized. It takes the first timestamp as a base time and then uses the RTP
timestamps for the output PTS.
When flush-stop arrives before we process the result of the _push() in the
loop function, we might pause even though we are not flushing anymore. Fix this
race by waiting for the srcpad loop function to completely pause after doing the
flush-start.
If we were not waiting for the missing seqnum when we insert the lost packet
event in the jitterbuffer, we end up not updating the next_seqnum and wait
forever for the lost packets to arrive. Instead, keep track of the amount of
packets contained by the jitterbuffer item and update the next expected
seqnum only after pushing the buffer/event. This makes sure we correctly handle
GAPS in the sequence numbers.
Always prepare a lost event in the jitterbuffer, it is to wake up and make the
pushing thread continue. We drop the event when we are not supposed to push lost
events downstream.
Schedule the lost event by placing it inside the jitterbuffer with the seqnum
that was lost so that the pushing thread can interleave and push it properly.
Make the jitterbuffer operate on a structure containing all the packet
information. This avoids mapping the buffer multiple times just to get the RTP
information. It will also make it possible to store other miniobjects such as
events later.
Improve the order of the timeout events, if there are timers with the same
timeout, we want to trigger the lowest seqnum first. For this we need to loop
over the complete array of timers to find the best one before triggering the
timeout.
First send the lost event, then update the next_seqnum counter and then
send the signal to the pushing thread that it can retry to push a buffer. This
avoids pushing out buffers before the lost event is pushed.
There is no need to unschedule the timer in flush-start, flush-stop will remove
the timers and unschedule.
Unschedule the current timer before attempting to join the timer thread.
Keep a separate delay in the timer so that we still know the original timestamp
of the packet that this timer refers to. We can then place the correct
running-time in the Retransmission event.
Instead of pushing the lost event from the chain function, schedule a timeout
that will push the lost event from the timer thread. This avoid blocking the
upstream thread while we push and sync the event.
When we have a large number of missing packets, generate one lost event for all
the packets that have no chance of being pushed out in time.
Fix and activate unit test for large gaps.
Keep track of the current time in the timeout loop.
Loop over all timers and trigger all the expired ones, we can do this in the
same loop that selects the new best timer.
Also update the timers when retransmission is disabled. We need to
do this because when we added LOST timers when we detected missing packets and
we need to remove those timers when the packet finally arrives.
We keep the DTS and PTS in running-time inside the jitterbuffer. Make sure to
transform it back to a buffer timestamp before pushing out the buffer.
Fixes https://bugzilla.gnome.org/show_bug.cgi?id=707931
Don't update the next RTCP check time in all cases but only when we
reconsidered. This avoids delaying sending a full RTCP packet when we
are doing early feedback.
Restructure handling of incomming packet and the gap with the expected seqnum
and register all timers from the _chain function.
Convert a timer to a LOST packet timer when the max amount of retransmission
requests has been reached.
If we change the seqnum of an existing timer and we were waiting for
that timer, unschedule it. If we change the timeout of an existing timer and we
were waiting on it, only unschedule when the new time is smaller.
Make the jitterbuffer schedule the timeouts based on the DTS instead
of the PTS. This makes it all smoother with reordered frames and gives
the decoder time to reorder the frames in time.
Refactor the jitterbuffer code. Make separate function for peeking a buffer,
pushing the next buffer, waiting for timeouts and handling the timeouts.
The main loop now tries to push as many buffers as it can until it runs out of
buffers or when it detects a seqnum discont. Then it will wait for some event to
happen before attempting to push more buffers.
Make methods to register timeouts in an array. These timeouts are registered
when we detect a missing packet, sync for the first packet or when we find an
estimation for the end-of-stream.
This greatly simplifies and clarifies the code and also makes it possible to
register more complicated timeout schemes later.
Make a method to suggest the best available SSRC. This is the SSRC of the last
created internal source and is used to instruct upstream to produce this
SSRC.
Store the BYE reason in our internal source object. Rename the methods on the
source object a little because now the BYE can be received in RTCP or
set when the session wants to send BYE.
rtpbin can now send a custom in-band downstream event which informs
downstream that the bin has received an RTCP SR packet. This is useful
for applications which want to drop the initial unsynchronized received
RTP packets.
Fixes https://bugzilla.gnome.org/show_bug.cgi?id=703560
Signed-off-by: Carlos Rafael Giani <dv@pseudoterminal.org>
Don't throw away the first RTCP packet if it arrives before the first
RTP packet but remember and use it to signal sync once we get the
RTP packet.
See https://bugzilla.gnome.org/show_bug.cgi?id=691400
Also send stream-start and segment event on the RTCP pad.
We don't need to send anything on the sync_src pad because we
already forwarded all incomming events.
