mirror of
https://gitlab.freedesktop.org/gstreamer/gstreamer.git
synced 2024-12-20 07:16:55 +00:00
docs: design: remove, moved to gst-docs
This commit is contained in:
Tim-Philipp Müller
parent
b87e5b9c5e
commit
866fefbf5a
4 changed files with 0 additions and 272 deletions
|
|
@ -1,8 +0,0 @@
|
||||||
SUBDIRS =
|
|
||||||
|
|
||||||
|
|
||||||
EXTRA_DIST = \
|
|
||||||
design-rtpauxiliary.txt \
|
|
||||||
design-rtcollision.txt \
|
|
||||||
design-rtpretransmission.txt
|
|
||||||
|
|
||||||
|
|
@ -1,85 +0,0 @@
|
||||||
RTP auxiliary stream design
|
|
||||||
|
|
||||||
|
|
||||||
auxiliary elements
|
|
||||||
------------------
|
|
||||||
|
|
||||||
There are two kind of auxiliary elements, sender and receiver.
|
|
||||||
Let's call them rtpauxsend and rtpauxreceive.
|
|
||||||
|
|
||||||
rtpauxsend has always one sink pad and can have unlimited requested src pads.
|
|
||||||
If only src pad then it works in SSRC-multiplexed mode, if several src pads
|
|
||||||
then it works in session multiplexed mode.
|
|
||||||
|
|
||||||
rtpauxreceive has always one ssrc pad and can have unlimited requested sink pads.
|
|
||||||
If only one sink pad then it works in SSRC-multiplexed mode, if several sink pads
|
|
||||||
then it works in session multiplexed mode.
|
|
||||||
|
|
||||||
|
|
||||||
rtpbin and auxiliary elements
|
|
||||||
----------------------------
|
|
||||||
|
|
||||||
-- basic mecanism
|
|
||||||
|
|
||||||
rtpbin knows for which session ids the given auxiliary element belong to.
|
|
||||||
It's done through "set-aux-send", for rtpauxsend kind, and through
|
|
||||||
"set-aux-receive" for rtpauxreceive kind.
|
|
||||||
You can call those signals as much as needed for each auxiliary element.
|
|
||||||
So for aux elements that work in SSRC-multiplexed mode this signal action is
|
|
||||||
called only one time.
|
|
||||||
|
|
||||||
The user has to call those action signals before to request the differents
|
|
||||||
rtpbin pads.
|
|
||||||
rtpbin is in charge to link those auxiliary elements with the sessions,
|
|
||||||
and on receiver side, rtpbin has also to handle the link with ssrcdemux.
|
|
||||||
|
|
||||||
rtpbin never knows if the given rtpauxsend is actually a rtprtxsend element
|
|
||||||
or another aux element.
|
|
||||||
rtpbin never knows if the given rtpauxreceive is actually a rtprtxreceive
|
|
||||||
element or another aux element.
|
|
||||||
rtpbin has to be kept generic so that more aux elements can be added later
|
|
||||||
without changing rtpbin.
|
|
||||||
|
|
||||||
It's currently not possible to use rtpbin with auxiliary stream from gst-launch.
|
|
||||||
We can discuss about having the ability for rtpbin to instanciate itself
|
|
||||||
the special aux elements rtprtxsend and rtprtxreceive but they need to be
|
|
||||||
configured ("payload-type" and "payload-types" properties) to make retransmission
|
|
||||||
work. So having several rtprtxsend and rtprtxreceive in a rtpbin would require
|
|
||||||
a lot of properties to manage them form rtpbin.
|
|
||||||
And for each auxiliary elements.
|
|
||||||
|
|
||||||
If you want to use rtprtxreceive and rtprtpsend from gst-launch you have to use
|
|
||||||
rtpsession, ssrcdemux and rtpjitterbuffer elements yourself.
|
|
||||||
See gtk-doc of rtprtxreceive for an example.
|
|
||||||
|
|
||||||
-- requesting the rtpbin's pads on the pipeline receiver side
|
|
||||||
|
|
||||||
If rtpauxreceive is set for session, i, j, k then it has to call
|
|
||||||
rtpbin::"set-aux-receive" 3 times giving those ids and this aux element.
|
|
||||||
It has to be done before requesting the recv_rtp_sink_i, recv_rtp_sink_j, recv_rtp_sink_k.
|
|
||||||
For a concrete case rtprtxreceive, if the user wants it for session i, then it has
|
|
||||||
to call rtpbin::"set-aux-receive" one time giving i and this aux element.
|
|
||||||
Then the user can request recv_rtp_sink_i pad.
|
|
||||||
|
|
||||||
Calling rtpbin::"set-aux-receive" does not create the session. It add the given
|
|
||||||
session id and aux element to a hashtable(key:session id, value: aux element).
|
|
||||||
Then when the user ask for rtpbin.recv_rtp_sink_i, rtpbin lookup if there is an
|
|
||||||
aux element for this i session id. If yes it requests a sink pad to this aux
|
|
||||||
element and links it with the recv_rtp_src pad of the new gstrtpsession.
|
|
||||||
rtpbin also checks that this aux element is connected only one time to ssrcdemux.
|
|
||||||
Because rtpauxreceive has only one source pad.
|
|
||||||
Each call to request rtpbin.recv_rtp_sink_k will also creates rtpbin.recv_rtp_src_k_ssrc_pt
|
|
||||||
as usual. So that the user have it when then it requests rtpbin. (from gst-launch) or
|
|
||||||
using on_rtpbinreceive_pad_added callback from an application.
|
|
||||||
|
|
||||||
-- requesting the rtpbin's pads on the pipeline sender side
|
|
||||||
|
|
||||||
For the sender this is similar but a bit more complicated to implement.
|
|
||||||
When the user asks for rtpbin.send_rtp_sink_i, rtpbin will lookup in its
|
|
||||||
second map (key:session id, value: aux send element). If there is one aux
|
|
||||||
element, then it will set the sink pad of this aux sender element to be the ghost
|
|
||||||
pad rtpbin.send_rtp_sink_i that the user asked. rtpbin will also request a src
|
|
||||||
pad of this aux element to connect it to gstrtpsession_i. It will automatically
|
|
||||||
create rtpbin.send_rtp_src_i the usuall way. Then if the user asks
|
|
||||||
rtpbin.send_rtp_src_k, then rtpbin will also lookup in that map and request
|
|
||||||
another source pad of the aux element and connect it to the new gstrtpsession_k.
|
|
||||||
|
|
@ -1,30 +0,0 @@
|
||||||
RTP collision design
|
|
||||||
|
|
||||||
|
|
||||||
GstRTPCollision
|
|
||||||
---------------
|
|
||||||
|
|
||||||
Custon upstream event which contains the ssrc marked as collided.
|
|
||||||
|
|
||||||
This event is generated on both pipeline sender and receiver side by
|
|
||||||
the gstrtpsession element when it detects a conflict between ssrc.
|
|
||||||
(same session id and same ssrc)
|
|
||||||
|
|
||||||
It's an upstream event so that means this event is for now only
|
|
||||||
useful on pipeline sender side. Because elements generating packets with the
|
|
||||||
collided SSRC are placed upstream from the gstrtpsession.
|
|
||||||
|
|
||||||
rtppayloader
|
|
||||||
------------
|
|
||||||
|
|
||||||
When handling a GstRTPCollision event, the rtppayloader has to choose another
|
|
||||||
ssrc.
|
|
||||||
|
|
||||||
|
|
||||||
BYE only the corresponding source, not the whole session.
|
|
||||||
---------------------------------------------------------
|
|
||||||
|
|
||||||
When a collision happens for the given ssrc, the associated source is marked
|
|
||||||
bye. But we make sure that the whole session is not itself set bye.
|
|
||||||
Because internally, gstrtpsession can manages several sources and all have
|
|
||||||
their own distinct ssrc.
|
|
||||||
|
|
@ -1,149 +0,0 @@
|
||||||
RTP retransmission design
|
|
||||||
|
|
||||||
|
|
||||||
GstRTPRetransmissionRequest
|
|
||||||
---------------------------
|
|
||||||
|
|
||||||
Custom upstream event which mainly contains the ssrc and the seqnum of the
|
|
||||||
packet which is asked to be retransmisted.
|
|
||||||
|
|
||||||
On the pipeline receiver side this event is generated by the
|
|
||||||
gstrtpjitterbuffer element. Then it is translated to a NACK to be sent over
|
|
||||||
the network.
|
|
||||||
|
|
||||||
On the pipeline sender side, this event is generated by the gstrtpsession
|
|
||||||
element when it receives a NACK from the network.
|
|
||||||
|
|
||||||
|
|
||||||
rtprtxsend element
|
|
||||||
------------------
|
|
||||||
|
|
||||||
-- basic mechanism
|
|
||||||
|
|
||||||
rtprtxsend keeps a history of rtp packets that it has already sent.
|
|
||||||
When it receives the event GstRTPRetransmissionRequest from the downstream
|
|
||||||
gstrtpsession element, it loopkup the requested seqnum in its stored packets.
|
|
||||||
If the packet is present in its history, it will create a RTX packet according
|
|
||||||
to RFC 4588. Then this rtx packet is pushed to its src pad as other packets.
|
|
||||||
|
|
||||||
rtprtxsend works in SSRC-multiplexed mode, so it has one always sink and
|
|
||||||
src pad.
|
|
||||||
|
|
||||||
-- building retransmission packet fron original packet
|
|
||||||
|
|
||||||
A rtx packet is mostly the same as an orignal packet, except it has its own
|
|
||||||
ssrc and its own seqnum. That's why rtprtxsend works in SSRC-multiplexed mode.
|
|
||||||
It also means that the same session is used.
|
|
||||||
Another difference between rtx packet and its original is that it inserts the
|
|
||||||
original seqnum (OSN: 2 bytes) at the beginning of the payload.
|
|
||||||
Also rtprtxsend builds rtx packet without padding, to let other elements do that.
|
|
||||||
The last difference is the payload type. For now the user has to set it through
|
|
||||||
the rtx-payload-type property. Later it will be automatically retreive this
|
|
||||||
information from SDP. See fmtp field as specifies in the RPC4588
|
|
||||||
(a=fmtp:99 apt=98) fmtp is the payload type of the retransmission stream
|
|
||||||
and apt the payload type of its associated master stream.
|
|
||||||
|
|
||||||
-- restransmission ssrc and seqnum
|
|
||||||
|
|
||||||
To choose rtx_ssrc it randomly selects a number between 0 and 2^32-1 until
|
|
||||||
it is different than master_ssrc.
|
|
||||||
rtx_seqnum is randomly selected between 0 and 2^16-1
|
|
||||||
|
|
||||||
-- deeper in the stored buffer history
|
|
||||||
|
|
||||||
For the history it uses a GSequence with 2^15-1 as its maximum size.
|
|
||||||
Which is resonable as the default value is 100.
|
|
||||||
It contains the packets in reverse order they have been sent
|
|
||||||
(head:newest, tail:oldest)
|
|
||||||
GSequence allows to add and remove an element in constant time (like a queue).
|
|
||||||
Also GSequence allows to do a binary search when rtprtxsend lookup in its
|
|
||||||
history.
|
|
||||||
It's important if it receives a lot of requests or if the history is large.
|
|
||||||
|
|
||||||
-- pending rtx packets
|
|
||||||
|
|
||||||
When looking up in its history, if seqnum is found then it pushes the buffer
|
|
||||||
into a GQueue to its tail.
|
|
||||||
Before to send the current master stream packet, rtprtxsend sends all the
|
|
||||||
buffers which are in this GQueue. Taking care of converting them to rtx
|
|
||||||
packets.
|
|
||||||
This way, rtx packets are sent in the same order they have been requested.
|
|
||||||
(g_list_foreach traverse the queue from head to tail)
|
|
||||||
The GQueue is cleared between sending 2 master stream packets.
|
|
||||||
So for this GQueue to contain more than one element, it means that rtprtxsend
|
|
||||||
receives more than one rtx request between sending 2 master packets.
|
|
||||||
|
|
||||||
-- collision
|
|
||||||
|
|
||||||
When handling a GstRTPCollision event, if the ssrc is its rtx ssrc then
|
|
||||||
rtprtxsend clear its history and its pending retransmission queue.
|
|
||||||
Then it chooses a rtx_ssrc until it's different than master ssrc.
|
|
||||||
If the GstRTPCollision event does not contain its rtx ssrc, for example
|
|
||||||
its master ssrc or other, then it just forwards the event to upstream.
|
|
||||||
So that it can be handled by the rtppayloader.
|
|
||||||
|
|
||||||
|
|
||||||
rtprtxreceive element
|
|
||||||
------------------
|
|
||||||
|
|
||||||
-- basic mechanism
|
|
||||||
|
|
||||||
The same rtprtxreceive instance can receive several master streams and several
|
|
||||||
retransmission streams.
|
|
||||||
So it will try to dynamically associate a rtx ssrc with its master ssrc.
|
|
||||||
So that it can reconstruct the original from the proper rtx packet.
|
|
||||||
|
|
||||||
The algorithm is based on the fact that seqnums of different streams
|
|
||||||
(considering all master and all rtx streams) evolve at a different rate.
|
|
||||||
It means that the initial seqnum is random for each one and the offset could
|
|
||||||
also be different. So that they are statistically all different at a given
|
|
||||||
time. If bad luck then the association is delayed to the next rtx request.
|
|
||||||
|
|
||||||
The algorithm also needs to know if a given packet is a rtx packet or not.
|
|
||||||
To know this information there is the rtx-payload-types property. For now the
|
|
||||||
user as to configure it but later it will be automatically retreive this
|
|
||||||
information from SDP.
|
|
||||||
It needs to know if the current packet is rtx or not in order to know if
|
|
||||||
it can extract the OSN from the payload. Otherwise it would extract the OSN
|
|
||||||
even on master streams which means nothing and so it could do bad things.
|
|
||||||
In theory maybe it could work but we have this information in SDP so why not
|
|
||||||
using it to avoid bad associations.
|
|
||||||
|
|
||||||
Note that it also means that several master streams can have the same payload
|
|
||||||
type. And also several rtx streams can have the same payload type.
|
|
||||||
So the information from SDP which gives us which rtx payload type belong to
|
|
||||||
a give master payload type is not enough to do the association between rtx ssrc
|
|
||||||
and master ssrc.
|
|
||||||
|
|
||||||
rtprtxreceive works in SSRC-multiplexed mode, so it has one always sink and
|
|
||||||
src pad.
|
|
||||||
|
|
||||||
-- deeper in the association algorithm
|
|
||||||
|
|
||||||
When it receives a GstRTPRetransmissionRequest event it will remember the ssrc
|
|
||||||
and the seqnum from this request.
|
|
||||||
|
|
||||||
On incoming packets, if the packet has its ssrc already associated then it
|
|
||||||
knows if the ssrc is an rtx ssrc or a master stream ssrc.
|
|
||||||
If this is a rtx packet then it recontructs the original and pushs the result to
|
|
||||||
src pad as if it was a master packet.
|
|
||||||
|
|
||||||
If the ssrc is not yet associated rtprtxreceive checks the payload type.
|
|
||||||
if the packet has its payload type marked as rtx then it will extract the OSN
|
|
||||||
(original seqnum number) and lookup in its stored requests if a seqnum matchs.
|
|
||||||
If found, then it associates the current ssrc to the master ssrc marked in the
|
|
||||||
request. If not found it just drops the packet.
|
|
||||||
Then it removes the request from the stored requests.
|
|
||||||
|
|
||||||
If there are 2 requests with the same seqnum and different ssrc, then the
|
|
||||||
couple seqnum,ssrc is removed from the stored requests.
|
|
||||||
A stored request actually means that actually the couple seqnum,ssrc is stored.
|
|
||||||
If it's happens the request is droped but it avoids to do bad associations.
|
|
||||||
In this case the association is just delayed to the next request.
|
|
||||||
|
|
||||||
-- building original packet from rtx packet
|
|
||||||
|
|
||||||
Header, extensions, payload and padding are mostly the same. Except that the
|
|
||||||
OSN is removed from the payload. Then ssrc, seqnum, and original payload type
|
|
||||||
are correctly set. Original payload type is actually also stored when the
|
|
||||||
rtx request is handled.
|
|
||||||
Loading…
Reference in a new issue