mirror of
https://gitlab.freedesktop.org/gstreamer/gstreamer.git
synced 2024-12-19 23:06:49 +00:00
docs: design: remove, moved to gst-docs
This commit is contained in:
Tim-Philipp Müller
parent
b87e5b9c5e
commit
866fefbf5a
4 changed files with 0 additions and 272 deletions
|
|
@ -1,8 +0,0 @@
|
|||
SUBDIRS =
|
||||
|
||||
|
||||
EXTRA_DIST = \
|
||||
design-rtpauxiliary.txt \
|
||||
design-rtcollision.txt \
|
||||
design-rtpretransmission.txt
|
||||
|
||||
|
|
@ -1,85 +0,0 @@
|
|||
RTP auxiliary stream design
|
||||
|
||||
|
||||
auxiliary elements
|
||||
------------------
|
||||
|
||||
There are two kind of auxiliary elements, sender and receiver.
|
||||
Let's call them rtpauxsend and rtpauxreceive.
|
||||
|
||||
rtpauxsend has always one sink pad and can have unlimited requested src pads.
|
||||
If only src pad then it works in SSRC-multiplexed mode, if several src pads
|
||||
then it works in session multiplexed mode.
|
||||
|
||||
rtpauxreceive has always one ssrc pad and can have unlimited requested sink pads.
|
||||
If only one sink pad then it works in SSRC-multiplexed mode, if several sink pads
|
||||
then it works in session multiplexed mode.
|
||||
|
||||
|
||||
rtpbin and auxiliary elements
|
||||
----------------------------
|
||||
|
||||
-- basic mecanism
|
||||
|
||||
rtpbin knows for which session ids the given auxiliary element belong to.
|
||||
It's done through "set-aux-send", for rtpauxsend kind, and through
|
||||
"set-aux-receive" for rtpauxreceive kind.
|
||||
You can call those signals as much as needed for each auxiliary element.
|
||||
So for aux elements that work in SSRC-multiplexed mode this signal action is
|
||||
called only one time.
|
||||
|
||||
The user has to call those action signals before to request the differents
|
||||
rtpbin pads.
|
||||
rtpbin is in charge to link those auxiliary elements with the sessions,
|
||||
and on receiver side, rtpbin has also to handle the link with ssrcdemux.
|
||||
|
||||
rtpbin never knows if the given rtpauxsend is actually a rtprtxsend element
|
||||
or another aux element.
|
||||
rtpbin never knows if the given rtpauxreceive is actually a rtprtxreceive
|
||||
element or another aux element.
|
||||
rtpbin has to be kept generic so that more aux elements can be added later
|
||||
without changing rtpbin.
|
||||
|
||||
It's currently not possible to use rtpbin with auxiliary stream from gst-launch.
|
||||
We can discuss about having the ability for rtpbin to instanciate itself
|
||||
the special aux elements rtprtxsend and rtprtxreceive but they need to be
|
||||
configured ("payload-type" and "payload-types" properties) to make retransmission
|
||||
work. So having several rtprtxsend and rtprtxreceive in a rtpbin would require
|
||||
a lot of properties to manage them form rtpbin.
|
||||
And for each auxiliary elements.
|
||||
|
||||
If you want to use rtprtxreceive and rtprtpsend from gst-launch you have to use
|
||||
rtpsession, ssrcdemux and rtpjitterbuffer elements yourself.
|
||||
See gtk-doc of rtprtxreceive for an example.
|
||||
|
||||
-- requesting the rtpbin's pads on the pipeline receiver side
|
||||
|
||||
If rtpauxreceive is set for session, i, j, k then it has to call
|
||||
rtpbin::"set-aux-receive" 3 times giving those ids and this aux element.
|
||||
It has to be done before requesting the recv_rtp_sink_i, recv_rtp_sink_j, recv_rtp_sink_k.
|
||||
For a concrete case rtprtxreceive, if the user wants it for session i, then it has
|
||||
to call rtpbin::"set-aux-receive" one time giving i and this aux element.
|
||||
Then the user can request recv_rtp_sink_i pad.
|
||||
|
||||
Calling rtpbin::"set-aux-receive" does not create the session. It add the given
|
||||
session id and aux element to a hashtable(key:session id, value: aux element).
|
||||
Then when the user ask for rtpbin.recv_rtp_sink_i, rtpbin lookup if there is an
|
||||
aux element for this i session id. If yes it requests a sink pad to this aux
|
||||
element and links it with the recv_rtp_src pad of the new gstrtpsession.
|
||||
rtpbin also checks that this aux element is connected only one time to ssrcdemux.
|
||||
Because rtpauxreceive has only one source pad.
|
||||
Each call to request rtpbin.recv_rtp_sink_k will also creates rtpbin.recv_rtp_src_k_ssrc_pt
|
||||
as usual. So that the user have it when then it requests rtpbin. (from gst-launch) or
|
||||
using on_rtpbinreceive_pad_added callback from an application.
|
||||
|
||||
-- requesting the rtpbin's pads on the pipeline sender side
|
||||
|
||||
For the sender this is similar but a bit more complicated to implement.
|
||||
When the user asks for rtpbin.send_rtp_sink_i, rtpbin will lookup in its
|
||||
second map (key:session id, value: aux send element). If there is one aux
|
||||
element, then it will set the sink pad of this aux sender element to be the ghost
|
||||
pad rtpbin.send_rtp_sink_i that the user asked. rtpbin will also request a src
|
||||
pad of this aux element to connect it to gstrtpsession_i. It will automatically
|
||||
create rtpbin.send_rtp_src_i the usuall way. Then if the user asks
|
||||
rtpbin.send_rtp_src_k, then rtpbin will also lookup in that map and request
|
||||
another source pad of the aux element and connect it to the new gstrtpsession_k.
|
||||
|
|
@ -1,30 +0,0 @@
|
|||
RTP collision design
|
||||
|
||||
|
||||
GstRTPCollision
|
||||
---------------
|
||||
|
||||
Custon upstream event which contains the ssrc marked as collided.
|
||||
|
||||
This event is generated on both pipeline sender and receiver side by
|
||||
the gstrtpsession element when it detects a conflict between ssrc.
|
||||
(same session id and same ssrc)
|
||||
|
||||
It's an upstream event so that means this event is for now only
|
||||
useful on pipeline sender side. Because elements generating packets with the
|
||||
collided SSRC are placed upstream from the gstrtpsession.
|
||||
|
||||
rtppayloader
|
||||
------------
|
||||
|
||||
When handling a GstRTPCollision event, the rtppayloader has to choose another
|
||||
ssrc.
|
||||
|
||||
|
||||
BYE only the corresponding source, not the whole session.
|
||||
---------------------------------------------------------
|
||||
|
||||
When a collision happens for the given ssrc, the associated source is marked
|
||||
bye. But we make sure that the whole session is not itself set bye.
|
||||
Because internally, gstrtpsession can manages several sources and all have
|
||||
their own distinct ssrc.
|
||||
|
|
@ -1,149 +0,0 @@
|
|||
RTP retransmission design
|
||||
|
||||
|
||||
GstRTPRetransmissionRequest
|
||||
---------------------------
|
||||
|
||||
Custom upstream event which mainly contains the ssrc and the seqnum of the
|
||||
packet which is asked to be retransmisted.
|
||||
|
||||
On the pipeline receiver side this event is generated by the
|
||||
gstrtpjitterbuffer element. Then it is translated to a NACK to be sent over
|
||||
the network.
|
||||
|
||||
On the pipeline sender side, this event is generated by the gstrtpsession
|
||||
element when it receives a NACK from the network.
|
||||
|
||||
|
||||
rtprtxsend element
|
||||
------------------
|
||||
|
||||
-- basic mechanism
|
||||
|
||||
rtprtxsend keeps a history of rtp packets that it has already sent.
|
||||
When it receives the event GstRTPRetransmissionRequest from the downstream
|
||||
gstrtpsession element, it loopkup the requested seqnum in its stored packets.
|
||||
If the packet is present in its history, it will create a RTX packet according
|
||||
to RFC 4588. Then this rtx packet is pushed to its src pad as other packets.
|
||||
|
||||
rtprtxsend works in SSRC-multiplexed mode, so it has one always sink and
|
||||
src pad.
|
||||
|
||||
-- building retransmission packet fron original packet
|
||||
|
||||
A rtx packet is mostly the same as an orignal packet, except it has its own
|
||||
ssrc and its own seqnum. That's why rtprtxsend works in SSRC-multiplexed mode.
|
||||
It also means that the same session is used.
|
||||
Another difference between rtx packet and its original is that it inserts the
|
||||
original seqnum (OSN: 2 bytes) at the beginning of the payload.
|
||||
Also rtprtxsend builds rtx packet without padding, to let other elements do that.
|
||||
The last difference is the payload type. For now the user has to set it through
|
||||
the rtx-payload-type property. Later it will be automatically retreive this
|
||||
information from SDP. See fmtp field as specifies in the RPC4588
|
||||
(a=fmtp:99 apt=98) fmtp is the payload type of the retransmission stream
|
||||
and apt the payload type of its associated master stream.
|
||||
|
||||
-- restransmission ssrc and seqnum
|
||||
|
||||
To choose rtx_ssrc it randomly selects a number between 0 and 2^32-1 until
|
||||
it is different than master_ssrc.
|
||||
rtx_seqnum is randomly selected between 0 and 2^16-1
|
||||
|
||||
-- deeper in the stored buffer history
|
||||
|
||||
For the history it uses a GSequence with 2^15-1 as its maximum size.
|
||||
Which is resonable as the default value is 100.
|
||||
It contains the packets in reverse order they have been sent
|
||||
(head:newest, tail:oldest)
|
||||
GSequence allows to add and remove an element in constant time (like a queue).
|
||||
Also GSequence allows to do a binary search when rtprtxsend lookup in its
|
||||
history.
|
||||
It's important if it receives a lot of requests or if the history is large.
|
||||
|
||||
-- pending rtx packets
|
||||
|
||||
When looking up in its history, if seqnum is found then it pushes the buffer
|
||||
into a GQueue to its tail.
|
||||
Before to send the current master stream packet, rtprtxsend sends all the
|
||||
buffers which are in this GQueue. Taking care of converting them to rtx
|
||||
packets.
|
||||
This way, rtx packets are sent in the same order they have been requested.
|
||||
(g_list_foreach traverse the queue from head to tail)
|
||||
The GQueue is cleared between sending 2 master stream packets.
|
||||
So for this GQueue to contain more than one element, it means that rtprtxsend
|
||||
receives more than one rtx request between sending 2 master packets.
|
||||
|
||||
-- collision
|
||||
|
||||
When handling a GstRTPCollision event, if the ssrc is its rtx ssrc then
|
||||
rtprtxsend clear its history and its pending retransmission queue.
|
||||
Then it chooses a rtx_ssrc until it's different than master ssrc.
|
||||
If the GstRTPCollision event does not contain its rtx ssrc, for example
|
||||
its master ssrc or other, then it just forwards the event to upstream.
|
||||
So that it can be handled by the rtppayloader.
|
||||
|
||||
|
||||
rtprtxreceive element
|
||||
------------------
|
||||
|
||||
-- basic mechanism
|
||||
|
||||
The same rtprtxreceive instance can receive several master streams and several
|
||||
retransmission streams.
|
||||
So it will try to dynamically associate a rtx ssrc with its master ssrc.
|
||||
So that it can reconstruct the original from the proper rtx packet.
|
||||
|
||||
The algorithm is based on the fact that seqnums of different streams
|
||||
(considering all master and all rtx streams) evolve at a different rate.
|
||||
It means that the initial seqnum is random for each one and the offset could
|
||||
also be different. So that they are statistically all different at a given
|
||||
time. If bad luck then the association is delayed to the next rtx request.
|
||||
|
||||
The algorithm also needs to know if a given packet is a rtx packet or not.
|
||||
To know this information there is the rtx-payload-types property. For now the
|
||||
user as to configure it but later it will be automatically retreive this
|
||||
information from SDP.
|
||||
It needs to know if the current packet is rtx or not in order to know if
|
||||
it can extract the OSN from the payload. Otherwise it would extract the OSN
|
||||
even on master streams which means nothing and so it could do bad things.
|
||||
In theory maybe it could work but we have this information in SDP so why not
|
||||
using it to avoid bad associations.
|
||||
|
||||
Note that it also means that several master streams can have the same payload
|
||||
type. And also several rtx streams can have the same payload type.
|
||||
So the information from SDP which gives us which rtx payload type belong to
|
||||
a give master payload type is not enough to do the association between rtx ssrc
|
||||
and master ssrc.
|
||||
|
||||
rtprtxreceive works in SSRC-multiplexed mode, so it has one always sink and
|
||||
src pad.
|
||||
|
||||
-- deeper in the association algorithm
|
||||
|
||||
When it receives a GstRTPRetransmissionRequest event it will remember the ssrc
|
||||
and the seqnum from this request.
|
||||
|
||||
On incoming packets, if the packet has its ssrc already associated then it
|
||||
knows if the ssrc is an rtx ssrc or a master stream ssrc.
|
||||
If this is a rtx packet then it recontructs the original and pushs the result to
|
||||
src pad as if it was a master packet.
|
||||
|
||||
If the ssrc is not yet associated rtprtxreceive checks the payload type.
|
||||
if the packet has its payload type marked as rtx then it will extract the OSN
|
||||
(original seqnum number) and lookup in its stored requests if a seqnum matchs.
|
||||
If found, then it associates the current ssrc to the master ssrc marked in the
|
||||
request. If not found it just drops the packet.
|
||||
Then it removes the request from the stored requests.
|
||||
|
||||
If there are 2 requests with the same seqnum and different ssrc, then the
|
||||
couple seqnum,ssrc is removed from the stored requests.
|
||||
A stored request actually means that actually the couple seqnum,ssrc is stored.
|
||||
If it's happens the request is droped but it avoids to do bad associations.
|
||||
In this case the association is just delayed to the next request.
|
||||
|
||||
-- building original packet from rtx packet
|
||||
|
||||
Header, extensions, payload and padding are mostly the same. Except that the
|
||||
OSN is removed from the payload. Then ssrc, seqnum, and original payload type
|
||||
are correctly set. Original payload type is actually also stored when the
|
||||
rtx request is handled.
|
||||
Loading…
Reference in a new issue