gst-plugins-rs

mirror of https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs.git synced 2024-12-28 04:50:30 +00:00

History

François Laignel 6319d104a8 Take advantage of `Into<Option<_>>` args Commit `24b7cfc8` applied changes related to nullability as declared by gir. One consequence was that some functions signature ended up requiring users to pass `Some(val)` when they could use `val` before. This commit applies changes on `gstreamer-rs` which, will honoring the nullability stil allow users to pass `val` for the few affected functions. This commit also fixes the signature for `Element::request_new_pad` which was updated upstream.		2022-10-21 11:54:24 +02:00
..
src	Take advantage of `Into<Option<_>>` args	2022-10-21 11:54:24 +02:00
tests	Update everything for element factory builder API changes	2022-10-19 19:43:29 +03:00
build.rs	Add RaptorQ RTP FEC plugins	2022-07-20 13:34:58 +00:00
Cargo.toml	build: Update versions to be 0.9.0-alpha.1	2022-10-04 21:27:23 +05:30
LICENSE-MPL-2.0	Add RaptorQ RTP FEC plugins	2022-07-20 13:34:58 +00:00
README.md	Add RaptorQ RTP FEC plugins	2022-07-20 13:34:58 +00:00

README.md

Introduction

This is GStreamer implementation of RaptorQ FEC for RTP streams.

The sender element produces requested number X of repair packets from K RTP packets. The receiver only needs:

K of any repair or RTP packets to recover all the data with 99% probability
K + 1 of any repair or RTP packets to recover all the data with 99.99% probability,
K + 2 of any repair or RTP packets to recover all the data with 99.9999% probability etc.

Relevant documents:

Sender/Receiver Example

    gst-launch-1.0 \
        rtpbin name=rtp fec-encoders='fec,0="raptorqenc\ mtu=1356\ symbol-size=192";' \
        uridecodebin uri=file:///path/to/video/file ! x264enc key-int-max=60 tune=zerolatency ! \
          queue ! mpegtsmux ! rtpmp2tpay ssrc=0 ! \
        rtp.send_rtp_sink_0 rtp.send_rtp_src_0 ! udpsink host=127.0.0.1 port=5000 \
        rtp.send_fec_src_0_0 ! udpsink host=127.0.0.1 port=5002 async=false

    gst-launch-1.0 \
        rtpbin latency=200 fec-decoders='fec,0="raptorqdec";' name=rtp \
        udpsrc address=127.0.0.1 port=5002 \
          caps="application/x-rtp, payload=96, raptor-scheme-id=(string)6, repair-window=(string)1000000, t=(string)192" ! \
        queue ! rtp.recv_fec_sink_0_0 \
        udpsrc address=127.0.0.1 port=5000 \
           caps="application/x-rtp, media=video, clock-rate=90000, encoding-name=mp2t, payload=33" ! \
        queue ! netsim drop-probability=0.05 ! rtp.recv_rtp_sink_0 \
        rtp. ! decodebin ! videoconvert ! queue ! autovideosink

Implementation Details

Encoder Element

The encoder element stores the copy of original RTP packets internally until it receives the number of packets that are requested to be protected together. At this point it creates a Source Block that is passed to RaptorQ Encoder. Source Block is constructed by concatenating ADUIs (Application Data Unit Information) sometimes also called SPI (Source Packet Information). Each ADUI contains:

Header with Flow ID - F(I) and Length Indication for the packet - L(I),
UDP payload, this a complete RTP packet with header,
Padding bytes if required,

            T                T                T                T
    <----------------><--------------><---------------><---------------->
    +----+--------+-----------------------+-----------------------------+
    |F[0]|  L[0]  |        ADU[0]         |            Pad[0]           |
    +----+--------+----------+------------+-----------------------------+
    |F[1]|  L[1]  | ADU[1]   |                         Pad[1]           |
    +----+--------+----------+------------------------------------------+
    |F[2]|  L[2]  |                    ADU[2]                           |
    +----+--------+------+----------------------------------------------+
    |F[3]|  L[3]  |ADU[3]|                             Pad[3]           |
    +----+--------+------+----------------------------------------------+
    \_________________________________  ________________________________/
                                      \/
                             RaptorQ FEC encoding
    
    +-------------------------------------------------------------------+
    |                              Repair 4                             |
    +-------------------------------------------------------------------+
    .                                                                   .
    .                                                                   .
    +-------------------------------------------------------------------+
    |                              Repair 7                             |
    +-------------------------------------------------------------------+
    
    T - Symbol Size
    F - Flow ID
    L - Length Indication
    ADU - Application Data Unit (RTP packet)

Encoder element creates requested number of packets for a given Source Block. The repair packets are send during repair-window which is configurable parameter. E.g. if encoder element produces 5 repair packets and repair-window is set to 500ms, a first repair packet is send 100ms after the last protected packet, second at 200ms and the last at repair-window.

Each repair packet except the symbols that are required to recover missing source packets, contains also the information about the Source Block:

I - Initial sequence number of the Source Block,
Lp - ADUI length in symbols,
Lb - Source Block Length in symbols,

Decoder Element

Decoder element stores the copy of received RTP packets, and push original packet downstream immediately. If all the RTP packets have been received, the buffered media packets are dropped. If any packets are missing, the receiver checks if it has enough buffered media and repair packets to perform decoding. If that's the case it tries to recover missing packets by building the Source Block following the same rules as sender, except it skips missing packets and append repair packets to the block instead.

Because the receiver element does not introduce latency, the recovered packets are send out of sequence, and it requires a rtpjitterbuffer to be chained downstream. The rtpjitterbuffer needs to be configured with high enough latency.

The receiver to determine which media packets belongs to Source Blocks uses the information that can be retrieved from any of the repair packets. Then media packets with Sequence Numbers: I + Lb/Lp - 1 inclusive, are considered during building a Source Block.

The receiver uses repair-window that is signaled by the sender, and its own repair-window-tolerance parameter to decide for how long it should wait for the corresponding repair packets before giving up. The wait time is repair-window + repair-window-tolerance.