gst-plugins-rs/net/raptorq/README.md

Introduction

This is GStreamer implementation of RaptorQ FEC for RTP streams.

The sender element produces requested number X of repair packets from K RTP packets. The receiver only needs:

• K of any repair or RTP packets to recover all the data with 99% probability
• K + 1 of any repair or RTP packets to recover all the data with 99.99% probability,
• K + 2 of any repair or RTP packets to recover all the data with 99.9999% probability etc.

Relevant documents:

• RFC6363 - Forward Error Correction (FEC) Framework
• RFC6681 - Raptor Forward Error Correction (FEC) Schemes for FECFRAME
• RFC6682 - RTP Payload Format for Raptor Forward Error Correction (FEC)

Sender/Receiver Example

    gst-launch-1.0 \
        rtpbin name=rtp fec-encoders='fec,0="raptorqenc\ mtu=1356\ symbol-size=192";' \
        uridecodebin uri=file:///path/to/video/file ! x264enc key-int-max=60 tune=zerolatency ! \
          queue ! mpegtsmux ! rtpmp2tpay ssrc=0 ! \
        rtp.send_rtp_sink_0 rtp.send_rtp_src_0 ! udpsink host=127.0.0.1 port=5000 \
        rtp.send_fec_src_0_0 ! udpsink host=127.0.0.1 port=5002 async=false

    gst-launch-1.0 \
        rtpbin latency=200 fec-decoders='fec,0="raptorqdec";' name=rtp \
        udpsrc address=127.0.0.1 port=5002 \
          caps="application/x-rtp, payload=96, raptor-scheme-id=(string)6, repair-window=(string)1000000, t=(string)192" ! \
        queue ! rtp.recv_fec_sink_0_0 \
        udpsrc address=127.0.0.1 port=5000 \
           caps="application/x-rtp, media=video, clock-rate=90000, encoding-name=mp2t, payload=33" ! \
        queue ! netsim drop-probability=0.05 ! rtp.recv_rtp_sink_0 \
        rtp. ! decodebin ! videoconvert ! queue ! autovideosink

Implementation Details

Encoder Element

The encoder element stores the copy of original RTP packets internally until it receives the number of packets that are requested to be protected together. At this point it creates a Source Block that is passed to RaptorQ Encoder. Source Block is constructed by concatenating ADUIs (Application Data Unit Information) sometimes also called SPI (Source Packet Information). Each ADUI contains:

• Header with Flow ID - F(I) and Length Indication for the packet - L(I),
• UDP payload, this a complete RTP packet with header,
• Padding bytes if required,

            T                T                T                T
    <----------------><--------------><---------------><---------------->
    +----+--------+-----------------------+-----------------------------+
    |F[0]|  L[0]  |        ADU[0]         |            Pad[0]           |
    +----+--------+----------+------------+-----------------------------+
    |F[1]|  L[1]  | ADU[1]   |                         Pad[1]           |
    +----+--------+----------+------------------------------------------+
    |F[2]|  L[2]  |                    ADU[2]                           |
    +----+--------+------+----------------------------------------------+
    |F[3]|  L[3]  |ADU[3]|                             Pad[3]           |
    +----+--------+------+----------------------------------------------+
    \_________________________________  ________________________________/
                                      \/
                             RaptorQ FEC encoding
    
    +-------------------------------------------------------------------+
    |                              Repair 4                             |
    +-------------------------------------------------------------------+
    .                                                                   .
    .                                                                   .
    +-------------------------------------------------------------------+
    |                              Repair 7                             |
    +-------------------------------------------------------------------+
    
    T - Symbol Size
    F - Flow ID
    L - Length Indication
    ADU - Application Data Unit (RTP packet)

Encoder element creates requested number of packets for a given Source Block. The repair packets are send during repair-window which is configurable parameter. E.g. if encoder element produces 5 repair packets and repair-window is set to 500ms, a first repair packet is send 100ms after the last protected packet, second at 200ms and the last at repair-window.

Each repair packet except the symbols that are required to recover missing source packets, contains also the information about the Source Block:

• I - Initial sequence number of the Source Block,
• Lp - ADUI length in symbols,
• Lb - Source Block Length in symbols,

Decoder Element

Decoder element stores the copy of received RTP packets, and push original packet downstream immediately. If all the RTP packets have been received, the buffered media packets are dropped. If any packets are missing, the receiver checks if it has enough buffered media and repair packets to perform decoding. If that's the case it tries to recover missing packets by building the Source Block following the same rules as sender, except it skips missing packets and append repair packets to the block instead.

Because the receiver element does not introduce latency, the recovered packets are send out of sequence, and it requires a rtpjitterbuffer to be chained downstream. The rtpjitterbuffer needs to be configured with high enough latency.

The receiver to determine which media packets belongs to Source Blocks uses the information that can be retrieved from any of the repair packets. Then media packets with Sequence Numbers: I + Lb/Lp - 1 inclusive, are considered during building a Source Block.

The receiver uses repair-window that is signaled by the sender, and its own repair-window-tolerance parameter to decide for how long it should wait for the corresponding repair packets before giving up. The wait time is repair-window + repair-window-tolerance.