gstreamer/docs/design-audiosinks.txt

Audiosink design
----------------

Requirements:

 - must operate chain based.
   Most simple playback pipelines will push audio from the decoders
   into the audio sink.
 
 - must operate getrange based
   Most professional audio applications will operate in a mode where
   the audio sink pulls samples from the pipeline. This is typically
   done in a callback from the audiosink requesting N samples. The
   callback is either scheduled from a thread or from an interrupt
   from the audio hardware device. 

 - Exact sample accurate clocks.
   the audiosink must be able to provide a clock that is sample 
   accurate even if samples are dropped or when discontinuities are
   found in the stream.

 - Exact timing of playback.
   The audiosink must be able to play samples at their exact times.

 - use DMA access when possible.
   When the hardware can do DMA we should use it. This should also
   work over bufferpools to avoid data copying to/from kernel space.


Design:

 The design is based on a set of base classes and the concept of a
 ringbuffer of samples.

   +-----------+   - provide preroll, rendering, timing
   + basesink  +   - caps nego
   +-----+-----+
         |
   +-----V----------+   - manages ringbuffer
   + baseaudiosink  +   - manages scheduling (push/pull)
   +-----+----------+   - manages clock/query/seek
         |              - manages scheduling of samples in the ringbuffer
         |              - manages caps parsing
         |
   +-----V------+   - default ringbuffer implementation with a GThread
   + audiosink  +   - subclasses provide open/read/close methods
   +------------+

  The ringbuffer is a contiguous piece of memory divided into segtotal
  pieces of segments. Each segment has segsize bytes.

         play position                    write position
           v                                 v
   +---+---+---+-------------------------------------+----------+
   + 0 | 1 | 2 | ....                                | segtotal |
   +---+---+---+-------------------------------------+----------+
   <--->
     segsize bytes = N samples * bytes_per_sample.

  
  The ringbuffer has a play and write position, which is expressed in
  segments. The play position is where the device is currently reading
  samples and the write position is where new samples can be written
  into the buffer.

  The latency of the ringbuffer is the distance between the play and
  write position. The lowest latency is the size of a segment, thus
  smaller segment sizes allow for lower latency. 

  The ringbuffer can be put to the PLAYING or STOPPED state. 
  
  In the STOPPED state no samples are played to the device and the play
  pointer does not advance. 
  
  In the PLAYING state samples are written to the device and the ringbuffer 
  should call a configurable callback after each segment is written to the
  device. In this state the play pointer is advanced after each segment is
  written.

  A write operation to the ringbuffer will put new samples in the ringbuffer.
  If there is not enough space in the ringbuffer, the write operation will 
  block.  The playback of the buffer never stops, even if the buffer is 
  empty. When the buffer is empty, silence is played by the device.

  The ringbuffer is implemented with lockfree atomic operations, especially
  on the reading side so that low-latency operations are possible.


Scheduling:

  - chain based mode:

   In chain based mode, bytes are written into the ringbuffer. This operation
   will eventually block when the ringbuffer is filled. 
  
   When no samples arrive in time, the ringbuffer will play silence. Each 
   buffer that arrives will be placed into the ringbuffer at the correct 
   times. This means that dropping samples or inserting silence is done
   automatically and very accurate and independend of the play pointer.
   
   In this mode, the ringbuffer is usually kept as full as possible. When 
   using a small buffer (small segsize and segtotal), the latency for audio 
   to start from the sink to when it is played can be kept low but at least
   one context switch has to be made between read and write.

  - getrange based mode

    In getrange based mode, the baseaudiosink will use the callback function
    of the ringbuffer to get a segsize samples from the peer element. These
    samples will then be placed in the ringbuffer at the next play position.
    It is assumed that the getrange function returns fast enough to fill the
    ringbuffer before the play pointer reaches the write pointer.
  
    In this mode, the ringbuffer is usually kept as empty as possible. There
    is no context switch needed between the elements that create the samples
    and the actual writing of the samples to the device.


DMA mode:

  - Elements that can do DMA based access to the audio device have to subclass
    from the GstBaseAudioSink class and wrap the DMA ringbuffer in a subclass
    of GstRingBuffer.
    
    The ringbuffer subclass should trigger a callback after writing or playing
    each sample to the device. This callback can be triggered from a thread or
    from a signal from the audio device. 


Clocks:

   The GstBaseAudioSink class will use the ringbuffer to act as a clock provider.
   It can do this by using the play pointer and the delay to calculate the
   clock time.