mirror of
https://gitlab.freedesktop.org/gstreamer/gstreamer.git
synced 2024-11-19 08:11:16 +00:00
133 lines
5.1 KiB
Markdown
133 lines
5.1 KiB
Markdown
## Audiosink design
|
|
|
|
### Requirements
|
|
|
|
- must operate chain based. Most simple playback pipelines will push
|
|
audio from the decoders into the audio sink.
|
|
|
|
- must operate getrange based Most professional audio applications
|
|
will operate in a mode where the audio sink pulls samples from the
|
|
pipeline. This is typically done in a callback from the audiosink
|
|
requesting N samples. The callback is either scheduled from a thread
|
|
or from an interrupt from the audio hardware device.
|
|
|
|
- Exact sample accurate clocks. the audiosink must be able to provide
|
|
a clock that is sample accurate even if samples are dropped or when
|
|
discontinuities are found in the stream.
|
|
|
|
- Exact timing of playback. The audiosink must be able to play samples
|
|
at their exact times.
|
|
|
|
- use DMA access when possible. When the hardware can do DMA we should
|
|
use it. This should also work over bufferpools to avoid data copying
|
|
to/from kernel space.
|
|
|
|
### Design
|
|
|
|
The design is based on a set of base classes and the concept of a
|
|
ringbuffer of samples.
|
|
|
|
```
|
|
+-----------+ - provide preroll, rendering, timing
|
|
+ basesink + - caps nego
|
|
+-----+-----+
|
|
|
|
|
+-----V----------+ - manages ringbuffer
|
|
+ audiobasesink + - manages scheduling (push/pull)
|
|
+-----+----------+ - manages clock/query/seek
|
|
| - manages scheduling of samples in the ringbuffer
|
|
| - manages caps parsing
|
|
|
|
|
+-----V------+ - default ringbuffer implementation with a GThread
|
|
+ audiosink + - subclasses provide open/read/close methods
|
|
+------------+
|
|
```
|
|
|
|
The ringbuffer is a contiguous piece of memory divided into segtotal
|
|
pieces of segments. Each segment has segsize bytes.
|
|
|
|
```
|
|
play position
|
|
v
|
|
+---+---+---+-------------------------------------+----------+
|
|
+ 0 | 1 | 2 | .... | segtotal |
|
|
+---+---+---+-------------------------------------+----------+
|
|
<--->
|
|
segsize bytes = N samples * bytes_per_sample.
|
|
```
|
|
|
|
The ringbuffer has a play position, which is expressed in segments. The
|
|
play position is where the device is currently reading samples from the
|
|
buffer.
|
|
|
|
The ringbuffer can be put to the `PLAYING` or `STOPPED` state.
|
|
|
|
In the `STOPPED` state no samples are played to the device and the play
|
|
pointer does not advance.
|
|
|
|
In the `PLAYING` state samples are written to the device and the
|
|
ringbuffer should call a configurable callback after each segment is
|
|
written to the device. In this state the play pointer is advanced after
|
|
each segment is written.
|
|
|
|
A write operation to the ringbuffer will put new samples in the
|
|
ringbuffer. If there is not enough space in the ringbuffer, the write
|
|
operation will block. The playback of the buffer never stops, even if
|
|
the buffer is empty. When the buffer is empty, silence is played by the
|
|
device.
|
|
|
|
The ringbuffer is implemented with lockfree atomic operations,
|
|
especially on the reading side so that low-latency operations are
|
|
possible.
|
|
|
|
Whenever new samples are to be put into the ringbuffer, the position of
|
|
the read pointer is taken. The required write position is taken and the
|
|
diff is made between the required and actual position. If the difference
|
|
is `< 0`, the sample is too late. If the difference is bigger than
|
|
segtotal, the writing part has to wait for the play pointer to advance.
|
|
|
|
### Scheduling
|
|
|
|
#### chain based mode
|
|
|
|
In chain based mode, bytes are written into the ringbuffer. This
|
|
operation will eventually block when the ringbuffer is filled.
|
|
|
|
When no samples arrive in time, the ringbuffer will play silence. Each
|
|
buffer that arrives will be placed into the ringbuffer at the correct
|
|
times. This means that dropping samples or inserting silence is done
|
|
automatically and very accurate and independent of the play pointer.
|
|
|
|
In this mode, the ringbuffer is usually kept as full as possible. When
|
|
using a small buffer (small segsize and segtotal), the latency for audio
|
|
to start from the sink to when it is played can be kept low but at least
|
|
one context switch has to be made between read and write.
|
|
|
|
#### getrange based mode
|
|
|
|
In getrange based mode, the audiobasesink will use the callback
|
|
function of the ringbuffer to get a segsize samples from the peer
|
|
element. These samples will then be placed in the ringbuffer at the
|
|
next play position. It is assumed that the getrange function returns
|
|
fast enough to fill the ringbuffer before the play pointer reaches
|
|
the write pointer.
|
|
|
|
In this mode, the ringbuffer is usually kept as empty as possible.
|
|
There is no context switch needed between the elements that create
|
|
the samples and the actual writing of the samples to the device.
|
|
|
|
#### DMA mode
|
|
|
|
Elements that can do DMA based access to the audio device have to
|
|
subclass from the `GstAudioBaseSink` class and wrap the DMA ringbuffer
|
|
in a subclass of `GstRingBuffer`.
|
|
|
|
The ringbuffer subclass should trigger a callback after writing or
|
|
playing each sample to the device. This callback can be triggered
|
|
from a thread or from a signal from the audio device.
|
|
|
|
### Clocks
|
|
|
|
The `GstAudioBaseSink` class will use the ringbuffer to act as a clock
|
|
provider. It can do this by using the play pointer and the delay to
|
|
calculate the clock time.
|