gstreamer/subprojects/gst-docs/markdown/additional/design/streams.md

Streams

This document describes the objects that are passed from element to element in the streaming thread.

Stream objects

The following objects are to be expected in the streaming thread:

• events, including:
  • STREAM_START (START)
  • SEGMENT (SEGMENT)
  • EOS * (EOS)
  • TAG (T)
• buffers * (B)

Objects marked with * need to be synchronised to the clock in sinks and live sources.

Typical stream

A typical stream starts with a STREAM_START event that marks the start of the stream, followed by a SEGMENT event that marks the buffer timestamp range. After that buffers are sent one after the other. After the last buffer an EOS event marks the end of the stream. No more buffers are to be processed after the EOS event.

+-----+-------+ +-++-+     +-+ +---+
|START|SEGMENT| |B||B| ... |B| |EOS|
+-----+-------+ +-++-+     +-+ +---+

1. STREAM_START

   • marks the start of a stream; unlike the SEGMENT event, there will be no STREAM_START event after flushing seeks.

2. SEGMENT, rate, start/stop, time. (See also Segments)

   • marks valid buffer timestamp range (start, stop)
   • marks stream_time of buffers (time). This is the stream time of buffers with a timestamp of S.start.
   • marks playback rate (rate). This is the required playback rate.
   • marks applied rate (applied_rate). This is the already applied playback rate. (See also trickmodes)
   • marks running_time of buffers. This is the time used to synchronize against the clock.

3. N buffers

   • displayable buffers are between start/stop of the SEGMENT (S). Buffers outside the segment range should be dropped or clipped.

   • running_time: For each buffer, a monotonically increasing value that can be used to synchronize against the clock (See also synchronisation).

     if (S.rate > 0.0)
       running_time = (B.timestamp - S.start) / ABS (S.rate) + S.base;
     else
       running_time = (S.stop - B.timestamp) / ABS (S.rate) + S.base;
     

   • stream_time: The current position in the stream, between 0 and duration.

     stream_time = (B.timestamp - S.start) * ABS (S.applied_rate) + S.time;
     

4. EOS

   • marks the end of data, nothing is to be expected after EOS, elements should refuse more data and return GST_FLOW_EOS. A FLUSH_STOP event clears the EOS state of an element.

Elements

These events are generated typically either by the GstBaseSrc class for sources operating in push mode, or by a parser/demuxer operating in pull-mode and pushing parsed/demuxed data downstream.