mirror of
https://gitlab.freedesktop.org/gstreamer/gstreamer.git
synced 2024-12-23 08:46:40 +00:00
docs/design/: Docs updates.
Original commit message from CVS: * docs/design/draft-ghostpads.txt: * docs/design/draft-push-pull.txt: * docs/design/draft-query.txt: * docs/design/part-overview.txt: Docs updates. Added general overview doc. Added draft ghostpad replacement idea.
This commit is contained in:
parent
489398a1db
commit
a7f3844adf
5 changed files with 629 additions and 3 deletions
|
@ -1,3 +1,11 @@
|
|||
2005-05-24 Wim Taymans <wim@fluendo.com>
|
||||
|
||||
* docs/design/draft-ghostpads.txt:
|
||||
* docs/design/draft-push-pull.txt:
|
||||
* docs/design/draft-query.txt:
|
||||
* docs/design/part-overview.txt:
|
||||
Docs updates, added general overview doc.
|
||||
|
||||
2005-05-21 David Schleef <ds@schleef.org>
|
||||
|
||||
* docs/gst/tmpl/old/GstBin.sgml:
|
||||
|
|
108
docs/design/draft-ghostpads.txt
Normal file
108
docs/design/draft-ghostpads.txt
Normal file
|
@ -0,0 +1,108 @@
|
|||
Ghostpads
|
||||
---------
|
||||
|
||||
Status:
|
||||
|
||||
DRAFT.
|
||||
|
||||
|
||||
Purpose:
|
||||
|
||||
To create compound elements that look and behave like real elements
|
||||
one needs to be able to expose pads on the element that are not
|
||||
implemented by the element itself but by one of its components.
|
||||
|
||||
ex1:
|
||||
|
||||
Consider an rtp receiver element. It uses 2 UDP input ports to
|
||||
capture RTP and RTCP messages using UDP. These messages are
|
||||
then processed by an rtpsession element, which will reorder, make
|
||||
statistics, and generate the RTP and RTCP messages.
|
||||
|
||||
This element can be build from existing elements as shown in the
|
||||
figure below.
|
||||
|
||||
+------------------------------------+
|
||||
| rtpsrc |
|
||||
| |
|
||||
| +--------+ +---------+ |
|
||||
| | udpsrc | | rtpsess | |
|
||||
| | port1 --- ----------------
|
||||
| +--------+ | | |
|
||||
| +--------+ | | +---------+ |
|
||||
| | udpsrc | | | | udpsink | |
|
||||
| | port2 --- --- port3 | |
|
||||
| +--------+ +---------+ +---------+ |
|
||||
+------------------------------------+
|
||||
|
||||
The element has one output pad that contains the raw RTP data. This
|
||||
pad will be connected to the next element that will decode the RTP
|
||||
data. Since this pad is actually from the rtpsession element, there
|
||||
has to be a way to expose this pad on the rtpsrc element.
|
||||
|
||||
|
||||
Current implementation:
|
||||
|
||||
Version 0.8 creates a new GstGhostPad type that extends GstPad and has
|
||||
a link to the real pad internally.
|
||||
|
||||
Any operation on a pad potentially requires to check if this pad is a
|
||||
GhostPad and if so, follow the pointer to the real pad to perform the
|
||||
pad operation.
|
||||
|
||||
|
||||
Current problem:
|
||||
|
||||
- following the ghostpad to resolve the real pad adds code.
|
||||
- operations on pads are not performed on the ghostpad but on the
|
||||
real pad. This means:
|
||||
- pads are not linked to a ghostpad but to the real pad. For
|
||||
compound objects, the link is not really performed to the
|
||||
pad of the compound element but with some internal pad.
|
||||
- for state changes, it is hard to follow linked elements upstream
|
||||
bacause pads enter bins to the real element.
|
||||
|
||||
|
||||
Proposal:
|
||||
|
||||
- The pad still has one parent, the element that owns (has sinked) the
|
||||
pad. the GST_OBJECT_PARENT() is the real parent.
|
||||
|
||||
- The pad receives a GList of ghostparents. Ghostparents are sorted by depth.
|
||||
|
||||
- When a pad is ghostparented to another element, the element is added to
|
||||
the ghostparent list of the pad. The pad is added to the padlist of the
|
||||
element.
|
||||
|
||||
- A pad cannot be ghostparented to an element that is not a parent of the
|
||||
pad parent element. this ensures that all ghostpads do not skip an hierarchy.
|
||||
|
||||
- When a pad is removed from the parent element, it is also removed from all
|
||||
the ghostparents. This is easy to do as you can follow the ghostparent list of
|
||||
that pad.
|
||||
|
||||
- When an element is removed from another element, all the elements pads
|
||||
ghostparented to parents of the element are removed as well. This is easy
|
||||
to do with the sorted ghostparent list of the pad.
|
||||
|
||||
|
||||
Consequences:
|
||||
|
||||
- All pads are Real pads.
|
||||
- All operations are performed on the real pads. This improves performance
|
||||
and code simplicity
|
||||
- more checks can be done. Linking nested elements can be done sanely.
|
||||
compount elements that expose usable pads must ghostpad them before they
|
||||
can be linked.
|
||||
- All linked pads have a common grandparent.
|
||||
|
||||
|
||||
Issues:
|
||||
|
||||
- Name of ghostpad cannot be changed and is the name of the element. This could
|
||||
be solved by temporarily renaming the pad when ghostpadding. This could then
|
||||
again result in the element being confused about the padnames or the
|
||||
element having two pads with the same name.
|
||||
|
||||
|
||||
|
|
@ -1,6 +1,10 @@
|
|||
DRAFT push-pull scheduling
|
||||
--------------------------
|
||||
|
||||
Status
|
||||
|
||||
DRAFT
|
||||
|
||||
Observations:
|
||||
|
||||
- The main scheduling mode is chain based scheduling where the source
|
||||
|
@ -34,12 +38,13 @@ Types of pull based scheduling:
|
|||
- some sources can do random access (file source, ...)
|
||||
|
||||
- some sources can read a random number of bytes but not at a random
|
||||
offset. (audio cards, ...)
|
||||
offset. (audio cards, ...) Audio cards using a ringbuffer can
|
||||
however do random access in the ringbuffer.
|
||||
|
||||
- some sources can do random access in a range of bytes but not in
|
||||
another range. (a caching network source).
|
||||
|
||||
- some sources can do a fixed size data but without an offset.
|
||||
- some sources can do a fixed size data and without an offset.
|
||||
(video sources, ...)
|
||||
|
||||
|
||||
|
@ -54,7 +59,8 @@ Current scheduling decision:
|
|||
Problems:
|
||||
|
||||
- core makes a touch desicion without knowing anything about the
|
||||
element.
|
||||
element. Some elements are able to deal with a pull_range()
|
||||
without offset while others need full random access.
|
||||
|
||||
|
||||
Requirements:
|
||||
|
|
|
@ -1,6 +1,10 @@
|
|||
DRAFT Query
|
||||
-----------
|
||||
|
||||
Status
|
||||
|
||||
Implemented, move me to design...
|
||||
|
||||
Purpose
|
||||
|
||||
Queries are used to get information about the stream.
|
||||
|
|
500
docs/design/part-overview.txt
Normal file
500
docs/design/part-overview.txt
Normal file
|
@ -0,0 +1,500 @@
|
|||
Overview
|
||||
--------
|
||||
|
||||
This part gives an overview of the design of GStreamer with references to
|
||||
the more detailed explanations of the different topics.
|
||||
|
||||
This document is intented for people that want to have a global overview of
|
||||
the inner workings of GStreamer.
|
||||
|
||||
|
||||
Introduction
|
||||
------------
|
||||
|
||||
GStreamer is a set of libraries and plugins that can be used to implement various
|
||||
multimedia applications ranging from desktop players, audio/video recorders,
|
||||
multimedia servers, transcoders, etc.
|
||||
|
||||
Applications are built by constructing a pipeline composed of elements. An element
|
||||
is an object that performs some action on a multimedia stream such as:
|
||||
|
||||
- read a file
|
||||
- decode or encoder between formats
|
||||
- capture from a hardware device
|
||||
- render to a hardware device
|
||||
- mix or multiplex multiple streams
|
||||
|
||||
Elements have input and output pads called sink and source pads in GStreamer. An
|
||||
application links elements together on pads to construct a pipeline. Below is
|
||||
an example of an ogg/vorbis playback pipeline.
|
||||
|
||||
+-----------------------------------------------------------+
|
||||
| pipeline |
|
||||
| +---------+ +----------+ +-----------+ +----------+ |
|
||||
| | filesrc | | oggdemux | | vorbisdec | | alsasink | |
|
||||
| | src-sink src-sink src-sink | |
|
||||
| +---------+ +----------+ +-----------+ +----------+ |
|
||||
+-----------------------------------------------------------+
|
||||
|
||||
The filesrc element reads data from a file on disk. The oggdemux element parses
|
||||
the data and sends the compressed audio data to the vorbisdec element. The
|
||||
vorbisdec element decodes the compressed data and sends it to the alsasink
|
||||
element. The alsasink element sends the samples to the audio card for playback.
|
||||
|
||||
The task of the application is to construct a pipeline as above using existing
|
||||
elements. This is further explained in the pipeline building topic.
|
||||
|
||||
The application does not have to manage any of the complexities of the
|
||||
actual dataflow/decoding/conversions/synchronsiation etc. but only calls high
|
||||
level functions on the pipeline object such as PLAY/PAUSE/STOP.
|
||||
|
||||
The application also receives messages and notifications from the pipeline such
|
||||
as metadata, warning or error messages.
|
||||
|
||||
If the application needs more control over the graph it is possible to directly
|
||||
access the elements and pads in the pipeline.
|
||||
|
||||
|
||||
Design overview
|
||||
---------------
|
||||
|
||||
GStreamer design goals include:
|
||||
|
||||
- Process large amounts of data quickly
|
||||
- Allow fully multithreaded processing
|
||||
- Ability to deal with multiple formats
|
||||
- Synchronize different dataflows
|
||||
- Ability to deal with multiple devices
|
||||
|
||||
The capabilities presented to the application depends on the number of elements
|
||||
installed on the system and their functionality.
|
||||
|
||||
The GStreamer core is designed to be media agnostic but provides many features
|
||||
to elements to describe media formats.
|
||||
|
||||
|
||||
Elements
|
||||
--------
|
||||
|
||||
The smallest building blocks in a pipeline are elements. An element provides a
|
||||
number of pads which can be source or sinkpads. Sourcepads provide data and
|
||||
sinkpads consume data. Below is an example of an ogg demuxer element that has
|
||||
one pad that takes (sinks) data and two source pads that produce data.
|
||||
|
||||
+-----------+
|
||||
| oggdemux |
|
||||
| src0
|
||||
sink src1
|
||||
+-----------+
|
||||
|
||||
An element can be in four different states: NULL, READY, PAUSED, PLAYING. In the
|
||||
NULL and READY state, the element is not processing any data. In the PLAYING state
|
||||
it is processing data. The intermediate PAUSED state is used to preroll data in
|
||||
the pipeline. A state change can be performed with gst_element_set_state().
|
||||
|
||||
An element always goes through all the intermediate state changes. This means that
|
||||
when en element is in the READY state and is put to PLAYING, it will first go
|
||||
through the intermediate PAUSED state.
|
||||
|
||||
An element state change to PAUSED will activate the pads of the element. First the
|
||||
source pads are activated, then the sinkpads. When the pads are activated, the
|
||||
pad activate function is called. Some pads will start a thread or some other
|
||||
mechanism to start producing or consuming data.
|
||||
|
||||
The PAUSED state is special as it is used to preroll data in the pipeline. The purpose
|
||||
is to fill all connected elements in the pipeline with data so that the subsequent
|
||||
PLAYING state change happens very quickly. Some elements will therefore not complete
|
||||
the state change to PAUSED before they have received enough data. Sink elements are
|
||||
required to only complete the state change to PAUSED after receiving the first data.
|
||||
|
||||
Normally the state changes of elements are coordinated by the pipeline as explained
|
||||
in [part-states.txt].
|
||||
|
||||
Different categories of elements exist:
|
||||
|
||||
- source elements, these are elements that do not consume data but only provide data
|
||||
for the pipeline.
|
||||
- sink elements, these are elements that do not produce data but renders data to
|
||||
an output device.
|
||||
- transform elements, these elements transform an input stream in a certain format
|
||||
into a stream of another format. Encoder/decoder/converters are examples.
|
||||
- demuxer elements, these elements parse a stream and produce several output streams.
|
||||
- mixer/muxer elements, combine several input streams into one output stream.
|
||||
|
||||
Other categories of elements can be constructed.
|
||||
|
||||
|
||||
Bins
|
||||
----
|
||||
|
||||
A bin is an element subclass and acts as a container for other elements so that multiple
|
||||
elements can be combined into one element.
|
||||
|
||||
A bin coordinates its children's state changes as explained later. It also distributes
|
||||
events and various other functionality to elements.
|
||||
|
||||
A bin can have its own source and sinkpads by ghostpadding one or more of its children's
|
||||
pads to itself.
|
||||
|
||||
Below is a picture of a bin with two elements. The sinkpad of one element is ghostpadded
|
||||
to the bin.
|
||||
|
||||
+---------------------------+
|
||||
| bin |
|
||||
| +--------+ +-------+ |
|
||||
| | | | | |
|
||||
| /sink src-sink | |
|
||||
sink +--------+ +-------+ |
|
||||
+---------------------------+
|
||||
|
||||
|
||||
Pipeline
|
||||
--------
|
||||
|
||||
A pipeline is a special bin subclass that provides the following features to its
|
||||
children:
|
||||
|
||||
- Select and manage a clock
|
||||
- Provide means for elements to comunicate with the application by the Bus.
|
||||
- Manage the global state of the elements such as Errors and end-of-stream.
|
||||
|
||||
Normally the application creates one pipeline that will manage all the elements
|
||||
in the application.
|
||||
|
||||
|
||||
Dataflow and buffers
|
||||
--------------------
|
||||
|
||||
GStreamer supports two possible types of dataflow, the push and pull model. In the
|
||||
push model, an upstream element sends data to a downstream element by calling a
|
||||
method on a sinkpad. In the pull model, a downstream element requests data from
|
||||
an upstream element by calling a method on a source pad.
|
||||
|
||||
The most common dataflow is the push model. The pull model can be used in specific
|
||||
circumstances by demuxer elements. The pull model can also be used by low latency
|
||||
audio applications.
|
||||
|
||||
The data passed between pads is encapsulated in Buffers. The buffer contains a
|
||||
pointer to the actual data and also metadata describing the data. This metadata
|
||||
includes:
|
||||
|
||||
- timestamp of the data, this is the time instance at which the data was captured
|
||||
or the time at which the data should be played back.
|
||||
- offset of the data: a media specific offset, this could be samples for audio or
|
||||
frames for video.
|
||||
- the duration of the data in time.
|
||||
- the media type of the data described with caps, these are key/value pairs that
|
||||
describe the media type in a unique way.
|
||||
|
||||
When an element whishes to send a buffer to another element is does this using one
|
||||
of the pads that is linked to a pad of the other element. In the push model, a
|
||||
buffer is pushed to the peer pad with gst_pad_push(). In the pull model, a buffer
|
||||
is pulled from the peer with the gst_pad_pull_region() function.
|
||||
|
||||
Before an element pushes out a buffer, it should make sure that the peer element
|
||||
can understand the buffer contents. It does this by querying the peer element
|
||||
for the supported formats and by selecting a suitable common format. The selected
|
||||
format is then attached to the buffer with gst_buffer_set_caps() before pushing
|
||||
out the buffer.
|
||||
|
||||
When an element pad receives a buffer, if has to check if it understands the media
|
||||
type of the buffer before starting processing it.
|
||||
|
||||
Both gst_pad_push() and gst_pad_pull_range() have a return value indicating wether
|
||||
the operation succeeded. An error code means that no more data should be send
|
||||
to that pad. A source element that initiates the data flow in a thread typically
|
||||
pauses the producing thread when this happens.
|
||||
|
||||
A buffer can be created with gst_buffer_new() or by requesting a usable buffer
|
||||
from the peer pad using gst_pad_alloc_buffer(). Using the second method, it is
|
||||
possible for the peer element to suggest the element to produce data in another
|
||||
format by attaching another media type caps to the buffer.
|
||||
|
||||
The process of selecting a media type and attaching it to the buffers is called
|
||||
caps negotiation.
|
||||
|
||||
|
||||
Caps
|
||||
----
|
||||
|
||||
A media type (Caps) is described using a generic list of key/value pairs. The key is
|
||||
a string and the value can be a single/list/range of int/float/string.
|
||||
|
||||
Caps that have no ranges/list or other variable parts are said to be fixed and
|
||||
can be used to put on a buffer.
|
||||
|
||||
Caps with variables in them are used to describe possible media types that can be
|
||||
handled by a pad.
|
||||
|
||||
|
||||
Dataflow and events
|
||||
-------------------
|
||||
|
||||
Parallel to the dataflow is a flow of events. Unlike the buffers, events can pass
|
||||
upstream and downstream. Some events only travel upstream others only downstream.
|
||||
|
||||
the events are used to denote special conditions in the dataflow such as EOS or
|
||||
to inform plugins of special events such as flushing or seeking.
|
||||
|
||||
|
||||
Pipeline construction
|
||||
---------------------
|
||||
|
||||
The application starts by creating a Pipeline element using gst_pipeline_new ().
|
||||
Elements are added to and removed from the pipeline with gst_bin_add() and
|
||||
gst_bin_remove().
|
||||
|
||||
After adding the elements, the pads of an element can be retrieved with
|
||||
gst_element_get_pad(). Pads can then be linked together with gst_pad_link().
|
||||
|
||||
Some elements create new pads when actual dataflow is happening in the pipeline.
|
||||
With g_signal_connect() one can receive a notification when an element has created
|
||||
a pad. These new pads can then be linked to other unlinked pads.
|
||||
|
||||
Some elements cannot be linked together because they operate on different
|
||||
incompatible data types. The possible datatypes a pad can provide or consume can
|
||||
be retrieved with gst_pad_get_caps().
|
||||
|
||||
Below is a simple mp3 playback pipeline that we constructed. We will use this
|
||||
pipeline in further examples.
|
||||
|
||||
+-------------------------------------------+
|
||||
| pipeline |
|
||||
| +---------+ +----------+ +----------+ |
|
||||
| | filesrc | | mp3dec | | alsasink | |
|
||||
| | src-sink src-sink | |
|
||||
| +---------+ +----------+ +----------+ |
|
||||
+-------------------------------------------+
|
||||
|
||||
|
||||
Pipeline clock
|
||||
--------------
|
||||
|
||||
One of the important functions of the pipeline is to select a global clock
|
||||
for all the elements in the pipeline.
|
||||
|
||||
Before the pipeline is set to PAUSED, the pipeline asks each element if they can
|
||||
provide a clock. The clock is selected in the following order:
|
||||
|
||||
- If the application selected a clock, us that one.
|
||||
- If a source element provides a clock, use that clock.
|
||||
- Select a clock from any other element that provides a clock, start with the
|
||||
sinks.
|
||||
- If no element provides a clock a default system clock is used for the pipeline.
|
||||
|
||||
In a typical playback pipeline this will select the clock provided by a sink element
|
||||
such as an audio sink.
|
||||
|
||||
|
||||
Pipeline states
|
||||
---------------
|
||||
|
||||
When all the pads are linked or signals have been connected, the pipeline can
|
||||
be put in the PAUSED state to start dataflow.
|
||||
|
||||
When a bin (and hence a pipeline) performs a state change, it will change the state
|
||||
of all its children. The pipeline will change the state of its children from the
|
||||
sink elements to the source elements, this to make sure that no upstream element
|
||||
produces data to an element that is not yet ready to accept it.
|
||||
|
||||
In the mp3 playback pipeline, the state of the elements is changed in the order
|
||||
alsasink, mp3dec, filesrc.
|
||||
|
||||
All intermediate states are traversed for each element resulting in the following
|
||||
chain of state changes:
|
||||
|
||||
alsasink to READY: the audio device is opened
|
||||
mp3dec to READY: the decoding library is initialized
|
||||
filesrc to READY: the file is opened
|
||||
alsasink to PAUSED: alsasink is a sink and returns ASYNC because it did not receive
|
||||
data yet.
|
||||
mp3dec to PAUSED: nothing happens
|
||||
filesrc to PAUSED: a thread is started to push data to mp3dec
|
||||
|
||||
At this point data flows from filesrc to mp3dec and alsasink. Since mp3dec is PAUSED,
|
||||
it accepts the data from filesrc on the sinkpad and starts decoding the compressed
|
||||
data to raw audio samples.
|
||||
|
||||
The mp3 decoder figures out the samplerate, the number of channels and other audio
|
||||
properties of the raw audio samples, puts the decoded samples into a Buffer,
|
||||
attaches the media type caps to the buffer and pushes this buffer to the next
|
||||
element.
|
||||
|
||||
Alsasink then receives the buffer, inspects the caps and reconfigures itself to process
|
||||
the buffer. Since it received the first buffer of samples, it completes the state change
|
||||
to the PAUSED state. At this point the pipeline is prerolled and all elements have
|
||||
samples.
|
||||
|
||||
Since alsasink is now in the PAUSED state it blocks while receiving the first buffer. This
|
||||
effectively blocks both mp3dec and filesrc in their gst_pad_push().
|
||||
|
||||
Since all elements now return SUCCESS from the gst_element_get_state() function,
|
||||
the pipeline can be put in the PLAYING state.
|
||||
|
||||
Before going to PLAYING, the pipeline samples the current time of the clock. This is
|
||||
the base time. It then distributes this time to all elements. Elements can then
|
||||
synchronize against the clock using the buffer timestamp+base time.
|
||||
|
||||
The following chain of state changes then takes place:
|
||||
|
||||
alsasink to PLAYING: the samples are played to the audio device
|
||||
mp3dec to PLAYING: nothing happens
|
||||
filesrc to PLAYING: nothing happens
|
||||
|
||||
|
||||
Pipeline status
|
||||
---------------
|
||||
|
||||
The pipeline informs the application of any special events that occur in the
|
||||
pipeline with the bus. The bus is an object that the pipeline provides and that
|
||||
can be retrieved with gst_pipeline_get_bus().
|
||||
|
||||
The bus can be polled or added to the glib mainloop.
|
||||
|
||||
The bus is distributed to all elements added to the pipeline. The elements use the bus
|
||||
to post messages on. Various message types exist such as ERRORS, WARNINGS, EOS,
|
||||
STATE_CHANGED, etc..
|
||||
|
||||
The pipeline handles EOS messages received from elements in a special way. It will
|
||||
only forward the message to the application when all sink elements have posted an
|
||||
EOS message.
|
||||
|
||||
Other methods for obtaining the pipeline status include the Query functionality that
|
||||
can be performed with gst_element_query() on the pipeline. This type of query
|
||||
is usefull for obtaining information about the current position and total time of
|
||||
the pipeline. It can also be used to query for the supported seeking formats and
|
||||
ranges.
|
||||
|
||||
|
||||
Pipeline EOS
|
||||
------------
|
||||
|
||||
When the source filter encounters the end of the stream, it sends an EOS event to
|
||||
the peer element. This event will then travel downstream to all of the connected
|
||||
elements to inform them of the EOS. The element is not supposed to accept any more
|
||||
data after receiving an EOS event on a sinkpad.
|
||||
|
||||
The source element stops sending data after sending the EOS message.
|
||||
|
||||
The EOS even will eventually arrive in the sink element. The sink will then post
|
||||
an EOS message on the bus to inform the pipeline that a particular stream has
|
||||
finished. When all sinks have reported EOS, the pipeline forwards the EOS message
|
||||
to the application.
|
||||
|
||||
When in EOS, the pipeline remains in the playing state, if is the application
|
||||
responsability to PAUSE or READY the pipeline.
|
||||
|
||||
|
||||
Pipeline READY
|
||||
--------------
|
||||
|
||||
When a running pipeline is set from the RUNNING to READY the following actions
|
||||
occur in the pipeline:
|
||||
|
||||
alsasink to PAUSED: alsasink blocks and completes the state change on the
|
||||
next sample. If the element was EOS, it does not wait for
|
||||
a sample to complete the state change.
|
||||
mp3dec to PAUSED: nothing
|
||||
filesrc to PAUSED: nothing
|
||||
|
||||
Going to the intermediate PAUSED state will block all elements in the _push()
|
||||
functions.
|
||||
|
||||
Some elements might be performing blocking operations in the PLAYING state that
|
||||
must be unblocked when they go into the PAUSED state. This makes sure that the
|
||||
state change happens very fast.
|
||||
|
||||
In the next PAUSED to READY state change the pipeline has to shut down and all
|
||||
streaming threads must stop sending data. This happens in the following sequence:
|
||||
|
||||
alsasink to READY: alsasink unblocks from the _chain() function and returns a
|
||||
WRONG_STATE return value to the peer element. The sinkpad is
|
||||
deactivated and becomes unusable for sending more data.
|
||||
mp3dec to READY: the pads are deactivated and the state change completes when
|
||||
mp3dec leaves its _chain() function.
|
||||
filesrc to PAUSED: the pads are deactivated and the thread is paused.
|
||||
|
||||
The upstream elements finish their chain() function because the downstream element
|
||||
returned an error code from the _push() functions. These error codes are eventually
|
||||
returned to the element that started the streaming thread (filesrc), which pauses
|
||||
the thread and completes the state change.
|
||||
|
||||
This sequence of events ensure that all elements are unblocked and all streaming
|
||||
threads stopped.
|
||||
|
||||
|
||||
Pipeline seeking
|
||||
----------------
|
||||
|
||||
Seeking in the pipeline requires a very specific order of operations to make
|
||||
sure that the elements remain synchronized and that the seek is performed with
|
||||
a minimal amount of latency.
|
||||
|
||||
An application issues a seek event on the pipeline using gst_element_send_event()
|
||||
on the pipeline element. The event can be a seek event in any of the formats
|
||||
supported by the elements.
|
||||
|
||||
The pipeline first pauses the pipeline to speed up the seek operations.
|
||||
|
||||
The pipeline then issues the seek event to all sink elements. The sink then forwards
|
||||
the seek event upstream until some element can perform the seek operation, which is
|
||||
typically the source or demuxer element. All intermediate elements can transform the
|
||||
requested seek offset to another format, this way a decoder element can transform a
|
||||
seek to a frame number to a timestamp, for example.
|
||||
|
||||
When the seek event reaches an element that will perform the seek operation, that
|
||||
element performs the following steps.
|
||||
|
||||
1) send a FLUSH event to all downstream and upstream peer elements.
|
||||
2) make sure the streaming thread is not running.
|
||||
3) perform the seek operation
|
||||
4) send a FLUSH done event to all downstream and upstream peer elements.
|
||||
5) send DISCONT event to inform all elements of the new position and to complete
|
||||
the seek.
|
||||
|
||||
In step 1) all dowstream elements have to return from any blocking operations
|
||||
and have to refuse any further buffers or events different from a FLUSH done.
|
||||
|
||||
The first step ensures that the streaming thread eventually unblocks and that
|
||||
step 2) can be performed. At this point, dataflow is completely stopped in the
|
||||
pipeline.
|
||||
|
||||
In step 3) the element performs the seek to the requested position.
|
||||
|
||||
In step 4) all peer elements are allowed to accept data again and streaming
|
||||
can continue from the new position. A FLUSH done event is sent to all the peer
|
||||
elements so that they accept new data again and restart their streaming threads.
|
||||
|
||||
Step 5) informs all elements of the new position in the stream. After that the
|
||||
event function returns back to the application. and the streaming threads start
|
||||
to produce new data.
|
||||
|
||||
Since the pipeline is still PAUSED, this will preroll the next media sample in the
|
||||
sinks.
|
||||
|
||||
The last step in the seek operation is then to adjust the media time of the pipeline
|
||||
to the new position and to set the pipeline back to PLAYING.
|
||||
|
||||
The sequence of events in out mp3 playback example.
|
||||
|
||||
| a) seek on pipeline
|
||||
| b) PAUSE pipeline
|
||||
+----------------------------------V--------+
|
||||
| pipeline | c) seek on sink
|
||||
| +---------+ +----------+ +---V------+ |
|
||||
| | filesrc | | mp3dec | | alsasink | |
|
||||
| | src-sink src-sink | |
|
||||
| +---------+ +----------+ +----|-----+ |
|
||||
+-----------------------------------|-------+
|
||||
<------------------------+
|
||||
d) seek travels upstream
|
||||
|
||||
--------------------------> 1) FLUSH event
|
||||
| 2) stop streaming
|
||||
| 3) perform seek
|
||||
--------------------------> 4) FLUSH done event
|
||||
--------------------------> 5) DISCONT event
|
||||
|
||||
| e) update stream time
|
||||
| f) PLAY pipeline
|
||||
|
Loading…
Reference in a new issue