mirror of
https://gitlab.freedesktop.org/gstreamer/gstreamer.git
synced 2024-11-22 17:51:16 +00:00
docs/design/: Some doc updates. Start renaming from stream_time to running_time where it was used wrongly.
Original commit message from CVS: * docs/design/part-TODO.txt: * docs/design/part-activation.txt: * docs/design/part-block.txt: * docs/design/part-buffering.txt: * docs/design/part-clocks.txt: * docs/design/part-element-source.txt: * docs/design/part-events.txt: * docs/design/part-gstbin.txt: * docs/design/part-gstbus.txt: * docs/design/part-gstpipeline.txt: * docs/design/part-live-source.txt: * docs/design/part-messages.txt: * docs/design/part-overview.txt: * docs/design/part-qos.txt: * docs/design/part-query.txt: * docs/design/part-states.txt: * docs/design/part-trickmodes.txt: Some doc updates. Start renaming from stream_time to running_time where it was used wrongly.
This commit is contained in:
Wim Taymans
parent
2d4c842ab7
commit
46d2c100f8
18 changed files with 287 additions and 141 deletions
22
ChangeLog
22
ChangeLog
|
|
@ -1,3 +1,25 @@
|
|||
2007-02-15 Wim Taymans <wim@fluendo.com>
|
||||
|
||||
* docs/design/part-TODO.txt:
|
||||
* docs/design/part-activation.txt:
|
||||
* docs/design/part-block.txt:
|
||||
* docs/design/part-buffering.txt:
|
||||
* docs/design/part-clocks.txt:
|
||||
* docs/design/part-element-source.txt:
|
||||
* docs/design/part-events.txt:
|
||||
* docs/design/part-gstbin.txt:
|
||||
* docs/design/part-gstbus.txt:
|
||||
* docs/design/part-gstpipeline.txt:
|
||||
* docs/design/part-live-source.txt:
|
||||
* docs/design/part-messages.txt:
|
||||
* docs/design/part-overview.txt:
|
||||
* docs/design/part-qos.txt:
|
||||
* docs/design/part-query.txt:
|
||||
* docs/design/part-states.txt:
|
||||
* docs/design/part-trickmodes.txt:
|
||||
Some doc updates. Start renaming from stream_time to running_time where
|
||||
it was used wrongly.
|
||||
|
||||
2007-02-15 Wim Taymans <wim@fluendo.com>
|
||||
|
||||
* libs/gst/base/gstbasesrc.c: (gst_base_src_default_query):
|
||||
|
|
|
|||
|
|
@ -22,11 +22,13 @@ API/ABI
|
|||
- use | instead of + as divider in serialization of Flags
|
||||
(gstvalue/gststructure)
|
||||
|
||||
- with the addition the PREROLLED message, we can probably get rid of the
|
||||
STATE_DIRTY message.
|
||||
|
||||
|
||||
IMPLEMENTATION
|
||||
--------------
|
||||
|
||||
- implement latency calculation for live sources.
|
||||
|
||||
- implement more QOS, see part-qos.txt.
|
||||
|
||||
- implement BUFFERSIZE.
|
||||
|
|
|
|||
|
|
@ -6,6 +6,10 @@ sink-to-source order. As elements undergo the READY->PAUSED transition,
|
|||
their pads are activated so as to prepare for data flow. Some pads will
|
||||
start tasks to drive the data flow.
|
||||
|
||||
An element activates its pads from sourcepads to sinkpads. This to make
|
||||
sure that when the sinkpads are activated an ready to accept data, the
|
||||
sourcepads are already active to pass the data downstream.
|
||||
|
||||
Pads can be activated in one of two modes, PUSH and PULL. PUSH pads are
|
||||
the normal case, where the source pad in a link sends data to the sink
|
||||
pad via gst_pad_push(). PULL pads instead have sink pads request data
|
||||
|
|
@ -71,12 +75,14 @@ pushing data.
|
|||
Deactivation
|
||||
------------
|
||||
|
||||
Pad deactivation occurs when a pad is unlinked, or when its parent goes
|
||||
into the READY state. gst_pad_set_active() is called with a FALSE
|
||||
argument, which then calls activate_push() or activate_pull() with a
|
||||
FALSE argument, depending on the activation mode of the pad.
|
||||
Pad deactivation occurs when its parent goes into the READY state or when the
|
||||
pad is deactivated explicitly by the application or element.
|
||||
gst_pad_set_active() is called with a FALSE argument, which then calls
|
||||
activate_push() or activate_pull() with a FALSE argument, depending on the
|
||||
activation mode of the pad.
|
||||
|
||||
Mode switching
|
||||
--------------
|
||||
|
||||
Changing from push to pull modes needs a bit of thought.
|
||||
Changing from push to pull modes needs a bit of thought. This is actually
|
||||
possible and implemented but not yet documented here.
|
||||
|
|
|
|||
|
|
@ -137,7 +137,7 @@ Use cases:
|
|||
5a) optionally element2 can now be set to NULL.
|
||||
6) link element4 and element3
|
||||
7) link element1 and element4 (FIXME, how about letting element4 know
|
||||
about the currently running segment?)
|
||||
about the currently running segment?, see issues.)
|
||||
8) unblock element1 src
|
||||
|
||||
The same flow can be used to replace an element in a PAUSED pipeline. Only
|
||||
|
|
@ -150,5 +150,18 @@ Use cases:
|
|||
be implemented with a helper function in the future.
|
||||
|
||||
|
||||
Issues
|
||||
------
|
||||
|
||||
When an EOS event has passed a pad and the pad is set to blocked, the block will
|
||||
never happen because no data is going to flow anymore. One possibility is to
|
||||
keep track of the pad's EOS state and make the block succeed immediatly. This is
|
||||
not yet implemenented.
|
||||
|
||||
When dynamically reconnecting pads, some events (like NEWSEGMENT, EOS,
|
||||
TAGS, ...) are not yet retransmitted to the newly connected element. It's
|
||||
unclear if this can be done by core automatically by caching those events and
|
||||
resending them on a relink. It might also be possible that this needs a
|
||||
GstFlowReturn value from the event function, in which case the implementation
|
||||
must be delayed for after 0.11, when we can break API/ABI.
|
||||
|
||||
|
|
|
|||
|
|
@ -51,3 +51,8 @@ application should not set the pipeline to PLAYING before a BUFFERING message
|
|||
with 100 percent value is received, which might only happen after the pipeline
|
||||
prerolled.
|
||||
|
||||
|
||||
Incremental download
|
||||
--------------------
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -1,13 +1,6 @@
|
|||
Clocks
|
||||
------
|
||||
|
||||
To synchronize the different elements, the GstPipeline is responsible for
|
||||
selecting and distributing a global GstClock for all the elements in it.
|
||||
|
||||
This selection happens whenever an element is added or removed from the
|
||||
pipeline. Whever the clock changes in a pipeline, a NEW_CLOCK message is
|
||||
posted on the bus signaling the new clock to the application.
|
||||
|
||||
The GstClock returns a monotonically increasing time with the method
|
||||
_get_time(). Its accuracy and base time depends on the specific clock
|
||||
implementation but time is always expessed in nanoseconds. Since the
|
||||
|
|
@ -17,28 +10,53 @@ times.
|
|||
The time reported by the clock is called the absolute time.
|
||||
|
||||
|
||||
Clock Selection
|
||||
---------------
|
||||
|
||||
To synchronize the different elements, the GstPipeline is responsible for
|
||||
selecting and distributing a global GstClock for all the elements in it.
|
||||
|
||||
This selection happens whenever the pipeline goes to PLAYING. Whenever an
|
||||
element is added/removed from the pipeline, this selection will be redone in the
|
||||
next state change to PLAYING. Adding an element that can provide a clock will
|
||||
post a GST_MESSAGE_CLOCK_PROVIDE message on the bus to inform parent bins of the
|
||||
fact that a clock recalculation is needed.
|
||||
|
||||
When a clock is selected, a NEW_CLOCK message is posted on the bus signaling the
|
||||
clock to the application.
|
||||
|
||||
When the element that provided the clock is removed from the pipeline, a
|
||||
CLOCK_LOST message is posted. The application must then set the pipeline to
|
||||
PAUSED and PLAYING again in order to let the pipeline select a new clock
|
||||
and distribute a new base time.
|
||||
|
||||
|
||||
Time in GStreamer
|
||||
-----------------
|
||||
|
||||
The absolute time is used to calculate the stream time. The stream time
|
||||
The absolute time is used to calculate the running time. The running time
|
||||
is defined as follows:
|
||||
|
||||
- If the pipeline is NULL/READY, the stream time is undefined.
|
||||
- In PAUSED, the stream time remains at the time when it was last
|
||||
PAUSED. When the stream is PAUSED for the first time, the stream time
|
||||
- If the pipeline is NULL/READY, the running time is undefined.
|
||||
- In PAUSED, the running time remains at the time when it was last
|
||||
PAUSED. When the stream is PAUSED for the first time, the running time
|
||||
is 0.
|
||||
- In PLAYING, the stream time is the delta between the absolute time
|
||||
- In PLAYING, the running time is the delta between the absolute time
|
||||
and the base time. The base time is defined as the absolute time minus
|
||||
the stream time at the time when the pipeline is set to PLAYING.
|
||||
- after a seek, the stream time is set to 0 (see part-seeking.txt)
|
||||
the running time at the time when the pipeline is set to PLAYING.
|
||||
- after a flushing seek, the running time is set to 0 (see part-seeking.txt)
|
||||
|
||||
The stream time is completely managed by the GstPipeline object using the
|
||||
GstClock absolute time.
|
||||
The running time is completely managed by the GstPipeline object using the
|
||||
GstClock absolute time. It basically represents the elapsed time that the
|
||||
pipeline spend in the PLAYING state.
|
||||
|
||||
|
||||
Timestamps
|
||||
----------
|
||||
|
||||
NOTE: this info is inaccurate, see part-synchronisation.txt for how the clock
|
||||
time is used to synchronize buffer timestamps.
|
||||
|
||||
The combination of the last NEWSEGMENT event and the buffer timestamps
|
||||
express the presentation stream time of the buffer. The stream time
|
||||
of a buffer is calculated as follows:
|
||||
|
|
|
|||
|
|
@ -7,10 +7,53 @@ does typically not have any sink (input) pads.
|
|||
Typical source elements include:
|
||||
|
||||
- file readers
|
||||
- network elements
|
||||
- network elements (live or not)
|
||||
- capture elements (video/audio/...)
|
||||
- generators (signals/video/audio/...)
|
||||
|
||||
|
||||
Live sources
|
||||
------------
|
||||
|
||||
A source is said to be a live source when it has the following property:
|
||||
|
||||
* temporarily stopping reading from the source causes data to be lost.
|
||||
|
||||
In general when this property holds, the source also produces data at a fixed
|
||||
rate. Most sources have a limit to the rate at which they can deliver data, which
|
||||
might be faster or slower than the consumption rate. this property however does
|
||||
not make them a live source.
|
||||
|
||||
Let's look at some example sources.
|
||||
|
||||
- file readers: you can PAUSE without losing data. There is however a limit to
|
||||
how fast you can read from this source. This limit is usually much higher
|
||||
than the consumption rate. In some cases it might be slower (an NFS share,
|
||||
for example) in which case you might need to use some buffering
|
||||
(see part-buffering.txt).
|
||||
|
||||
- http network element: you can PAUSE without data loss. Depending on the
|
||||
available network bandwidth, consumption rate might be higher than production
|
||||
rate in which case buffering should be used (see part-buffering.txt).
|
||||
|
||||
- audio source: pausing the audio capture will lead to lost data. this source
|
||||
is therefore definatly live. In addition, an audio source will produce data
|
||||
at a fixed rate (the samplerate). Also depending on the buffersize, this
|
||||
source will introduce a latency (see part-latency.txt).
|
||||
|
||||
- udp network source: Pausing the receiving part will lead to lost data. This
|
||||
source is therefore a live source. Also in a typical case the udp packets
|
||||
will be received at a certain rate, which might be difficult to guess because
|
||||
of network jitter. This source does not necessarily introduce latency on its
|
||||
own.
|
||||
|
||||
- dvb source: PAUSING this element will lead to data loss, it's a live source
|
||||
similar to a UDP source.
|
||||
|
||||
|
||||
Source types
|
||||
------------
|
||||
|
||||
A source element can operate in three ways:
|
||||
|
||||
- it is fully seekable, this means that random access can be performed
|
||||
|
|
@ -22,15 +65,14 @@ A source element can operate in three ways:
|
|||
A video source always provides the same amount of data (one video
|
||||
frame). Note that this is not a fully seekable source.
|
||||
|
||||
- it is a live source, this means that data arrives when it is ready.
|
||||
An example of this is a video or network source.
|
||||
- it is a live source, see above.
|
||||
|
||||
When writing a source, one has to look at how the source can operate to
|
||||
decide on the scheduling methods to implement on the source.
|
||||
|
||||
- fully seekable sources implement a getrange function on the source pad.
|
||||
|
||||
- sources that can give N bytes but cannot do seeking also implement a
|
||||
- sources that can give N bytes but cannot do seeking also implement a
|
||||
getrange function but state that they cannot do random access.
|
||||
|
||||
- sources that are purely live sources implement a task to push out
|
||||
|
|
|
|||
|
|
@ -19,8 +19,12 @@ Different types of events exist to implement various functionalities.
|
|||
GST_EVENT_QOS: A notification of the quality of service of the stream
|
||||
GST_EVENT_SEEK: A seek should be performed to a new position in the stream
|
||||
GST_EVENT_NAVIGATION: A navigation event.
|
||||
GST_EVENT_DRAIN: Play all data downstream before returning.
|
||||
GST_EVENT_LATENCY: Configure the latency in a pipeline
|
||||
|
||||
* GST_EVENT_DRAIN: Play all data downstream before returning.
|
||||
|
||||
* not yet implemented, under investigation, might be needed to do still frames
|
||||
in DVD.
|
||||
|
||||
FLUSH_START/STOP
|
||||
----------------
|
||||
|
|
@ -53,6 +57,12 @@ For elements that use the pullrange function, they send both flush events to
|
|||
the upstream pads in the same way to make sure that the pullrange function
|
||||
unlocks and any pending buffers are cleared in the upstream elements.
|
||||
|
||||
A FLUSH_STOP event will also clear any configured synchronisation information
|
||||
like NEWSEGMENT events. After a FLUSH_STOP, any element that performs
|
||||
synchronisation to the clock will therefore need a NEWSEGMENT event (which makes
|
||||
the running_time start from 0 again) and will therefore also need a new
|
||||
base_time (see part-clocks.txt and part-synchronisation.txt).
|
||||
|
||||
|
||||
EOS
|
||||
---
|
||||
|
|
@ -64,7 +74,7 @@ in the streaming thread but can also be sent from the application thread.
|
|||
An EOS event sent on a srcpad returns GST_FLOW_UNEXPECTED.
|
||||
|
||||
The downstream element should forward the EOS event to its downstream peer
|
||||
elements. This way the event will eventually reach the renderers which should
|
||||
elements. This way the event will eventually reach the sinks which should
|
||||
then post an EOS message on the bus when in PLAYING.
|
||||
|
||||
An element might want to flush its internally queued data before forwarding
|
||||
|
|
@ -86,14 +96,14 @@ sending data.
|
|||
An element that sends EOS on a pad should stop sending data on that pad. Source
|
||||
elements typically pause() their task for that purpose.
|
||||
|
||||
By default, a GstBin collects all EOS events from all its sinks before
|
||||
By default, a GstBin collects all EOS messages from all its sinks before
|
||||
posting the EOS message to its parent.
|
||||
|
||||
The EOS is only posted on the bus by the sink elements in the PLAYING state. If
|
||||
the EOS event is received in the PAUSED state, it is queued until the element
|
||||
goes to PLAYING.
|
||||
|
||||
A seek event on an element flushes all pending EOS messages.
|
||||
A FLUSH_STOP event on an element flushes the EOS state and all pending EOS messages.
|
||||
|
||||
|
||||
NEWSEGMENT
|
||||
|
|
@ -148,56 +158,57 @@ BUFFERSIZE
|
|||
|
||||
An element can suggest a buffersize for downstream elements. This is
|
||||
typically done by elements that produce data on multiple source pads
|
||||
such as demuxers.
|
||||
such as demuxers. This event is currently not yet defined or used.
|
||||
|
||||
|
||||
QOS
|
||||
---
|
||||
|
||||
A QOS, or quality of service message, is generated in an element to report
|
||||
to the upstream elements about the current quality of the stream. This
|
||||
is typically done by the sinks that measure the amount of framedrops they
|
||||
have. (see part-qos.txt)
|
||||
to the upstream elements about the current quality of real-time performance
|
||||
the stream. This is typically done by the sinks that measure the amount of
|
||||
framedrops they have. (see part-qos.txt)
|
||||
|
||||
|
||||
SEEK
|
||||
----
|
||||
|
||||
A seek event is issued by the application to start playback of a new
|
||||
position in the stream. It is called form the application thread and
|
||||
travels upstream.
|
||||
A seek event is issued by the application to configure the playback range
|
||||
of a stream. It is called form the application thread and travels upstream.
|
||||
|
||||
The seek event contains the new start and end position of playback
|
||||
after the seek is performed. Optionally the end position can be left
|
||||
The seek event contains the new start and stop position of playback
|
||||
after the seek is performed. Optionally the stop position can be left
|
||||
at -1 to continue playback to the end of the stream. The seek event
|
||||
also contains the new playback rate of the stream, 1.0 is normal playback,
|
||||
2.0 double speed and negative values mean backwards playback.
|
||||
|
||||
A seek usually flushes the graph to minimize latency after the seek. This
|
||||
behaviour is triggered by using the SEEK_FLUSH flag.
|
||||
behaviour is triggered by using the SEEK_FLUSH flag on the seek event.
|
||||
|
||||
The seek event is passed along from element to element until it reaches
|
||||
an element that can perform the seek. No intermediate element is allowed
|
||||
to assume that a seek to this location will happen. It is allowed to
|
||||
modify the start and stop times if it needs to do so. this is typically
|
||||
the case if a seek is requested for a non-time position.
|
||||
The seek event usually starts from the sink elements and travels upstream
|
||||
from element to element until it reaches an element that can perform the
|
||||
seek. No intermediate element is allowed to assume that a seek to this
|
||||
location will happen. It is allowed to modify the start and stop times if it
|
||||
needs to do so. this is typically the case if a seek is requested for a
|
||||
non-time position.
|
||||
|
||||
The actual seek is performed in the application thread so that success
|
||||
or failure can be reported as a return value of the seek event. It is
|
||||
therefore important that before executing the seek, the element acquires
|
||||
the STREAM_LOCK so that the streaming thread and the seek gets serialized.
|
||||
the STREAM_LOCK so that the streaming thread and the seek get serialized.
|
||||
|
||||
The general flow of executing the seek with FLUSH is as follows:
|
||||
|
||||
1) unblock the streaming threads, they could be blocked in a chain
|
||||
function. This is done by sending a FLUSH_START on all srcpads.
|
||||
function. This is done by sending a FLUSH_START on all srcpads or by pausing
|
||||
the streaming task, depending on the seek FLUSH flag.
|
||||
The flush will make sure that all downstream elements unlock and
|
||||
that control will return to this element chain/loop function.
|
||||
We cannot lock the STREAM_LOCK before doing this since it might
|
||||
cause a deadlock.
|
||||
|
||||
2) acquire the STREAM_LOCK. This will work since the chain/loop function
|
||||
was unlocked in step 1).
|
||||
was unlocked/paused in step 1).
|
||||
|
||||
3) perform the seek. since the STREAM_LOCK is held, the streaming thread
|
||||
will wait for the seek to complete. Most likely, the stream thread
|
||||
|
|
@ -205,8 +216,8 @@ The general flow of executing the seek with FLUSH is as follows:
|
|||
|
||||
4) send a FLUSH_STOP event to all peer elements to allow streaming again.
|
||||
|
||||
5) send a NEWSEGMENT event to signal the new buffer timestamp base time.
|
||||
This can also be done from the streaming thread.
|
||||
5) create a NEWSEGMENT event to signal the new buffer timestamp base time.
|
||||
This event must be queued to be send by the streaming thread.
|
||||
|
||||
6) start stopped tasks and unlock the STREAM_LOCK, dataflow will continue
|
||||
now from the new position.
|
||||
|
|
@ -223,9 +234,22 @@ of a navigation event such as a mouse movement or button click.
|
|||
Navigation events travel upstream.
|
||||
|
||||
|
||||
LATENCY
|
||||
-------
|
||||
|
||||
A latency event is used to configure a certain latency in the pipeline. It
|
||||
contains a single GstClockTime with the required latency. The latency value is
|
||||
calculated by the pipeline and distributed to all sink elements before they are
|
||||
set to PLAYING. The sinks will add the configured latency value to the
|
||||
timestamps of the buffer in order to delay their presentation.
|
||||
See also part-latency.txt.
|
||||
|
||||
|
||||
DRAIN
|
||||
-----
|
||||
|
||||
This event is not yet implemented.
|
||||
|
||||
Drain event indicates that upstream is about to perform a real-time event, such
|
||||
as pausing to present an interactive menu or such, and needs to wait for all
|
||||
data it has sent to be played-out in the sink.
|
||||
|
|
|
|||
|
|
@ -65,8 +65,8 @@ When a bin goes to READY it will clear all cached messages.
|
|||
EOS
|
||||
---
|
||||
|
||||
The sink elements will post an EOS event on the bus when they reach EOS. The
|
||||
EOS message is only posted to the bus when the element is in PLAYING.
|
||||
The sink elements will post an EOS message on the bus when they reach EOS. The
|
||||
EOS message is only posted to the bus when the sink element is in PLAYING.
|
||||
|
||||
The bin collects all EOS messages and forwards it to the application as
|
||||
soon as all the sinks have posted an EOS.
|
||||
|
|
@ -89,6 +89,18 @@ The cached SEGMENT_START/STOP messages are cleared when going to READY.
|
|||
DURATION
|
||||
--------
|
||||
|
||||
When a DURATION query is performed on a bin, it will forward the query to all
|
||||
its sink elements. The bin will calculate the total duration as the MAX of all
|
||||
returned durations and will then cache the result so that any further query can
|
||||
use the cached version. The reason for caching the result is because the
|
||||
duration of a stream typically does not change that often.
|
||||
|
||||
A GST_MESSAGE_DURATION posted by an element will clear the cached duration value
|
||||
so that the bin will query the sinks again. This message is typically posted by
|
||||
elements that calculate the duration of the stream based on some average
|
||||
bitrate, which might change while playing the stream. The DURATION message is
|
||||
posted to the application, which can then fetch the updated DURATION.
|
||||
|
||||
|
||||
Subclassing
|
||||
-----------
|
||||
|
|
|
|||
|
|
@ -7,7 +7,8 @@ a first-in first-out way from the streaming threads to the application.
|
|||
Since the application typically only wants to deal with delivery of these
|
||||
messages from one thread, the GstBus will marshall the messages between
|
||||
different threads. This is important since the actual streaming of media
|
||||
is done in another thread than the application.
|
||||
is done in another thread than the application. It is also important to not
|
||||
block the streaming thread while the application deals with the message.
|
||||
|
||||
The GstBus provides support for GSource based notifications. This makes it
|
||||
possible to handle the delivery in the glib mainloop. Different GSources
|
||||
|
|
|
|||
|
|
@ -6,11 +6,9 @@ children with a clock.
|
|||
|
||||
A GstPipeline also provides a toplevel GstBus (see part-gstbus.txt)
|
||||
|
||||
The pipeline also calculates the stream time based on the selected
|
||||
The pipeline also calculates the running_time based on the selected
|
||||
clock (see part-clocks.txt).
|
||||
|
||||
The pipeline manages the seek operation for the application.
|
||||
|
||||
|
||||
State changes
|
||||
-------------
|
||||
|
|
@ -22,26 +20,30 @@ GstBin, the pipeline performs the following actions during a state change:
|
|||
- set the bus to non-flushing
|
||||
|
||||
- READY -> PAUSED:
|
||||
- reset the stream time to 0
|
||||
- reset the running_time to 0
|
||||
|
||||
- PAUSED -> PLAYING:
|
||||
- Select and a clock.
|
||||
- calculate base time using the stream time.
|
||||
- set clock and base time on all elements before performing the
|
||||
- calculate base_time using the running_time.
|
||||
- set clock and base_time on all elements before performing the
|
||||
state change.
|
||||
|
||||
- PAUSED -> PLAYING:
|
||||
- calculate the stream time when the pipeline was stopped.
|
||||
- calculate the running_time when the pipeline was stopped.
|
||||
|
||||
- READY -> NULL:
|
||||
- set the bus to flushing (when auto-flushing is enabled)
|
||||
|
||||
The running_time represents the total elapsed time, measured in clock units,
|
||||
that the pipeline spent in the PLAYING state.
|
||||
|
||||
|
||||
Clock selection
|
||||
---------------
|
||||
|
||||
Since all of the children of a GstPipeline must use the same clock, the
|
||||
pipeline must select a clock.
|
||||
pipeline must select a clock. This clock selection happens when the pipeline
|
||||
gopes to the PLAYING state.
|
||||
|
||||
The default clock selection algorithm works as follows:
|
||||
|
||||
|
|
@ -54,7 +56,7 @@ The default clock selection algorithm works as follows:
|
|||
* since this selection procedure happens in the PAUSED->PLAYING
|
||||
state change, all the sinks are prerolled and we can thus be sure
|
||||
that each sink is linked to some upstream element.
|
||||
* in the case of a live pipeline (NO_PREROLL), the sink will not
|
||||
* in the case of a live pipeline (NO_PREROLL), the sink will not yet
|
||||
be prerolled and the selection process will select the clock of
|
||||
a more upstream element.
|
||||
|
||||
|
|
@ -72,19 +74,3 @@ possible.
|
|||
The _auto_clock() method removes the fixed clock and reactivates the auto-
|
||||
matic clock selection algorithm described above.
|
||||
|
||||
|
||||
Seeking
|
||||
-------
|
||||
|
||||
When performing a seek on the pipeline element using gst_element_send_event(),
|
||||
the pipeline performs the following actions:
|
||||
|
||||
- record the current state of the pipeline.
|
||||
- set the pipeline to paused if a FLUSHING seek is requested
|
||||
- send the seek event to all sinks
|
||||
- when a FLUSH seek was performed successfully, the stream_time is set 0 again.
|
||||
- restore old state of the pipeline.
|
||||
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -1,6 +1,9 @@
|
|||
Live sources
|
||||
------------
|
||||
|
||||
A live source is a source that cannot be arbitrarily PAUSED without losing
|
||||
data.
|
||||
|
||||
A live source such as an element capturing audio or video need to be handled
|
||||
in a special way. It does not make sense to start the dataflow in the PAUSED
|
||||
state for those devices as the user might wait a long time between going from
|
||||
|
|
@ -26,7 +29,7 @@ a blocking wait.
|
|||
Scheduling
|
||||
----------
|
||||
|
||||
Live sources can not produce data in the paused state. They block in the
|
||||
Live sources will not produce data in the paused state. They block in the
|
||||
getrange function or in the loop function until they go to PLAYING.
|
||||
|
||||
|
||||
|
|
@ -39,19 +42,6 @@ the first sample of data and the last sample. This means that when the
|
|||
buffer arrives at the sink, it will already be late and will be dropped.
|
||||
|
||||
The latency is the time it takes to construct one buffer of data. This latency
|
||||
could be exposed by latency queries.
|
||||
is exposed with a LATENCY query.
|
||||
|
||||
Theses latency queries could to be done by the managing pipeline for all sinks.
|
||||
In that case they can only be done after the meassurements have been taken (all
|
||||
are prerolled). Thus in pipeline:state_changed:PAUSED_TO_PLAYING we need
|
||||
get the max-latency and set this as a sync-offset in all sinks. The problem is
|
||||
that in a live pipeline, we set the pipeline to PLAYING before waiting for the
|
||||
sinks to preroll.
|
||||
|
||||
Another possibility would be to configure a fixed latency in a pipeline that
|
||||
would automatically be configured on any sink in the case of a NO_PREROLL
|
||||
element. For decoupled elements this is practically the only viable way to
|
||||
introduce enough latency that does not starve the sinks.
|
||||
|
||||
The current latency can also be measured in the sinks and the pipeline could
|
||||
optimize it to the lowest possible latency.
|
||||
See part-latency.txt.
|
||||
|
|
|
|||
|
|
@ -55,9 +55,14 @@ Message types
|
|||
An element changed state in the pipeline. The message carries the old, new
|
||||
and pending state of the element.
|
||||
|
||||
GST_MESSAGE_STATE_DIRTY:
|
||||
|
||||
An internal message used to instruct a pipeline hierarchy that a state
|
||||
recalculation must be performed because of an ASYNC state change completed.
|
||||
|
||||
GST_MESSAGE_STEP_DONE:
|
||||
|
||||
An element stepping frames has finished.
|
||||
An element stepping frames has finished. This is currently not used.
|
||||
|
||||
GST_MESSAGE_CLOCK_PROVIDE:
|
||||
|
||||
|
|
@ -75,12 +80,15 @@ Message types
|
|||
GST_MESSAGE_STRUCTURE_CHANGE:
|
||||
|
||||
The pipeline changed of structure, This means elements were added or removed or
|
||||
pads were linked or unlinked.
|
||||
pads were linked or unlinked. This messages is not yet used.
|
||||
|
||||
GST_MESSAGE_STREAM_STATUS:
|
||||
|
||||
An element posted information about the stream it is handling. This could include
|
||||
information about the length of the stream.
|
||||
Posted by an element when it start/stop/pauses a streaming task. It
|
||||
contains information about the reason why the stream state changed along
|
||||
with the thread id. The application can use this information to detect
|
||||
failures in streaming threads. It can also be used to adjust streaming
|
||||
thread priorities by the application.
|
||||
|
||||
GST_MESSAGE_APPLICATION:
|
||||
|
||||
|
|
@ -100,4 +108,22 @@ Message types
|
|||
An element or bin completed playback of a segment. This message is only posted
|
||||
on the bus if a SEGMENT seek is performed on a pipeline.
|
||||
|
||||
GST_MESSAGE_DURATION:
|
||||
|
||||
An element posts this message when it has detected or updated the stream duration.
|
||||
|
||||
GST_MESSAGE_LOST_PREROLL:
|
||||
|
||||
Posted by sinks when they lose their preroll buffer in the PAUSED or PLAYING
|
||||
state caused by a FLUSH_START event.
|
||||
|
||||
GST_MESSAGE_PREROLLED:
|
||||
|
||||
Posted by sinks when they receive the first data buffer.
|
||||
|
||||
GST_MESSAGE_LATENCY:
|
||||
|
||||
Posted by elements when the latency in a pipeline changed and a new global
|
||||
latency should be calculated by the pipeline or application.
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -47,7 +47,7 @@ Introduction
|
|||
|
||||
Downstream and upstream are the terms used to describe the direction in the
|
||||
Pipeline. From source to sink is called "downstream" and "upstream" is
|
||||
from sink to source.
|
||||
from sink to source. Dataflow always happens downstream.
|
||||
|
||||
The task of the application is to construct a pipeline as above using existing
|
||||
elements. This is further explained in the pipeline building topic.
|
||||
|
|
@ -162,8 +162,10 @@ Pipeline
|
|||
A pipeline is a special bin subclass that provides the following features to its
|
||||
children:
|
||||
|
||||
- Select and manage a clock.
|
||||
- Manage stream time based on the selected clock.
|
||||
- Select and manage a global clock for all its children.
|
||||
- Manage running_time based on the selected clock. Running_time is the elapsed
|
||||
time the pipeline spent in the PLAYING state and is used for
|
||||
synchronisation.
|
||||
- Provide means for elements to comunicate with the application by the GstBus.
|
||||
- Manage the global state of the elements such as Errors and end-of-stream.
|
||||
|
||||
|
|
@ -226,6 +228,7 @@ Dataflow and buffers
|
|||
The process of selecting a media type and attaching it to the buffers is called
|
||||
caps negotiation.
|
||||
|
||||
|
||||
Caps
|
||||
----
|
||||
|
||||
|
|
@ -243,7 +246,7 @@ Dataflow and events
|
|||
-------------------
|
||||
|
||||
Parallel to the dataflow is a flow of events. Unlike the buffers, events can pass
|
||||
upstream and downstream. Some events only travel upstream others only downstream.
|
||||
both upstream and downstream. Some events only travel upstream others only downstream.
|
||||
|
||||
the events are used to denote special conditions in the dataflow such as EOS or
|
||||
to inform plugins of special events such as flushing or seeking.
|
||||
|
|
@ -311,13 +314,13 @@ Pipeline clock
|
|||
a sink element such as an audio sink.
|
||||
|
||||
In capture pipelines, this will typically select the clock of the data producer, which
|
||||
can in most cases not control the rate at which it produces data.
|
||||
in most cases can not control the rate at which it produces data.
|
||||
|
||||
|
||||
Pipeline states
|
||||
---------------
|
||||
|
||||
When all the pads are linked or signals have been connected, the pipeline can
|
||||
When all the pads are linked and signals have been connected, the pipeline can
|
||||
be put in the PAUSED state to start dataflow.
|
||||
|
||||
When a bin (and hence a pipeline) performs a state change, it will change the state
|
||||
|
|
@ -361,7 +364,7 @@ Pipeline states
|
|||
the pipeline can be put in the PLAYING state.
|
||||
|
||||
Before going to PLAYING, the pipeline select a clock and samples the current time of
|
||||
the clock. This is the base time. It then distributes this time to all elements.
|
||||
the clock. This is the base_time. It then distributes this time to all elements.
|
||||
Elements can then synchronize against the clock using the buffer timestamp+base time.
|
||||
|
||||
The following chain of state changes then takes place:
|
||||
|
|
@ -404,14 +407,14 @@ Pipeline EOS
|
|||
data after receiving an EOS event on a sinkpad.
|
||||
|
||||
The element providing the streaming thread stops sending data after sending the
|
||||
EOS message.
|
||||
EOS event.
|
||||
|
||||
The EOS even will eventually arrive in the sink element. The sink will then post
|
||||
The EOS event will eventually arrive in the sink element. The sink will then post
|
||||
an EOS message on the bus to inform the pipeline that a particular stream has
|
||||
finished. When all sinks have reported EOS, the pipeline forwards the EOS message
|
||||
to the application.
|
||||
|
||||
When in EOS, the pipeline remains in the playing state, it is the applications
|
||||
When in EOS, the pipeline remains in the PLAYING state, it is the applications
|
||||
responsability to PAUSE or READY the pipeline. The application can also issue
|
||||
a seek, for example.
|
||||
|
||||
|
|
|
|||
|
|
@ -31,7 +31,7 @@ QoS event
|
|||
The QoS event travels upstream and contains the following fields:
|
||||
|
||||
- timestamp: The timestamp on the buffer that generated the QoS
|
||||
event. These timestamps are expressed in total running time in
|
||||
event. These timestamps are expressed in total running_time in
|
||||
the sink so that the value is ever increasing.
|
||||
|
||||
- jitter: The difference of that timestamp against the current clock time.
|
||||
|
|
@ -223,11 +223,11 @@ values:
|
|||
The processing time and the average buffer durations will be used to
|
||||
calculate a proportion.
|
||||
|
||||
the processing time in system time is compared to render time to decide if
|
||||
The processing time in system time is compared to render time to decide if
|
||||
the majority of the time is spend upstream or in the sink itself. This value
|
||||
is used to decide flood or starvation.
|
||||
|
||||
the number of rendered and dropped buffers is used to query stats on the sink.
|
||||
The number of rendered and dropped buffers is used to query stats on the sink.
|
||||
|
||||
A QoS message with the most current values is sent upstream for each buffer
|
||||
that was received by the sink.
|
||||
|
|
@ -293,5 +293,5 @@ less demanding source material. In case of a network stream, a switch could be d
|
|||
to a lower or higher quality stream or additional enhancement layers could be used
|
||||
or ignored.
|
||||
|
||||
Live sources will automatically drop data when it takes too long to prcess the data
|
||||
Live sources will automatically drop data when it takes too long to process the data
|
||||
that the element pushes out.
|
||||
|
|
|
|||
|
|
@ -1,10 +1,6 @@
|
|||
DRAFT Query
|
||||
-----------
|
||||
|
||||
Status
|
||||
|
||||
Implemented, move me to design...
|
||||
|
||||
Purpose
|
||||
|
||||
Queries are used to get information about the stream.
|
||||
|
|
@ -61,33 +57,33 @@ Proposed types
|
|||
|
||||
- GST_QUERY_SEEKING:
|
||||
|
||||
- get info on how seeking can be done
|
||||
- getrange, with/without offset/size
|
||||
- ranges where seeking is efficient (for caching network sources)
|
||||
- flags describing seeking behaviour (forward, backward, segments,
|
||||
get info on how seeking can be done
|
||||
- getrange, with/without offset/size
|
||||
- ranges where seeking is efficient (for caching network sources)
|
||||
- flags describing seeking behaviour (forward, backward, segments,
|
||||
play backwards, ...)
|
||||
|
||||
- GST_QUERY_POSITION:
|
||||
|
||||
- get info on current position of the stream
|
||||
- start position
|
||||
- current position
|
||||
- end position
|
||||
- length
|
||||
get info on current position of the stream in stream_time.
|
||||
|
||||
- GST_QUERY_DURATION:
|
||||
|
||||
get info on the total duration of the stream.
|
||||
|
||||
- GST_QUERY_LATENCY:
|
||||
|
||||
- get amount of buffering
|
||||
get amount of latency introduced in the pipeline.
|
||||
|
||||
- GST_QUERY_SEGMENT:
|
||||
|
||||
get info about the currently configured playback segment.
|
||||
|
||||
- GST_QUERY_CONVERT:
|
||||
|
||||
- convert format/value to another format/value pair.
|
||||
convert format/value to another format/value pair.
|
||||
|
||||
- GST_QUERY_FORMATS:
|
||||
|
||||
- return list of supported formats.
|
||||
|
||||
Also????
|
||||
|
||||
- GST_QUERY_CAPS:
|
||||
return list of supported formats that can be used for GST_QUERY_CONVERT.
|
||||
|
||||
|
|
|
|||
|
|
@ -41,7 +41,7 @@ the following state changes are possible:
|
|||
when they have enough information. It is a requirement for sinks to
|
||||
return ASYNC and complete the state change when they receive the first
|
||||
buffer or EOS event (prerol). Sinks also block the dataflow when in PAUSED.
|
||||
- a pipeline resets the stream time to 0.
|
||||
- a pipeline resets the running_time to 0.
|
||||
- live sources return NO_PREROLL and don't generate data.
|
||||
|
||||
PAUSED -> PLAYING
|
||||
|
|
@ -49,8 +49,8 @@ the following state changes are possible:
|
|||
- The pipeline selects a clock and distributes this to all the children
|
||||
before setting them to PLAYING. This means that it is only alowed to
|
||||
synchronize on the clock in the PLAYING state.
|
||||
- The pipeline uses the clock and the stream time to calculate the base time.
|
||||
The base time is distributed to all children when performing the state
|
||||
- The pipeline uses the clock and the running_time to calculate the base_time.
|
||||
The base_time is distributed to all children when performing the state
|
||||
change.
|
||||
- sink elements stop blocking on the preroll buffer or event and start
|
||||
rendering the data.
|
||||
|
|
@ -62,8 +62,8 @@ the following state changes are possible:
|
|||
|
||||
PLAYING -> PAUSED
|
||||
- most elements ignore this state change.
|
||||
- The pipeline calculates the stream time based on the last selected clock
|
||||
and the base time. It stores this information to continue playback when
|
||||
- The pipeline calculates the running_time based on the last selected clock
|
||||
and the base_time. It stores this information to continue playback when
|
||||
going back to the PLAYING state.
|
||||
- sinks unblock any clock wait calls.
|
||||
- when a sink did not have a pending buffer to play, it returns ASYNC from
|
||||
|
|
|
|||
|
|
@ -195,7 +195,7 @@ For plugins the following rules apply:
|
|||
|
||||
Notes:
|
||||
|
||||
- The clock/stream_time keeps running forward.
|
||||
- The clock/running_time keeps running forward.
|
||||
- backwards playback potentially uses a lot of memory as frames and undecoded
|
||||
data gets buffered.
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue