mirror of
https://gitlab.freedesktop.org/gstreamer/gstreamer.git
synced 2025-06-06 07:28:53 +00:00
design-docs: add html output using asciidoc
Unify the ad-hoc markup to be asciidoc style in many places. Add a "html" target to Makefile to generate the output.
This commit is contained in:
Stefan Kost
parent
1c50dcd54f
commit
16ce2d4ea4
46 changed files with 1637 additions and 1589 deletions
|
|
@ -47,6 +47,22 @@ EXTRA_DIST = \
|
||||||
part-TODO.txt \
|
part-TODO.txt \
|
||||||
part-trickmodes.txt
|
part-trickmodes.txt
|
||||||
|
|
||||||
|
CLEANFILES = index.html index.txt
|
||||||
|
|
||||||
|
html:
|
||||||
|
if ! test -z `which asciidoc`; then \
|
||||||
|
echo >index.txt "GStreamer design"; \
|
||||||
|
echo >>index.txt "================"; \
|
||||||
|
echo >>index.txt "The Gstreamer developers"; \
|
||||||
|
echo >>index.txt "Version $(PACKAGE_VERSION)"; \
|
||||||
|
echo >>index.txt ""; \
|
||||||
|
( cd $(srcdir) && \
|
||||||
|
cat >>$(abs_builddir)/index.txt $(EXTRA_DIST) ); \
|
||||||
|
asciidoc -o index.html index.txt; \
|
||||||
|
else \
|
||||||
|
echo "need asciidoc to generate html"; \
|
||||||
|
fi;
|
||||||
|
|
||||||
upload:
|
upload:
|
||||||
@echo nothing to upload
|
@echo nothing to upload
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -23,7 +23,7 @@ Some examples of metadata:
|
||||||
|
|
||||||
|
|
||||||
Requirements
|
Requirements
|
||||||
------------
|
~~~~~~~~~~~~
|
||||||
|
|
||||||
- It must be fast
|
- It must be fast
|
||||||
* allocation, free, low fragmentation
|
* allocation, free, low fragmentation
|
||||||
|
|
@ -41,7 +41,7 @@ Requirements
|
||||||
|
|
||||||
|
|
||||||
GstMiniObject
|
GstMiniObject
|
||||||
-------------
|
~~~~~~~~~~~~~
|
||||||
|
|
||||||
We make GstMiniObject a simple refcounted C structure and also a GLib boxed
|
We make GstMiniObject a simple refcounted C structure and also a GLib boxed
|
||||||
type. The following fields will be in the structure:
|
type. The following fields will be in the structure:
|
||||||
|
|
@ -66,7 +66,7 @@ objects.
|
||||||
|
|
||||||
|
|
||||||
GstEvent, GstCaps, GstQuery, GstMessage
|
GstEvent, GstCaps, GstQuery, GstMessage
|
||||||
---------------------------------------
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
Have the new GstMiniObject be the first field in these objects. They will probably
|
Have the new GstMiniObject be the first field in these objects. They will probably
|
||||||
also replace the copy and free functions with their own implementations.
|
also replace the copy and free functions with their own implementations.
|
||||||
|
|
@ -77,7 +77,7 @@ the custom functions.
|
||||||
|
|
||||||
|
|
||||||
GstBuffer
|
GstBuffer
|
||||||
---------
|
~~~~~~~~~
|
||||||
|
|
||||||
A GstMiniObject will be the parent instance of the GstBuffer object, which is a
|
A GstMiniObject will be the parent instance of the GstBuffer object, which is a
|
||||||
regular C structure.
|
regular C structure.
|
||||||
|
|
@ -105,7 +105,7 @@ Buffers point to a GstCaps structure that contains the caps of the buffer data.
|
||||||
|
|
||||||
|
|
||||||
GstBufferMeta
|
GstBufferMeta
|
||||||
-------------
|
~~~~~~~~~~~~~
|
||||||
|
|
||||||
A GstBufferMeta is a structure as follows:
|
A GstBufferMeta is a structure as follows:
|
||||||
|
|
||||||
|
|
@ -208,7 +208,7 @@ GstBufferMetaMemory | | next ---+
|
||||||
. .
|
. .
|
||||||
|
|
||||||
API examples
|
API examples
|
||||||
------------
|
~~~~~~~~~~~~
|
||||||
|
|
||||||
Buffers are created using the normal gst_buffer_new functions. The standard fields
|
Buffers are created using the normal gst_buffer_new functions. The standard fields
|
||||||
are initialized as usual. A memory area that is bigger than the structure size
|
are initialized as usual. A memory area that is bigger than the structure size
|
||||||
|
|
@ -276,7 +276,7 @@ possible simple API would look like this:
|
||||||
|
|
||||||
|
|
||||||
Memory management
|
Memory management
|
||||||
-----------------
|
~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
* allocation
|
* allocation
|
||||||
|
|
||||||
|
|
@ -316,7 +316,7 @@ Memory management
|
||||||
|
|
||||||
|
|
||||||
Subbuffers
|
Subbuffers
|
||||||
----------
|
~~~~~~~~~~
|
||||||
|
|
||||||
Subbuffers are a first class feature of the GstBuffer.
|
Subbuffers are a first class feature of the GstBuffer.
|
||||||
|
|
||||||
|
|
@ -334,7 +334,7 @@ timing metadata needs to be reset to NONE when the start offset is different.
|
||||||
|
|
||||||
|
|
||||||
Serialization
|
Serialization
|
||||||
-------------
|
~~~~~~~~~~~~~
|
||||||
|
|
||||||
When buffer should be sent over the wire or be serialized in GDP, we need a way
|
When buffer should be sent over the wire or be serialized in GDP, we need a way
|
||||||
to perform custom serialization and deserialization on the metadata.
|
to perform custom serialization and deserialization on the metadata.
|
||||||
|
|
@ -345,7 +345,7 @@ and endianness.
|
||||||
|
|
||||||
|
|
||||||
Transformations
|
Transformations
|
||||||
---------------
|
~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
After certain transformations, the metadata on a buffer might not be relevant
|
After certain transformations, the metadata on a buffer might not be relevant
|
||||||
anymore.
|
anymore.
|
||||||
|
|
@ -368,7 +368,7 @@ so on).
|
||||||
|
|
||||||
|
|
||||||
Other use cases
|
Other use cases
|
||||||
---------------
|
~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
Making the GstBufferMetaMemory (for making the buffer point to the associated
|
Making the GstBufferMetaMemory (for making the buffer point to the associated
|
||||||
memory region) as metadata on a GstBuffer, as opposed to making it an integral
|
memory region) as metadata on a GstBuffer, as opposed to making it an integral
|
||||||
|
|
@ -397,7 +397,7 @@ implementations add (or use, in the case of a file reader) the memory metadata.
|
||||||
|
|
||||||
|
|
||||||
Relationship with GstCaps
|
Relationship with GstCaps
|
||||||
-------------------------
|
~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
The difference between GstCaps, used in negotiation, and the metadata is not
|
The difference between GstCaps, used in negotiation, and the metadata is not
|
||||||
clearly defined.
|
clearly defined.
|
||||||
|
|
@ -415,7 +415,7 @@ video resolution while the implementation details would be inside the metadata.
|
||||||
|
|
||||||
|
|
||||||
Compatibility
|
Compatibility
|
||||||
-------------
|
~~~~~~~~~~~~~
|
||||||
|
|
||||||
We need to make sure that elements exchange metadata that they both understand,
|
We need to make sure that elements exchange metadata that they both understand,
|
||||||
This is particulary important when the metadata describes the data layout in
|
This is particulary important when the metadata describes the data layout in
|
||||||
|
|
@ -436,16 +436,16 @@ that are metadata aware to set a flag on their pads; any buffer passed on that
|
||||||
pad will be converted to the old layout when the flag is not set.
|
pad will be converted to the old layout when the flag is not set.
|
||||||
|
|
||||||
|
|
||||||
Notes:
|
Notes
|
||||||
------
|
~~~~~
|
||||||
|
|
||||||
Some structures that we need to be able to add to buffers.
|
Some structures that we need to be able to add to buffers.
|
||||||
|
|
||||||
Clean Aperture
|
* Clean Aperture
|
||||||
Arbitrary Matrix Transform
|
* Arbitrary Matrix Transform
|
||||||
Aspect ratio
|
* Aspect ratio
|
||||||
Pan/crop/zoom
|
* Pan/crop/zoom
|
||||||
Video strides
|
* Video strides
|
||||||
|
|
||||||
Some of these overlap, we need to find a minimal set of metadata structures that
|
Some of these overlap, we need to find a minimal set of metadata structures that
|
||||||
allows us to define all use cases.
|
allows us to define all use cases.
|
||||||
|
|
|
||||||
|
|
@ -1,189 +1,187 @@
|
||||||
Element Klass definition
|
Element Klass definition
|
||||||
------------------------
|
------------------------
|
||||||
|
|
||||||
Status:
|
Purpose
|
||||||
|
~~~~~~~
|
||||||
|
|
||||||
DRAFT.
|
Applications should be able to retrieve elements from the registry of existing
|
||||||
|
elements based on specific capabilities or features of the element.
|
||||||
|
|
||||||
|
A playback application might want to retrieve all the elements that can be
|
||||||
|
used for visualisation, for example, or a video editor might want to select
|
||||||
|
all video effect filters.
|
||||||
|
|
||||||
Purpose:
|
The topic of defining the klass of elements should be based on use cases.
|
||||||
|
|
||||||
Applications should be able to retrieve elements from the registry of existing
|
A list of classes that are used in a installation can be generated using:
|
||||||
elements based on specific capabilities or features of the element.
|
gst-inspect-0.10 -a | grep -ho Class:.* | cut -c8- | sed "s/\//\\n/g" | sort | uniq
|
||||||
|
|
||||||
A playback application might want to retrieve all the elements that can be
|
Proposal
|
||||||
used for visualisation, for example, or a video editor might want to select
|
~~~~~~~~
|
||||||
all video effect filters.
|
|
||||||
|
|
||||||
The topic of defining the klass of elements should be based on use cases.
|
The GstElementDetails contains a field named klass that is a pointer to a
|
||||||
|
string describing the element type.
|
||||||
|
|
||||||
A list of classes that are used in a installation can be generated using:
|
In this document we describe the format and contents of the string. Elements
|
||||||
gst-inspect-0.10 -a | grep -ho Class:.* | cut -c8- | sed "s/\//\\n/g" | sort | uniq
|
should adhere to this specification although that is not enforced to allow
|
||||||
|
for wild (application specific) customisation.
|
||||||
|
|
||||||
Proposal:
|
1) string format
|
||||||
|
|
||||||
The GstElementDetails contains a field named klass that is a pointer to a
|
<keyword>['/'<keyword]*
|
||||||
string describing the element type.
|
|
||||||
|
|
||||||
In this document we describe the format and contents of the string. Elements
|
The string consists of an _unordered_ list of keywords separated with a '/'
|
||||||
should adhere to this specification although that is not enforced to allow
|
character. While the / suggests a hierarchy, this is _not_ the case.
|
||||||
for wild (application specific) customisation.
|
|
||||||
|
|
||||||
1) string format
|
2) keyword categories
|
||||||
|
|
||||||
<keyword>['/'<keyword]*
|
- functional
|
||||||
|
|
||||||
The string consists of an _unordered_ list of keywords separated with a '/'
|
Categories are base on _intended usage_ of the element. Some elements
|
||||||
character. While the / suggests a hierarchy, this is _not_ the case.
|
might have other side-effects (especially for filers/effects). The purpose
|
||||||
|
is to list enough keywords so that applications can do meaningfull filtering,
|
||||||
|
not to completely describe the functionality, that is expressed in caps etc..
|
||||||
|
|
||||||
2) keyword categories
|
* Source : produces data
|
||||||
|
* Sink : consumes data
|
||||||
|
* Filter : filters/transforms data, no modification on the data is
|
||||||
|
intended (although it might be unavoidable). The
|
||||||
|
filter can decide on input and output caps independently
|
||||||
|
of the stream contents (GstBaseTransform).
|
||||||
|
* Effect : applies an effect to some data, changes to data are
|
||||||
|
intended. Examples are colorbalance, volume. These
|
||||||
|
elements can also be implemented with GstBaseTransform.
|
||||||
|
* Demuxer : splits audio, video, ... from a stream
|
||||||
|
* Muxer : interleave audio, video, ... into one stream, this is
|
||||||
|
like mixing but without losing or degrading each separate
|
||||||
|
input stream. The reverse operation is possible with a
|
||||||
|
Demuxer that reproduces the exact same input streams.
|
||||||
|
* Decoder : decodes encoded data into a raw format, there is typically
|
||||||
|
no relation between input caps and output caps. The output
|
||||||
|
caps are defined in the stream data. This separates the
|
||||||
|
Decoder from the Filter and Effect.
|
||||||
|
* Encoder : encodes raw data into an encoded format.
|
||||||
|
* Mixer : combine audio, video, .. this is like muxing but with
|
||||||
|
applying some algorithm so that the individual streams
|
||||||
|
are not extractable anymore, there is therefore no
|
||||||
|
reverse operation to mixing. (audio mixer, video mixer, ...)
|
||||||
|
* Converter : convert audio into video, text to audio, ... The converter
|
||||||
|
typically works on raw types only. The source media type
|
||||||
|
is listed first.
|
||||||
|
* Analyzer : reports about the stream contents.
|
||||||
|
* Control : controls some aspect of a hardware device
|
||||||
|
* Extracter : extracts tags/headers from a stream
|
||||||
|
* Formatter : adds tags/headers to a stream
|
||||||
|
* Connector : allows for new connections in the pipeline. (tee, ...)
|
||||||
|
* ...
|
||||||
|
|
||||||
- functional
|
- Based on media type
|
||||||
|
|
||||||
Categories are base on _intended usage_ of the element. Some elements
|
Purpose is to make a selection for elements operating on the different
|
||||||
might have other side-effects (especially for filers/effects). The purpose
|
types of media. An audio application must be able to filter out the
|
||||||
is to list enough keywords so that applications can do meaningfull filtering,
|
elements operating on audio, for example.
|
||||||
not to completely describe the functionality, that is expressed in caps etc..
|
|
||||||
|
|
||||||
* Source : produces data
|
* Audio : operates on audio data
|
||||||
* Sink : consumes data
|
* Video : operates on video data
|
||||||
* Filter : filters/transforms data, no modification on the data is
|
* Image : operates on image data. Usually this media type can also
|
||||||
intended (although it might be unavoidable). The
|
be used to make a video stream in which case it is added
|
||||||
filter can decide on input and output caps independently
|
together with the Video media type.
|
||||||
of the stream contents (GstBaseTransform).
|
* Text : operates on text data
|
||||||
* Effect : applies an effect to some data, changes to data are
|
* Metadata : operates on metadata
|
||||||
intended. Examples are colorbalance, volume. These
|
* ...
|
||||||
elements can also be implemented with GstBaseTransform.
|
|
||||||
* Demuxer : splits audio, video, ... from a stream
|
|
||||||
* Muxer : interleave audio, video, ... into one stream, this is
|
|
||||||
like mixing but without losing or degrading each separate
|
|
||||||
input stream. The reverse operation is possible with a
|
|
||||||
Demuxer that reproduces the exact same input streams.
|
|
||||||
* Decoder : decodes encoded data into a raw format, there is typically
|
|
||||||
no relation between input caps and output caps. The output
|
|
||||||
caps are defined in the stream data. This separates the
|
|
||||||
Decoder from the Filter and Effect.
|
|
||||||
* Encoder : encodes raw data into an encoded format.
|
|
||||||
* Mixer : combine audio, video, .. this is like muxing but with
|
|
||||||
applying some algorithm so that the individual streams
|
|
||||||
are not extractable anymore, there is therefore no
|
|
||||||
reverse operation to mixing. (audio mixer, video mixer, ...)
|
|
||||||
* Converter : convert audio into video, text to audio, ... The converter
|
|
||||||
typically works on raw types only. The source media type
|
|
||||||
is listed first.
|
|
||||||
* Analyzer : reports about the stream contents.
|
|
||||||
* Control : controls some aspect of a hardware device
|
|
||||||
* Extracter : extracts tags/headers from a stream
|
|
||||||
* Formatter : adds tags/headers to a stream
|
|
||||||
* Connector : allows for new connections in the pipeline. (tee, ...)
|
|
||||||
* ...
|
|
||||||
|
|
||||||
- Based on media type
|
- Extra features
|
||||||
|
|
||||||
Purpose is to make a selection for elements operating on the different
|
The purpose is to further specialize the element, mostly for
|
||||||
types of media. An audio application must be able to filter out the
|
application specific needs.
|
||||||
elements operating on audio, for example.
|
|
||||||
|
|
||||||
* Audio : operates on audio data
|
* Network : element is used in networked situations
|
||||||
* Video : operates on video data
|
* Protocol : implements some protocol (RTSP, HTTP, ...)
|
||||||
* Image : operates on image data. Usually this media type can also
|
* Payloader : encapsulate as payload (RTP, RDT,.. )
|
||||||
be used to make a video stream in which case it is added
|
* Depayloader : strip a payload (RTP, RDT,.. )
|
||||||
together with the Video media type.
|
* RTP : intended to be used in RTP applications
|
||||||
* Text : operates on text data
|
* Device : operates on some hardware device (disk, network,
|
||||||
* Metadata : operates on metadata
|
audio card, video card, usb, ...)
|
||||||
* ...
|
* Visualisation : intended to be used for audio visualisation
|
||||||
|
* Debug : intended usage is more for debugging purposes.
|
||||||
|
|
||||||
- Extra features
|
- Categories found, but not yet in one of the above lists
|
||||||
|
|
||||||
The purpose is to further specialize the element, mostly for
|
* Bin : playbin, decodebin, bin, pipeline
|
||||||
application specific needs.
|
* Codec : lots of decoders, encoder, demuxers
|
||||||
|
should be removed?
|
||||||
|
* Generic : should be removed?
|
||||||
|
* File : like network, should go to Extra?
|
||||||
|
* Editor : gnonlin, textoverlays
|
||||||
|
* DVD, GDP, LADSPA, Parser, Player, Subtitle, Testing, ...
|
||||||
|
|
||||||
* Network : element is used in networked situations
|
3) suggested order:
|
||||||
* Protocol : implements some protocol (RTSP, HTTP, ...)
|
|
||||||
* Payloader : encapsulate as payload (RTP, RDT,.. )
|
|
||||||
* Depayloader : strip a payload (RTP, RDT,.. )
|
|
||||||
* RTP : intended to be used in RTP applications
|
|
||||||
* Device : operates on some hardware device (disk, network,
|
|
||||||
audio card, video card, usb, ...)
|
|
||||||
* Visualisation : intended to be used for audio visualisation
|
|
||||||
* Debug : intended usage is more for debugging purposes.
|
|
||||||
|
|
||||||
- Categories found, but not yet in one of the above lists
|
<functional>[/<media type>]*[/<extra...>]*
|
||||||
|
|
||||||
* Bin : playbin, decodebin, bin, pipeline
|
4) examples:
|
||||||
* Codec : lots of decoders, encoder, demuxers
|
|
||||||
should be removed?
|
|
||||||
* Generic : should be removed?
|
|
||||||
* File : like network, should go to Extra?
|
|
||||||
* Editor : gnonlin, textoverlays
|
|
||||||
* DVD, GDP, LADSPA, Parser, Player, Subtitle, Testing, ...
|
|
||||||
|
|
||||||
3) suggested order:
|
apedemux : Extracter/Metadata
|
||||||
|
audiotestsrc : Source/Audio
|
||||||
|
autoaudiosink : Sink/Audio/Device
|
||||||
|
cairotimeoverlay : Mixer/Video/Text
|
||||||
|
dvdec : Decoder/Video
|
||||||
|
dvdemux : Demuxer
|
||||||
|
goom : Converter/Audio/Video
|
||||||
|
id3demux : Extracter/Metadata
|
||||||
|
udpsrc : Source/Network/Protocol/Device
|
||||||
|
videomixer : Mixer/Video
|
||||||
|
ffmpegcolorspace : Filter/Video (intended use to convert video with as little
|
||||||
|
visible change as possible)
|
||||||
|
vertigotv : Effect/Video (intended use is to change the video)
|
||||||
|
volume : Effect/Audio (intended use is to change the audio data)
|
||||||
|
vorbisdec : Decoder/Audio
|
||||||
|
vorbisenc : Encoder/Audio
|
||||||
|
oggmux : Muxer
|
||||||
|
adder : Mixer/Audio
|
||||||
|
videobox : Effect/Video
|
||||||
|
alsamixer : Control/Audio/Device
|
||||||
|
audioconvert : Filter/Audio
|
||||||
|
audioresample : Filter/Audio
|
||||||
|
xvimagesink : Sink/Video/Device
|
||||||
|
navseek : Filter/Debug
|
||||||
|
decodebin : Decoder/Demuxer
|
||||||
|
level : Filter/Analyzer/Audio
|
||||||
|
tee : Connector/Debug
|
||||||
|
|
||||||
<functional>[/<media type>]*[/<extra...>]*
|
5) open issues:
|
||||||
|
|
||||||
4) examples:
|
- how to differencial physical devices from logical ones?
|
||||||
|
|
||||||
apedemux : Extracter/Metadata
|
|
||||||
audiotestsrc : Source/Audio
|
|
||||||
autoaudiosink : Sink/Audio/Device
|
autoaudiosink : Sink/Audio/Device
|
||||||
cairotimeoverlay : Mixer/Video/Text
|
alsasink : Sink/Audio/Device
|
||||||
dvdec : Decoder/Video
|
|
||||||
dvdemux : Demuxer
|
|
||||||
goom : Converter/Audio/Video
|
|
||||||
id3demux : Extracter/Metadata
|
|
||||||
udpsrc : Source/Network/Protocol/Device
|
|
||||||
videomixer : Mixer/Video
|
|
||||||
ffmpegcolorspace : Filter/Video (intended use to convert video with as little
|
|
||||||
visible change as possible)
|
|
||||||
vertigotv : Effect/Video (intended use is to change the video)
|
|
||||||
volume : Effect/Audio (intended use is to change the audio data)
|
|
||||||
vorbisdec : Decoder/Audio
|
|
||||||
vorbisenc : Encoder/Audio
|
|
||||||
oggmux : Muxer
|
|
||||||
adder : Mixer/Audio
|
|
||||||
videobox : Effect/Video
|
|
||||||
alsamixer : Control/Audio/Device
|
|
||||||
audioconvert : Filter/Audio
|
|
||||||
audioresample : Filter/Audio
|
|
||||||
xvimagesink : Sink/Video/Device
|
|
||||||
navseek : Filter/Debug
|
|
||||||
decodebin : Decoder/Demuxer
|
|
||||||
level : Filter/Analyzer/Audio
|
|
||||||
tee : Connector/Debug
|
|
||||||
|
|
||||||
5) open issues:
|
Use cases
|
||||||
|
~~~~~~~~~
|
||||||
|
|
||||||
- how to differencial physical devices from logical ones?
|
- get a list of all elements implementing a video effect (pitivi):
|
||||||
autoaudiosink : Sink/Audio/Device
|
|
||||||
alsasink : Sink/Audio/Device
|
|
||||||
|
|
||||||
Use cases:
|
klass.contains (Effect & Video)
|
||||||
|
|
||||||
- get a list of all elements implementing a video effect (pitivi):
|
- get list of muxers (pitivi):
|
||||||
|
|
||||||
klass.contains (Effect & Video)
|
klass.contains (Muxer)
|
||||||
|
|
||||||
- get list of muxers (pitivi):
|
- get list of video encoders (pitivi):
|
||||||
|
|
||||||
klass.contains (Muxer)
|
klass.contains (Encoder & video)
|
||||||
|
|
||||||
- get list of video encoders (pitivi):
|
- Get a list of all audio/video visualisations (totem):
|
||||||
|
|
||||||
klass.contains (Encoder & video)
|
klass.contains (Visualisation)
|
||||||
|
|
||||||
- Get a list of all audio/video visualisations (totem):
|
- Get a list of all decoders/demuxer/metadata parsers/vis (playbin):
|
||||||
|
|
||||||
klass.contains (Visualisation)
|
klass.contains (Visualisation | Demuxer | Decoder | (Extractor & Metadata))
|
||||||
|
|
||||||
- Get a list of all decoders/demuxer/metadata parsers/vis (playbin):
|
- Get a list of elements that can capture from an audio device (gst-properties):
|
||||||
|
|
||||||
klass.contains (Visualisation | Demuxer | Decoder | (Extractor & Metadata))
|
klass.contains (Source & Audio & Device)
|
||||||
|
|
||||||
- Get a list of elements that can capture from an audio device (gst-properties):
|
* filters out audiotestsrc, since it is not a device
|
||||||
|
|
||||||
klass.contains (Source & Audio & Device)
|
|
||||||
|
|
||||||
* filters out audiotestsrc, since it is not a device
|
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -6,7 +6,7 @@ additions.
|
||||||
|
|
||||||
|
|
||||||
Supported Metadata standards
|
Supported Metadata standards
|
||||||
----------------------------
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
The paragraphs below list supported native metadata standards sorted by type and
|
The paragraphs below list supported native metadata standards sorted by type and
|
||||||
then in alphabetical order. Some standards have been extended to support
|
then in alphabetical order. Some standards have been extended to support
|
||||||
|
|
@ -94,7 +94,7 @@ http://wiki.creativecommons.org/Tracker_CC_Indexing
|
||||||
|
|
||||||
|
|
||||||
Current Metadata handling
|
Current Metadata handling
|
||||||
-------------------------
|
~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
When reading files, demuxers or parsers extract the metadata. It will be sent as
|
When reading files, demuxers or parsers extract the metadata. It will be sent as
|
||||||
a GST_MESSAGE_TAG to the GstBus and GST_EVENT_TAG to downstream elements.
|
a GST_MESSAGE_TAG to the GstBus and GST_EVENT_TAG to downstream elements.
|
||||||
|
|
@ -130,10 +130,11 @@ metadata.
|
||||||
|
|
||||||
|
|
||||||
Issues
|
Issues
|
||||||
------
|
~~~~~~
|
||||||
|
|
||||||
Unknown/Unmapped metadata
|
Unknown/Unmapped metadata
|
||||||
- - - - - - - - - - - - -
|
^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
Right now GStreamer can lose metadata when transcoding, remuxing content. This
|
Right now GStreamer can lose metadata when transcoding, remuxing content. This
|
||||||
can happend as we don't map all metadata fields to generic ones.
|
can happend as we don't map all metadata fields to generic ones.
|
||||||
|
|
||||||
|
|
@ -159,7 +160,8 @@ We would basically need this for each container format.
|
||||||
See also https://bugzilla.gnome.org/show_bug.cgi?id=345352
|
See also https://bugzilla.gnome.org/show_bug.cgi?id=345352
|
||||||
|
|
||||||
Lost metadata
|
Lost metadata
|
||||||
- - - - - - -
|
^^^^^^^^^^^^^
|
||||||
|
|
||||||
A case slighly different from the previous is that when an application sets a
|
A case slighly different from the previous is that when an application sets a
|
||||||
GstTagList on a pipeline. Right elements consuming tags do not report which tags
|
GstTagList on a pipeline. Right elements consuming tags do not report which tags
|
||||||
have been consumed. Especially when using elements that make metadata
|
have been consumed. Especially when using elements that make metadata
|
||||||
|
|
@ -181,7 +183,8 @@ of elements and their tags. As a convenience we could flatten the list of tags
|
||||||
for the top-level element (if the query was sent to a bin) and add that.
|
for the top-level element (if the query was sent to a bin) and add that.
|
||||||
|
|
||||||
Tags are per Element
|
Tags are per Element
|
||||||
- - - - - - - - - -
|
^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
In many cases we want tags per stream. Even metadata standards like mp4/3gp
|
In many cases we want tags per stream. Even metadata standards like mp4/3gp
|
||||||
metadata supports that. Right now GST_MESSAGE_SRC(tags) is the element. We tried
|
metadata supports that. Right now GST_MESSAGE_SRC(tags) is the element. We tried
|
||||||
changing that to the pad, but that broke applications.
|
changing that to the pad, but that broke applications.
|
||||||
|
|
@ -189,7 +192,7 @@ Also we miss the symmetric functionality in GstTagSetter. This interface is
|
||||||
usually implemented by elements.
|
usually implemented by elements.
|
||||||
|
|
||||||
Open bugs
|
Open bugs
|
||||||
- - - - -
|
^^^^^^^^^
|
||||||
|
|
||||||
https://bugzilla.gnome.org/buglist.cgi?query_format=advanced;short_desc=tag;bug_status=UNCONFIRMED;bug_status=NEW;bug_status=ASSIGNED;bug_status=REOPENED;bug_status=NEEDINFO;short_desc_type=allwordssubstr;product=GStreamer
|
https://bugzilla.gnome.org/buglist.cgi?query_format=advanced;short_desc=tag;bug_status=UNCONFIRMED;bug_status=NEW;bug_status=ASSIGNED;bug_status=REOPENED;bug_status=NEEDINFO;short_desc_type=allwordssubstr;product=GStreamer
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -20,7 +20,7 @@ peeked buffers (a queue element after sink, but that would change pull to push).
|
||||||
|
|
||||||
|
|
||||||
Design
|
Design
|
||||||
------
|
~~~~~~
|
||||||
|
|
||||||
The plan is that applications can do the following:
|
The plan is that applications can do the following:
|
||||||
pipeline = "filesrc ! tagbin"
|
pipeline = "filesrc ! tagbin"
|
||||||
|
|
@ -38,7 +38,8 @@ The plan is that applications can do the following:
|
||||||
decodebin
|
decodebin
|
||||||
|
|
||||||
Interface
|
Interface
|
||||||
---------
|
~~~~~~~~~
|
||||||
|
|
||||||
* gboolean iface property "tag-reading"
|
* gboolean iface property "tag-reading"
|
||||||
Switches the element to tagreading mode. Needed if normal element implement
|
Switches the element to tagreading mode. Needed if normal element implement
|
||||||
that behaviour. Elements will skip parsing unneeded data, don't build a
|
that behaviour. Elements will skip parsing unneeded data, don't build a
|
||||||
|
|
@ -47,7 +48,7 @@ Interface
|
||||||
Equivalent of EOS.
|
Equivalent of EOS.
|
||||||
|
|
||||||
Use Cases
|
Use Cases
|
||||||
---------
|
~~~~~~~~~
|
||||||
|
|
||||||
* mp3 with id3- and apetags
|
* mp3 with id3- and apetags
|
||||||
* plug id3demux ! apedemux
|
* plug id3demux ! apedemux
|
||||||
|
|
@ -57,7 +58,8 @@ Use Cases
|
||||||
* plug vorbisdec or special vorbiscomment reader
|
* plug vorbisdec or special vorbiscomment reader
|
||||||
|
|
||||||
Additional Thoughts
|
Additional Thoughts
|
||||||
-------------------
|
~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
* would it make sense to have 2-phase tag-reading (property on tagbin and/or
|
* would it make sense to have 2-phase tag-reading (property on tagbin and/or
|
||||||
tagread elements)
|
tagread elements)
|
||||||
* 1st phase: get tag-data that are directly embedded in the data
|
* 1st phase: get tag-data that are directly embedded in the data
|
||||||
|
|
@ -73,23 +75,25 @@ Additional Thoughts
|
||||||
gst_tag_cache_store_tag_data (GstTagCache *self, const gchar *uri, GstTagList *tags);
|
gst_tag_cache_store_tag_data (GstTagCache *self, const gchar *uri, GstTagList *tags);
|
||||||
|
|
||||||
Tests
|
Tests
|
||||||
-----
|
~~~~~
|
||||||
|
|
||||||
* write a generic test for parsers/demuxers to ensure they send tags until they
|
* write a generic test for parsers/demuxers to ensure they send tags until they
|
||||||
reached PAUSED (elements need to parse file for prerolling anyway):
|
reached PAUSED (elements need to parse file for prerolling anyway):
|
||||||
set pipeline to paused, check for tags, set to playing, error out if tags come
|
set pipeline to paused, check for tags, set to playing, error out if tags come
|
||||||
after paused
|
after paused
|
||||||
|
|
||||||
Code Locations
|
Code Locations
|
||||||
--------------
|
~~~~~~~~~~~~~~
|
||||||
|
|
||||||
* tagreadbin -> gst-plugins-base/gst/tagread
|
* tagreadbin -> gst-plugins-base/gst/tagread
|
||||||
* tagreaderiface -> gst-plugins-base/gst-libs/gst/tag
|
* tagreaderiface -> gst-plugins-base/gst-libs/gst/tag
|
||||||
|
|
||||||
Reuse
|
Reuse
|
||||||
-----
|
~~~~~
|
||||||
|
|
||||||
ogg : gst-plugins-base/ext/ogg
|
* ogg : gst-plugins-base/ext/ogg
|
||||||
avi : gst-plugins-good/gst/avi
|
* avi : gst-plugins-good/gst/avi
|
||||||
mp3 : gst-plugins-good/gst/id3demux
|
* mp3 : gst-plugins-good/gst/id3demux
|
||||||
wav : gst-plugins-good/gst/wavparse
|
* wav : gst-plugins-good/gst/wavparse
|
||||||
qt : gst-plugins-bad/gst/qtdemux
|
* qt : gst-plugins-bad/gst/qtdemux
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -10,7 +10,7 @@ This design decision has implications for the usage of the API and the objects
|
||||||
which this document explains.
|
which this document explains.
|
||||||
|
|
||||||
MT safety techniques
|
MT safety techniques
|
||||||
--------------------
|
~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
Several design patterns are used to guarantee object consistency in GStreamer.
|
Several design patterns are used to guarantee object consistency in GStreamer.
|
||||||
This is an overview of the methods used in various GStreamer subsystems.
|
This is an overview of the methods used in various GStreamer subsystems.
|
||||||
|
|
@ -140,7 +140,7 @@ Compare and swap
|
||||||
|
|
||||||
|
|
||||||
Objects
|
Objects
|
||||||
-------
|
~~~~~~~
|
||||||
|
|
||||||
* Locking involved:
|
* Locking involved:
|
||||||
|
|
||||||
|
|
@ -414,3 +414,4 @@ Objects
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
gst_iterator_free (it);
|
gst_iterator_free (it);
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,5 +1,8 @@
|
||||||
|
TODO - Future Development
|
||||||
|
-------------------------
|
||||||
|
|
||||||
API/ABI
|
API/ABI
|
||||||
-------
|
~~~~~~~
|
||||||
|
|
||||||
- implement return values from events in addition to the gboolean. This should
|
- implement return values from events in addition to the gboolean. This should
|
||||||
be done by making the event contain a GstStructure with input/output values,
|
be done by making the event contain a GstStructure with input/output values,
|
||||||
|
|
@ -66,7 +69,7 @@ API/ABI
|
||||||
|
|
||||||
|
|
||||||
IMPLEMENTATION
|
IMPLEMENTATION
|
||||||
--------------
|
~~~~~~~~~~~~~~
|
||||||
|
|
||||||
- implement more QOS, see part-qos.txt.
|
- implement more QOS, see part-qos.txt.
|
||||||
|
|
||||||
|
|
@ -76,7 +79,7 @@ IMPLEMENTATION
|
||||||
|
|
||||||
|
|
||||||
DESIGN
|
DESIGN
|
||||||
------
|
~~~~~~
|
||||||
|
|
||||||
- unlinking pads in the PAUSED state needs to make sure the stream thread is not
|
- unlinking pads in the PAUSED state needs to make sure the stream thread is not
|
||||||
executing code. Can this be done with a flush to unlock all downstream chain
|
executing code. Can this be done with a flush to unlock all downstream chain
|
||||||
|
|
|
||||||
|
|
@ -1,5 +1,8 @@
|
||||||
Pad activation
|
Pad (de)activation
|
||||||
--------------
|
------------------
|
||||||
|
|
||||||
|
Activation
|
||||||
|
~~~~~~~~~~
|
||||||
|
|
||||||
When changing states, a bin will set the state on all of its children in
|
When changing states, a bin will set the state on all of its children in
|
||||||
sink-to-source order. As elements undergo the READY->PAUSED transition,
|
sink-to-source order. As elements undergo the READY->PAUSED transition,
|
||||||
|
|
@ -73,7 +76,7 @@ already in PAUSED with an active sink pad by the time fakesrc starts
|
||||||
pushing data.
|
pushing data.
|
||||||
|
|
||||||
Deactivation
|
Deactivation
|
||||||
------------
|
~~~~~~~~~~~~
|
||||||
|
|
||||||
Pad deactivation occurs when its parent goes into the READY state or when the
|
Pad deactivation occurs when its parent goes into the READY state or when the
|
||||||
pad is deactivated explicitly by the application or element.
|
pad is deactivated explicitly by the application or element.
|
||||||
|
|
@ -82,7 +85,8 @@ activate_push() or activate_pull() with a FALSE argument, depending on the
|
||||||
activation mode of the pad.
|
activation mode of the pad.
|
||||||
|
|
||||||
Mode switching
|
Mode switching
|
||||||
--------------
|
~~~~~~~~~~~~~~
|
||||||
|
|
||||||
Changing from push to pull modes needs a bit of thought. This is actually
|
Changing from push to pull modes needs a bit of thought. This is actually
|
||||||
possible and implemented but not yet documented here.
|
possible and implemented but not yet documented here.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -18,66 +18,69 @@ and will succeed when the following events happen on the pad:
|
||||||
|
|
||||||
|
|
||||||
Flushing
|
Flushing
|
||||||
--------
|
~~~~~~~~
|
||||||
|
|
||||||
The flushing event is used to clear any data out of the
|
The flushing event is used to clear any data out of the
|
||||||
downstream elements.
|
downstream elements.
|
||||||
|
|
||||||
* Generic case
|
Generic case
|
||||||
|
^^^^^^^^^^^^
|
||||||
|
|
||||||
Consider the following pipeline:
|
Consider the following pipeline:
|
||||||
|
|
||||||
.-----. .-------. .-------.
|
.-----. .-------. .-------.
|
||||||
| src | | elem1 |\/ | elem2 |
|
| src | | elem1 |\/ | elem2 |
|
||||||
| src -> sink src -> sink src ....
|
| src -> sink src -> sink src ....
|
||||||
'-----' '-------'/\ '-------'
|
'-----' '-------'/\ '-------'
|
||||||
|
|
||||||
Where elem1.src is blocked. If the pad block is taken (the callback
|
Where elem1.src is blocked. If the pad block is taken (the callback
|
||||||
is called or the sync block returned) no data is flowing in elem2.sink.
|
is called or the sync block returned) no data is flowing in elem2.sink.
|
||||||
In this situation, the streaming thread is blocked on a GCond and is
|
In this situation, the streaming thread is blocked on a GCond and is
|
||||||
waiting to be unblocked.
|
waiting to be unblocked.
|
||||||
|
|
||||||
When sending a flushing seek upstream on elem1.src, the FLUSH_START and
|
When sending a flushing seek upstream on elem1.src, the FLUSH_START and
|
||||||
will temporary unblock the streaming thread and make all pad functions that
|
will temporary unblock the streaming thread and make all pad functions that
|
||||||
triggers a block (_push/_alloc_buffer/_push_event/_pull_range) return
|
triggers a block (_push/_alloc_buffer/_push_event/_pull_range) return
|
||||||
GST_FLOW_WRONG_STATE. This will then eventually pause the streaming thread
|
GST_FLOW_WRONG_STATE. This will then eventually pause the streaming thread
|
||||||
and release the STREAM_LOCK.
|
and release the STREAM_LOCK.
|
||||||
|
|
||||||
Since no STREAM lock is taken after the pad block it is not needed to send
|
|
||||||
the FLUSH_START event further downstream.
|
|
||||||
|
|
||||||
The FLUSH_STOP will set the srcpad to non-flushing again and is dropped
|
Since no STREAM lock is taken after the pad block it is not needed to send
|
||||||
for the same reason. From then on, the new data after the flushing seek
|
the FLUSH_START event further downstream.
|
||||||
will be queued when the pad block is taken again.
|
|
||||||
|
|
||||||
* Case where the stream is blocking downstream
|
The FLUSH_STOP will set the srcpad to non-flushing again and is dropped
|
||||||
|
for the same reason. From then on, the new data after the flushing seek
|
||||||
|
will be queued when the pad block is taken again.
|
||||||
|
|
||||||
The example above is only valid if the elem1.src pad is really blocking
|
Case where the stream is blocking downstream
|
||||||
(callback called or sync block returned).
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
In the case where the stream is blocking further downstream (on elem2.src
|
|
||||||
for example, or on a blocking queue), extra care has to be taken.
|
|
||||||
|
|
||||||
Consider the following pipeline:
|
The example above is only valid if the elem1.src pad is really blocking
|
||||||
|
(callback called or sync block returned).
|
||||||
|
|
||||||
|
In the case where the stream is blocking further downstream (on elem2.src
|
||||||
|
for example, or on a blocking queue), extra care has to be taken.
|
||||||
|
|
||||||
|
Consider the following pipeline:
|
||||||
|
|
||||||
.-----. .-------. .-------.
|
.-----. .-------. .-------.
|
||||||
| src | | elem1 |\/ | elem2 |
|
| src | | elem1 |\/ | elem2 |
|
||||||
| src -> sink src -> sink src .... Blocking somewhere downstream
|
| src -> sink src -> sink src .... Blocking somewhere downstream
|
||||||
'-----' '-------'/\ '-------'
|
'-----' '-------'/\ '-------'
|
||||||
|
|
||||||
A pad block has been requested by the user on elem1.src , but since the stream
|
A pad block has been requested by the user on elem1.src , but since the stream
|
||||||
is blocking somewhere downstream, the callback is not called or the sync block
|
is blocking somewhere downstream, the callback is not called or the sync block
|
||||||
does not return.
|
does not return.
|
||||||
|
|
||||||
In order for the block to happen, a FLUSH_START needs to be sent directly on
|
In order for the block to happen, a FLUSH_START needs to be sent directly on
|
||||||
the downstream blocking element/pad so that it releases the stream lock, and it
|
the downstream blocking element/pad so that it releases the stream lock, and it
|
||||||
gives a chance for the elem1.src pad to block.
|
gives a chance for the elem1.src pad to block.
|
||||||
|
|
||||||
|
|
||||||
Use cases:
|
Use cases
|
||||||
----------
|
~~~~~~~~~
|
||||||
|
|
||||||
* Prerolling a partial pipeline
|
Prerolling a partial pipeline
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
.---------. .---------. .----------.
|
.---------. .---------. .----------.
|
||||||
| filesrc | | demuxer | .-----. | decoder1 |
|
| filesrc | | demuxer | .-----. | decoder1 |
|
||||||
|
|
@ -89,27 +92,28 @@ Use cases:
|
||||||
'---------' '-----' '----------' X
|
'---------' '-----' '----------' X
|
||||||
|
|
||||||
|
|
||||||
The purpose is to create the pipeline dynamically up to the
|
The purpose is to create the pipeline dynamically up to the
|
||||||
decoders but not yet connect them to a sink and without losing
|
decoders but not yet connect them to a sink and without losing
|
||||||
any data.
|
any data.
|
||||||
|
|
||||||
To do this, the source pads of the decoders is blocked so that no
|
To do this, the source pads of the decoders is blocked so that no
|
||||||
events or buffers can escape and we don't interrupt the stream.
|
events or buffers can escape and we don't interrupt the stream.
|
||||||
|
|
||||||
When all of the dynamic pad are created (no-more-pads emited by the
|
When all of the dynamic pad are created (no-more-pads emited by the
|
||||||
branching point, ie, the demuxer or the queues filled) and the pads
|
branching point, ie, the demuxer or the queues filled) and the pads
|
||||||
are blocked (blocked callback received) the pipeline is completely
|
are blocked (blocked callback received) the pipeline is completely
|
||||||
prerolled.
|
prerolled.
|
||||||
|
|
||||||
It should then be possible to perform the following actions on the
|
It should then be possible to perform the following actions on the
|
||||||
prerolled pipeline:
|
prerolled pipeline:
|
||||||
|
|
||||||
- query duration/position
|
- query duration/position
|
||||||
- perform a flushing seek to preroll a new position
|
- perform a flushing seek to preroll a new position
|
||||||
- connect other elements and unblock the blocked pads.
|
- connect other elements and unblock the blocked pads.
|
||||||
|
|
||||||
|
|
||||||
* dynamically switching an element in a PLAYING pipeline.
|
dynamically switching an element in a PLAYING pipeline
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
|
||||||
.----------. .----------. .----------.
|
.----------. .----------. .----------.
|
||||||
|
|
@ -121,8 +125,8 @@ Use cases:
|
||||||
sink src
|
sink src
|
||||||
'----------'
|
'----------'
|
||||||
|
|
||||||
The purpose is to replace element2 with element4 in the PLAYING
|
The purpose is to replace element2 with element4 in the PLAYING
|
||||||
pipeline.
|
pipeline.
|
||||||
|
|
||||||
1) block element1 src pad. This can be done async.
|
1) block element1 src pad. This can be done async.
|
||||||
2) wait for block to happen. at that point nothing is flowing between
|
2) wait for block to happen. at that point nothing is flowing between
|
||||||
|
|
@ -144,18 +148,18 @@ Use cases:
|
||||||
element should at least be PAUSED.
|
element should at least be PAUSED.
|
||||||
9) unblock element1 src
|
9) unblock element1 src
|
||||||
|
|
||||||
The same flow can be used to replace an element in a PAUSED pipeline. Only
|
The same flow can be used to replace an element in a PAUSED pipeline. Only
|
||||||
special care has to be taken when performing step 2) which has to be done
|
special care has to be taken when performing step 2) which has to be done
|
||||||
async or it might deadlock. In the async callback one can then perform the
|
async or it might deadlock. In the async callback one can then perform the
|
||||||
steps from 3). In a playing pipeline one can of course use the async block
|
steps from 3). In a playing pipeline one can of course use the async block
|
||||||
as well, so that there is a generic method for both PAUSED and PLAYING.
|
as well, so that there is a generic method for both PAUSED and PLAYING.
|
||||||
|
|
||||||
The same flow works as well for any chain of multiple elements and might
|
The same flow works as well for any chain of multiple elements and might
|
||||||
be implemented with a helper function in the future.
|
be implemented with a helper function in the future.
|
||||||
|
|
||||||
|
|
||||||
Issues
|
Issues
|
||||||
------
|
~~~~~~
|
||||||
|
|
||||||
When an EOS event has passed a pad and the pad is set to blocked, the block will
|
When an EOS event has passed a pad and the pad is set to blocked, the block will
|
||||||
never happen because no data is going to flow anymore. One possibility is to
|
never happen because no data is going to flow anymore. One possibility is to
|
||||||
|
|
|
||||||
|
|
@ -21,7 +21,7 @@ We want to be able to implement the following features:
|
||||||
- easy (backward compatible) application notification of buffering
|
- easy (backward compatible) application notification of buffering
|
||||||
- the possibility for the application to do more complex buffering
|
- the possibility for the application to do more complex buffering
|
||||||
|
|
||||||
Some use cases:
|
Some use cases:
|
||||||
|
|
||||||
* Stream buffering:
|
* Stream buffering:
|
||||||
|
|
||||||
|
|
@ -112,7 +112,7 @@ We want to be able to implement the following features:
|
||||||
|
|
||||||
|
|
||||||
Messages
|
Messages
|
||||||
--------
|
~~~~~~~~
|
||||||
|
|
||||||
A GST_MESSAGE_BUFFERING must be posted on the bus when playback temporarily
|
A GST_MESSAGE_BUFFERING must be posted on the bus when playback temporarily
|
||||||
stops to buffer and when buffering finishes. When percentage field in the
|
stops to buffer and when buffering finishes. When percentage field in the
|
||||||
|
|
@ -137,27 +137,24 @@ are added to the buffering message:
|
||||||
|
|
||||||
"avg-in-rate", G_TYPE_INT
|
"avg-in-rate", G_TYPE_INT
|
||||||
- gives the average input buffering speed in bytes/second. -1 is unknown.
|
- gives the average input buffering speed in bytes/second. -1 is unknown.
|
||||||
|
|
||||||
This is the average number of bytes per second that is received on the
|
This is the average number of bytes per second that is received on the
|
||||||
buffering element input (sink) pads. It is a measurement of the network
|
buffering element input (sink) pads. It is a measurement of the network
|
||||||
speed in most cases.
|
speed in most cases.
|
||||||
|
|
||||||
"avg-out-rate", G_TYPE_INT
|
"avg-out-rate", G_TYPE_INT
|
||||||
- gives the average consumption speed in bytes/second. -1 is unknown.
|
- gives the average consumption speed in bytes/second. -1 is unknown.
|
||||||
|
|
||||||
This is the average number of bytes per second that is consumed by the
|
This is the average number of bytes per second that is consumed by the
|
||||||
downstream element of the buffering element.
|
downstream element of the buffering element.
|
||||||
|
|
||||||
"buffering-left", G_TYPE_INT64
|
"buffering-left", G_TYPE_INT64
|
||||||
- gives the estimated time that bufferring will take in milliseconds.
|
- gives the estimated time that bufferring will take in milliseconds.
|
||||||
-1 unknown.
|
-1 unknown.
|
||||||
|
This is measured based on the avg-in-rate and the filled level of the
|
||||||
This is measured based on the avg-in-rate and the filled level of the
|
queue. The application can use this hint to update the GUI about the
|
||||||
queue. The application can use this hint to update the GUI about the
|
estimated remaining time that buffering will take.
|
||||||
estimated remaining time that buffering will take.
|
|
||||||
|
|
||||||
Application
|
Application
|
||||||
-----------
|
~~~~~~~~~~~
|
||||||
|
|
||||||
While data is buffered, the pipeline should remain in the PAUSED state. It is
|
While data is buffered, the pipeline should remain in the PAUSED state. It is
|
||||||
also possible that more data should be buffered while the pipeline is PLAYING,
|
also possible that more data should be buffered while the pipeline is PLAYING,
|
||||||
|
|
@ -180,7 +177,7 @@ time to resume playback to get uninterrupted playback.
|
||||||
|
|
||||||
|
|
||||||
Buffering Query
|
Buffering Query
|
||||||
---------------
|
~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
In addition to the BUFFERING messages posted by the buffering elements we want
|
In addition to the BUFFERING messages posted by the buffering elements we want
|
||||||
to be able to query the same information from the application. We also want to
|
to be able to query the same information from the application. We also want to
|
||||||
|
|
@ -222,7 +219,7 @@ and newest item (expressed in "format") in the buffer.
|
||||||
|
|
||||||
|
|
||||||
Defaults
|
Defaults
|
||||||
--------
|
~~~~~~~~
|
||||||
|
|
||||||
Some defaults for common elements:
|
Some defaults for common elements:
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -21,17 +21,18 @@ for the following extra functionality:
|
||||||
|
|
||||||
|
|
||||||
Use cases
|
Use cases
|
||||||
---------
|
~~~~~~~~~
|
||||||
|
|
||||||
A typical use case for multimedia pipelines is to append or remove 'headers'
|
A typical use case for multimedia pipelines is to append or remove 'headers'
|
||||||
from packets of data.
|
from packets of data.
|
||||||
|
|
||||||
* Generating RTP packets from h264 video
|
Generating RTP packets from h264 video
|
||||||
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
We receive as input a GstBuffer with an encoded h264 image and we need to
|
We receive as input a GstBuffer with an encoded h264 image and we need to
|
||||||
create RTP packets containing this h264 data as the payload. We typically need
|
create RTP packets containing this h264 data as the payload. We typically need
|
||||||
to fragment the h264 data into multiple packets, each with their own RTP and
|
to fragment the h264 data into multiple packets, each with their own RTP and
|
||||||
payload specific header.
|
payload specific header.
|
||||||
|
|
||||||
+-------+-------+---------------------------+--------+
|
+-------+-------+---------------------------+--------+
|
||||||
input H264 buffer: | NALU1 | NALU2 | ..... | NALUx |
|
input H264 buffer: | NALU1 | NALU2 | ..... | NALUx |
|
||||||
|
|
@ -41,25 +42,25 @@ from packets of data.
|
||||||
+-+ +-------+ +-+ +-------+ +-+ +-------+
|
+-+ +-------+ +-+ +-------+ +-+ +-------+
|
||||||
output bufferlist: | | | NALU1 | | | | NALU2 | .... | | | NALUx |
|
output bufferlist: | | | NALU1 | | | | NALU2 | .... | | | NALUx |
|
||||||
+-+ +-------+ +-+ +-------+ +-+ +-------+
|
+-+ +-------+ +-+ +-------+ +-+ +-------+
|
||||||
|
: : : :
|
||||||
\-----------/ \-----------/
|
\-----------/ \-----------/
|
||||||
group 1 group 2
|
group 1 group 2
|
||||||
|
|
||||||
The output bufferlist consists of x groups consisting of an RTP payload header
|
The output bufferlist consists of x groups consisting of an RTP payload header
|
||||||
and a subbuffer of the original input H264 buffer. Since the rtp headers and
|
and a subbuffer of the original input H264 buffer. Since the rtp headers and
|
||||||
the h264 data don't need to be contiguous in memory, we can avoid to memcpy the
|
the h264 data don't need to be contiguous in memory, we can avoid to memcpy the
|
||||||
h264 data into the rtp packets.
|
h264 data into the rtp packets.
|
||||||
|
|
||||||
Since we can generate a bufferlist with multiple groups, we can push all the
|
Since we can generate a bufferlist with multiple groups, we can push all the
|
||||||
RTP packets for the input data to the next element in one operation.
|
RTP packets for the input data to the next element in one operation.
|
||||||
|
|
||||||
A typical udpsink will then use something like sendmsg to send the groups on
|
A typical udpsink will then use something like sendmsg to send the groups on
|
||||||
the network inside one UDP packet. This will further avoid having to memcpy
|
the network inside one UDP packet. This will further avoid having to memcpy
|
||||||
data into contiguous memory.
|
data into contiguous memory.
|
||||||
|
|
||||||
|
|
||||||
API
|
API
|
||||||
---
|
~~~
|
||||||
|
|
||||||
The GstBufferList is an opaque data structure and is operated on using an
|
The GstBufferList is an opaque data structure and is operated on using an
|
||||||
iterator. It derives from GstMiniObject so that it has basic refcounting and
|
iterator. It derives from GstMiniObject so that it has basic refcounting and
|
||||||
|
|
@ -78,7 +79,7 @@ group.
|
||||||
|
|
||||||
|
|
||||||
Metadata
|
Metadata
|
||||||
--------
|
~~~~~~~~
|
||||||
|
|
||||||
Each of the buffers inside the bufferlist can have metadata assiociated with it.
|
Each of the buffers inside the bufferlist can have metadata assiociated with it.
|
||||||
|
|
||||||
|
|
@ -92,3 +93,4 @@ first group in the bufferlist. This means that:
|
||||||
|
|
||||||
This allows for efficient (re)timestamping and re-typing (caps) of a group of
|
This allows for efficient (re)timestamping and re-typing (caps) of a group of
|
||||||
buffers without having to modify each of the buffer's metadata.
|
buffers without having to modify each of the buffer's metadata.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -11,7 +11,7 @@ The time reported by the clock is called the absolute_time.
|
||||||
|
|
||||||
|
|
||||||
Clock Selection
|
Clock Selection
|
||||||
---------------
|
~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
To synchronize the different elements, the GstPipeline is responsible for
|
To synchronize the different elements, the GstPipeline is responsible for
|
||||||
selecting and distributing a global GstClock for all the elements in it.
|
selecting and distributing a global GstClock for all the elements in it.
|
||||||
|
|
@ -35,7 +35,7 @@ PLAYING and is described in part-states.txt.
|
||||||
|
|
||||||
|
|
||||||
Clock features
|
Clock features
|
||||||
--------------
|
~~~~~~~~~~~~~~
|
||||||
|
|
||||||
The clock supports periodic and single shot clock notifications both
|
The clock supports periodic and single shot clock notifications both
|
||||||
synchronous and asynchronous.
|
synchronous and asynchronous.
|
||||||
|
|
@ -73,7 +73,7 @@ the clock is not PLAYING.
|
||||||
|
|
||||||
|
|
||||||
Clock implementations
|
Clock implementations
|
||||||
---------------------
|
~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
The GStreamer core provides a GstSystemClock based on the system time.
|
The GStreamer core provides a GstSystemClock based on the system time.
|
||||||
Asynchronous callbacks are scheduled from an internal thread.
|
Asynchronous callbacks are scheduled from an internal thread.
|
||||||
|
|
|
||||||
|
|
@ -1,5 +1,5 @@
|
||||||
Documentation conventions
|
Documentation conventions
|
||||||
=========================
|
-------------------------
|
||||||
|
|
||||||
Due to the potential for exponential growth, several abbreviating conventions will be used throughout this
|
Due to the potential for exponential growth, several abbreviating conventions will be used throughout this
|
||||||
documentation. These conventions have grown primarily from extremely in-depth discussions of the architecure in IRC.
|
documentation. These conventions have grown primarily from extremely in-depth discussions of the architecure in IRC.
|
||||||
|
|
@ -7,21 +7,21 @@ This has verified the safety of these conventions, if used properly. There are
|
||||||
context is rigorously observed.
|
context is rigorously observed.
|
||||||
|
|
||||||
Object classes
|
Object classes
|
||||||
--------------
|
~~~~~~~~~~~~~~
|
||||||
|
|
||||||
Since everything starts with Gst, we will generally refer to objects by the shorter name, i.e. Element or Pad. These
|
Since everything starts with Gst, we will generally refer to objects by the shorter name, i.e. Element or Pad. These
|
||||||
names will always have their first letter capitalized.
|
names will always have their first letter capitalized.
|
||||||
|
|
||||||
Function names
|
Function names
|
||||||
--------------
|
~~~~~~~~~~~~~~
|
||||||
|
|
||||||
Within the context of a given object, functions defined in that object's header and/or source file will have their
|
Within the context of a given object, functions defined in that object's header and/or source file will have their
|
||||||
object-specific prefix stripped. For instance, gst_element_add_pad() would be referred to as simply _add_pad(). Note
|
object-specific prefix stripped. For instance, gst_element_add_pad() would be referred to as simply _add_pad(). Note
|
||||||
that the trailing parentheses should always be present, but sometimes may not be. A prefixing underscore (_) will
|
that the trailing parentheses should always be present, but sometimes may not be. A prefixing underscore (_) will
|
||||||
always tell you it's a function, however, regardless of the presence or absence of the trailing parentheses.
|
always tell you it's a function, however, regardless of the presence or absence of the trailing parentheses.
|
||||||
|
|
||||||
#defines and enums
|
defines and enums
|
||||||
------------------
|
~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
Values and macros defined as enums and preprocessor macros will be referred to in all capitals, as per their
|
Values and macros defined as enums and preprocessor macros will be referred to in all capitals, as per their
|
||||||
definition. This includes object flags and element states, as well as general enums. Examples are the states NULL,
|
definition. This includes object flags and element states, as well as general enums. Examples are the states NULL,
|
||||||
|
|
@ -31,23 +31,25 @@ element flags should be cross-checked with the header, as there are currently tw
|
||||||
_FLAGS_ in the middle.
|
_FLAGS_ in the middle.
|
||||||
|
|
||||||
Drawing conventions
|
Drawing conventions
|
||||||
===================
|
~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
When drawing pictures the folowing conventions apply:
|
When drawing pictures the folowing conventions apply:
|
||||||
|
|
||||||
objects
|
objects
|
||||||
-------
|
^^^^^^^
|
||||||
|
|
||||||
|
Objects are drawn with a box like:
|
||||||
|
|
||||||
Objects are drawn with a box like
|
|
||||||
+------+
|
+------+
|
||||||
| |
|
| |
|
||||||
+------+
|
+------+
|
||||||
|
|
||||||
|
|
||||||
pointers
|
pointers
|
||||||
--------
|
^^^^^^^^
|
||||||
|
|
||||||
a pointer to an object.
|
a pointer to an object.
|
||||||
|
|
||||||
+-----+
|
+-----+
|
||||||
*--->| |
|
*--->| |
|
||||||
+-----+
|
+-----+
|
||||||
|
|
@ -58,7 +60,7 @@ an invalid pointer, this is a pointer that should not be used.
|
||||||
|
|
||||||
|
|
||||||
elements
|
elements
|
||||||
--------
|
^^^^^^^^
|
||||||
|
|
||||||
+----------+
|
+----------+
|
||||||
| name |
|
| name |
|
||||||
|
|
@ -66,16 +68,10 @@ elements
|
||||||
+----------+
|
+----------+
|
||||||
|
|
||||||
pad links
|
pad links
|
||||||
---------
|
^^^^^^^^^
|
||||||
|
|
||||||
-----+ +---
|
-----+ +---
|
||||||
| |
|
| |
|
||||||
src--sink
|
src--sink
|
||||||
-----+ +---
|
-----+ +---
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -11,3 +11,4 @@ following features of gstreamer:
|
||||||
- pad blocking (part-block.txt)
|
- pad blocking (part-block.txt)
|
||||||
- playback segments.
|
- playback segments.
|
||||||
- streaming vs application threads.
|
- streaming vs application threads.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -13,7 +13,7 @@ Sinks are harder to construct than other element types as they are
|
||||||
treated specially by the GStreamer core.
|
treated specially by the GStreamer core.
|
||||||
|
|
||||||
state changes
|
state changes
|
||||||
-------------
|
~~~~~~~~~~~~~
|
||||||
|
|
||||||
A sink always returns ASYNC from the state change to PAUSED, this
|
A sink always returns ASYNC from the state change to PAUSED, this
|
||||||
includes a state change from READY->PAUSED and PLAYING->PAUSED. The
|
includes a state change from READY->PAUSED and PLAYING->PAUSED. The
|
||||||
|
|
@ -29,7 +29,7 @@ and take no part in the preroll procedure.
|
||||||
Events other than EOS do not complete the preroll stage.
|
Events other than EOS do not complete the preroll stage.
|
||||||
|
|
||||||
sink overview
|
sink overview
|
||||||
-------------
|
~~~~~~~~~~~~~
|
||||||
|
|
||||||
- TODO: PREROLL_LOCK can be removed and we can safely use the STREAM_LOCK.
|
- TODO: PREROLL_LOCK can be removed and we can safely use the STREAM_LOCK.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -13,7 +13,7 @@ Typical source elements include:
|
||||||
|
|
||||||
|
|
||||||
Live sources
|
Live sources
|
||||||
------------
|
~~~~~~~~~~~~
|
||||||
|
|
||||||
A source is said to be a live source when it has the following property:
|
A source is said to be a live source when it has the following property:
|
||||||
|
|
||||||
|
|
@ -52,7 +52,7 @@ Let's look at some example sources.
|
||||||
|
|
||||||
|
|
||||||
Source types
|
Source types
|
||||||
------------
|
~~~~~~~~~~~~
|
||||||
|
|
||||||
A source element can operate in three ways:
|
A source element can operate in three ways:
|
||||||
|
|
||||||
|
|
@ -90,36 +90,36 @@ source's state change function is called.
|
||||||
|
|
||||||
|
|
||||||
Source base classes
|
Source base classes
|
||||||
-------------------
|
~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
GstBaseSrc:
|
GstBaseSrc:
|
||||||
|
|
||||||
This base class provides an implementation of a random access source and
|
This base class provides an implementation of a random access source and
|
||||||
is very well suited for file reader like sources.
|
is very well suited for file reader like sources.
|
||||||
|
|
||||||
|
|
||||||
GstPushSrc:
|
GstPushSrc:
|
||||||
|
|
||||||
Base class for block-based sources. This class is mostly useful for
|
Base class for block-based sources. This class is mostly useful for
|
||||||
elements that cannot do random access, or at least very slowly. The
|
elements that cannot do random access, or at least very slowly. The
|
||||||
source usually prefers to push out a fixed size buffer.
|
source usually prefers to push out a fixed size buffer.
|
||||||
|
|
||||||
Classes extending this base class will usually be scheduled in a push
|
Classes extending this base class will usually be scheduled in a push
|
||||||
based mode. It the peer accepts to operate without offsets and withing
|
based mode. It the peer accepts to operate without offsets and withing
|
||||||
the limits of the allowed block size, this class can operate in getrange
|
the limits of the allowed block size, this class can operate in getrange
|
||||||
based mode automatically.
|
based mode automatically.
|
||||||
|
|
||||||
The subclass should extend the methods from the baseclass in
|
The subclass should extend the methods from the baseclass in
|
||||||
addition to the create method. If the source is seekable, it
|
addition to the create method. If the source is seekable, it
|
||||||
needs to override GstBaseSrc::event() in addition to
|
needs to override GstBaseSrc::event() in addition to
|
||||||
GstBaseSrc::is_seekable() in order to retrieve the seek offset,
|
GstBaseSrc::is_seekable() in order to retrieve the seek offset,
|
||||||
which is the offset of the next buffer to be requested.
|
which is the offset of the next buffer to be requested.
|
||||||
|
|
||||||
Flushing, scheduling and sync is all handled by this base class.
|
Flushing, scheduling and sync is all handled by this base class.
|
||||||
|
|
||||||
|
|
||||||
Timestamps
|
Timestamps
|
||||||
----------
|
~~~~~~~~~~
|
||||||
|
|
||||||
A non-live source should timestamp the buffers it produces starting from 0. If
|
A non-live source should timestamp the buffers it produces starting from 0. If
|
||||||
it is not possible to timestamp every buffer (filesrc), the source is allowed to
|
it is not possible to timestamp every buffer (filesrc), the source is allowed to
|
||||||
|
|
@ -134,3 +134,4 @@ the pipeline, which is expressed as:
|
||||||
With absolute_time the time obtained from the global pipeline with
|
With absolute_time the time obtained from the global pipeline with
|
||||||
gst_clock_get_time() and base_time being the time of that clock when the
|
gst_clock_get_time() and base_time being the time of that clock when the
|
||||||
pipeline was last set to PLAYING.
|
pipeline was last set to PLAYING.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -49,7 +49,7 @@ each of the above use cases. We focus mostly on the buffer allocation strategies
|
||||||
and caps negotiation.
|
and caps negotiation.
|
||||||
|
|
||||||
Processing
|
Processing
|
||||||
----------
|
~~~~~~~~~~
|
||||||
|
|
||||||
A transform has 2 main processing functions:
|
A transform has 2 main processing functions:
|
||||||
|
|
||||||
|
|
@ -94,10 +94,10 @@ When no functions are provided, we can only process in passthrough mode.
|
||||||
|
|
||||||
|
|
||||||
Negotiation
|
Negotiation
|
||||||
-----------
|
~~~~~~~~~~~
|
||||||
|
|
||||||
Typical (re)negotiation of the transform element in push mode always goes from
|
Typical (re)negotiation of the transform element in push mode always goes from
|
||||||
sink to src, this means triggers the following sequence:
|
sink to src, this means triggers the following sequence:
|
||||||
|
|
||||||
- the sinkpad receives a buffer with new caps, this triggers the setcaps
|
- the sinkpad receives a buffer with new caps, this triggers the setcaps
|
||||||
function on the sinkpad before handing the buffer to transform.
|
function on the sinkpad before handing the buffer to transform.
|
||||||
|
|
@ -108,7 +108,7 @@ Negotiation
|
||||||
target src caps
|
target src caps
|
||||||
- the transform processes and sets the output caps on the src pad
|
- the transform processes and sets the output caps on the src pad
|
||||||
|
|
||||||
We call this downstream negotiation (DN) and it goes roughly like this:
|
We call this downstream negotiation (DN) and it goes roughly like this:
|
||||||
|
|
||||||
sinkpad transform srcpad
|
sinkpad transform srcpad
|
||||||
setcaps() | | |
|
setcaps() | | |
|
||||||
|
|
@ -119,10 +119,10 @@ Negotiation
|
||||||
| <configure caps> <-| |
|
| <configure caps> <-| |
|
||||||
|
|
||||||
|
|
||||||
These steps configure the element for a transformation from the input caps to
|
These steps configure the element for a transformation from the input caps to
|
||||||
the output caps.
|
the output caps.
|
||||||
|
|
||||||
The transform has 3 function to perform the negotiation:
|
The transform has 3 function to perform the negotiation:
|
||||||
|
|
||||||
- transform_caps():
|
- transform_caps():
|
||||||
|
|
||||||
|
|
@ -140,17 +140,17 @@ Negotiation
|
||||||
Configure the transform for a transformation between src caps and dest
|
Configure the transform for a transformation between src caps and dest
|
||||||
caps. Both caps are guaranteed to be fixed caps.
|
caps. Both caps are guaranteed to be fixed caps.
|
||||||
|
|
||||||
If no transform_caps() is defined, we can only perform the identity transform,
|
If no transform_caps() is defined, we can only perform the identity transform,
|
||||||
by default.
|
by default.
|
||||||
|
|
||||||
If no set_caps() is defined, we don't care about caps. In that case we also
|
If no set_caps() is defined, we don't care about caps. In that case we also
|
||||||
assume nothing is going to write to the buffer and we don't enforce a writable
|
assume nothing is going to write to the buffer and we don't enforce a writable
|
||||||
buffer for the transform_ip function, when present.
|
buffer for the transform_ip function, when present.
|
||||||
|
|
||||||
One common function that we need for the transform element is to find the best
|
One common function that we need for the transform element is to find the best
|
||||||
transform from one format (src) to another (dest). Since the function is
|
transform from one format (src) to another (dest). Since the function is
|
||||||
bidirectional, we will use the src->dest negotiation. Some requirements of this
|
bidirectional, we will use the src->dest negotiation. Some requirements of this
|
||||||
function are:
|
function are:
|
||||||
|
|
||||||
- has a fixed src caps
|
- has a fixed src caps
|
||||||
- finds a fixed dest caps that the transform element can transform to
|
- finds a fixed dest caps that the transform element can transform to
|
||||||
|
|
@ -158,7 +158,7 @@ Negotiation
|
||||||
- the transform function prefers to make src caps == dest caps
|
- the transform function prefers to make src caps == dest caps
|
||||||
- the transform function can optionally fixate dest caps.
|
- the transform function can optionally fixate dest caps.
|
||||||
|
|
||||||
The find_transform() function goes like this:
|
The find_transform() function goes like this:
|
||||||
|
|
||||||
- start from src aps, these caps are fixed.
|
- start from src aps, these caps are fixed.
|
||||||
- check if the caps are acceptable for us as src caps. This is usually
|
- check if the caps are acceptable for us as src caps. This is usually
|
||||||
|
|
@ -177,8 +177,8 @@ Negotiation
|
||||||
- if we run out of caps, we fail to find a transform.
|
- if we run out of caps, we fail to find a transform.
|
||||||
- if we found a destination caps, configure the transform with set_caps().
|
- if we found a destination caps, configure the transform with set_caps().
|
||||||
|
|
||||||
After this negotiation process, the transform element is usually in a steady
|
After this negotiation process, the transform element is usually in a steady
|
||||||
state. We can identify these steady states:
|
state. We can identify these steady states:
|
||||||
|
|
||||||
- src and sink pads both have the same caps. Note that when the caps are equal
|
- src and sink pads both have the same caps. Note that when the caps are equal
|
||||||
on both pads, the input and output buffers automatically have the same size.
|
on both pads, the input and output buffers automatically have the same size.
|
||||||
|
|
@ -266,14 +266,14 @@ Negotiation
|
||||||
input buffer is transformed into the output buffer. The flow is exactly
|
input buffer is transformed into the output buffer. The flow is exactly
|
||||||
the same as the case with the same-caps negotiation. (DCC)
|
the same as the case with the same-caps negotiation. (DCC)
|
||||||
|
|
||||||
We can immeditatly observe that the copy transform states will need to
|
We can immeditatly observe that the copy transform states will need to
|
||||||
allocate a buffer from a downstream element using pad-alloc. When the transform
|
allocate a buffer from a downstream element using pad-alloc. When the transform
|
||||||
element is receiving a non-writable buffer in the in-place state, it will also
|
element is receiving a non-writable buffer in the in-place state, it will also
|
||||||
need to perform a pad-alloc. There is no reason why the passthrough state would
|
need to perform a pad-alloc. There is no reason why the passthrough state would
|
||||||
perform a pad-alloc. This is important because upstream re-negotiation can only
|
perform a pad-alloc. This is important because upstream re-negotiation can only
|
||||||
happen when the transform uses pad-alloc for all outgoing buffers.
|
happen when the transform uses pad-alloc for all outgoing buffers.
|
||||||
|
|
||||||
This steady state changes when one of the following actions occur:
|
This steady state changes when one of the following actions occur:
|
||||||
|
|
||||||
- the sink pad receives new caps, this triggers the above downstream
|
- the sink pad receives new caps, this triggers the above downstream
|
||||||
renegotation process, see above for the flow.
|
renegotation process, see above for the flow.
|
||||||
|
|
@ -284,9 +284,9 @@ Negotiation
|
||||||
for example). This essentially clears the current steady state and
|
for example). This essentially clears the current steady state and
|
||||||
triggers the downstream and upstream renegotiation process.
|
triggers the downstream and upstream renegotiation process.
|
||||||
|
|
||||||
Parallel to the downstream negotiation process there is an upstream negotiation
|
Parallel to the downstream negotiation process there is an upstream negotiation
|
||||||
process. The handling and proxy of buffer-alloc is the most comple part of the
|
process. The handling and proxy of buffer-alloc is the most comple part of the
|
||||||
transform element. This upstream negotiation process has 3 cases: (UN)
|
transform element. This upstream negotiation process has 3 cases: (UN)
|
||||||
|
|
||||||
- upstream calls the buffer-alloc function of the transform sinkpad and this
|
- upstream calls the buffer-alloc function of the transform sinkpad and this
|
||||||
call is proxied downstream (UNP)
|
call is proxied downstream (UNP)
|
||||||
|
|
@ -295,13 +295,13 @@ Negotiation
|
||||||
- the transform calls the pad-alloc function downstream to allocate a new
|
- the transform calls the pad-alloc function downstream to allocate a new
|
||||||
output buffer (but not because of a proxied buffer-alloc) (UNA)
|
output buffer (but not because of a proxied buffer-alloc) (UNA)
|
||||||
|
|
||||||
The case where the pad-alloc is called because an output buffer must be
|
The case where the pad-alloc is called because an output buffer must be
|
||||||
generated in the chain function is handled above in the copy-transform and the
|
generated in the chain function is handled above in the copy-transform and the
|
||||||
in-place transform when the input buffer is not writable or the input buffer
|
in-place transform when the input buffer is not writable or the input buffer
|
||||||
size is smaller than the output size.
|
size is smaller than the output size.
|
||||||
|
|
||||||
We are left with the last case (proxy an incomming pad-alloc or not). We have 2
|
We are left with the last case (proxy an incomming pad-alloc or not). We have 2
|
||||||
possibilities here:
|
possibilities here:
|
||||||
|
|
||||||
- pad-alloc is called with the same caps as are currently being handled by
|
- pad-alloc is called with the same caps as are currently being handled by
|
||||||
the transform on the sinkcaps. Note that this will only be true when the
|
the transform on the sinkcaps. Note that this will only be true when the
|
||||||
|
|
@ -400,14 +400,14 @@ Negotiation
|
||||||
<----------------------------------| |
|
<----------------------------------| |
|
||||||
| | |
|
| | |
|
||||||
|
|
||||||
In order to perform passthrough buffer-alloc or pad-alloc, we need to be able
|
In order to perform passthrough buffer-alloc or pad-alloc, we need to be able
|
||||||
to get the size of the output buffer after the transform.
|
to get the size of the output buffer after the transform.
|
||||||
|
|
||||||
For passthrough buffer-alloc, this is trivial: the input size equals the output
|
|
||||||
size.
|
|
||||||
|
|
||||||
For the copy transform or the in-place transform we need additional function to
|
For passthrough buffer-alloc, this is trivial: the input size equals the output
|
||||||
retrieve the size. There are two functions:
|
size.
|
||||||
|
|
||||||
|
For the copy transform or the in-place transform we need additional function to
|
||||||
|
retrieve the size. There are two functions:
|
||||||
|
|
||||||
- transform_size()
|
- transform_size()
|
||||||
|
|
||||||
|
|
@ -426,35 +426,35 @@ Negotiation
|
||||||
|
|
||||||
|
|
||||||
Issues
|
Issues
|
||||||
------
|
~~~~~~
|
||||||
|
|
||||||
passthrough and in-place transforms (with writable buffers) never need to
|
passthrough and in-place transforms (with writable buffers) never need to
|
||||||
perform a pad-alloc on the srcpad. This means that if upstream negotiation
|
perform a pad-alloc on the srcpad. This means that if upstream negotiation
|
||||||
happens, the transform element will never know about it.
|
happens, the transform element will never know about it.
|
||||||
|
|
||||||
The transform element will keep therefore track of the allocation pattern of
|
The transform element will keep therefore track of the allocation pattern of
|
||||||
the peer elements. We can see the following cases:
|
the peer elements. We can see the following cases:
|
||||||
|
|
||||||
- upstream peer calls buffer-alloc on the sinkpad of the transform. In some
|
- upstream peer calls buffer-alloc on the sinkpad of the transform. In some
|
||||||
cases (see above) this call gets proxied or not.
|
cases (see above) this call gets proxied or not.
|
||||||
|
|
||||||
- upstream peer does never call buffer-alloc.
|
- upstream peer does never call buffer-alloc.
|
||||||
|
|
||||||
We will keeps state about this allocation pattern and perform the following in
|
We will keeps state about this allocation pattern and perform the following in
|
||||||
each case respectively:
|
each case respectively:
|
||||||
|
|
||||||
- Upstream calls buffer-alloc: In passthrough and (some) in-place we proxy
|
- Upstream calls buffer-alloc: In passthrough and (some) in-place we proxy
|
||||||
this call onto the downstream element. If the caps are changed, we mark
|
this call onto the downstream element. If the caps are changed, we mark
|
||||||
a flag that we will require a new pad-alloc for the output of the next
|
a flag that we will require a new pad-alloc for the output of the next
|
||||||
output buffer.
|
output buffer.
|
||||||
|
|
||||||
- upstream peer does not call buffer-alloc: We always perform a pad-alloc
|
- upstream peer does not call buffer-alloc: We always perform a pad-alloc
|
||||||
when processing buffers. We can further optimize by only looking at the
|
when processing buffers. We can further optimize by only looking at the
|
||||||
returned caps instead of doing a full, needless buffer copy.
|
returned caps instead of doing a full, needless buffer copy.
|
||||||
|
|
||||||
|
|
||||||
Use cases
|
Use cases
|
||||||
---------
|
~~~~~~~~~
|
||||||
|
|
||||||
videotestsrc ! ximagesink
|
videotestsrc ! ximagesink
|
||||||
|
|
||||||
|
|
@ -478,7 +478,8 @@ Use cases
|
||||||
- resizing the videosink makes videoscale perform the scaling.
|
- resizing the videosink makes videoscale perform the scaling.
|
||||||
|
|
||||||
Problematic
|
Problematic
|
||||||
-----------
|
~~~~~~~~~~~
|
||||||
|
|
||||||
filesrc location=~/media/moveyourfeet.mov ! decodebin !
|
filesrc location=~/media/moveyourfeet.mov ! decodebin !
|
||||||
ffmpegcolorspace ! videoscale ! ffmpegcolorspace ! ximagesink -v
|
ffmpegcolorspace ! videoscale ! ffmpegcolorspace ! ximagesink -v
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -27,7 +27,7 @@ Different types of events exist to implement various functionalities.
|
||||||
in DVD.
|
in DVD.
|
||||||
|
|
||||||
FLUSH_START/STOP
|
FLUSH_START/STOP
|
||||||
----------------
|
~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
A flush event is sent both downstream and upstream to clear any pending data
|
A flush event is sent both downstream and upstream to clear any pending data
|
||||||
from the pipeline. This might be needed to make the graph more responsive
|
from the pipeline. This might be needed to make the graph more responsive
|
||||||
|
|
@ -65,7 +65,7 @@ base_time (see part-clocks.txt and part-synchronisation.txt).
|
||||||
|
|
||||||
|
|
||||||
EOS
|
EOS
|
||||||
---
|
~~~
|
||||||
|
|
||||||
The EOS event can only be sent on a sinkpad. It is typically emited by the
|
The EOS event can only be sent on a sinkpad. It is typically emited by the
|
||||||
source element when it has finished sending data. This event is mainly sent
|
source element when it has finished sending data. This event is mainly sent
|
||||||
|
|
@ -107,7 +107,7 @@ A FLUSH_STOP event on an element flushes the EOS state and all pending EOS messa
|
||||||
|
|
||||||
|
|
||||||
NEWSEGMENT
|
NEWSEGMENT
|
||||||
-------------
|
~~~~~~~~~~
|
||||||
|
|
||||||
A newsegment event is sent downstream by an element to indicate that the following
|
A newsegment event is sent downstream by an element to indicate that the following
|
||||||
group of buffers start and end at the specified positions. The newsegment event
|
group of buffers start and end at the specified positions. The newsegment event
|
||||||
|
|
@ -146,7 +146,7 @@ make the buffer timestamps increasing (part-segments.txt).
|
||||||
|
|
||||||
|
|
||||||
TAG
|
TAG
|
||||||
---
|
~~~
|
||||||
|
|
||||||
The tag event is sent downstream when an element has discovered metadata
|
The tag event is sent downstream when an element has discovered metadata
|
||||||
tags in a media file. Encoders can use this event to adjust their tagging
|
tags in a media file. Encoders can use this event to adjust their tagging
|
||||||
|
|
@ -154,15 +154,17 @@ system. A tag is serialized with buffers.
|
||||||
|
|
||||||
|
|
||||||
BUFFERSIZE
|
BUFFERSIZE
|
||||||
----------
|
~~~~~~~~~~
|
||||||
|
|
||||||
|
NOTE: This event is not yet implemented.
|
||||||
|
|
||||||
An element can suggest a buffersize for downstream elements. This is
|
An element can suggest a buffersize for downstream elements. This is
|
||||||
typically done by elements that produce data on multiple source pads
|
typically done by elements that produce data on multiple source pads
|
||||||
such as demuxers. This event is currently not yet defined nor used.
|
such as demuxers.
|
||||||
|
|
||||||
|
|
||||||
QOS
|
QOS
|
||||||
---
|
~~~
|
||||||
|
|
||||||
A QOS, or quality of service message, is generated in an element to report
|
A QOS, or quality of service message, is generated in an element to report
|
||||||
to the upstream elements about the current quality of real-time performance
|
to the upstream elements about the current quality of real-time performance
|
||||||
|
|
@ -171,7 +173,7 @@ of framedrops they have. (see part-qos.txt)
|
||||||
|
|
||||||
|
|
||||||
SEEK
|
SEEK
|
||||||
----
|
~~~~
|
||||||
|
|
||||||
A seek event is issued by the application to configure the playback range
|
A seek event is issued by the application to configure the playback range
|
||||||
of a stream. It is called form the application thread and travels upstream.
|
of a stream. It is called form the application thread and travels upstream.
|
||||||
|
|
@ -227,7 +229,7 @@ part-seeking.txt.
|
||||||
|
|
||||||
|
|
||||||
NAVIGATION
|
NAVIGATION
|
||||||
----------
|
~~~~~~~~~~~
|
||||||
|
|
||||||
A navigation event is generated by a sink element to signal the elements
|
A navigation event is generated by a sink element to signal the elements
|
||||||
of a navigation event such as a mouse movement or button click.
|
of a navigation event such as a mouse movement or button click.
|
||||||
|
|
@ -235,7 +237,7 @@ Navigation events travel upstream.
|
||||||
|
|
||||||
|
|
||||||
LATENCY
|
LATENCY
|
||||||
-------
|
~~~~~~~
|
||||||
|
|
||||||
A latency event is used to configure a certain latency in the pipeline. It
|
A latency event is used to configure a certain latency in the pipeline. It
|
||||||
contains a single GstClockTime with the required latency. The latency value is
|
contains a single GstClockTime with the required latency. The latency value is
|
||||||
|
|
@ -246,9 +248,9 @@ timestamps of the buffer in order to delay their presentation.
|
||||||
|
|
||||||
|
|
||||||
DRAIN
|
DRAIN
|
||||||
-----
|
~~~~~
|
||||||
|
|
||||||
This event is not yet implemented.
|
NOTE: This event is not yet implemented.
|
||||||
|
|
||||||
Drain event indicates that upstream is about to perform a real-time event, such
|
Drain event indicates that upstream is about to perform a real-time event, such
|
||||||
as pausing to present an interactive menu or such, and needs to wait for all
|
as pausing to present an interactive menu or such, and needs to wait for all
|
||||||
|
|
|
||||||
|
|
@ -26,7 +26,7 @@ segments of data in PLAYING.
|
||||||
|
|
||||||
|
|
||||||
Use Cases
|
Use Cases
|
||||||
---------
|
~~~~~~~~~
|
||||||
|
|
||||||
* frame stepping in video only pipeline in PAUSED
|
* frame stepping in video only pipeline in PAUSED
|
||||||
|
|
||||||
|
|
@ -92,10 +92,10 @@ Use Cases
|
||||||
|
|
||||||
|
|
||||||
events
|
events
|
||||||
------
|
~~~~~~
|
||||||
|
|
||||||
A new GST_EVENT_STEP event is introduced to start the step operation.
|
A new GST_EVENT_STEP event is introduced to start the step operation.
|
||||||
The step event is created with the following fields in the structure:
|
The step event is created with the following fields in the structure:
|
||||||
|
|
||||||
"format", GST_TYPE_FORMAT
|
"format", GST_TYPE_FORMAT
|
||||||
The format of the step units
|
The format of the step units
|
||||||
|
|
@ -146,29 +146,29 @@ events
|
||||||
flag is passed to the corresponding GST_MESSAGE_STEP_DONE.
|
flag is passed to the corresponding GST_MESSAGE_STEP_DONE.
|
||||||
|
|
||||||
|
|
||||||
The application will create a STEP event to start or stop the stepping
|
The application will create a STEP event to start or stop the stepping
|
||||||
operation. Both stepping in PAUSED and PLAYING can be performed by means of
|
operation. Both stepping in PAUSED and PLAYING can be performed by means of
|
||||||
the flush flag.
|
the flush flag.
|
||||||
|
|
||||||
The event is usually sent to the pipeline, which will typically distribute the
|
The event is usually sent to the pipeline, which will typically distribute the
|
||||||
event to all of its sinks. For some use cases, like frame stepping on video
|
event to all of its sinks. For some use cases, like frame stepping on video
|
||||||
frames only, the event should only be sent to the video sink and upon reception
|
frames only, the event should only be sent to the video sink and upon reception
|
||||||
of the STEP_DONE message, one can step the other sinks to align the streams
|
of the STEP_DONE message, one can step the other sinks to align the streams
|
||||||
again.
|
again.
|
||||||
|
|
||||||
For large stepping amounts, there needs to be enough queueing in front of all
|
For large stepping amounts, there needs to be enough queueing in front of all
|
||||||
the sinks. If large steps need to be performed, they can be split up into
|
the sinks. If large steps need to be performed, they can be split up into
|
||||||
smaller step operations using the "intermediate" flag on the step.
|
smaller step operations using the "intermediate" flag on the step.
|
||||||
|
|
||||||
Since the step event does not update the base_time of any of the elements, the
|
Since the step event does not update the base_time of any of the elements, the
|
||||||
sinks should keep track of the amount of stepped data in order to remain
|
sinks should keep track of the amount of stepped data in order to remain
|
||||||
synchronized against the clock.
|
synchronized against the clock.
|
||||||
|
|
||||||
|
|
||||||
messages
|
messages
|
||||||
--------
|
~~~~~~~~
|
||||||
|
|
||||||
A GST_MESSAGE_STEP_START is created. It contains the following fields.
|
A GST_MESSAGE_STEP_START is created. It contains the following fields.
|
||||||
|
|
||||||
"active"
|
"active"
|
||||||
If the step was queued or activated.
|
If the step was queued or activated.
|
||||||
|
|
@ -188,7 +188,7 @@ messages
|
||||||
"intermediate", G_TYPE_BOOLEAN
|
"intermediate", G_TYPE_BOOLEAN
|
||||||
If this is an intermediate step operation that queued/activated.
|
If this is an intermediate step operation that queued/activated.
|
||||||
|
|
||||||
The STEP_START message is emited 2 times:
|
The STEP_START message is emited 2 times:
|
||||||
|
|
||||||
* first when an element received the STEP event and queued it. The "active"
|
* first when an element received the STEP event and queued it. The "active"
|
||||||
field will be FALSE in this case.
|
field will be FALSE in this case.
|
||||||
|
|
@ -197,12 +197,12 @@ messages
|
||||||
field is TRUE in this case. After this message is emited, the application
|
field is TRUE in this case. After this message is emited, the application
|
||||||
can queue a new step operation.
|
can queue a new step operation.
|
||||||
|
|
||||||
The purpose of this message is to find out how many elements participate in the
|
The purpose of this message is to find out how many elements participate in the
|
||||||
step operation and to queue new step operations at the earliest possible
|
step operation and to queue new step operations at the earliest possible
|
||||||
moment.
|
moment.
|
||||||
|
|
||||||
A new GST_MESSAGE_STEP_DONE message is created. It contains the following
|
A new GST_MESSAGE_STEP_DONE message is created. It contains the following
|
||||||
fields:
|
fields:
|
||||||
|
|
||||||
"format", GST_TYPE_FORMAT
|
"format", GST_TYPE_FORMAT
|
||||||
The format of the step units that completed.
|
The format of the step units that completed.
|
||||||
|
|
@ -225,25 +225,24 @@ messages
|
||||||
"eos", G_TYPE_BOOLEAN
|
"eos", G_TYPE_BOOLEAN
|
||||||
The step ended because of EOS.
|
The step ended because of EOS.
|
||||||
|
|
||||||
The message is emited by the element that performs the step operation. The
|
The message is emited by the element that performs the step operation. The
|
||||||
purpose is to return the duration in GST_FORMAT_TIME of the stepped media. This
|
purpose is to return the duration in GST_FORMAT_TIME of the stepped media. This
|
||||||
especially interesting to align other stream in case of stepping frames on the
|
especially interesting to align other stream in case of stepping frames on the
|
||||||
video sink element.
|
video sink element.
|
||||||
|
|
||||||
|
|
||||||
Direction switch
|
Direction switch
|
||||||
----------------
|
~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
When quickly switching between a forwards and a backwards step of, for example,
|
When quickly switching between a forwards and a backwards step of, for example,
|
||||||
one video frame, we need either:
|
one video frame, we need either:
|
||||||
|
|
||||||
a) issue a new seek to change the direction from the current position.
|
a) issue a new seek to change the direction from the current position.
|
||||||
b) cache a certain number of stepped frames and walk the cache.
|
b) cache a certain number of stepped frames and walk the cache.
|
||||||
|
|
||||||
option a) might be very slow.
|
option a) might be very slow.
|
||||||
For option b) we would ideally like to offload this caching functionality to a
|
For option b) we would ideally like to offload this caching functionality to a
|
||||||
separate element, which means that we need to forward the STEP event upstream.
|
separate element, which means that we need to forward the STEP event upstream.
|
||||||
It's unclear how this could work in a generic way. What is a demuxer supposed
|
It's unclear how this could work in a generic way. What is a demuxer supposed
|
||||||
to do when it received a step event? a flushing seek to what stream position?
|
to do when it received a step event? a flushing seek to what stream position?
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -7,7 +7,7 @@ GstElement. A GstBin provides a GstBus for the children and collates messages
|
||||||
from them.
|
from them.
|
||||||
|
|
||||||
Add/removing elements
|
Add/removing elements
|
||||||
---------------------
|
~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
The basic functionality of a bin is to add and remove GstElements to/from it.
|
The basic functionality of a bin is to add and remove GstElements to/from it.
|
||||||
gst_bin_add() and gst_bin_remove() perform these operations respectively.
|
gst_bin_add() and gst_bin_remove() perform these operations respectively.
|
||||||
|
|
@ -17,7 +17,7 @@ relations.txt).
|
||||||
|
|
||||||
|
|
||||||
Retrieving elements
|
Retrieving elements
|
||||||
-------------------
|
~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
GstBin provides a number of functions to retrieve one or more children from
|
GstBin provides a number of functions to retrieve one or more children from
|
||||||
itself. A few examples of the provided functions:
|
itself. A few examples of the provided functions:
|
||||||
|
|
@ -28,7 +28,7 @@ gst_bin_iterate_elements() returns an iterator to all the children.
|
||||||
|
|
||||||
|
|
||||||
element management
|
element management
|
||||||
------------------
|
~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
The most important function of the GstBin is to distribute all GstElement
|
The most important function of the GstBin is to distribute all GstElement
|
||||||
operations on itself to all of its children. This includes:
|
operations on itself to all of its children. This includes:
|
||||||
|
|
@ -41,7 +41,7 @@ The state change distribution is the most complex and is explained in
|
||||||
part-states.txt.
|
part-states.txt.
|
||||||
|
|
||||||
GstBus
|
GstBus
|
||||||
------
|
~~~~~~
|
||||||
|
|
||||||
The GstBin creates a GstBus for its children and distributes it when child
|
The GstBin creates a GstBus for its children and distributes it when child
|
||||||
elements are added to the bin. The bin attaches a sync handler to receive
|
elements are added to the bin. The bin attaches a sync handler to receive
|
||||||
|
|
@ -63,7 +63,7 @@ When a bin goes to READY it will clear all cached messages.
|
||||||
|
|
||||||
|
|
||||||
EOS
|
EOS
|
||||||
---
|
~~~
|
||||||
|
|
||||||
The sink elements will post an EOS message on the bus when they reach EOS. The
|
The sink elements will post an EOS message on the bus when they reach EOS. The
|
||||||
EOS message is only posted to the bus when the sink element is in PLAYING.
|
EOS message is only posted to the bus when the sink element is in PLAYING.
|
||||||
|
|
@ -77,7 +77,7 @@ to PLAYING again.
|
||||||
|
|
||||||
|
|
||||||
SEGMENT_START/DONE
|
SEGMENT_START/DONE
|
||||||
------------------
|
~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
A bin collects SEGMENT_START messages but does not post them to the application.
|
A bin collects SEGMENT_START messages but does not post them to the application.
|
||||||
It counts the number of SEGMENT_START messages and posts a SEGMENT_STOP message
|
It counts the number of SEGMENT_START messages and posts a SEGMENT_STOP message
|
||||||
|
|
@ -87,7 +87,7 @@ The cached SEGMENT_START/STOP messages are cleared when going to READY.
|
||||||
|
|
||||||
|
|
||||||
DURATION
|
DURATION
|
||||||
--------
|
~~~~~~~~
|
||||||
|
|
||||||
When a DURATION query is performed on a bin, it will forward the query to all
|
When a DURATION query is performed on a bin, it will forward the query to all
|
||||||
its sink elements. The bin will calculate the total duration as the MAX of all
|
its sink elements. The bin will calculate the total duration as the MAX of all
|
||||||
|
|
@ -103,7 +103,7 @@ posted to the application, which can then fetch the updated DURATION.
|
||||||
|
|
||||||
|
|
||||||
Subclassing
|
Subclassing
|
||||||
-----------
|
~~~~~~~~~~~
|
||||||
|
|
||||||
Subclasses of GstBin are free to implement their own add/remove implementations.
|
Subclasses of GstBin are free to implement their own add/remove implementations.
|
||||||
It is a good idea to update the GList of children so that the _iterate() functions
|
It is a good idea to update the GList of children so that the _iterate() functions
|
||||||
|
|
|
||||||
|
|
@ -38,3 +38,4 @@ to flushing, ie. the bus will drop all existing and new messages on the bus,
|
||||||
This is necessary because bus messages hold references to the bin/pipeline
|
This is necessary because bus messages hold references to the bin/pipeline
|
||||||
or its elements, so there are circular references that need to be broken if
|
or its elements, so there are circular references that need to be broken if
|
||||||
one ever wants to be able to destroy a bin or pipeline properly.
|
one ever wants to be able to destroy a bin or pipeline properly.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,14 +1,5 @@
|
||||||
gstelement
|
|
||||||
name
|
|
||||||
pads
|
|
||||||
state
|
|
||||||
loopfunction
|
|
||||||
threadstate
|
|
||||||
manager
|
|
||||||
ghostpads
|
|
||||||
|
|
||||||
GstElement
|
GstElement
|
||||||
==========
|
----------
|
||||||
|
|
||||||
The Element is the most important object in the entire GStreamer system, as it
|
The Element is the most important object in the entire GStreamer system, as it
|
||||||
defines the structure of the pipeline. Elements include sources, filters,
|
defines the structure of the pipeline. Elements include sources, filters,
|
||||||
|
|
@ -26,7 +17,7 @@ Element. This allows deeply nested pipelines, and the possibility of
|
||||||
"black-box" meta-elements.
|
"black-box" meta-elements.
|
||||||
|
|
||||||
Name
|
Name
|
||||||
----
|
~~~~
|
||||||
|
|
||||||
All elements are named, and while they should ideally be unique in any given
|
All elements are named, and while they should ideally be unique in any given
|
||||||
pipeline, they do not have to be. The only guaranteed unique name for an
|
pipeline, they do not have to be. The only guaranteed unique name for an
|
||||||
|
|
@ -39,7 +30,7 @@ or name of an element is changed.
|
||||||
|
|
||||||
|
|
||||||
Pads
|
Pads
|
||||||
----
|
~~~~
|
||||||
|
|
||||||
GstPads are the property of a given GstElement. They provide the connection
|
GstPads are the property of a given GstElement. They provide the connection
|
||||||
capability, with allowing arbitrary structure in the graph. For any Element
|
capability, with allowing arbitrary structure in the graph. For any Element
|
||||||
|
|
@ -67,10 +58,12 @@ pad = gst_element_get_pad(element,"padname"):
|
||||||
|
|
||||||
|
|
||||||
Ghost Pads
|
Ghost Pads
|
||||||
----------
|
~~~~~~~~~~
|
||||||
|
|
||||||
|
More info in part-gstghostpad.txt.
|
||||||
|
|
||||||
State
|
State
|
||||||
-----
|
~~~~~
|
||||||
|
|
||||||
An element has a state. More info in part-states.txt.
|
An element has a state. More info in part-states.txt.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,11 +1,11 @@
|
||||||
Ghostpads
|
Ghostpads
|
||||||
---------
|
---------
|
||||||
|
|
||||||
GhostPads are used to build complex compound elements out of
|
GhostPads are used to build complex compound elements out of
|
||||||
existing elements. They are used to expose internal element pads
|
existing elements. They are used to expose internal element pads
|
||||||
on the complex element.
|
on the complex element.
|
||||||
|
|
||||||
Some design requirements
|
Some design requirements
|
||||||
|
|
||||||
- Must look like a real GstPad on both sides.
|
- Must look like a real GstPad on both sides.
|
||||||
- target of Ghostpad must be changeable
|
- target of Ghostpad must be changeable
|
||||||
|
|
@ -346,23 +346,22 @@ Ghostpads
|
||||||
|
|
||||||
|
|
||||||
Activation
|
Activation
|
||||||
==========
|
~~~~~~~~~~
|
||||||
|
|
||||||
Sometimes ghost pads should proxy activation functions. This thingie
|
Sometimes ghost pads should proxy activation functions. This thingie
|
||||||
attempts to explain how it should work in the different cases.
|
attempts to explain how it should work in the different cases.
|
||||||
|
|
||||||
|
+---+ +----+ +----+ +----+
|
||||||
+---+ +----+ +----+ +----+
|
| A +-----+ B | | C |-------+ D |
|
||||||
| A +-----+ B | | C |-------+ D |
|
+---+ +---=+ +=---+ +----+
|
||||||
+---+ +---=+ +=---+ +----+
|
+--=-----------------------------=-+
|
||||||
+--=-----------------------------=-+
|
| +=---+ +----+ +----+ +---=+ |
|
||||||
| +=---+ +----+ +----+ +---=+ |
|
| | a +---+ b ==== c +--+ d | |
|
||||||
| | a +---+ b ==== c +--+ d | |
|
| +----+ +----+ +----+ +----+ |
|
||||||
| +----+ +----+ +----+ +----+ |
|
| |
|
||||||
| |
|
+----------------------------------+
|
||||||
+----------------------------------+
|
state change goes from right to left
|
||||||
state change goes from right to left
|
<-----------------------------------------------------------
|
||||||
<-----------------------------------------------------------
|
|
||||||
|
|
||||||
All of the labeled boxes are pads. The dashes (---) show pad links, and
|
All of the labeled boxes are pads. The dashes (---) show pad links, and
|
||||||
the double-lines (===) are internal connections. The box around a, b, c,
|
the double-lines (===) are internal connections. The box around a, b, c,
|
||||||
|
|
@ -375,10 +374,10 @@ Now, in the state change from READY to PAUSED, assuming the pipeline
|
||||||
does not have a live source, all of the pads will end up activated at
|
does not have a live source, all of the pads will end up activated at
|
||||||
the end. There are 4 possible activation modes:
|
the end. There are 4 possible activation modes:
|
||||||
|
|
||||||
1) AD and ab in PUSH, cd and CD in PUSH
|
1) AD and ab in PUSH, cd and CD in PUSH
|
||||||
2) AD and ab in PUSH, cd and CD in PULL
|
2) AD and ab in PUSH, cd and CD in PULL
|
||||||
3) AD and ab in PULL, cd and CD in PUSH
|
3) AD and ab in PULL, cd and CD in PUSH
|
||||||
4) AD and ab in PULL, cd and CD in PULL
|
4) AD and ab in PULL, cd and CD in PULL
|
||||||
|
|
||||||
When activating (1), the state change algorithm will first visit the
|
When activating (1), the state change algorithm will first visit the
|
||||||
parent of D and activate D in push mode. Then it visits the bin. The bin
|
parent of D and activate D in push mode. Then it visits the bin. The bin
|
||||||
|
|
|
||||||
|
|
@ -1,10 +1,10 @@
|
||||||
GstObject
|
GstObject
|
||||||
=========
|
---------
|
||||||
|
|
||||||
The base class for the entire GStreamer hierarchy is the GstObject.
|
The base class for the entire GStreamer hierarchy is the GstObject.
|
||||||
|
|
||||||
Parentage
|
Parentage
|
||||||
---------
|
~~~~~~~~~
|
||||||
|
|
||||||
A pointer is available to store the current parent of the object. This is one
|
A pointer is available to store the current parent of the object. This is one
|
||||||
of the two fundamental requirements for a hierarchical system such as GStreamer
|
of the two fundamental requirements for a hierarchical system such as GStreamer
|
||||||
|
|
@ -20,7 +20,8 @@ allows for new additions later.
|
||||||
|
|
||||||
|
|
||||||
Naming
|
Naming
|
||||||
------
|
~~~~~~
|
||||||
|
|
||||||
- names of objects cannot be changed when they are parented
|
- names of objects cannot be changed when they are parented
|
||||||
- names of objects should be unique across parent
|
- names of objects should be unique across parent
|
||||||
- set_name() can fail because of this
|
- set_name() can fail because of this
|
||||||
|
|
@ -36,7 +37,7 @@ Naming
|
||||||
|
|
||||||
|
|
||||||
Locking
|
Locking
|
||||||
-------
|
~~~~~~~
|
||||||
|
|
||||||
The GstObject contains the necessary primitives to lock the object in a
|
The GstObject contains the necessary primitives to lock the object in a
|
||||||
thread-safe manner. This will be used to provide general thread-safety as
|
thread-safe manner. This will be used to provide general thread-safety as
|
||||||
|
|
@ -62,7 +63,7 @@ GstObject.
|
||||||
|
|
||||||
|
|
||||||
Locking order
|
Locking order
|
||||||
-------------
|
~~~~~~~~~~~~~
|
||||||
|
|
||||||
In parent-child situations the lock of the parent must always be taken first
|
In parent-child situations the lock of the parent must always be taken first
|
||||||
before taking the lock of the child. It is NOT allowed to hold the child
|
before taking the lock of the child. It is NOT allowed to hold the child
|
||||||
|
|
@ -76,7 +77,7 @@ parent-child relation (eg. pads), an explictic locking order has to be defined.
|
||||||
|
|
||||||
|
|
||||||
Path Generation
|
Path Generation
|
||||||
---------------
|
~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
Due to the base nature of the GstObject, it becomes the only reasonable place
|
Due to the base nature of the GstObject, it becomes the only reasonable place
|
||||||
to put this particular function (_get_path_string). It will generate a string
|
to put this particular function (_get_path_string). It will generate a string
|
||||||
|
|
@ -84,7 +85,7 @@ describing the parent hierarchy of a given GstObject.
|
||||||
|
|
||||||
|
|
||||||
Flags
|
Flags
|
||||||
-----
|
~~~~~
|
||||||
|
|
||||||
Each object in the GStreamer object hierarchy can have flags associated with it,
|
Each object in the GStreamer object hierarchy can have flags associated with it,
|
||||||
which are used to describe a state or a feature of the object.
|
which are used to describe a state or a feature of the object.
|
||||||
|
|
@ -92,7 +93,7 @@ GstObject has flags to mark its lifecycle: FLOATING and DISPOSING.
|
||||||
|
|
||||||
|
|
||||||
Class signals
|
Class signals
|
||||||
-------------
|
~~~~~~~~~~~~~
|
||||||
|
|
||||||
It is possible to know when a new object is loaded by connecting to the
|
It is possible to know when a new object is loaded by connecting to the
|
||||||
GstObjectClass signal. This feature is not very much used and might be removed
|
GstObjectClass signal. This feature is not very much used and might be removed
|
||||||
|
|
|
||||||
|
|
@ -13,7 +13,7 @@ The pipeline will calculate a global latency for the elements in the pipeline.
|
||||||
(See also part-latency.txt).
|
(See also part-latency.txt).
|
||||||
|
|
||||||
State changes
|
State changes
|
||||||
-------------
|
~~~~~~~~~~~~~
|
||||||
|
|
||||||
In addition to the normal state change procedure of its parent class
|
In addition to the normal state change procedure of its parent class
|
||||||
GstBin, the pipeline performs the following actions during a state change:
|
GstBin, the pipeline performs the following actions during a state change:
|
||||||
|
|
@ -43,7 +43,7 @@ The running_time is set to 0 after a flushing seek.
|
||||||
|
|
||||||
|
|
||||||
Clock selection
|
Clock selection
|
||||||
---------------
|
~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
Since all of the children of a GstPipeline must use the same clock, the
|
Since all of the children of a GstPipeline must use the same clock, the
|
||||||
pipeline must select a clock. This clock selection happens when the pipeline
|
pipeline must select a clock. This clock selection happens when the pipeline
|
||||||
|
|
@ -80,7 +80,7 @@ matic clock selection algorithm described above.
|
||||||
|
|
||||||
|
|
||||||
GstBus
|
GstBus
|
||||||
------
|
~~~~~~
|
||||||
|
|
||||||
A GstPipeline provides a GstBus to the application. The bus can be retrieved
|
A GstPipeline provides a GstBus to the application. The bus can be retrieved
|
||||||
with gst_pipeline_get_bus() and can then be used to retrieve messages posted by
|
with gst_pipeline_get_bus() and can then be used to retrieve messages posted by
|
||||||
|
|
|
||||||
|
|
@ -35,7 +35,7 @@ application needs (required minimal latency).
|
||||||
|
|
||||||
|
|
||||||
Pipelines without latency compensation
|
Pipelines without latency compensation
|
||||||
--------------------------------------
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
We show some examples to demonstrate the problem of latency in typical
|
We show some examples to demonstrate the problem of latency in typical
|
||||||
capture pipelines.
|
capture pipelines.
|
||||||
|
|
@ -196,7 +196,7 @@ capture pipelines.
|
||||||
|
|
||||||
|
|
||||||
State Changes revised
|
State Changes revised
|
||||||
---------------------
|
~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
As a first step in a generic solution we propose to modify the state changes so
|
As a first step in a generic solution we propose to modify the state changes so
|
||||||
that no sink is set to PLAYING before it is prerolled.
|
that no sink is set to PLAYING before it is prerolled.
|
||||||
|
|
@ -229,7 +229,7 @@ implications:
|
||||||
|
|
||||||
|
|
||||||
Latency compensation
|
Latency compensation
|
||||||
--------------------
|
~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
As an extension to the revised state changes we can perform latency calculation
|
As an extension to the revised state changes we can perform latency calculation
|
||||||
and compensation before we proceed to the PLAYING state.
|
and compensation before we proceed to the PLAYING state.
|
||||||
|
|
@ -277,7 +277,7 @@ the same for all sinks, all sinks will render data relatively synchronised.
|
||||||
|
|
||||||
|
|
||||||
Flushing a playing pipeline
|
Flushing a playing pipeline
|
||||||
---------------------------
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
Using the new state change mechanism we can implement resynchronisation after an
|
Using the new state change mechanism we can implement resynchronisation after an
|
||||||
uncontrolled FLUSH in (part of) a pipeline. Indeed, when a flush is performed on
|
uncontrolled FLUSH in (part of) a pipeline. Indeed, when a flush is performed on
|
||||||
|
|
@ -306,7 +306,7 @@ to perform additional latency calculations and adjustments before doing this.
|
||||||
|
|
||||||
|
|
||||||
Dynamically adjusting latency
|
Dynamically adjusting latency
|
||||||
-----------------------------
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
An element that want to change the latency in the pipeline can do this by
|
An element that want to change the latency in the pipeline can do this by
|
||||||
posting a LATENCY message on the bus. This message instructs the pipeline to:
|
posting a LATENCY message on the bus. This message instructs the pipeline to:
|
||||||
|
|
|
||||||
|
|
@ -28,14 +28,14 @@ a blocking wait.
|
||||||
|
|
||||||
|
|
||||||
Scheduling
|
Scheduling
|
||||||
----------
|
~~~~~~~~~~
|
||||||
|
|
||||||
Live sources will not produce data in the paused state. They block in the
|
Live sources will not produce data in the paused state. They block in the
|
||||||
getrange function or in the loop function until they go to PLAYING.
|
getrange function or in the loop function until they go to PLAYING.
|
||||||
|
|
||||||
|
|
||||||
Latency
|
Latency
|
||||||
-------
|
~~~~~~~
|
||||||
|
|
||||||
The live source timestamps its data with the time of the clock at the
|
The live source timestamps its data with the time of the clock at the
|
||||||
time the data was captured. Normally it will take some time to capture
|
time the data was captured. Normally it will take some time to capture
|
||||||
|
|
@ -49,7 +49,7 @@ See part-latency.txt.
|
||||||
|
|
||||||
|
|
||||||
Timestamps
|
Timestamps
|
||||||
----------
|
~~~~~~~~~~
|
||||||
|
|
||||||
Live sources always timestamp their buffers with the running_time of the
|
Live sources always timestamp their buffers with the running_time of the
|
||||||
pipeline. This is needed to be able to match the timestamps of different live
|
pipeline. This is needed to be able to match the timestamps of different live
|
||||||
|
|
|
||||||
|
|
@ -14,122 +14,123 @@ application using the GstBus (See also part-gstbus.txt and part-gstpipeline.txt)
|
||||||
|
|
||||||
|
|
||||||
Message types
|
Message types
|
||||||
-------------
|
~~~~~~~~~~~~~
|
||||||
|
|
||||||
GST_MESSAGE_EOS:
|
GST_MESSAGE_EOS:
|
||||||
|
|
||||||
Posted by sink elements. This message is posted to the application when
|
Posted by sink elements. This message is posted to the application when
|
||||||
all the sinks in a pipeline posted an EOS message. When performing a seek,
|
all the sinks in a pipeline posted an EOS message. When performing a seek,
|
||||||
the EOS state of the pipeline and sinks is undone.
|
the EOS state of the pipeline and sinks is undone.
|
||||||
|
|
||||||
|
GST_MESSAGE_ERROR:
|
||||||
|
|
||||||
|
An element in the pipeline got into an error state. The message carries
|
||||||
|
a GError and a debug string describing the error. This usually means that
|
||||||
|
part of the pipeline is not streaming anymore.
|
||||||
|
|
||||||
|
GST_MESSAGE_WARNING:
|
||||||
|
|
||||||
|
An element in the pipeline encountered a condition that made it produce a
|
||||||
|
warning. This could be a recoverable decoding error or some other non fatal
|
||||||
|
event. The pipeline continues streaming after a warning.
|
||||||
|
|
||||||
|
GST_MESSAGE_INFO:
|
||||||
|
|
||||||
|
An element produced an informational message.
|
||||||
|
|
||||||
|
GST_MESSAGE_TAG:
|
||||||
|
|
||||||
|
An element decoded metadata about the stream. The message carries a GstTagList
|
||||||
|
with the tag information.
|
||||||
|
|
||||||
|
GST_MESSAGE_BUFFERING:
|
||||||
|
|
||||||
|
An element is buffering data and that could potentially take some time. This
|
||||||
|
message is typically emited by elements that perform some sort of network
|
||||||
|
buffering. While the pipeline is buffering it should remain in the PAUSED
|
||||||
|
state. When the buffering is finished, it can resume PLAYING.
|
||||||
|
|
||||||
|
GST_MESSAGE_STATE_CHANGED:
|
||||||
|
|
||||||
|
An element changed state in the pipeline. The message carries the old, new
|
||||||
|
and pending state of the element.
|
||||||
|
|
||||||
|
GST_MESSAGE_STATE_DIRTY:
|
||||||
|
|
||||||
GST_MESSAGE_ERROR:
|
An internal message used to instruct a pipeline hierarchy that a state
|
||||||
|
recalculation must be performed because of an ASYNC state change completed.
|
||||||
|
This message is not used anymore.
|
||||||
|
|
||||||
|
GST_MESSAGE_STEP_DONE:
|
||||||
|
|
||||||
|
An element stepping frames has finished. This is currently not used.
|
||||||
|
|
||||||
|
GST_MESSAGE_CLOCK_PROVIDE:
|
||||||
|
|
||||||
|
An element notifies it capability of providing a clock for the pipeline.
|
||||||
|
|
||||||
|
GST_MESSAGE_CLOCK_LOST:
|
||||||
|
|
||||||
|
The current clock as selected by the pipeline became unusable. The pipeline
|
||||||
|
will select a new clock on the next PLAYING state change.
|
||||||
|
|
||||||
|
GST_MESSAGE_NEW_CLOCK:
|
||||||
|
|
||||||
|
A new clock was selected for the pipeline.
|
||||||
|
|
||||||
|
GST_MESSAGE_STRUCTURE_CHANGE:
|
||||||
|
|
||||||
|
The pipeline changed of structure, This means elements were added or removed or
|
||||||
|
pads were linked or unlinked. This messages is not yet used.
|
||||||
|
|
||||||
|
GST_MESSAGE_STREAM_STATUS:
|
||||||
|
|
||||||
|
Posted by an element when it start/stop/pauses a streaming task. It
|
||||||
|
contains information about the reason why the stream state changed along
|
||||||
|
with the thread id. The application can use this information to detect
|
||||||
|
failures in streaming threads. It can also be used to adjust streaming
|
||||||
|
thread priorities by the application.
|
||||||
|
|
||||||
An element in the pipeline got into an error state. The message carries
|
GST_MESSAGE_APPLICATION:
|
||||||
a GError and a debug string describing the error. This usually means that
|
|
||||||
part of the pipeline is not streaming anymore.
|
|
||||||
|
|
||||||
GST_MESSAGE_WARNING:
|
|
||||||
|
|
||||||
An element in the pipeline encountered a condition that made it produce a
|
|
||||||
warning. This could be a recoverable decoding error or some other non fatal
|
|
||||||
event. The pipeline continues streaming after a warning.
|
|
||||||
|
|
||||||
GST_MESSAGE_INFO:
|
|
||||||
|
|
||||||
An element produced an informational message.
|
The application posted a message. This message must be used when the
|
||||||
|
application posts a message on the bus.
|
||||||
GST_MESSAGE_TAG:
|
|
||||||
|
|
||||||
An element decoded metadata about the stream. The message carries a GstTagList
|
GST_MESSAGE_ELEMENT:
|
||||||
with the tag information.
|
|
||||||
|
|
||||||
GST_MESSAGE_BUFFERING:
|
|
||||||
|
|
||||||
An element is buffering data and that could potentially take some time. This
|
|
||||||
message is typically emited by elements that perform some sort of network
|
|
||||||
buffering. While the pipeline is buffering it should remain in the PAUSED
|
|
||||||
state. When the buffering is finished, it can resume PLAYING.
|
|
||||||
|
|
||||||
GST_MESSAGE_STATE_CHANGED:
|
|
||||||
|
|
||||||
An element changed state in the pipeline. The message carries the old, new
|
Element-specific message, see the specific element's documentation
|
||||||
and pending state of the element.
|
|
||||||
|
|
||||||
GST_MESSAGE_STATE_DIRTY:
|
|
||||||
|
|
||||||
An internal message used to instruct a pipeline hierarchy that a state
|
|
||||||
recalculation must be performed because of an ASYNC state change completed.
|
|
||||||
This message is not used anymore.
|
|
||||||
|
|
||||||
GST_MESSAGE_STEP_DONE:
|
GST_MESSAGE_SEGMENT_START:
|
||||||
|
|
||||||
An element stepping frames has finished. This is currently not used.
|
An element started playback of a new segment. This message is not forwarded
|
||||||
|
the the application but is used internally to schedule SEGMENT_DONE messages.
|
||||||
GST_MESSAGE_CLOCK_PROVIDE:
|
|
||||||
|
|
||||||
An element notifies it capability of providing a clock for the pipeline.
|
GST_MESSAGE_SEGMENT_DONE:
|
||||||
|
|
||||||
GST_MESSAGE_CLOCK_LOST:
|
An element or bin completed playback of a segment. This message is only posted
|
||||||
|
on the bus if a SEGMENT seek is performed on a pipeline.
|
||||||
|
|
||||||
The current clock as selected by the pipeline became unusable. The pipeline
|
GST_MESSAGE_DURATION:
|
||||||
will select a new clock on the next PLAYING state change.
|
|
||||||
|
|
||||||
GST_MESSAGE_NEW_CLOCK:
|
An element posts this message when it has detected or updated the stream duration.
|
||||||
|
|
||||||
A new clock was selected for the pipeline.
|
GST_MESSAGE_ASYNC_START:
|
||||||
|
|
||||||
GST_MESSAGE_STRUCTURE_CHANGE:
|
Posted by sinks when they start an asynchronous state change.
|
||||||
|
|
||||||
The pipeline changed of structure, This means elements were added or removed or
|
GST_MESSAGE_ASYNC_DONE:
|
||||||
pads were linked or unlinked. This messages is not yet used.
|
|
||||||
|
|
||||||
GST_MESSAGE_STREAM_STATUS:
|
Posted by sinks when they receive the first data buffer and complete the
|
||||||
|
asynchronous state change.
|
||||||
|
|
||||||
Posted by an element when it start/stop/pauses a streaming task. It
|
GST_MESSAGE_LATENCY:
|
||||||
contains information about the reason why the stream state changed along
|
|
||||||
with the thread id. The application can use this information to detect
|
|
||||||
failures in streaming threads. It can also be used to adjust streaming
|
|
||||||
thread priorities by the application.
|
|
||||||
|
|
||||||
GST_MESSAGE_APPLICATION:
|
|
||||||
|
|
||||||
The application posted a message. This message must be used when the
|
Posted by elements when the latency in a pipeline changed and a new global
|
||||||
application posts a message on the bus.
|
latency should be calculated by the pipeline or application.
|
||||||
|
|
||||||
GST_MESSAGE_ELEMENT:
|
GST_MESSAGE_REQUEST_STATE:
|
||||||
|
|
||||||
Element-specific message, see the specific element's documentation
|
Posted by elements when they want to change the state of the pipeline they
|
||||||
|
are in. A typical use case would be an audio sink that requests the pipeline
|
||||||
|
to pause in order to play a higher priority stream.
|
||||||
|
|
||||||
GST_MESSAGE_SEGMENT_START:
|
|
||||||
|
|
||||||
An element started playback of a new segment. This message is not forwarded
|
|
||||||
the the application but is used internally to schedule SEGMENT_DONE messages.
|
|
||||||
|
|
||||||
GST_MESSAGE_SEGMENT_DONE:
|
|
||||||
|
|
||||||
An element or bin completed playback of a segment. This message is only posted
|
|
||||||
on the bus if a SEGMENT seek is performed on a pipeline.
|
|
||||||
|
|
||||||
GST_MESSAGE_DURATION:
|
|
||||||
|
|
||||||
An element posts this message when it has detected or updated the stream duration.
|
|
||||||
|
|
||||||
GST_MESSAGE_ASYNC_START:
|
|
||||||
|
|
||||||
Posted by sinks when they start an asynchronous state change.
|
|
||||||
|
|
||||||
GST_MESSAGE_ASYNC_DONE:
|
|
||||||
|
|
||||||
Posted by sinks when they receive the first data buffer and complete the
|
|
||||||
asynchronous state change.
|
|
||||||
|
|
||||||
GST_MESSAGE_LATENCY:
|
|
||||||
|
|
||||||
Posted by elements when the latency in a pipeline changed and a new global
|
|
||||||
latency should be calculated by the pipeline or application.
|
|
||||||
|
|
||||||
GST_MESSAGE_REQUEST_STATE:
|
|
||||||
|
|
||||||
Posted by elements when they want to change the state of the pipeline they
|
|
||||||
are in. A typical use case would be an audio sink that requests the pipeline
|
|
||||||
to pause in order to play a higher priority stream.
|
|
||||||
|
|
|
||||||
|
|
@ -1,5 +1,5 @@
|
||||||
What to do when a plugin is missing
|
What to do when a plugin is missing
|
||||||
===================================
|
-----------------------------------
|
||||||
|
|
||||||
The mechanism and API described in this document requires GStreamer core and
|
The mechanism and API described in this document requires GStreamer core and
|
||||||
gst-plugins-base versions >= 0.10.12. Further information on some aspects of
|
gst-plugins-base versions >= 0.10.12. Further information on some aspects of
|
||||||
|
|
|
||||||
|
|
@ -13,7 +13,7 @@ themselves to different mechanisms to achieve this goal. As it is more
|
||||||
common we describe push mode negotiation first.
|
common we describe push mode negotiation first.
|
||||||
|
|
||||||
Push-mode negotiation
|
Push-mode negotiation
|
||||||
---------------------
|
~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
Push-mode negotiation happens when elements want to push buffers and
|
Push-mode negotiation happens when elements want to push buffers and
|
||||||
need to decide on the format. This is called downstream negotiation
|
need to decide on the format. This is called downstream negotiation
|
||||||
|
|
@ -168,10 +168,10 @@ videotestsrc ! queue ! xvimagesink
|
||||||
|
|
||||||
|
|
||||||
Pull-mode negotiation
|
Pull-mode negotiation
|
||||||
---------------------
|
~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
Rationale
|
Rationale
|
||||||
.........
|
^^^^^^^^^
|
||||||
|
|
||||||
A pipeline in pull mode has different negotiation needs than one
|
A pipeline in pull mode has different negotiation needs than one
|
||||||
activated in push mode. Push mode is optimized for two use cases:
|
activated in push mode. Push mode is optimized for two use cases:
|
||||||
|
|
@ -237,7 +237,7 @@ but before the sink has spawned the pulling thread.
|
||||||
|
|
||||||
|
|
||||||
Mechanism
|
Mechanism
|
||||||
.........
|
^^^^^^^^^
|
||||||
|
|
||||||
The sink determines that the upstream elements support pull based scheduling by
|
The sink determines that the upstream elements support pull based scheduling by
|
||||||
calling gst_pad_check_pull_range().
|
calling gst_pad_check_pull_range().
|
||||||
|
|
@ -276,3 +276,4 @@ GST_FLOW_NOT_NEGOTIATED. Because of the low-latency requirements,
|
||||||
changing caps in an activate pull-mode pipeline is not supported, as it
|
changing caps in an activate pull-mode pipeline is not supported, as it
|
||||||
might require e.g. the sound card to reconfigure its hardware buffers,
|
might require e.g. the sound card to reconfigure its hardware buffers,
|
||||||
and start capsnego again.
|
and start capsnego again.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,32 +1,32 @@
|
||||||
Overview
|
Overview
|
||||||
--------
|
--------
|
||||||
|
|
||||||
This part gives an overview of the design of GStreamer with references to
|
This part gives an overview of the design of GStreamer with references to
|
||||||
the more detailed explanations of the different topics.
|
the more detailed explanations of the different topics.
|
||||||
|
|
||||||
This document is intented for people that want to have a global overview of
|
This document is intented for people that want to have a global overview of
|
||||||
the inner workings of GStreamer.
|
the inner workings of GStreamer.
|
||||||
|
|
||||||
|
|
||||||
Introduction
|
Introduction
|
||||||
------------
|
~~~~~~~~~~~~
|
||||||
|
|
||||||
GStreamer is a set of libraries and plugins that can be used to implement various
|
GStreamer is a set of libraries and plugins that can be used to implement various
|
||||||
multimedia applications ranging from desktop players, audio/video recorders,
|
multimedia applications ranging from desktop players, audio/video recorders,
|
||||||
multimedia servers, transcoders, etc.
|
multimedia servers, transcoders, etc.
|
||||||
|
|
||||||
Applications are built by constructing a pipeline composed of elements. An element
|
Applications are built by constructing a pipeline composed of elements. An element
|
||||||
is an object that performs some action on a multimedia stream such as:
|
is an object that performs some action on a multimedia stream such as:
|
||||||
|
|
||||||
- read a file
|
- read a file
|
||||||
- decode or encode between formats
|
- decode or encode between formats
|
||||||
- capture from a hardware device
|
- capture from a hardware device
|
||||||
- render to a hardware device
|
- render to a hardware device
|
||||||
- mix or multiplex multiple streams
|
- mix or multiplex multiple streams
|
||||||
|
|
||||||
Elements have input and output pads called sink and source pads in GStreamer. An
|
Elements have input and output pads called sink and source pads in GStreamer. An
|
||||||
application links elements together on pads to construct a pipeline. Below is
|
application links elements together on pads to construct a pipeline. Below is
|
||||||
an example of an ogg/vorbis playback pipeline.
|
an example of an ogg/vorbis playback pipeline.
|
||||||
|
|
||||||
+-----------------------------------------------------------+
|
+-----------------------------------------------------------+
|
||||||
| ----------> downstream -------------------> |
|
| ----------> downstream -------------------> |
|
||||||
|
|
@ -40,54 +40,54 @@ Introduction
|
||||||
| <---------< upstream <-------------------< |
|
| <---------< upstream <-------------------< |
|
||||||
+-----------------------------------------------------------+
|
+-----------------------------------------------------------+
|
||||||
|
|
||||||
The filesrc element reads data from a file on disk. The oggdemux element parses
|
The filesrc element reads data from a file on disk. The oggdemux element parses
|
||||||
the data and sends the compressed audio data to the vorbisdec element. The
|
the data and sends the compressed audio data to the vorbisdec element. The
|
||||||
vorbisdec element decodes the compressed data and sends it to the alsasink
|
vorbisdec element decodes the compressed data and sends it to the alsasink
|
||||||
element. The alsasink element sends the samples to the audio card for playback.
|
element. The alsasink element sends the samples to the audio card for playback.
|
||||||
|
|
||||||
Downstream and upstream are the terms used to describe the direction in the
|
Downstream and upstream are the terms used to describe the direction in the
|
||||||
Pipeline. From source to sink is called "downstream" and "upstream" is
|
Pipeline. From source to sink is called "downstream" and "upstream" is
|
||||||
from sink to source. Dataflow always happens downstream.
|
from sink to source. Dataflow always happens downstream.
|
||||||
|
|
||||||
The task of the application is to construct a pipeline as above using existing
|
The task of the application is to construct a pipeline as above using existing
|
||||||
elements. This is further explained in the pipeline building topic.
|
elements. This is further explained in the pipeline building topic.
|
||||||
|
|
||||||
The application does not have to manage any of the complexities of the
|
The application does not have to manage any of the complexities of the
|
||||||
actual dataflow/decoding/conversions/synchronsiation etc. but only calls high
|
actual dataflow/decoding/conversions/synchronsiation etc. but only calls high
|
||||||
level functions on the pipeline object such as PLAY/PAUSE/STOP.
|
level functions on the pipeline object such as PLAY/PAUSE/STOP.
|
||||||
|
|
||||||
The application also receives messages and notifications from the pipeline such
|
The application also receives messages and notifications from the pipeline such
|
||||||
as metadata, warning, error and EOS messages.
|
as metadata, warning, error and EOS messages.
|
||||||
|
|
||||||
If the application needs more control over the graph it is possible to directly
|
If the application needs more control over the graph it is possible to directly
|
||||||
access the elements and pads in the pipeline.
|
access the elements and pads in the pipeline.
|
||||||
|
|
||||||
|
|
||||||
Design overview
|
Design overview
|
||||||
---------------
|
~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
GStreamer design goals include:
|
GStreamer design goals include:
|
||||||
|
|
||||||
- Process large amounts of data quickly
|
- Process large amounts of data quickly
|
||||||
- Allow fully multithreaded processing
|
- Allow fully multithreaded processing
|
||||||
- Ability to deal with multiple formats
|
- Ability to deal with multiple formats
|
||||||
- Synchronize different dataflows
|
- Synchronize different dataflows
|
||||||
- Ability to deal with multiple devices
|
- Ability to deal with multiple devices
|
||||||
|
|
||||||
|
The capabilities presented to the application depends on the number of elements
|
||||||
|
installed on the system and their functionality.
|
||||||
|
|
||||||
|
The GStreamer core is designed to be media agnostic but provides many features
|
||||||
|
to elements to describe media formats.
|
||||||
|
|
||||||
The capabilities presented to the application depends on the number of elements
|
|
||||||
installed on the system and their functionality.
|
|
||||||
|
|
||||||
The GStreamer core is designed to be media agnostic but provides many features
|
|
||||||
to elements to describe media formats.
|
|
||||||
|
|
||||||
|
|
||||||
Elements
|
Elements
|
||||||
--------
|
~~~~~~~~
|
||||||
|
|
||||||
The smallest building blocks in a pipeline are elements. An element provides a
|
The smallest building blocks in a pipeline are elements. An element provides a
|
||||||
number of pads which can be source or sinkpads. Sourcepads provide data and
|
number of pads which can be source or sinkpads. Sourcepads provide data and
|
||||||
sinkpads consume data. Below is an example of an ogg demuxer element that has
|
sinkpads consume data. Below is an example of an ogg demuxer element that has
|
||||||
one pad that takes (sinks) data and two source pads that produce data.
|
one pad that takes (sinks) data and two source pads that produce data.
|
||||||
|
|
||||||
+-----------+
|
+-----------+
|
||||||
| oggdemux |
|
| oggdemux |
|
||||||
|
|
@ -95,57 +95,57 @@ Elements
|
||||||
sink src1
|
sink src1
|
||||||
+-----------+
|
+-----------+
|
||||||
|
|
||||||
An element can be in four different states: NULL, READY, PAUSED, PLAYING. In the
|
An element can be in four different states: NULL, READY, PAUSED, PLAYING. In the
|
||||||
NULL and READY state, the element is not processing any data. In the PLAYING state
|
NULL and READY state, the element is not processing any data. In the PLAYING state
|
||||||
it is processing data. The intermediate PAUSED state is used to preroll data in
|
it is processing data. The intermediate PAUSED state is used to preroll data in
|
||||||
the pipeline. A state change can be performed with gst_element_set_state().
|
the pipeline. A state change can be performed with gst_element_set_state().
|
||||||
|
|
||||||
An element always goes through all the intermediate state changes. This means that
|
An element always goes through all the intermediate state changes. This means that
|
||||||
when en element is in the READY state and is put to PLAYING, it will first go
|
when en element is in the READY state and is put to PLAYING, it will first go
|
||||||
through the intermediate PAUSED state.
|
through the intermediate PAUSED state.
|
||||||
|
|
||||||
An element state change to PAUSED will activate the pads of the element. First the
|
An element state change to PAUSED will activate the pads of the element. First the
|
||||||
source pads are activated, then the sinkpads. When the pads are activated, the
|
source pads are activated, then the sinkpads. When the pads are activated, the
|
||||||
pad activate function is called. Some pads will start a thread (GstTask) or some
|
pad activate function is called. Some pads will start a thread (GstTask) or some
|
||||||
other mechanism to start producing or consuming data.
|
other mechanism to start producing or consuming data.
|
||||||
|
|
||||||
The PAUSED state is special as it is used to preroll data in the pipeline. The purpose
|
The PAUSED state is special as it is used to preroll data in the pipeline. The purpose
|
||||||
is to fill all connected elements in the pipeline with data so that the subsequent
|
is to fill all connected elements in the pipeline with data so that the subsequent
|
||||||
PLAYING state change happens very quickly. Some elements will therefore not complete
|
PLAYING state change happens very quickly. Some elements will therefore not complete
|
||||||
the state change to PAUSED before they have received enough data. Sink elements are
|
the state change to PAUSED before they have received enough data. Sink elements are
|
||||||
required to only complete the state change to PAUSED after receiving the first data.
|
required to only complete the state change to PAUSED after receiving the first data.
|
||||||
|
|
||||||
Normally the state changes of elements are coordinated by the pipeline as explained
|
Normally the state changes of elements are coordinated by the pipeline as explained
|
||||||
in [part-states.txt].
|
in [part-states.txt].
|
||||||
|
|
||||||
Different categories of elements exist:
|
Different categories of elements exist:
|
||||||
|
|
||||||
- source elements, these are elements that do not consume data but only provide data
|
- source elements, these are elements that do not consume data but only provide data
|
||||||
for the pipeline.
|
for the pipeline.
|
||||||
- sink elements, these are elements that do not produce data but renders data to
|
- sink elements, these are elements that do not produce data but renders data to
|
||||||
an output device.
|
an output device.
|
||||||
- transform elements, these elements transform an input stream in a certain format
|
- transform elements, these elements transform an input stream in a certain format
|
||||||
into a stream of another format. Encoder/decoder/converters are examples.
|
into a stream of another format. Encoder/decoder/converters are examples.
|
||||||
- demuxer elements, these elements parse a stream and produce several output streams.
|
- demuxer elements, these elements parse a stream and produce several output streams.
|
||||||
- mixer/muxer elements, combine several input streams into one output stream.
|
- mixer/muxer elements, combine several input streams into one output stream.
|
||||||
|
|
||||||
Other categories of elements can be constructed (see part-klass.txt).
|
Other categories of elements can be constructed (see part-klass.txt).
|
||||||
|
|
||||||
|
|
||||||
Bins
|
Bins
|
||||||
----
|
~~~~
|
||||||
|
|
||||||
A bin is an element subclass and acts as a container for other elements so that multiple
|
A bin is an element subclass and acts as a container for other elements so that multiple
|
||||||
elements can be combined into one element.
|
elements can be combined into one element.
|
||||||
|
|
||||||
A bin coordinates its children's state changes as explained later. It also distributes
|
A bin coordinates its children's state changes as explained later. It also distributes
|
||||||
events and various other functionality to elements.
|
events and various other functionality to elements.
|
||||||
|
|
||||||
A bin can have its own source and sinkpads by ghostpadding one or more of its children's
|
A bin can have its own source and sinkpads by ghostpadding one or more of its children's
|
||||||
pads to itself.
|
pads to itself.
|
||||||
|
|
||||||
Below is a picture of a bin with two elements. The sinkpad of one element is ghostpadded
|
Below is a picture of a bin with two elements. The sinkpad of one element is ghostpadded
|
||||||
to the bin.
|
to the bin.
|
||||||
|
|
||||||
+---------------------------+
|
+---------------------------+
|
||||||
| bin |
|
| bin |
|
||||||
|
|
@ -157,131 +157,131 @@ Bins
|
||||||
|
|
||||||
|
|
||||||
Pipeline
|
Pipeline
|
||||||
--------
|
~~~~~~~~
|
||||||
|
|
||||||
A pipeline is a special bin subclass that provides the following features to its
|
A pipeline is a special bin subclass that provides the following features to its
|
||||||
children:
|
children:
|
||||||
|
|
||||||
|
- Select and manage a global clock for all its children.
|
||||||
|
- Manage running_time based on the selected clock. Running_time is the elapsed
|
||||||
|
time the pipeline spent in the PLAYING state and is used for
|
||||||
|
synchronisation.
|
||||||
|
- Manage latency in the pipeline.
|
||||||
|
- Provide means for elements to comunicate with the application by the GstBus.
|
||||||
|
- Manage the global state of the elements such as Errors and end-of-stream.
|
||||||
|
|
||||||
|
Normally the application creates one pipeline that will manage all the elements
|
||||||
|
in the application.
|
||||||
|
|
||||||
- Select and manage a global clock for all its children.
|
|
||||||
- Manage running_time based on the selected clock. Running_time is the elapsed
|
|
||||||
time the pipeline spent in the PLAYING state and is used for
|
|
||||||
synchronisation.
|
|
||||||
- Manage latency in the pipeline.
|
|
||||||
- Provide means for elements to comunicate with the application by the GstBus.
|
|
||||||
- Manage the global state of the elements such as Errors and end-of-stream.
|
|
||||||
|
|
||||||
Normally the application creates one pipeline that will manage all the elements
|
|
||||||
in the application.
|
|
||||||
|
|
||||||
|
|
||||||
Dataflow and buffers
|
Dataflow and buffers
|
||||||
--------------------
|
~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
GStreamer supports two possible types of dataflow, the push and pull model. In the
|
GStreamer supports two possible types of dataflow, the push and pull model. In the
|
||||||
push model, an upstream element sends data to a downstream element by calling a
|
push model, an upstream element sends data to a downstream element by calling a
|
||||||
method on a sinkpad. In the pull model, a downstream element requests data from
|
method on a sinkpad. In the pull model, a downstream element requests data from
|
||||||
an upstream element by calling a method on a source pad.
|
an upstream element by calling a method on a source pad.
|
||||||
|
|
||||||
The most common dataflow is the push model. The pull model can be used in specific
|
The most common dataflow is the push model. The pull model can be used in specific
|
||||||
circumstances by demuxer elements. The pull model can also be used by low latency
|
circumstances by demuxer elements. The pull model can also be used by low latency
|
||||||
audio applications.
|
audio applications.
|
||||||
|
|
||||||
The data passed between pads is encapsulated in Buffers. The buffer contains a
|
The data passed between pads is encapsulated in Buffers. The buffer contains a
|
||||||
pointer to the actual data and also metadata describing the data. This metadata
|
pointer to the actual data and also metadata describing the data. This metadata
|
||||||
includes:
|
includes:
|
||||||
|
|
||||||
- timestamp of the data, this is the time instance at which the data was captured
|
|
||||||
or the time at which the data should be played back.
|
|
||||||
- offset of the data: a media specific offset, this could be samples for audio or
|
|
||||||
frames for video.
|
|
||||||
- the duration of the data in time.
|
|
||||||
- the media type of the data described with caps, these are key/value pairs that
|
|
||||||
describe the media type in a unique way.
|
|
||||||
- additional flags describing special properties of the data such as
|
|
||||||
discontinuities or delta units.
|
|
||||||
|
|
||||||
When an element whishes to send a buffer to another element is does this using one
|
|
||||||
of the pads that is linked to a pad of the other element. In the push model, a
|
|
||||||
buffer is pushed to the peer pad with gst_pad_push(). In the pull model, a buffer
|
|
||||||
is pulled from the peer with the gst_pad_pull_range() function.
|
|
||||||
|
|
||||||
Before an element pushes out a buffer, it should make sure that the peer element
|
|
||||||
can understand the buffer contents. It does this by querying the peer element
|
|
||||||
for the supported formats and by selecting a suitable common format. The selected
|
|
||||||
format is then attached to the buffer with gst_buffer_set_caps() before pushing
|
|
||||||
out the buffer.
|
|
||||||
|
|
||||||
When an element pad receives a buffer, if has to check if it understands the media
|
|
||||||
type of the buffer before starting processing it. The GStreamer core does this
|
|
||||||
automatically and will call the gst_pad_set_caps() function of the element before
|
|
||||||
sending the buffer to the element.
|
|
||||||
|
|
||||||
Both gst_pad_push() and gst_pad_pull_range() have a return value indicating whether
|
|
||||||
the operation succeeded. An error code means that no more data should be sent
|
|
||||||
to that pad. A source element that initiates the data flow in a thread typically
|
|
||||||
pauses the producing thread when this happens.
|
|
||||||
|
|
||||||
A buffer can be created with gst_buffer_new() or by requesting a usable buffer
|
- timestamp of the data, this is the time instance at which the data was captured
|
||||||
from the peer pad using gst_pad_alloc_buffer(). Using the second method, it is
|
or the time at which the data should be played back.
|
||||||
possible for the peer element to suggest the element to produce data in another
|
- offset of the data: a media specific offset, this could be samples for audio or
|
||||||
format by attaching another media type caps to the buffer.
|
frames for video.
|
||||||
|
- the duration of the data in time.
|
||||||
|
- the media type of the data described with caps, these are key/value pairs that
|
||||||
|
describe the media type in a unique way.
|
||||||
|
- additional flags describing special properties of the data such as
|
||||||
|
discontinuities or delta units.
|
||||||
|
|
||||||
The process of selecting a media type and attaching it to the buffers is called
|
When an element whishes to send a buffer to another element is does this using one
|
||||||
caps negotiation.
|
of the pads that is linked to a pad of the other element. In the push model, a
|
||||||
|
buffer is pushed to the peer pad with gst_pad_push(). In the pull model, a buffer
|
||||||
|
is pulled from the peer with the gst_pad_pull_range() function.
|
||||||
|
|
||||||
|
Before an element pushes out a buffer, it should make sure that the peer element
|
||||||
|
can understand the buffer contents. It does this by querying the peer element
|
||||||
|
for the supported formats and by selecting a suitable common format. The selected
|
||||||
|
format is then attached to the buffer with gst_buffer_set_caps() before pushing
|
||||||
|
out the buffer.
|
||||||
|
|
||||||
|
When an element pad receives a buffer, if has to check if it understands the media
|
||||||
|
type of the buffer before starting processing it. The GStreamer core does this
|
||||||
|
automatically and will call the gst_pad_set_caps() function of the element before
|
||||||
|
sending the buffer to the element.
|
||||||
|
|
||||||
|
Both gst_pad_push() and gst_pad_pull_range() have a return value indicating whether
|
||||||
|
the operation succeeded. An error code means that no more data should be sent
|
||||||
|
to that pad. A source element that initiates the data flow in a thread typically
|
||||||
|
pauses the producing thread when this happens.
|
||||||
|
|
||||||
|
A buffer can be created with gst_buffer_new() or by requesting a usable buffer
|
||||||
|
from the peer pad using gst_pad_alloc_buffer(). Using the second method, it is
|
||||||
|
possible for the peer element to suggest the element to produce data in another
|
||||||
|
format by attaching another media type caps to the buffer.
|
||||||
|
|
||||||
|
The process of selecting a media type and attaching it to the buffers is called
|
||||||
|
caps negotiation.
|
||||||
|
|
||||||
|
|
||||||
Caps
|
Caps
|
||||||
----
|
~~~~
|
||||||
|
|
||||||
A media type (Caps) is described using a generic list of key/value pairs. The key is
|
|
||||||
a string and the value can be a single/list/range of int/float/string.
|
|
||||||
|
|
||||||
Caps that have no ranges/list or other variable parts are said to be fixed and
|
A media type (Caps) is described using a generic list of key/value pairs. The key is
|
||||||
can be used to put on a buffer.
|
a string and the value can be a single/list/range of int/float/string.
|
||||||
|
|
||||||
Caps with variables in them are used to describe possible media types that can be
|
Caps that have no ranges/list or other variable parts are said to be fixed and
|
||||||
handled by a pad.
|
can be used to put on a buffer.
|
||||||
|
|
||||||
|
Caps with variables in them are used to describe possible media types that can be
|
||||||
|
handled by a pad.
|
||||||
|
|
||||||
|
|
||||||
Dataflow and events
|
Dataflow and events
|
||||||
-------------------
|
~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
Parallel to the dataflow is a flow of events. Unlike the buffers, events can pass
|
Parallel to the dataflow is a flow of events. Unlike the buffers, events can pass
|
||||||
both upstream and downstream. Some events only travel upstream others only downstream.
|
both upstream and downstream. Some events only travel upstream others only downstream.
|
||||||
|
|
||||||
the events are used to denote special conditions in the dataflow such as EOS or
|
the events are used to denote special conditions in the dataflow such as EOS or
|
||||||
to inform plugins of special events such as flushing or seeking.
|
to inform plugins of special events such as flushing or seeking.
|
||||||
|
|
||||||
Some events must be serialized with the buffer flow, others don't. Serialized
|
Some events must be serialized with the buffer flow, others don't. Serialized
|
||||||
events are inserted between the buffers. Non serialized events jump in front
|
events are inserted between the buffers. Non serialized events jump in front
|
||||||
of any buffers current being processed.
|
of any buffers current being processed.
|
||||||
|
|
||||||
An example of a serialized event is a TAG event that is inserted between buffers
|
An example of a serialized event is a TAG event that is inserted between buffers
|
||||||
to mark metadata for those buffers.
|
to mark metadata for those buffers.
|
||||||
|
|
||||||
An example of a non serialized event is the FLUSH event.
|
An example of a non serialized event is the FLUSH event.
|
||||||
|
|
||||||
|
|
||||||
Pipeline construction
|
Pipeline construction
|
||||||
---------------------
|
~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
The application starts by creating a Pipeline element using gst_pipeline_new ().
|
The application starts by creating a Pipeline element using gst_pipeline_new ().
|
||||||
Elements are added to and removed from the pipeline with gst_bin_add() and
|
Elements are added to and removed from the pipeline with gst_bin_add() and
|
||||||
gst_bin_remove().
|
gst_bin_remove().
|
||||||
|
|
||||||
After adding the elements, the pads of an element can be retrieved with
|
After adding the elements, the pads of an element can be retrieved with
|
||||||
gst_element_get_pad(). Pads can then be linked together with gst_pad_link().
|
gst_element_get_pad(). Pads can then be linked together with gst_pad_link().
|
||||||
|
|
||||||
Some elements create new pads when actual dataflow is happening in the pipeline.
|
Some elements create new pads when actual dataflow is happening in the pipeline.
|
||||||
With g_signal_connect() one can receive a notification when an element has created
|
With g_signal_connect() one can receive a notification when an element has created
|
||||||
a pad. These new pads can then be linked to other unlinked pads.
|
a pad. These new pads can then be linked to other unlinked pads.
|
||||||
|
|
||||||
Some elements cannot be linked together because they operate on different
|
Some elements cannot be linked together because they operate on different
|
||||||
incompatible data types. The possible datatypes a pad can provide or consume can
|
incompatible data types. The possible datatypes a pad can provide or consume can
|
||||||
be retrieved with gst_pad_get_caps().
|
be retrieved with gst_pad_get_caps().
|
||||||
|
|
||||||
Below is a simple mp3 playback pipeline that we constructed. We will use this
|
Below is a simple mp3 playback pipeline that we constructed. We will use this
|
||||||
pipeline in further examples.
|
pipeline in further examples.
|
||||||
|
|
||||||
+-------------------------------------------+
|
+-------------------------------------------+
|
||||||
| pipeline |
|
| pipeline |
|
||||||
|
|
@ -293,195 +293,195 @@ Pipeline construction
|
||||||
|
|
||||||
|
|
||||||
Pipeline clock
|
Pipeline clock
|
||||||
--------------
|
~~~~~~~~~~~~~~
|
||||||
|
|
||||||
One of the important functions of the pipeline is to select a global clock
|
One of the important functions of the pipeline is to select a global clock
|
||||||
for all the elements in the pipeline.
|
for all the elements in the pipeline.
|
||||||
|
|
||||||
The purpose of the clock is to provide a stricly increasing value at the rate
|
The purpose of the clock is to provide a stricly increasing value at the rate
|
||||||
of one GST_SECOND per second. Clock values are expressed in nanoseconds.
|
of one GST_SECOND per second. Clock values are expressed in nanoseconds.
|
||||||
Elements use the clock time to synchronize the playback of data.
|
Elements use the clock time to synchronize the playback of data.
|
||||||
|
|
||||||
Before the pipeline is set to PLAYING, the pipeline asks each element if they can
|
Before the pipeline is set to PLAYING, the pipeline asks each element if they can
|
||||||
provide a clock. The clock is selected in the following order:
|
provide a clock. The clock is selected in the following order:
|
||||||
|
|
||||||
- If the application selected a clock, use that one.
|
- If the application selected a clock, use that one.
|
||||||
- If a source element provides a clock, use that clock.
|
- If a source element provides a clock, use that clock.
|
||||||
- Select a clock from any other element that provides a clock, start with the
|
- Select a clock from any other element that provides a clock, start with the
|
||||||
sinks.
|
sinks.
|
||||||
- If no element provides a clock a default system clock is used for the pipeline.
|
- If no element provides a clock a default system clock is used for the pipeline.
|
||||||
|
|
||||||
In a typical playback pipeline this algorithm will select the clock provided by
|
In a typical playback pipeline this algorithm will select the clock provided by
|
||||||
a sink element such as an audio sink.
|
a sink element such as an audio sink.
|
||||||
|
|
||||||
In capture pipelines, this will typically select the clock of the data producer, which
|
In capture pipelines, this will typically select the clock of the data producer, which
|
||||||
in most cases can not control the rate at which it produces data.
|
in most cases can not control the rate at which it produces data.
|
||||||
|
|
||||||
|
|
||||||
Pipeline states
|
Pipeline states
|
||||||
---------------
|
~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
When all the pads are linked and signals have been connected, the pipeline can
|
When all the pads are linked and signals have been connected, the pipeline can
|
||||||
be put in the PAUSED state to start dataflow.
|
be put in the PAUSED state to start dataflow.
|
||||||
|
|
||||||
When a bin (and hence a pipeline) performs a state change, it will change the state
|
When a bin (and hence a pipeline) performs a state change, it will change the state
|
||||||
of all its children. The pipeline will change the state of its children from the
|
of all its children. The pipeline will change the state of its children from the
|
||||||
sink elements to the source elements, this to make sure that no upstream element
|
sink elements to the source elements, this to make sure that no upstream element
|
||||||
produces data to an element that is not yet ready to accept it.
|
produces data to an element that is not yet ready to accept it.
|
||||||
|
|
||||||
In the mp3 playback pipeline, the state of the elements is changed in the order
|
In the mp3 playback pipeline, the state of the elements is changed in the order
|
||||||
alsasink, mp3dec, filesrc.
|
alsasink, mp3dec, filesrc.
|
||||||
|
|
||||||
All intermediate states are traversed for each element resulting in the following
|
All intermediate states are traversed for each element resulting in the following
|
||||||
chain of state changes:
|
chain of state changes:
|
||||||
|
|
||||||
alsasink to READY: the audio device is probed
|
alsasink to READY: the audio device is probed
|
||||||
mp3dec to READY: nothing happens.
|
mp3dec to READY: nothing happens.
|
||||||
filesrc to READY: the file is probed
|
filesrc to READY: the file is probed
|
||||||
alsasink to PAUSED: the audio device is opened. alsasink is a sink and returns
|
alsasink to PAUSED: the audio device is opened. alsasink is a sink and returns
|
||||||
ASYNC because it did not receive data yet.
|
ASYNC because it did not receive data yet.
|
||||||
mp3dec to PAUSED: the decoding library is initialized
|
mp3dec to PAUSED: the decoding library is initialized
|
||||||
filesrc to PAUSED: the file is opened and a thread is started to push data to
|
filesrc to PAUSED: the file is opened and a thread is started to push data to
|
||||||
mp3dec
|
mp3dec
|
||||||
|
|
||||||
At this point data flows from filesrc to mp3dec and alsasink. Since mp3dec is PAUSED,
|
At this point data flows from filesrc to mp3dec and alsasink. Since mp3dec is PAUSED,
|
||||||
it accepts the data from filesrc on the sinkpad and starts decoding the compressed
|
it accepts the data from filesrc on the sinkpad and starts decoding the compressed
|
||||||
data to raw audio samples.
|
data to raw audio samples.
|
||||||
|
|
||||||
The mp3 decoder figures out the samplerate, the number of channels and other audio
|
The mp3 decoder figures out the samplerate, the number of channels and other audio
|
||||||
properties of the raw audio samples, puts the decoded samples into a Buffer,
|
properties of the raw audio samples, puts the decoded samples into a Buffer,
|
||||||
attaches the media type caps to the buffer and pushes this buffer to the next
|
attaches the media type caps to the buffer and pushes this buffer to the next
|
||||||
element.
|
element.
|
||||||
|
|
||||||
Alsasink then receives the buffer, inspects the caps and reconfigures itself to process
|
Alsasink then receives the buffer, inspects the caps and reconfigures itself to process
|
||||||
the buffer. Since it received the first buffer of samples, it completes the state change
|
the buffer. Since it received the first buffer of samples, it completes the state change
|
||||||
to the PAUSED state. At this point the pipeline is prerolled and all elements have
|
to the PAUSED state. At this point the pipeline is prerolled and all elements have
|
||||||
samples. Alsasink is now also capable of providing a clock to the pipeline.
|
samples. Alsasink is now also capable of providing a clock to the pipeline.
|
||||||
|
|
||||||
Since alsasink is now in the PAUSED state it blocks while receiving the first buffer. This
|
Since alsasink is now in the PAUSED state it blocks while receiving the first buffer. This
|
||||||
effectively blocks both mp3dec and filesrc in their gst_pad_push().
|
effectively blocks both mp3dec and filesrc in their gst_pad_push().
|
||||||
|
|
||||||
Since all elements now return SUCCESS from the gst_element_get_state() function,
|
Since all elements now return SUCCESS from the gst_element_get_state() function,
|
||||||
the pipeline can be put in the PLAYING state.
|
the pipeline can be put in the PLAYING state.
|
||||||
|
|
||||||
Before going to PLAYING, the pipeline select a clock and samples the current time of
|
Before going to PLAYING, the pipeline select a clock and samples the current time of
|
||||||
the clock. This is the base_time. It then distributes this time to all elements.
|
the clock. This is the base_time. It then distributes this time to all elements.
|
||||||
Elements can then synchronize against the clock using the buffer running_time +
|
Elements can then synchronize against the clock using the buffer running_time +
|
||||||
base_time (See also part-synchronisation.txt).
|
base_time (See also part-synchronisation.txt).
|
||||||
|
|
||||||
The following chain of state changes then takes place:
|
The following chain of state changes then takes place:
|
||||||
|
|
||||||
|
alsasink to PLAYING: the samples are played to the audio device
|
||||||
|
mp3dec to PLAYING: nothing happens
|
||||||
|
filesrc to PLAYING: nothing happens
|
||||||
|
|
||||||
alsasink to PLAYING: the samples are played to the audio device
|
|
||||||
mp3dec to PLAYING: nothing happens
|
|
||||||
filesrc to PLAYING: nothing happens
|
|
||||||
|
|
||||||
|
|
||||||
Pipeline status
|
Pipeline status
|
||||||
---------------
|
~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
The pipeline informs the application of any special events that occur in the
|
The pipeline informs the application of any special events that occur in the
|
||||||
pipeline with the bus. The bus is an object that the pipeline provides and that
|
pipeline with the bus. The bus is an object that the pipeline provides and that
|
||||||
can be retrieved with gst_pipeline_get_bus().
|
can be retrieved with gst_pipeline_get_bus().
|
||||||
|
|
||||||
The bus can be polled or added to the glib mainloop.
|
The bus can be polled or added to the glib mainloop.
|
||||||
|
|
||||||
The bus is distributed to all elements added to the pipeline. The elements use the bus
|
The bus is distributed to all elements added to the pipeline. The elements use the bus
|
||||||
to post messages on. Various message types exist such as ERRORS, WARNINGS, EOS,
|
to post messages on. Various message types exist such as ERRORS, WARNINGS, EOS,
|
||||||
STATE_CHANGED, etc..
|
STATE_CHANGED, etc..
|
||||||
|
|
||||||
The pipeline handles EOS messages received from elements in a special way. It will
|
The pipeline handles EOS messages received from elements in a special way. It will
|
||||||
only forward the message to the application when all sink elements have posted an
|
only forward the message to the application when all sink elements have posted an
|
||||||
EOS message.
|
EOS message.
|
||||||
|
|
||||||
Other methods for obtaining the pipeline status include the Query functionality that
|
Other methods for obtaining the pipeline status include the Query functionality that
|
||||||
can be performed with gst_element_query() on the pipeline. This type of query
|
can be performed with gst_element_query() on the pipeline. This type of query
|
||||||
is useful for obtaining information about the current position and total time of
|
is useful for obtaining information about the current position and total time of
|
||||||
the pipeline. It can also be used to query for the supported seeking formats and
|
the pipeline. It can also be used to query for the supported seeking formats and
|
||||||
ranges.
|
ranges.
|
||||||
|
|
||||||
|
|
||||||
Pipeline EOS
|
Pipeline EOS
|
||||||
------------
|
~~~~~~~~~~~~
|
||||||
|
|
||||||
When the source filter encounters the end of the stream, it sends an EOS event to
|
When the source filter encounters the end of the stream, it sends an EOS event to
|
||||||
the peer element. This event will then travel downstream to all of the connected
|
the peer element. This event will then travel downstream to all of the connected
|
||||||
elements to inform them of the EOS. The element is not supposed to accept any more
|
elements to inform them of the EOS. The element is not supposed to accept any more
|
||||||
data after receiving an EOS event on a sinkpad.
|
data after receiving an EOS event on a sinkpad.
|
||||||
|
|
||||||
The element providing the streaming thread stops sending data after sending the
|
The element providing the streaming thread stops sending data after sending the
|
||||||
EOS event.
|
EOS event.
|
||||||
|
|
||||||
The EOS event will eventually arrive in the sink element. The sink will then post
|
The EOS event will eventually arrive in the sink element. The sink will then post
|
||||||
an EOS message on the bus to inform the pipeline that a particular stream has
|
an EOS message on the bus to inform the pipeline that a particular stream has
|
||||||
finished. When all sinks have reported EOS, the pipeline forwards the EOS message
|
finished. When all sinks have reported EOS, the pipeline forwards the EOS message
|
||||||
to the application. The EOS message is only forwarded to the application in the
|
to the application. The EOS message is only forwarded to the application in the
|
||||||
PLAYING state.
|
PLAYING state.
|
||||||
|
|
||||||
When in EOS, the pipeline remains in the PLAYING state, it is the applications
|
When in EOS, the pipeline remains in the PLAYING state, it is the applications
|
||||||
responsability to PAUSE or READY the pipeline. The application can also issue
|
responsability to PAUSE or READY the pipeline. The application can also issue
|
||||||
a seek, for example.
|
a seek, for example.
|
||||||
|
|
||||||
|
|
||||||
Pipeline READY
|
Pipeline READY
|
||||||
--------------
|
~~~~~~~~~~~~~~
|
||||||
|
|
||||||
When a running pipeline is set from the PLAYING to READY state, the following
|
When a running pipeline is set from the PLAYING to READY state, the following
|
||||||
actions occur in the pipeline:
|
actions occur in the pipeline:
|
||||||
|
|
||||||
alsasink to PAUSED: alsasink blocks and completes the state change on the
|
|
||||||
next sample. If the element was EOS, it does not wait for
|
|
||||||
a sample to complete the state change.
|
|
||||||
mp3dec to PAUSED: nothing
|
|
||||||
filesrc to PAUSED: nothing
|
|
||||||
|
|
||||||
Going to the intermediate PAUSED state will block all elements in the _push()
|
alsasink to PAUSED: alsasink blocks and completes the state change on the
|
||||||
functions. This happens because the sink element blocks on the first buffer
|
next sample. If the element was EOS, it does not wait for
|
||||||
it receives.
|
a sample to complete the state change.
|
||||||
|
mp3dec to PAUSED: nothing
|
||||||
|
filesrc to PAUSED: nothing
|
||||||
|
|
||||||
Some elements might be performing blocking operations in the PLAYING state that
|
Going to the intermediate PAUSED state will block all elements in the _push()
|
||||||
must be unblocked when they go into the PAUSED state. This makes sure that the
|
functions. This happens because the sink element blocks on the first buffer
|
||||||
state change happens very fast.
|
it receives.
|
||||||
|
|
||||||
In the next PAUSED to READY state change the pipeline has to shut down and all
|
Some elements might be performing blocking operations in the PLAYING state that
|
||||||
streaming threads must stop sending data. This happens in the following sequence:
|
must be unblocked when they go into the PAUSED state. This makes sure that the
|
||||||
|
state change happens very fast.
|
||||||
|
|
||||||
alsasink to READY: alsasink unblocks from the _chain() function and returns a
|
In the next PAUSED to READY state change the pipeline has to shut down and all
|
||||||
WRONG_STATE return value to the peer element. The sinkpad is
|
streaming threads must stop sending data. This happens in the following sequence:
|
||||||
deactivated and becomes unusable for sending more data.
|
|
||||||
mp3dec to READY: the pads are deactivated and the state change completes when
|
alsasink to READY: alsasink unblocks from the _chain() function and returns a
|
||||||
mp3dec leaves its _chain() function.
|
WRONG_STATE return value to the peer element. The sinkpad is
|
||||||
filesrc to READY: the pads are deactivated and the thread is paused.
|
deactivated and becomes unusable for sending more data.
|
||||||
|
mp3dec to READY: the pads are deactivated and the state change completes when
|
||||||
|
mp3dec leaves its _chain() function.
|
||||||
|
filesrc to READY: the pads are deactivated and the thread is paused.
|
||||||
|
|
||||||
|
The upstream elements finish their chain() function because the downstream element
|
||||||
|
returned an error code (WRONG_STATE) from the _push() functions. These error codes
|
||||||
|
are eventually returned to the element that started the streaming thread (filesrc),
|
||||||
|
which pauses the thread and completes the state change.
|
||||||
|
|
||||||
|
This sequence of events ensure that all elements are unblocked and all streaming
|
||||||
|
threads stopped.
|
||||||
|
|
||||||
The upstream elements finish their chain() function because the downstream element
|
|
||||||
returned an error code (WRONG_STATE) from the _push() functions. These error codes
|
|
||||||
are eventually returned to the element that started the streaming thread (filesrc),
|
|
||||||
which pauses the thread and completes the state change.
|
|
||||||
|
|
||||||
This sequence of events ensure that all elements are unblocked and all streaming
|
|
||||||
threads stopped.
|
|
||||||
|
|
||||||
|
|
||||||
Pipeline seeking
|
Pipeline seeking
|
||||||
----------------
|
~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
Seeking in the pipeline requires a very specific order of operations to make
|
Seeking in the pipeline requires a very specific order of operations to make
|
||||||
sure that the elements remain synchronized and that the seek is performed with
|
sure that the elements remain synchronized and that the seek is performed with
|
||||||
a minimal amount of latency.
|
a minimal amount of latency.
|
||||||
|
|
||||||
An application issues a seek event on the pipeline using gst_element_send_event()
|
An application issues a seek event on the pipeline using gst_element_send_event()
|
||||||
on the pipeline element. The event can be a seek event in any of the formats
|
on the pipeline element. The event can be a seek event in any of the formats
|
||||||
supported by the elements.
|
supported by the elements.
|
||||||
|
|
||||||
The pipeline first pauses the pipeline to speed up the seek operations.
|
The pipeline first pauses the pipeline to speed up the seek operations.
|
||||||
|
|
||||||
The pipeline then issues the seek event to all sink elements. The sink then forwards
|
The pipeline then issues the seek event to all sink elements. The sink then forwards
|
||||||
the seek event upstream until some element can perform the seek operation, which is
|
the seek event upstream until some element can perform the seek operation, which is
|
||||||
typically the source or demuxer element. All intermediate elements can transform the
|
typically the source or demuxer element. All intermediate elements can transform the
|
||||||
requested seek offset to another format, this way a decoder element can transform a
|
requested seek offset to another format, this way a decoder element can transform a
|
||||||
seek to a frame number to a timestamp, for example.
|
seek to a frame number to a timestamp, for example.
|
||||||
|
|
||||||
When the seek event reaches an element that will perform the seek operation, that
|
When the seek event reaches an element that will perform the seek operation, that
|
||||||
element performs the following steps.
|
element performs the following steps.
|
||||||
|
|
||||||
1) send a FLUSH_START event to all downstream and upstream peer elements.
|
1) send a FLUSH_START event to all downstream and upstream peer elements.
|
||||||
2) make sure the streaming thread is not running. The streaming thread will
|
2) make sure the streaming thread is not running. The streaming thread will
|
||||||
|
|
@ -491,31 +491,31 @@ Pipeline seeking
|
||||||
5) send NEWSEGMENT event to inform all elements of the new position and to complete
|
5) send NEWSEGMENT event to inform all elements of the new position and to complete
|
||||||
the seek.
|
the seek.
|
||||||
|
|
||||||
In step 1) all dowstream elements have to return from any blocking operations
|
In step 1) all dowstream elements have to return from any blocking operations
|
||||||
and have to refuse any further buffers or events different from a FLUSH done.
|
and have to refuse any further buffers or events different from a FLUSH done.
|
||||||
|
|
||||||
The first step ensures that the streaming thread eventually unblocks and that
|
The first step ensures that the streaming thread eventually unblocks and that
|
||||||
step 2) can be performed. At this point, dataflow is completely stopped in the
|
step 2) can be performed. At this point, dataflow is completely stopped in the
|
||||||
pipeline.
|
pipeline.
|
||||||
|
|
||||||
In step 3) the element performs the seek to the requested position.
|
In step 3) the element performs the seek to the requested position.
|
||||||
|
|
||||||
In step 4) all peer elements are allowed to accept data again and streaming
|
In step 4) all peer elements are allowed to accept data again and streaming
|
||||||
can continue from the new position. A FLUSH done event is sent to all the peer
|
can continue from the new position. A FLUSH done event is sent to all the peer
|
||||||
elements so that they accept new data again and restart their streaming threads.
|
elements so that they accept new data again and restart their streaming threads.
|
||||||
|
|
||||||
Step 5) informs all elements of the new position in the stream. After that the
|
Step 5) informs all elements of the new position in the stream. After that the
|
||||||
event function returns back to the application. and the streaming threads start
|
event function returns back to the application. and the streaming threads start
|
||||||
to produce new data.
|
to produce new data.
|
||||||
|
|
||||||
Since the pipeline is still PAUSED, this will preroll the next media sample in the
|
Since the pipeline is still PAUSED, this will preroll the next media sample in the
|
||||||
sinks. The application can wait for this preroll to complete by performing a
|
sinks. The application can wait for this preroll to complete by performing a
|
||||||
_get_state() on the pipeline.
|
_get_state() on the pipeline.
|
||||||
|
|
||||||
The last step in the seek operation is then to adjust the stream time of the pipeline
|
The last step in the seek operation is then to adjust the stream time of the pipeline
|
||||||
to 0 and to set the pipeline back to PLAYING.
|
to 0 and to set the pipeline back to PLAYING.
|
||||||
|
|
||||||
The sequence of events in our mp3 playback example.
|
The sequence of events in our mp3 playback example.
|
||||||
|
|
||||||
| a) seek on pipeline
|
| a) seek on pipeline
|
||||||
| b) PAUSE pipeline
|
| b) PAUSE pipeline
|
||||||
|
|
@ -537,4 +537,4 @@ Pipeline seeking
|
||||||
|
|
||||||
| e) update stream time to 0
|
| e) update stream time to 0
|
||||||
| f) PLAY pipeline
|
| f) PLAY pipeline
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -19,7 +19,7 @@ sinks (see part-element-sink.txt)
|
||||||
|
|
||||||
|
|
||||||
Committing the state
|
Committing the state
|
||||||
--------------------
|
~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
When going to PAUSED and PLAYING a buffer should be queued in the pad. We also
|
When going to PAUSED and PLAYING a buffer should be queued in the pad. We also
|
||||||
make this requirement for going to PLAYING since a flush event in the PAUSED
|
make this requirement for going to PLAYING since a flush event in the PAUSED
|
||||||
|
|
@ -39,7 +39,7 @@ blocking wait.
|
||||||
|
|
||||||
|
|
||||||
Unlocking the preroll
|
Unlocking the preroll
|
||||||
---------------------
|
~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
The following conditions unlock the preroll:
|
The following conditions unlock the preroll:
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -23,7 +23,7 @@ srcpads implementing a getrange function will be the exception.
|
||||||
|
|
||||||
|
|
||||||
state changes
|
state changes
|
||||||
-------------
|
~~~~~~~~~~~~~
|
||||||
|
|
||||||
The GstBin sets the state of all the sink elements. These are the elements
|
The GstBin sets the state of all the sink elements. These are the elements
|
||||||
without source pads.
|
without source pads.
|
||||||
|
|
|
||||||
|
|
@ -18,7 +18,7 @@ made:
|
||||||
|
|
||||||
|
|
||||||
Sources of quality problems
|
Sources of quality problems
|
||||||
---------------------------
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
- High CPU load
|
- High CPU load
|
||||||
- Network problems
|
- Network problems
|
||||||
|
|
@ -26,7 +26,7 @@ Sources of quality problems
|
||||||
|
|
||||||
|
|
||||||
QoS event
|
QoS event
|
||||||
---------
|
~~~~~~~~~
|
||||||
|
|
||||||
The QoS event is generated by an element that synchronizes against the clock. It
|
The QoS event is generated by an element that synchronizes against the clock. It
|
||||||
travels upstream and contains the following fields:
|
travels upstream and contains the following fields:
|
||||||
|
|
@ -51,7 +51,7 @@ operations.
|
||||||
|
|
||||||
|
|
||||||
QoS message
|
QoS message
|
||||||
-----------
|
~~~~~~~~~~~
|
||||||
|
|
||||||
A QOS message is posted on the bus whenever an element decides to:
|
A QOS message is posted on the bus whenever an element decides to:
|
||||||
|
|
||||||
|
|
@ -122,7 +122,7 @@ relevant elements or baseclasses.
|
||||||
|
|
||||||
|
|
||||||
Collecting statistics
|
Collecting statistics
|
||||||
---------------------
|
~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
A buffer with timestamp B1 arrives in the sink at time T1. The buffer
|
A buffer with timestamp B1 arrives in the sink at time T1. The buffer
|
||||||
timestamp is then synchronized against the clock which yields a jitter J1
|
timestamp is then synchronized against the clock which yields a jitter J1
|
||||||
|
|
@ -179,7 +179,7 @@ An element that is not receiving enough data is said to be starved.
|
||||||
|
|
||||||
|
|
||||||
Element measurements
|
Element measurements
|
||||||
--------------------
|
~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
In addition to the measurements of the datarate of the upstream element, a
|
In addition to the measurements of the datarate of the upstream element, a
|
||||||
typical element must also measure its own performance. Global pipeline
|
typical element must also measure its own performance. Global pipeline
|
||||||
|
|
@ -389,3 +389,4 @@ Live sources will automatically drop data when it takes too long to process the
|
||||||
that the element pushes out.
|
that the element pushes out.
|
||||||
|
|
||||||
Live sources should post a QoS message when data is dropped.
|
Live sources should post a QoS message when data is dropped.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,12 +1,16 @@
|
||||||
DRAFT Query
|
Query
|
||||||
-----------
|
-----
|
||||||
|
|
||||||
|
NOTE: this is implemented as proposed
|
||||||
|
|
||||||
Purpose
|
Purpose
|
||||||
|
~~~~~~~
|
||||||
|
|
||||||
Queries are used to get information about the stream.
|
Queries are used to get information about the stream.
|
||||||
A query is started on a specific pad and travels up or downstream.
|
A query is started on a specific pad and travels up or downstream.
|
||||||
|
|
||||||
Types of queries
|
Types of queries
|
||||||
|
~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
- get length of stream
|
- get length of stream
|
||||||
- get position in stream
|
- get position in stream
|
||||||
|
|
@ -17,6 +21,7 @@ Types of queries
|
||||||
- query internal links.
|
- query internal links.
|
||||||
|
|
||||||
Current implementation
|
Current implementation
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
The current implementation of query requires pads to implement the
|
The current implementation of query requires pads to implement the
|
||||||
following functions:
|
following functions:
|
||||||
|
|
@ -36,12 +41,14 @@ Current implementation
|
||||||
|
|
||||||
|
|
||||||
Requirements
|
Requirements
|
||||||
|
~~~~~~~~~~~~
|
||||||
|
|
||||||
- multiple return values, grouped together when they make sense.
|
- multiple return values, grouped together when they make sense.
|
||||||
- one pad function to perform the query
|
- one pad function to perform the query
|
||||||
- extensible queries.
|
- extensible queries.
|
||||||
|
|
||||||
Proposition
|
Proposition
|
||||||
|
~~~~~~~~~~~
|
||||||
|
|
||||||
- define GstQuery extending GstMiniObject and containing a GstStructure (see GstMessage)
|
- define GstQuery extending GstMiniObject and containing a GstStructure (see GstMessage)
|
||||||
- define standard query types (see proposed types)
|
- define standard query types (see proposed types)
|
||||||
|
|
@ -54,6 +61,7 @@ Proposition
|
||||||
query is not supported.
|
query is not supported.
|
||||||
|
|
||||||
Proposed types
|
Proposed types
|
||||||
|
~~~~~~~~~~~~~~
|
||||||
|
|
||||||
- GST_QUERY_SEEKING:
|
- GST_QUERY_SEEKING:
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -5,487 +5,487 @@ This document describes the relations between objects that exist in GStreamer.
|
||||||
It will also describe the way of handling the relation wrt locking and
|
It will also describe the way of handling the relation wrt locking and
|
||||||
refcounting.
|
refcounting.
|
||||||
|
|
||||||
1) parent-child relation
|
parent-child relation
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
+---------+ +-------+
|
+---------+ +-------+
|
||||||
| parent | | child |
|
| parent | | child |
|
||||||
*--->| *----->| |
|
*--->| *----->| |
|
||||||
| F1|<-----* 1|
|
| F1|<-----* 1|
|
||||||
+---------+ +-------+
|
+---------+ +-------+
|
||||||
|
|
||||||
- properties
|
- properties
|
||||||
|
|
||||||
- parent has references to multiple children
|
- parent has references to multiple children
|
||||||
- child has reference to parent
|
- child has reference to parent
|
||||||
- reference fields protected with LOCK
|
- reference fields protected with LOCK
|
||||||
- the reference held by each child to the parent is
|
- the reference held by each child to the parent is
|
||||||
NOT reflected in the refcount of the parent.
|
NOT reflected in the refcount of the parent.
|
||||||
- the parent removes the floating flag of the child when taking
|
- the parent removes the floating flag of the child when taking
|
||||||
ownership.
|
ownership.
|
||||||
- the application has valid reference to parent
|
- the application has valid reference to parent
|
||||||
- creation/destruction requires two unnested locks and 1 refcount.
|
- creation/destruction requires two unnested locks and 1 refcount.
|
||||||
|
|
||||||
- usage in GStreamer
|
- usage in GStreamer
|
||||||
|
|
||||||
GstBin -> GstElement
|
GstBin -> GstElement
|
||||||
GstElement -> GstRealPad
|
GstElement -> GstRealPad
|
||||||
|
|
||||||
- lifecycle
|
- lifecycle
|
||||||
|
|
||||||
a) object creation
|
a) object creation
|
||||||
|
|
||||||
The application creates two object and holds a pointer
|
The application creates two object and holds a pointer
|
||||||
to them. The objects are initially FLOATING with a refcount
|
to them. The objects are initially FLOATING with a refcount
|
||||||
of 1.
|
of 1.
|
||||||
|
|
||||||
+---------+ +-------+
|
+---------+ +-------+
|
||||||
*--->| parent | *--->| child |
|
*--->| parent | *--->| child |
|
||||||
| * | | |
|
| * | | |
|
||||||
| F1| | * F1|
|
| F1| | * F1|
|
||||||
+---------+ +-------+
|
+---------+ +-------+
|
||||||
|
|
||||||
b) establishing the parent-child relationship
|
b) establishing the parent-child relationship
|
||||||
|
|
||||||
The application then calls a method on the parent object to take
|
The application then calls a method on the parent object to take
|
||||||
ownership of the child object. The parent performs the following
|
ownership of the child object. The parent performs the following
|
||||||
actions:
|
actions:
|
||||||
|
|
||||||
result = _set_parent (child, parent);
|
result = _set_parent (child, parent);
|
||||||
if (result) {
|
if (result) {
|
||||||
LOCK (parent);
|
LOCK (parent);
|
||||||
ref_pointer = child;
|
ref_pointer = child;
|
||||||
|
|
||||||
.. update other data structures ..
|
.. update other data structures ..
|
||||||
UNLOCK (parent);
|
UNLOCK (parent);
|
||||||
}
|
}
|
||||||
else {
|
else {
|
||||||
.. child had parent ..
|
.. child had parent ..
|
||||||
}
|
}
|
||||||
|
|
||||||
The _set_parent() method performs the following actions:
|
The _set_parent() method performs the following actions:
|
||||||
|
|
||||||
LOCK (child);
|
LOCK (child);
|
||||||
if (child->parent != NULL) {
|
if (child->parent != NULL) {
|
||||||
UNLOCK (child);
|
|
||||||
return FALSE;
|
|
||||||
}
|
|
||||||
if (IS_FLOATING (child)) {
|
|
||||||
UNSET (child, FLOATING);
|
|
||||||
}
|
|
||||||
else {
|
|
||||||
_ref (child);
|
|
||||||
}
|
|
||||||
child->parent = parent;
|
|
||||||
UNLOCK (child);
|
UNLOCK (child);
|
||||||
_signal (PARENT_SET, child, parent);
|
return FALSE;
|
||||||
return TRUE;
|
}
|
||||||
|
if (IS_FLOATING (child)) {
|
||||||
The function atomically checks if the child has no parent yet
|
UNSET (child, FLOATING);
|
||||||
and will set the parent if not. It will also sink the child, meaning
|
}
|
||||||
all floating references to the child are invalid now as it takes
|
else {
|
||||||
over the refcount of the object.
|
_ref (child);
|
||||||
|
}
|
||||||
|
child->parent = parent;
|
||||||
|
UNLOCK (child);
|
||||||
|
_signal (PARENT_SET, child, parent);
|
||||||
|
return TRUE;
|
||||||
|
|
||||||
|
The function atomically checks if the child has no parent yet
|
||||||
|
and will set the parent if not. It will also sink the child, meaning
|
||||||
|
all floating references to the child are invalid now as it takes
|
||||||
|
over the refcount of the object.
|
||||||
|
|
||||||
Visually:
|
Visually:
|
||||||
|
|
||||||
after _set_parent() returns TRUE:
|
after _set_parent() returns TRUE:
|
||||||
|
|
||||||
+---------+ +-------+
|
+---------+ +-------+
|
||||||
*---->| parent | *-//->| child |
|
*---->| parent | *-//->| child |
|
||||||
| * | | |
|
| * | | |
|
||||||
| F1|<-------------* 1|
|
| F1|<-------------* 1|
|
||||||
+---------+ +-------+
|
+---------+ +-------+
|
||||||
|
|
||||||
after parent updates ref_pointer to child.
|
after parent updates ref_pointer to child.
|
||||||
|
|
||||||
+---------+ +-------+
|
+---------+ +-------+
|
||||||
*---->| parent | *-//->| child |
|
*---->| parent | *-//->| child |
|
||||||
| *--------->| |
|
| *--------->| |
|
||||||
| F1|<---------* 1|
|
| F1|<---------* 1|
|
||||||
+---------+ +-------+
|
+---------+ +-------+
|
||||||
|
|
||||||
|
- only one parent is able to _sink the same object because the
|
||||||
|
_set_parent() method is atomic.
|
||||||
|
- since only one parent is able to _set_parent() the object, only
|
||||||
|
one will add a reference to the object.
|
||||||
|
- since the parent can hold multiple references to children, we don't
|
||||||
|
need to lock the parent when locking the child. Many threads can
|
||||||
|
call _set_parent() on the children with the same parent, the parent
|
||||||
|
can then add all those to its lists.
|
||||||
|
|
||||||
|
Note: that the signal is emited before the parent has added the
|
||||||
|
element to its internal data structures. This is not a problem
|
||||||
|
since the parent usually has his own signal to inform the app that
|
||||||
|
the child was reffed. One possible solution would be to update the
|
||||||
|
internal structure first and then perform a rollback if the _set_parent()
|
||||||
|
failed. This is not a good solution as iterators might grab the
|
||||||
|
'half-added' child too soon.
|
||||||
|
|
||||||
|
c) using the parent-child relationship
|
||||||
|
|
||||||
|
- since the initial floating reference to the child object became
|
||||||
|
invalid after giving it to the parent, any reference to a child
|
||||||
|
has at least a refcount > 1.
|
||||||
|
|
||||||
|
- this means that unreffing a child object cannot decrease the refcount
|
||||||
|
to 0. In fact, only the parent can destroy and dispose the child
|
||||||
|
object.
|
||||||
|
|
||||||
|
- given a reference to the child object, the parent pointer is only
|
||||||
|
valid when holding the child LOCK. Indeed, after unlocking the child
|
||||||
|
LOCK, the parent can unparent the child or the parent could even become
|
||||||
|
disposed. To avoid the parent dispose problem, when obtaining the
|
||||||
|
parent pointer, if should be reffed before releasing the child LOCK.
|
||||||
|
|
||||||
|
1) getting a reference to the parent.
|
||||||
|
|
||||||
- only one parent is able to _sink the same object because the
|
- a referece is held to the child, so it cannot be disposed.
|
||||||
_set_parent() method is atomic.
|
|
||||||
- since only one parent is able to _set_parent() the object, only
|
LOCK (child);
|
||||||
one will add a reference to the object.
|
parent = _ref (child->parent);
|
||||||
- since the parent can hold multiple references to children, we don't
|
UNLOCK (child);
|
||||||
need to lock the parent when locking the child. Many threads can
|
|
||||||
call _set_parent() on the children with the same parent, the parent
|
|
||||||
can then add all those to its lists.
|
|
||||||
|
|
||||||
Note: that the signal is emited before the parent has added the
|
.. use parent ..
|
||||||
element to its internal data structures. This is not a problem
|
|
||||||
since the parent usually has his own signal to inform the app that
|
|
||||||
the child was reffed. One possible solution would be to update the
|
|
||||||
internal structure first and then perform a rollback if the _set_parent()
|
|
||||||
failed. This is not a good solution as iterators might grab the
|
|
||||||
'half-added' child too soon.
|
|
||||||
|
|
||||||
c) using the parent-child relationship
|
|
||||||
|
|
||||||
- since the initial floating reference to the child object became
|
|
||||||
invalid after giving it to the parent, any reference to a child
|
|
||||||
has at least a refcount > 1.
|
|
||||||
|
|
||||||
- this means that unreffing a child object cannot decrease the refcount
|
_unref (parent);
|
||||||
to 0. In fact, only the parent can destroy and dispose the child
|
|
||||||
object.
|
|
||||||
|
|
||||||
- given a reference to the child object, the parent pointer is only
|
2) getting a reference to a child
|
||||||
valid when holding the child LOCK. Indeed, after unlocking the child
|
|
||||||
LOCK, the parent can unparent the child or the parent could even become
|
- a reference to a child can be obtained by reffing it before
|
||||||
disposed. To avoid the parent dispose problem, when obtaining the
|
adding it to the parent or by querying the parent.
|
||||||
parent pointer, if should be reffed before releasing the child LOCK.
|
|
||||||
|
- when requesting a child from the parent, a reference is held to
|
||||||
|
the parent so it cannot be disposed. The parent will use its
|
||||||
|
internal data structures to locate the child element and will
|
||||||
|
return a reference to it with an incremented refcount. The
|
||||||
|
requester should _unref() the child after usage.
|
||||||
|
|
||||||
|
|
||||||
|
d) destroying the parent-child relationship
|
||||||
|
|
||||||
|
- only the parent can actively destroy the parent-child relationship
|
||||||
|
this typically happens when a method is called on the parent to release
|
||||||
|
ownership of the child.
|
||||||
|
|
||||||
|
- a child shall never remove itself from the parent.
|
||||||
|
|
||||||
|
- since calling a method on the parent with the child as an argument
|
||||||
|
requires the caller to obtain a valid reference to the child, the child
|
||||||
|
refcount is at least > 1.
|
||||||
|
|
||||||
|
- the parent will perform the folowing actions:
|
||||||
|
|
||||||
|
LOCK (parent);
|
||||||
|
if (ref_pointer == child) {
|
||||||
|
ref_pointer = NULL;
|
||||||
|
|
||||||
|
.. update other data structures ..
|
||||||
|
UNLOCK (parent);
|
||||||
|
|
||||||
|
_unparent (child);
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
UNLOCK (parent);
|
||||||
|
.. not our child ..
|
||||||
|
}
|
||||||
|
|
||||||
|
The _unparent() method performs the following actions:
|
||||||
|
|
||||||
|
LOCK (child);
|
||||||
|
if (child->parent != NULL) {
|
||||||
|
child->parent = NULL;
|
||||||
|
UNLOCK (child);
|
||||||
|
_signal (PARENT_UNSET, child, parent);
|
||||||
|
|
||||||
|
_unref (child);
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
UNLOCK (child);
|
||||||
|
}
|
||||||
|
|
||||||
I) getting a reference to the parent.
|
Since the _unparent() method unrefs the child object, it is possible that
|
||||||
|
the child pointer is invalid after this function. If the parent wants to
|
||||||
- a referece is held to the child, so it cannot be disposed.
|
perform other actions on the child (such as signal emmision) it should
|
||||||
|
_ref() the child first.
|
||||||
LOCK (child);
|
|
||||||
parent = _ref (child->parent);
|
|
||||||
UNLOCK (child);
|
|
||||||
|
|
||||||
.. use parent ..
|
|
||||||
|
|
||||||
_unref (parent);
|
|
||||||
|
|
||||||
II) getting a reference to a child
|
|
||||||
|
|
||||||
- a reference to a child can be obtained by reffing it before
|
|
||||||
adding it to the parent or by querying the parent.
|
|
||||||
|
|
||||||
- when requesting a child from the parent, a reference is held to
|
|
||||||
the parent so it cannot be disposed. The parent will use its
|
|
||||||
internal data structures to locate the child element and will
|
|
||||||
return a reference to it with an incremented refcount. The
|
|
||||||
requester should _unref() the child after usage.
|
|
||||||
|
|
||||||
|
|
||||||
d) destroying the parent-child relationship
|
|
||||||
|
|
||||||
- only the parent can actively destroy the parent-child relationship
|
|
||||||
this typically happens when a method is called on the parent to release
|
|
||||||
ownership of the child.
|
|
||||||
|
|
||||||
- a child shall never remove itself from the parent.
|
|
||||||
|
|
||||||
- since calling a method on the parent with the child as an argument
|
|
||||||
requires the caller to obtain a valid reference to the child, the child
|
|
||||||
refcount is at least > 1.
|
|
||||||
|
|
||||||
- the parent will perform the folowing actions:
|
|
||||||
|
|
||||||
LOCK (parent);
|
|
||||||
if (ref_pointer == child) {
|
|
||||||
ref_pointer = NULL;
|
|
||||||
|
|
||||||
.. update other data structures ..
|
|
||||||
UNLOCK (parent);
|
|
||||||
|
|
||||||
_unparent (child);
|
|
||||||
}
|
|
||||||
else {
|
|
||||||
UNLOCK (parent);
|
|
||||||
.. not our child ..
|
|
||||||
}
|
|
||||||
|
|
||||||
The _unparent() method performs the following actions:
|
|
||||||
|
|
||||||
LOCK (child);
|
|
||||||
if (child->parent != NULL) {
|
|
||||||
child->parent = NULL;
|
|
||||||
UNLOCK (child);
|
|
||||||
_signal (PARENT_UNSET, child, parent);
|
|
||||||
|
|
||||||
_unref (child);
|
|
||||||
}
|
|
||||||
else {
|
|
||||||
UNLOCK (child);
|
|
||||||
}
|
|
||||||
|
|
||||||
Since the _unparent() method unrefs the child object, it is possible that
|
|
||||||
the child pointer is invalid after this function. If the parent wants to
|
|
||||||
perform other actions on the child (such as signal emmision) it should
|
|
||||||
_ref() the child first.
|
|
||||||
|
|
||||||
|
|
||||||
2) single-reffed relation
|
single-reffed relation
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
+---------+ +---------+
|
+---------+ +---------+
|
||||||
*--->| object1 | *--->| object2 |
|
*--->| object1 | *--->| object2 |
|
||||||
| *--------->| |
|
| *--------->| |
|
||||||
| 1| | 2|
|
| 1| | 2|
|
||||||
+---------+ +---------+
|
+---------+ +---------+
|
||||||
|
|
||||||
- properties
|
- properties
|
||||||
|
|
||||||
- one object has a reference to another
|
- one object has a reference to another
|
||||||
- reference field protected with LOCK
|
- reference field protected with LOCK
|
||||||
- the reference held by the object is reflected in the
|
- the reference held by the object is reflected in the
|
||||||
refcount of the other object.
|
refcount of the other object.
|
||||||
- typically the other object can be shared among multiple
|
- typically the other object can be shared among multiple
|
||||||
other objects where each ref is counted for in the
|
other objects where each ref is counted for in the
|
||||||
refcount.
|
refcount.
|
||||||
- no object has ownership of the other.
|
- no object has ownership of the other.
|
||||||
- either shared state or copy-on-write.
|
- either shared state or copy-on-write.
|
||||||
- creation/destruction requires one lock and one refcount.
|
- creation/destruction requires one lock and one refcount.
|
||||||
|
|
||||||
- usage
|
- usage
|
||||||
|
|
||||||
GstRealPad -> GstCaps
|
GstRealPad -> GstCaps
|
||||||
GstBuffer -> GstCaps
|
GstBuffer -> GstCaps
|
||||||
GstEvent -> GstCaps
|
GstEvent -> GstCaps
|
||||||
GstEvent -> GstObject
|
GstEvent -> GstObject
|
||||||
GstMessage -> GstCaps
|
GstMessage -> GstCaps
|
||||||
GstMessage -> GstObject
|
GstMessage -> GstObject
|
||||||
|
|
||||||
- lifecycle
|
- lifecycle
|
||||||
|
|
||||||
a) Two objects exist unlinked.
|
a) Two objects exist unlinked.
|
||||||
|
|
||||||
+---------+ +---------+
|
+---------+ +---------+
|
||||||
*--->| object1 | *--->| object2 |
|
*--->| object1 | *--->| object2 |
|
||||||
| * | | |
|
| * | | |
|
||||||
| 1| | 1|
|
| 1| | 1|
|
||||||
+---------+ +---------+
|
+---------+ +---------+
|
||||||
|
|
||||||
b) establishing the single-reffed relationship
|
b) establishing the single-reffed relationship
|
||||||
|
|
||||||
The second object is attached to the first one using a method
|
The second object is attached to the first one using a method
|
||||||
on the first object. The second object is reffed and a pointer
|
on the first object. The second object is reffed and a pointer
|
||||||
is updated in the first object using the following algorithm:
|
is updated in the first object using the following algorithm:
|
||||||
|
|
||||||
LOCK (object1);
|
LOCK (object1);
|
||||||
|
if (object1->pointer)
|
||||||
|
_unref (object1->pointer);
|
||||||
|
object1->pointer = _ref (object2);
|
||||||
|
UNLOCK (object1);
|
||||||
|
|
||||||
|
After releasing the lock on the first object is is not sure that
|
||||||
|
object2 is still reffed from object1.
|
||||||
|
|
||||||
|
+---------+ +---------+
|
||||||
|
*--->| object1 | *--->| object2 |
|
||||||
|
| *--------->| |
|
||||||
|
| 1| | 2|
|
||||||
|
+---------+ +---------+
|
||||||
|
|
||||||
|
c) using the single-reffed relationship
|
||||||
|
|
||||||
|
The only way to access object2 is by holding a ref to it or by
|
||||||
|
getting the reference from object1.
|
||||||
|
Reading the object pointed to by object1 can be done like this:
|
||||||
|
|
||||||
|
LOCK (object1);
|
||||||
|
object2 = object1->pointer;
|
||||||
|
_ref (object2);
|
||||||
|
UNLOCK (object1);
|
||||||
|
|
||||||
|
.. use object2 ...
|
||||||
|
_unref (object2);
|
||||||
|
|
||||||
|
Depending on the type of the object, modifications can be done either
|
||||||
|
with copy-on-write or directly into the object.
|
||||||
|
|
||||||
|
Copy on write can practically only be done like this:
|
||||||
|
|
||||||
|
LOCK (object1);
|
||||||
|
object2 = object1->pointer;
|
||||||
|
object2 = _copy_on_write (object2);
|
||||||
|
... make modifications to object2 ...
|
||||||
|
UNLOCK (object1);
|
||||||
|
|
||||||
|
Releasing the lock has only a very small window where the copy_on_write
|
||||||
|
actually does not perform a copy:
|
||||||
|
|
||||||
|
LOCK (object1);
|
||||||
|
object2 = object1->pointer;
|
||||||
|
_ref (object2);
|
||||||
|
UNLOCK (object1);
|
||||||
|
|
||||||
|
.. object2 now has at least 2 refcounts making the next
|
||||||
|
copy-on-write make a real copy, unless some other thread
|
||||||
|
writes another object2 to object1 here ...
|
||||||
|
|
||||||
|
object2 = _copy_on_write (object2);
|
||||||
|
|
||||||
|
.. make modifications to object2 ...
|
||||||
|
|
||||||
|
LOCK (object1);
|
||||||
|
if (object1->pointer != object2) {
|
||||||
if (object1->pointer)
|
if (object1->pointer)
|
||||||
_unref (object1->pointer);
|
_unref (object1->pointer);
|
||||||
object1->pointer = _ref (object2);
|
object1->pointer = gst_object_ref (object2);
|
||||||
UNLOCK (object1);
|
}
|
||||||
|
UNLOCK (object1);
|
||||||
|
|
||||||
After releasing the lock on the first object is is not sure that
|
d) destroying the single-reffed relationship
|
||||||
object2 is still reffed from object1.
|
|
||||||
|
The folowing algorithm removes the single-reffed link between
|
||||||
|
object1 and object2.
|
||||||
|
|
||||||
+---------+ +---------+
|
LOCK (object1);
|
||||||
*--->| object1 | *--->| object2 |
|
_unref (object1->pointer);
|
||||||
| *--------->| |
|
object1->pointer = NULL;
|
||||||
| 1| | 2|
|
UNLOCK (object1);
|
||||||
+---------+ +---------+
|
|
||||||
|
|
||||||
c) using the single-reffed relationship
|
|
||||||
|
|
||||||
The only way to access object2 is by holding a ref to it or by
|
|
||||||
getting the reference from object1.
|
|
||||||
Reading the object pointed to by object1 can be done like this:
|
|
||||||
|
|
||||||
LOCK (object1);
|
|
||||||
object2 = object1->pointer;
|
|
||||||
_ref (object2);
|
|
||||||
UNLOCK (object1);
|
|
||||||
|
|
||||||
.. use object2 ...
|
|
||||||
_unref (object2);
|
|
||||||
|
|
||||||
Depending on the type of the object, modifications can be done either
|
|
||||||
with copy-on-write or directly into the object.
|
|
||||||
|
|
||||||
Copy on write can practically only be done like this:
|
|
||||||
|
|
||||||
LOCK (object1);
|
|
||||||
object2 = object1->pointer;
|
|
||||||
object2 = _copy_on_write (object2);
|
|
||||||
... make modifications to object2 ...
|
|
||||||
UNLOCK (object1);
|
|
||||||
|
|
||||||
Releasing the lock has only a very small window where the copy_on_write
|
|
||||||
actually does not perform a copy:
|
|
||||||
|
|
||||||
LOCK (object1);
|
|
||||||
object2 = object1->pointer;
|
|
||||||
_ref (object2);
|
|
||||||
UNLOCK (object1);
|
|
||||||
|
|
||||||
.. object2 now has at least 2 refcounts making the next
|
|
||||||
copy-on-write make a real copy, unless some other thread
|
|
||||||
writes another object2 to object1 here ...
|
|
||||||
|
|
||||||
object2 = _copy_on_write (object2);
|
|
||||||
|
|
||||||
.. make modifications to object2 ...
|
|
||||||
|
|
||||||
LOCK (object1);
|
|
||||||
if (object1->pointer != object2) {
|
|
||||||
if (object1->pointer)
|
|
||||||
_unref (object1->pointer);
|
|
||||||
object1->pointer = gst_object_ref (object2);
|
|
||||||
}
|
|
||||||
UNLOCK (object1);
|
|
||||||
|
|
||||||
d) destroying the single-reffed relationship
|
|
||||||
|
|
||||||
The folowing algorithm removes the single-reffed link between
|
|
||||||
object1 and object2.
|
|
||||||
|
|
||||||
LOCK (object1);
|
|
||||||
_unref (object1->pointer);
|
|
||||||
object1->pointer = NULL;
|
|
||||||
UNLOCK (object1);
|
|
||||||
|
|
||||||
Which yields the following initial state again:
|
|
||||||
|
|
||||||
+---------+ +---------+
|
|
||||||
*--->| object1 | *--->| object2 |
|
|
||||||
| * | | |
|
|
||||||
| 1| | 1|
|
|
||||||
+---------+ +---------+
|
|
||||||
|
|
||||||
|
|
||||||
3) unreffed relation
|
|
||||||
|
|
||||||
+---------+ +---------+
|
|
||||||
*--->| object1 | *--->| object2 |
|
|
||||||
| *--------->| |
|
|
||||||
| 1|<---------* 1|
|
|
||||||
+---------+ +---------+
|
|
||||||
|
|
||||||
- properties
|
|
||||||
|
|
||||||
- two objects have references to eachother
|
|
||||||
- both objects can only have 1 reference to another object.
|
|
||||||
- reference fields protected with LOCK
|
|
||||||
- the references held by each object are NOT reflected in the
|
|
||||||
refcount of the other object.
|
|
||||||
- no object has ownership of the other.
|
|
||||||
- typically each object is owned by a different parent.
|
|
||||||
- creation/destruction requires two nested locks and no refcounts.
|
|
||||||
|
|
||||||
- usage
|
|
||||||
|
|
||||||
- This type of link is used when the link is less important than
|
|
||||||
the existance of the objects, If one of the objects is disposed, so
|
|
||||||
is the link.
|
|
||||||
|
|
||||||
GstRealPad <-> GstRealPad (srcpad lock taken first)
|
Which yields the following initial state again:
|
||||||
|
|
||||||
- lifecycle
|
+---------+ +---------+
|
||||||
|
*--->| object1 | *--->| object2 |
|
||||||
|
| * | | |
|
||||||
|
| 1| | 1|
|
||||||
|
+---------+ +---------+
|
||||||
|
|
||||||
a) Two objects exist unlinked.
|
|
||||||
|
|
||||||
+---------+ +---------+
|
unreffed relation
|
||||||
*--->| object1 | *--->| object2 |
|
~~~~~~~~~~~~~~~~~
|
||||||
| * | | |
|
|
||||||
| 1| | * 1|
|
+---------+ +---------+
|
||||||
+---------+ +---------+
|
*--->| object1 | *--->| object2 |
|
||||||
|
| *--------->| |
|
||||||
b) establishing the unreffed relationship
|
| 1|<---------* 1|
|
||||||
|
+---------+ +---------+
|
||||||
Since we need to take two locks, the order in which these locks are
|
|
||||||
taken is very important or we might cause deadlocks. This lock order
|
|
||||||
must be defined for all unreffed relations. In these examples we always
|
|
||||||
lock object1 first and then object2.
|
|
||||||
|
|
||||||
LOCK (object1);
|
|
||||||
LOCK (object2);
|
|
||||||
object2->refpointer = object1;
|
|
||||||
object1->refpointer = object2;
|
|
||||||
UNLOCK (object2);
|
|
||||||
UNLOCK (object1);
|
|
||||||
|
|
||||||
c) using the unreffed relationship
|
|
||||||
|
|
||||||
Reading requires taking one of the locks and reading the corresponing
|
|
||||||
object. Again we need to ref the object before releasing the lock.
|
|
||||||
|
|
||||||
LOCK (object1);
|
|
||||||
object2 = _ref (object1->refpointer);
|
|
||||||
UNLOCK (object1);
|
|
||||||
|
|
||||||
.. use object2 ..
|
|
||||||
_unref (object2);
|
|
||||||
|
|
||||||
d) destroying the unreffed relationship
|
|
||||||
|
|
||||||
Because of the lock order we need to be careful when destroying this
|
|
||||||
Relation.
|
|
||||||
|
|
||||||
When only a reference to object1 is held:
|
- properties
|
||||||
|
|
||||||
LOCK (object1);
|
- two objects have references to eachother
|
||||||
LOCK (object2);
|
- both objects can only have 1 reference to another object.
|
||||||
|
- reference fields protected with LOCK
|
||||||
|
- the references held by each object are NOT reflected in the
|
||||||
|
refcount of the other object.
|
||||||
|
- no object has ownership of the other.
|
||||||
|
- typically each object is owned by a different parent.
|
||||||
|
- creation/destruction requires two nested locks and no refcounts.
|
||||||
|
|
||||||
|
- usage
|
||||||
|
|
||||||
|
- This type of link is used when the link is less important than
|
||||||
|
the existance of the objects, If one of the objects is disposed, so
|
||||||
|
is the link.
|
||||||
|
|
||||||
|
GstRealPad <-> GstRealPad (srcpad lock taken first)
|
||||||
|
|
||||||
|
- lifecycle
|
||||||
|
|
||||||
|
a) Two objects exist unlinked.
|
||||||
|
|
||||||
|
+---------+ +---------+
|
||||||
|
*--->| object1 | *--->| object2 |
|
||||||
|
| * | | |
|
||||||
|
| 1| | * 1|
|
||||||
|
+---------+ +---------+
|
||||||
|
|
||||||
|
b) establishing the unreffed relationship
|
||||||
|
|
||||||
|
Since we need to take two locks, the order in which these locks are
|
||||||
|
taken is very important or we might cause deadlocks. This lock order
|
||||||
|
must be defined for all unreffed relations. In these examples we always
|
||||||
|
lock object1 first and then object2.
|
||||||
|
|
||||||
|
LOCK (object1);
|
||||||
|
LOCK (object2);
|
||||||
|
object2->refpointer = object1;
|
||||||
|
object1->refpointer = object2;
|
||||||
|
UNLOCK (object2);
|
||||||
|
UNLOCK (object1);
|
||||||
|
|
||||||
|
c) using the unreffed relationship
|
||||||
|
|
||||||
|
Reading requires taking one of the locks and reading the corresponing
|
||||||
|
object. Again we need to ref the object before releasing the lock.
|
||||||
|
|
||||||
|
LOCK (object1);
|
||||||
|
object2 = _ref (object1->refpointer);
|
||||||
|
UNLOCK (object1);
|
||||||
|
|
||||||
|
.. use object2 ..
|
||||||
|
_unref (object2);
|
||||||
|
|
||||||
|
d) destroying the unreffed relationship
|
||||||
|
|
||||||
|
Because of the lock order we need to be careful when destroying this
|
||||||
|
Relation.
|
||||||
|
|
||||||
|
When only a reference to object1 is held:
|
||||||
|
|
||||||
|
LOCK (object1);
|
||||||
|
LOCK (object2);
|
||||||
|
object1->refpointer->refpointer = NULL;
|
||||||
|
object1->refpointer = NULL;
|
||||||
|
UNLOCK (object2);
|
||||||
|
UNLOCK (object1);
|
||||||
|
|
||||||
|
When only a reference to object2 is held we need to get a handle to the
|
||||||
|
other object fist so that we can lock it first. There is a window where
|
||||||
|
we need to release all locks and the relation could be invalid. To solve
|
||||||
|
this we check the relation after grabbing both locks and retry if the
|
||||||
|
relation changed.
|
||||||
|
|
||||||
|
retry:
|
||||||
|
LOCK (object2);
|
||||||
|
object1 = _ref (object2->refpointer);
|
||||||
|
UNLOCK (object2);
|
||||||
|
.. things can change here ..
|
||||||
|
LOCK (object1);
|
||||||
|
LOCK (object2);
|
||||||
|
if (object1 == object2->refpointer) {
|
||||||
|
/* relation unchanged */
|
||||||
object1->refpointer->refpointer = NULL;
|
object1->refpointer->refpointer = NULL;
|
||||||
object1->refpointer = NULL;
|
object1->refpointer = NULL;
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
/* relation changed.. retry */
|
||||||
UNLOCK (object2);
|
UNLOCK (object2);
|
||||||
UNLOCK (object1);
|
UNLOCK (object1);
|
||||||
|
_unref (object1);
|
||||||
|
goto retry;
|
||||||
|
}
|
||||||
|
UNLOCK (object2);
|
||||||
|
UNLOCK (object1);
|
||||||
|
_unref (object1);
|
||||||
|
|
||||||
When only a reference to object2 is held we need to get a handle to the
|
When references are held to both objects. Note that it is not possible to
|
||||||
other object fist so that we can lock it first. There is a window where
|
get references to both objects with the locks released since when the
|
||||||
we need to release all locks and the relation could be invalid. To solve
|
references are taken and the locks are released, a concurrent update might
|
||||||
this we check the relation after grabbing both locks and retry if the
|
have changed the link, making the references not point to linked objects.
|
||||||
relation changed.
|
|
||||||
|
|
||||||
retry:
|
LOCK (object1);
|
||||||
LOCK (object2);
|
LOCK (object2);
|
||||||
object1 = _ref (object2->refpointer);
|
if (object1->refpointer == object2) {
|
||||||
UNLOCK (object2);
|
object2->refpointer = NULL;
|
||||||
.. things can change here ..
|
object1->refpointer = NULL;
|
||||||
LOCK (object1);
|
}
|
||||||
LOCK (object2);
|
else {
|
||||||
if (object1 == object2->refpointer) {
|
.. objects are not linked ..
|
||||||
/* relation unchanged */
|
}
|
||||||
object1->refpointer->refpointer = NULL;
|
UNLOCK (object2);
|
||||||
object1->refpointer = NULL;
|
UNLOCK (object1);
|
||||||
}
|
|
||||||
else {
|
|
||||||
/* relation changed.. retry */
|
|
||||||
UNLOCK (object2);
|
|
||||||
UNLOCK (object1);
|
|
||||||
_unref (object1);
|
|
||||||
goto retry;
|
|
||||||
}
|
|
||||||
UNLOCK (object2);
|
|
||||||
UNLOCK (object1);
|
|
||||||
_unref (object1);
|
|
||||||
|
|
||||||
When references are held to both objects. Note that it is not possible to
|
|
||||||
get references to both objects with the locks released since when the
|
|
||||||
references are taken and the locks are released, a concurrent update might
|
|
||||||
have changed the link, making the references not point to linked objects.
|
|
||||||
|
|
||||||
LOCK (object1);
|
|
||||||
LOCK (object2);
|
|
||||||
if (object1->refpointer == object2) {
|
|
||||||
object2->refpointer = NULL;
|
|
||||||
object1->refpointer = NULL;
|
|
||||||
}
|
|
||||||
else {
|
|
||||||
.. objects are not linked ..
|
|
||||||
}
|
|
||||||
UNLOCK (object2);
|
|
||||||
UNLOCK (object1);
|
|
||||||
|
|
||||||
|
|
||||||
4) double-reffed relation
|
double-reffed relation
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
+---------+ +---------+
|
+---------+ +---------+
|
||||||
*--->| object1 | *--->| object2 |
|
*--->| object1 | *--->| object2 |
|
||||||
| *--------->| |
|
| *--------->| |
|
||||||
| 2|<---------* 2|
|
| 2|<---------* 2|
|
||||||
+---------+ +---------+
|
+---------+ +---------+
|
||||||
|
|
||||||
- properties
|
|
||||||
|
|
||||||
- two objects have references to eachother
|
|
||||||
- reference fields protected with LOCK
|
|
||||||
- the references held by each object are reflected in the
|
|
||||||
refcount of the other object.
|
|
||||||
- no object has ownership of the other.
|
|
||||||
- typically each object is owned by a different parent.
|
|
||||||
- creation/destruction requires two locks and two refcounts.
|
|
||||||
|
|
||||||
- usage
|
|
||||||
|
|
||||||
Not used in GStreamer.
|
|
||||||
|
|
||||||
- lifecycle
|
|
||||||
|
|
||||||
|
- properties
|
||||||
|
|
||||||
|
- two objects have references to eachother
|
||||||
|
- reference fields protected with LOCK
|
||||||
|
- the references held by each object are reflected in the
|
||||||
|
refcount of the other object.
|
||||||
|
- no object has ownership of the other.
|
||||||
|
- typically each object is owned by a different parent.
|
||||||
|
- creation/destruction requires two locks and two refcounts.
|
||||||
|
|
||||||
|
- usage
|
||||||
|
|
||||||
|
Not used in GStreamer.
|
||||||
|
|
||||||
|
- lifecycle
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -5,7 +5,7 @@ The scheduling in GStreamer is based on pads actively pushing (producing) data o
|
||||||
pad pulling in data (consuming) from other pads.
|
pad pulling in data (consuming) from other pads.
|
||||||
|
|
||||||
Pushing
|
Pushing
|
||||||
-------
|
~~~~~~~
|
||||||
|
|
||||||
A pad can produce data and push it to the next pad. A pad that behaves this way
|
A pad can produce data and push it to the next pad. A pad that behaves this way
|
||||||
exposes a loop function that will be called repeadedly until it returns false.
|
exposes a loop function that will be called repeadedly until it returns false.
|
||||||
|
|
@ -20,7 +20,7 @@ This method of producing data is called the streaming mode since the producer
|
||||||
produces a constant stream of data.
|
produces a constant stream of data.
|
||||||
|
|
||||||
Pulling
|
Pulling
|
||||||
-------
|
~~~~~~~
|
||||||
|
|
||||||
Pads that operate in pulling mode can only pull data from a pad that exposes the
|
Pads that operate in pulling mode can only pull data from a pad that exposes the
|
||||||
pull_range function. In this case, the sink pad exposes a loop function that will be
|
pull_range function. In this case, the sink pad exposes a loop function that will be
|
||||||
|
|
@ -31,7 +31,7 @@ push function to push the result to the peer sinkpad.
|
||||||
|
|
||||||
|
|
||||||
Deciding the scheduling mode
|
Deciding the scheduling mode
|
||||||
----------------------------
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
When tha pad is activated, the _activate() function is called. The pad can then
|
When tha pad is activated, the _activate() function is called. The pad can then
|
||||||
choose to activate itself in push or pull mode depending on upstream
|
choose to activate itself in push or pull mode depending on upstream
|
||||||
|
|
@ -41,162 +41,163 @@ The GStreamer core will by default activate pads in push mode when there is no
|
||||||
activate function for the pad.
|
activate function for the pad.
|
||||||
|
|
||||||
The chain function
|
The chain function
|
||||||
------------------
|
~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
The chain function will be called when a upstream element perform a _push() on the pad.
|
The chain function will be called when a upstream element perform a _push() on the pad.
|
||||||
The upstream element can be another chain based element or a pushing source.
|
The upstream element can be another chain based element or a pushing source.
|
||||||
|
|
||||||
The getrange function
|
The getrange function
|
||||||
---------------------
|
~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
The getrange function is called when a peer pad perform a _pull_range() on the pad. This
|
The getrange function is called when a peer pad perform a _pull_range() on the pad. This
|
||||||
downstream pad can be a pulling element or another _pull_range() based element.
|
downstream pad can be a pulling element or another _pull_range() based element.
|
||||||
|
|
||||||
Plug-in techniques
|
Plug-in techniques
|
||||||
------------------
|
~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
Multi-sink elements
|
Multi-sink elements
|
||||||
-------------------
|
^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
Elements with multiple sinks can either expose a loop function on each of the pads to
|
||||||
|
actively pull_range data or they can expose a chain function on each pad.
|
||||||
|
|
||||||
|
Implementing a chain function is usually easy and allows for all possible scheduling
|
||||||
|
methods.
|
||||||
|
|
||||||
|
Pad select
|
||||||
|
----------
|
||||||
|
|
||||||
|
If the chain based sink wants to wait for one of the pads to receive a buffer, just
|
||||||
|
implement the action to perform in the chain function. Be aware that the action could
|
||||||
|
be performed in different threads and possibly simultaneously so grab the STREAM_LOCK.
|
||||||
|
|
||||||
|
Collect pads
|
||||||
|
------------
|
||||||
|
|
||||||
|
If the chain based sink pads all require one buffer before the element can operate on
|
||||||
|
the data, collect all the buffers in the chain function and perform the action when
|
||||||
|
all chainpads received the buffer.
|
||||||
|
|
||||||
|
In this case you probably also don't want to accept more data on a pad that has a buffer
|
||||||
|
queued. This can easily be done with the following code snippet:
|
||||||
|
|
||||||
|
static GstFlowReturn _chain (GstPad *pad, GstBuffer *buffer)
|
||||||
|
{
|
||||||
|
LOCK (mylock);
|
||||||
|
while (pad->store != NULL) {
|
||||||
|
WAIT (mycond, mylock);
|
||||||
|
}
|
||||||
|
pad->store = buffer;
|
||||||
|
SIGNAL (mycond);
|
||||||
|
UNLOCK (mylock);
|
||||||
|
|
||||||
|
return GST_FLOW_OK;
|
||||||
|
}
|
||||||
|
|
||||||
|
static void _pull (GstPad *pad, GstBuffer **buffer)
|
||||||
|
{
|
||||||
|
LOCK (mylock);
|
||||||
|
while (pad->store == NULL) {
|
||||||
|
WAIT (mycond, mylock);
|
||||||
|
}
|
||||||
|
**buffer = pad->store;
|
||||||
|
pad->store = NULL;
|
||||||
|
SIGNAL (mycond);
|
||||||
|
UNLOCK (mylock);
|
||||||
|
}
|
||||||
|
|
||||||
Elements with multiple sinks can either expose a loop function on each of the pads to
|
|
||||||
actively pull_range data or they can expose a chain function on each pad.
|
|
||||||
|
|
||||||
Implementing a chain function is usually easy and allows for all possible scheduling
|
|
||||||
methods.
|
|
||||||
|
|
||||||
Pad select
|
|
||||||
----------
|
|
||||||
|
|
||||||
If the chain based sink wants to wait for one of the pads to receive a buffer, just
|
|
||||||
implement the action to perform in the chain function. Be aware that the action could
|
|
||||||
be performed in different threads and possibly simultaneously so grab the STREAM_LOCK.
|
|
||||||
|
|
||||||
Collect pads
|
|
||||||
------------
|
|
||||||
|
|
||||||
If the chain based sink pads all require one buffer before the element can operate on
|
|
||||||
the data, collect all the buffers in the chain function and perform the action when
|
|
||||||
all chainpads received the buffer.
|
|
||||||
|
|
||||||
In this case you probably also don't want to accept more data on a pad that has a buffer
|
|
||||||
queued. This can easily be done with the following code snippet:
|
|
||||||
|
|
||||||
static GstFlowReturn _chain (GstPad *pad, GstBuffer *buffer)
|
|
||||||
{
|
|
||||||
LOCK (mylock);
|
|
||||||
while (pad->store != NULL) {
|
|
||||||
WAIT (mycond, mylock);
|
|
||||||
}
|
|
||||||
pad->store = buffer;
|
|
||||||
SIGNAL (mycond);
|
|
||||||
UNLOCK (mylock);
|
|
||||||
|
|
||||||
return GST_FLOW_OK;
|
|
||||||
}
|
|
||||||
|
|
||||||
static void _pull (GstPad *pad, GstBuffer **buffer)
|
|
||||||
{
|
|
||||||
LOCK (mylock);
|
|
||||||
while (pad->store == NULL) {
|
|
||||||
WAIT (mycond, mylock);
|
|
||||||
}
|
|
||||||
**buffer = pad->store;
|
|
||||||
pad->store = NULL;
|
|
||||||
SIGNAL (mycond);
|
|
||||||
UNLOCK (mylock);
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
Cases
|
Cases
|
||||||
-----
|
~~~~~
|
||||||
Inside the braces below the pads is stated what function the
|
|
||||||
pad support:
|
|
||||||
|
|
||||||
l: exposes a loop function, so it can act as a pushing source.
|
Inside the braces below the pads is stated what function the
|
||||||
g: exposes a getrange function
|
pad support:
|
||||||
c: exposes a chain function
|
|
||||||
|
|
||||||
following scheduling decisions are made based on the scheduling
|
l: exposes a loop function, so it can act as a pushing source.
|
||||||
methods exposed by the pads:
|
g: exposes a getrange function
|
||||||
|
c: exposes a chain function
|
||||||
|
|
||||||
(g) - (l): sinkpad will pull data from src
|
following scheduling decisions are made based on the scheduling
|
||||||
(l) - (c): srcpad actively pushes data to sinkpad
|
methods exposed by the pads:
|
||||||
() - (c): srcpad will push data to sinkpad.
|
|
||||||
|
|
||||||
() - () : not schedulable.
|
(g) - (l): sinkpad will pull data from src
|
||||||
() - (l): not schedulable.
|
(l) - (c): srcpad actively pushes data to sinkpad
|
||||||
(g) - () : not schedulable.
|
() - (c): srcpad will push data to sinkpad.
|
||||||
(g) - (c): not schedulable.
|
|
||||||
(l) - () : not schedulable.
|
|
||||||
(l) - (l): not schedulable
|
|
||||||
|
|
||||||
() - (g): impossible
|
() - () : not schedulable.
|
||||||
(g) - (g): impossible.
|
() - (l): not schedulable.
|
||||||
(l) - (g): impossible
|
(g) - () : not schedulable.
|
||||||
(c) - () : impossible
|
(g) - (c): not schedulable.
|
||||||
(c) - (g): impossible
|
(l) - () : not schedulable.
|
||||||
(c) - (l): impossible
|
(l) - (l): not schedulable
|
||||||
(c) - (c): impossible
|
|
||||||
|
|
||||||
+---------+ +------------+ +-----------+
|
() - (g): impossible
|
||||||
| filesrc | | mp3decoder | | audiosink |
|
(g) - (g): impossible.
|
||||||
| src--sink src--sink |
|
(l) - (g): impossible
|
||||||
+---------+ +------------+ +-----------+
|
(c) - () : impossible
|
||||||
(l-g) (c) () (c)
|
(c) - (g): impossible
|
||||||
|
(c) - (l): impossible
|
||||||
|
(c) - (c): impossible
|
||||||
|
|
||||||
When activating the pads:
|
+---------+ +------------+ +-----------+
|
||||||
|
| filesrc | | mp3decoder | | audiosink |
|
||||||
|
| src--sink src--sink |
|
||||||
|
+---------+ +------------+ +-----------+
|
||||||
|
(l-g) (c) () (c)
|
||||||
|
|
||||||
* audiosink has a chain function and the peer pad has no
|
When activating the pads:
|
||||||
loop function, no scheduling is done.
|
|
||||||
* mp3decoder and filesrc expose an (l) - (c) connection,
|
|
||||||
a thread is created to call the srcpad loop function.
|
|
||||||
|
|
||||||
+---------+ +------------+ +----------+
|
* audiosink has a chain function and the peer pad has no
|
||||||
| filesrc | | avidemuxer | | fakesink |
|
loop function, no scheduling is done.
|
||||||
| src--sink src--sink |
|
* mp3decoder and filesrc expose an (l) - (c) connection,
|
||||||
+---------+ +------------+ +----------+
|
a thread is created to call the srcpad loop function.
|
||||||
(l-g) (l) () (c)
|
|
||||||
|
|
||||||
* fakesink has a chain function and the peer pad has no
|
|
||||||
loop function, no scheduling is done.
|
|
||||||
* avidemuxer and filesrc expose an (g) - (l) connection,
|
|
||||||
a thread is created to call the sinkpad loop function.
|
|
||||||
|
|
||||||
+---------+ +----------+ +------------+ +----------+
|
+---------+ +------------+ +----------+
|
||||||
| filesrc | | identity | | avidemuxer | | fakesink |
|
| filesrc | | avidemuxer | | fakesink |
|
||||||
| src--sink src--sink src--sink |
|
| src--sink src--sink |
|
||||||
+---------+ +----------+ +------------+ +----------+
|
+---------+ +------------+ +----------+
|
||||||
(l-g) (c) () (l) () (c)
|
(l-g) (l) () (c)
|
||||||
|
|
||||||
|
* fakesink has a chain function and the peer pad has no
|
||||||
|
loop function, no scheduling is done.
|
||||||
|
* avidemuxer and filesrc expose an (g) - (l) connection,
|
||||||
|
a thread is created to call the sinkpad loop function.
|
||||||
|
|
||||||
* fakesink has a chain function and the peer pad has no
|
+---------+ +----------+ +------------+ +----------+
|
||||||
loop function, no scheduling is done.
|
| filesrc | | identity | | avidemuxer | | fakesink |
|
||||||
* avidemuxer and identity expose no schedulable connection so
|
| src--sink src--sink src--sink |
|
||||||
this pipeline is not schedulable.
|
+---------+ +----------+ +------------+ +----------+
|
||||||
|
(l-g) (c) () (l) () (c)
|
||||||
|
|
||||||
+---------+ +----------+ +------------+ +----------+
|
* fakesink has a chain function and the peer pad has no
|
||||||
| filesrc | | identity | | avidemuxer | | fakesink |
|
loop function, no scheduling is done.
|
||||||
| src--sink src--sink src--sink |
|
* avidemuxer and identity expose no schedulable connection so
|
||||||
+---------+ +----------+ +------------+ +----------+
|
this pipeline is not schedulable.
|
||||||
(l-g) (c-l) (g) (l) () (c)
|
|
||||||
|
|
||||||
* fakesink has a chain function and the peer pad has no
|
+---------+ +----------+ +------------+ +----------+
|
||||||
loop function, no scheduling is done.
|
| filesrc | | identity | | avidemuxer | | fakesink |
|
||||||
* avidemuxer and identity expose an (g) - (l) connection,
|
| src--sink src--sink src--sink |
|
||||||
a thread is created to call the sinkpad loop function.
|
+---------+ +----------+ +------------+ +----------+
|
||||||
* identity knows the srcpad is getrange based and uses the
|
(l-g) (c-l) (g) (l) () (c)
|
||||||
thread from avidemux to getrange data from filesrc.
|
|
||||||
|
|
||||||
+---------+ +----------+ +------------+ +----------+
|
* fakesink has a chain function and the peer pad has no
|
||||||
| filesrc | | identity | | oggdemuxer | | fakesink |
|
loop function, no scheduling is done.
|
||||||
| src--sink src--sink src--sink |
|
* avidemuxer and identity expose an (g) - (l) connection,
|
||||||
+---------+ +----------+ +------------+ +----------+
|
a thread is created to call the sinkpad loop function.
|
||||||
(l-g) (c) () (l-c) () (c)
|
* identity knows the srcpad is getrange based and uses the
|
||||||
|
thread from avidemux to getrange data from filesrc.
|
||||||
|
|
||||||
* fakesink has a chain function and the peer pad has no
|
+---------+ +----------+ +------------+ +----------+
|
||||||
loop function, no scheduling is done.
|
| filesrc | | identity | | oggdemuxer | | fakesink |
|
||||||
* oggdemuxer and identity expose an () - (l-c) connection,
|
| src--sink src--sink src--sink |
|
||||||
oggdemux has to operate in chain mode.
|
+---------+ +----------+ +------------+ +----------+
|
||||||
* identity chan only work chain based and so filesrc creates
|
(l-g) (c) () (l-c) () (c)
|
||||||
a thread to push data to identity.
|
|
||||||
|
* fakesink has a chain function and the peer pad has no
|
||||||
|
loop function, no scheduling is done.
|
||||||
|
* oggdemuxer and identity expose an () - (l-c) connection,
|
||||||
|
oggdemux has to operate in chain mode.
|
||||||
|
* identity chan only work chain based and so filesrc creates
|
||||||
|
a thread to push data to identity.
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -58,7 +58,7 @@ more important than accurately producing all frames.
|
||||||
|
|
||||||
|
|
||||||
Seeking in push based elements
|
Seeking in push based elements
|
||||||
------------------------------
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -66,19 +66,21 @@ Seeking in push based elements
|
||||||
|
|
||||||
|
|
||||||
Generating seeking events
|
Generating seeking events
|
||||||
-------------------------
|
~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
A seek event is created with gst_event_new_seek ().
|
A seek event is created with gst_event_new_seek ().
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
Seeking variants
|
||||||
|
~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
The different kinds of seeking methods and their internal workings are
|
The different kinds of seeking methods and their internal workings are
|
||||||
described below.
|
described below.
|
||||||
|
|
||||||
|
|
||||||
FLUSH seeking
|
FLUSH seeking
|
||||||
-------------
|
^^^^^^^^^^^^^
|
||||||
|
|
||||||
This is the most common way of performing a seek in a playback application.
|
This is the most common way of performing a seek in a playback application.
|
||||||
The application issues a seek on the pipeline and the new media is immediatly
|
The application issues a seek on the pipeline and the new media is immediatly
|
||||||
|
|
@ -86,7 +88,7 @@ played after the seek calls returns.
|
||||||
|
|
||||||
|
|
||||||
seeking without FLUSH
|
seeking without FLUSH
|
||||||
---------------------
|
^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
This seek type is typically performed after issuing segment seeks to finish
|
This seek type is typically performed after issuing segment seeks to finish
|
||||||
the playback of the pipeline.
|
the playback of the pipeline.
|
||||||
|
|
@ -97,13 +99,13 @@ sinks.
|
||||||
|
|
||||||
|
|
||||||
segment seeking with FLUSH
|
segment seeking with FLUSH
|
||||||
--------------------------
|
^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
This seek is typically performed when starting seamless looping.
|
This seek is typically performed when starting seamless looping.
|
||||||
|
|
||||||
|
|
||||||
segment seeking without FLUSH
|
segment seeking without FLUSH
|
||||||
-----------------------------
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
This seek is typically performed when continuing seamless looping.
|
This seek is typically performed when continuing seamless looping.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -29,7 +29,7 @@ outside of the segment and that might therefore be discarded or clipped.
|
||||||
|
|
||||||
|
|
||||||
Use case: FLUSHING seek
|
Use case: FLUSHING seek
|
||||||
-----------------------
|
~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
ex.
|
ex.
|
||||||
|
|
||||||
|
|
@ -95,13 +95,13 @@ Use case: FLUSHING seek
|
||||||
|
|
||||||
|
|
||||||
Use case: live stream
|
Use case: live stream
|
||||||
---------------------
|
~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
Use case: segment looping
|
Use case: segment looping
|
||||||
-------------------------
|
~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
Consider the case of a wav file with raw audio.
|
Consider the case of a wav file with raw audio.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -2,7 +2,8 @@ DRAFT Sparse Streams
|
||||||
--------------------
|
--------------------
|
||||||
|
|
||||||
Introduction
|
Introduction
|
||||||
------------
|
~~~~~~~~~~~~
|
||||||
|
|
||||||
In 0.8, there was some support for Sparse Streams through the use of
|
In 0.8, there was some support for Sparse Streams through the use of
|
||||||
FILLER events. These were used to mark gaps between buffers so that downstream
|
FILLER events. These were used to mark gaps between buffers so that downstream
|
||||||
elements could know not to expect any more data for that gap.
|
elements could know not to expect any more data for that gap.
|
||||||
|
|
@ -11,7 +12,8 @@ In 0.10, segment information conveyed through NEWSEGMENT events can be used
|
||||||
for the same purpose.
|
for the same purpose.
|
||||||
|
|
||||||
Use cases
|
Use cases
|
||||||
---------
|
~~~~~~~~~
|
||||||
|
|
||||||
1) Sub-title streams
|
1) Sub-title streams
|
||||||
Sub-title information from muxed formats such as Matroska or MPEG consist of
|
Sub-title information from muxed formats such as Matroska or MPEG consist of
|
||||||
irregular buffers spaced far apart compared to the other streams
|
irregular buffers spaced far apart compared to the other streams
|
||||||
|
|
@ -37,7 +39,8 @@ Use cases
|
||||||
application that uses noise-gating (to save bandwith).
|
application that uses noise-gating (to save bandwith).
|
||||||
|
|
||||||
Details
|
Details
|
||||||
-------
|
~~~~~~~
|
||||||
|
|
||||||
1) Sub-title streams
|
1) Sub-title streams
|
||||||
The main requirement here is to avoid stalling the pipeline between sub-title
|
The main requirement here is to avoid stalling the pipeline between sub-title
|
||||||
packets, and is effectively updating the minimum-timestamp for that stream.
|
packets, and is effectively updating the minimum-timestamp for that stream.
|
||||||
|
|
|
||||||
|
|
@ -6,7 +6,7 @@ collection has many potential pitfalls as far as the pointers go. Therefore,
|
||||||
some standards must be adhered to as far as who owns what.
|
some standards must be adhered to as far as who owns what.
|
||||||
|
|
||||||
Strings
|
Strings
|
||||||
-------
|
~~~~~~~
|
||||||
|
|
||||||
Arguments passed into a function are owned by the caller, and the function
|
Arguments passed into a function are owned by the caller, and the function
|
||||||
will make a copy of the string for its own internal use. The string should
|
will make a copy of the string for its own internal use. The string should
|
||||||
|
|
@ -21,7 +21,7 @@ original and should be freed after usage by the caller.
|
||||||
|
|
||||||
|
|
||||||
Objects
|
Objects
|
||||||
-------
|
~~~~~~~
|
||||||
|
|
||||||
Objects passed into a function are owned by the caller, any additional
|
Objects passed into a function are owned by the caller, any additional
|
||||||
reference held to the object after leaving the function should increase the
|
reference held to the object after leaving the function should increase the
|
||||||
|
|
@ -40,7 +40,7 @@ called should _free() or _unref() the object after usage.
|
||||||
|
|
||||||
|
|
||||||
Iterators
|
Iterators
|
||||||
---------
|
~~~~~~~~~
|
||||||
|
|
||||||
When retrieving multiple objects from an object an iterator should be used.
|
When retrieving multiple objects from an object an iterator should be used.
|
||||||
The iterator allows you to access the objects one after another while making
|
The iterator allows you to access the objects one after another while making
|
||||||
|
|
|
||||||
|
|
@ -1,5 +1,5 @@
|
||||||
States
|
States
|
||||||
======
|
------
|
||||||
|
|
||||||
Both elements and pads can be in different states. The states of the pads are
|
Both elements and pads can be in different states. The states of the pads are
|
||||||
linked to the state of the element so the design of the states is mainly
|
linked to the state of the element so the design of the states is mainly
|
||||||
|
|
@ -10,7 +10,7 @@ is initially instantiated, it is in the NULL state.
|
||||||
|
|
||||||
|
|
||||||
State definitions
|
State definitions
|
||||||
-----------------
|
~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
- NULL: This is the initial state of an element.
|
- NULL: This is the initial state of an element.
|
||||||
- READY: The element should be prepared to go to PAUSED.
|
- READY: The element should be prepared to go to PAUSED.
|
||||||
|
|
@ -24,7 +24,7 @@ a downwards state change.
|
||||||
|
|
||||||
|
|
||||||
State transitions
|
State transitions
|
||||||
-----------------
|
~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
the following state changes are possible:
|
the following state changes are possible:
|
||||||
|
|
||||||
|
|
@ -89,7 +89,7 @@ the following state changes are possible:
|
||||||
|
|
||||||
|
|
||||||
State variables
|
State variables
|
||||||
---------------
|
~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
An element has 4 state variables that are protected with the object LOCK:
|
An element has 4 state variables that are protected with the object LOCK:
|
||||||
|
|
||||||
|
|
@ -111,7 +111,7 @@ _set_state(), called the STATE_LOCK.
|
||||||
|
|
||||||
|
|
||||||
Setting state on elements
|
Setting state on elements
|
||||||
-------------------------
|
~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
The state of an element can be changed with _element_set_state(). When chaning
|
The state of an element can be changed with _element_set_state(). When chaning
|
||||||
the state of an element all intermediate states will also be set on the element
|
the state of an element all intermediate states will also be set on the element
|
||||||
|
|
@ -147,7 +147,7 @@ is immediatly returned to the caller.
|
||||||
|
|
||||||
|
|
||||||
Getting state of elements
|
Getting state of elements
|
||||||
-------------------------
|
~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
The _get_state() function takes 3 arguments, two pointers that will hold the
|
The _get_state() function takes 3 arguments, two pointers that will hold the
|
||||||
current and pending state and one GstClockTime that holds a timeout value. The
|
current and pending state and one GstClockTime that holds a timeout value. The
|
||||||
|
|
@ -184,7 +184,7 @@ function returns a GstElementStateReturn.
|
||||||
|
|
||||||
|
|
||||||
States in GstBin
|
States in GstBin
|
||||||
----------------
|
~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
A GstBin manages the state of its children. It does this by propagating the state
|
A GstBin manages the state of its children. It does this by propagating the state
|
||||||
changes performed on it to all of its children. The _set_state() function on a
|
changes performed on it to all of its children. The _set_state() function on a
|
||||||
|
|
@ -221,15 +221,15 @@ current state fields when it receives state messages from the children.
|
||||||
|
|
||||||
|
|
||||||
Implementing states in elements
|
Implementing states in elements
|
||||||
-------------------------------
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
READY
|
READY
|
||||||
-----
|
^^^^^
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
upward state change
|
upward state change
|
||||||
-------------------
|
~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
Upward state changes always return ASYNC either if the STATE_PENDING is
|
Upward state changes always return ASYNC either if the STATE_PENDING is
|
||||||
reached or not.
|
reached or not.
|
||||||
|
|
@ -262,7 +262,7 @@ Bin:
|
||||||
to STATE_PENDING
|
to STATE_PENDING
|
||||||
|
|
||||||
downward state change
|
downward state change
|
||||||
----------------------
|
~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
Downward state changes only return ASYNC if the final state is ASYNC.
|
Downward state changes only return ASYNC if the final state is ASYNC.
|
||||||
This is to make sure that it's not needed to wait for an element to
|
This is to make sure that it's not needed to wait for an element to
|
||||||
|
|
@ -296,7 +296,7 @@ Bin:
|
||||||
|
|
||||||
|
|
||||||
Locking overview (element)
|
Locking overview (element)
|
||||||
--------------------------
|
~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
* Element commiting SUCCESS
|
* Element commiting SUCCESS
|
||||||
|
|
||||||
|
|
@ -376,8 +376,8 @@ Locking overview (element)
|
||||||
| ...
|
| ...
|
||||||
STREAM_UNLOCK
|
STREAM_UNLOCK
|
||||||
|
|
||||||
*********************************************
|
Remarks
|
||||||
*********************************************
|
~~~~~~~
|
||||||
|
|
||||||
set_state cannot be called from multiple threads at the same time. The STATE_LOCK
|
set_state cannot be called from multiple threads at the same time. The STATE_LOCK
|
||||||
prevents this.
|
prevents this.
|
||||||
|
|
|
||||||
|
|
@ -26,7 +26,7 @@ We allow for the following scenarios:
|
||||||
|
|
||||||
|
|
||||||
Use cases
|
Use cases
|
||||||
---------
|
~~~~~~~~~
|
||||||
|
|
||||||
* boost the priority of the udp receiver streaming thread
|
* boost the priority of the udp receiver streaming thread
|
||||||
|
|
||||||
|
|
@ -56,7 +56,7 @@ Use cases
|
||||||
|
|
||||||
|
|
||||||
Messages
|
Messages
|
||||||
--------
|
~~~~~~~~
|
||||||
|
|
||||||
The existing STREAM_STATUS message will be further defined and implemented in
|
The existing STREAM_STATUS message will be further defined and implemented in
|
||||||
(selected) elements. The following fields will be contained in the message:
|
(selected) elements. The following fields will be contained in the message:
|
||||||
|
|
@ -101,7 +101,7 @@ Messages
|
||||||
|
|
||||||
|
|
||||||
Events
|
Events
|
||||||
------
|
~~~~~~
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -6,7 +6,7 @@ Streams
|
||||||
|
|
||||||
|
|
||||||
Stream objects
|
Stream objects
|
||||||
--------------
|
~~~~~~~~~~~~~~
|
||||||
|
|
||||||
The following objects are to be expected in the streaming thread:
|
The following objects are to be expected in the streaming thread:
|
||||||
|
|
||||||
|
|
@ -21,7 +21,7 @@ and live sources.
|
||||||
|
|
||||||
|
|
||||||
Typical stream
|
Typical stream
|
||||||
--------------
|
~~~~~~~~~~~~~~
|
||||||
|
|
||||||
A typical stream starts with a newsegment event that marks the
|
A typical stream starts with a newsegment event that marks the
|
||||||
buffer timestamp range. After that buffers are send one after the
|
buffer timestamp range. After that buffers are send one after the
|
||||||
|
|
|
||||||
|
|
@ -12,7 +12,7 @@ Synchronisation in a GstPipeline is achieved using the following 3 components:
|
||||||
|
|
||||||
|
|
||||||
A GstClock
|
A GstClock
|
||||||
----------
|
~~~~~~~~~~
|
||||||
|
|
||||||
This object provides a counter that represents the current time in nanoseconds.
|
This object provides a counter that represents the current time in nanoseconds.
|
||||||
This value is called the absolute_time.
|
This value is called the absolute_time.
|
||||||
|
|
@ -33,7 +33,7 @@ A GstClock always counts time upwards and does not necessarily start at 0.
|
||||||
|
|
||||||
|
|
||||||
Running time
|
Running time
|
||||||
------------
|
~~~~~~~~~~~~
|
||||||
|
|
||||||
After a pipeline selected a clock it will maintain the running_time based on the
|
After a pipeline selected a clock it will maintain the running_time based on the
|
||||||
selected clock. This running_time represents the total time spent in the PLAYING
|
selected clock. This running_time represents the total time spent in the PLAYING
|
||||||
|
|
@ -66,7 +66,7 @@ This value is monotonically increasing at the rate of the clock.
|
||||||
|
|
||||||
|
|
||||||
Timestamps
|
Timestamps
|
||||||
----------
|
~~~~~~~~~~
|
||||||
|
|
||||||
The GstBuffer timestamps and the preceeding NEW_SEGMENT event (See
|
The GstBuffer timestamps and the preceeding NEW_SEGMENT event (See
|
||||||
part-streams.txt) define a transformation of the buffer timestamps to
|
part-streams.txt) define a transformation of the buffer timestamps to
|
||||||
|
|
@ -112,7 +112,7 @@ NS.stop and NS.accum == 0).
|
||||||
|
|
||||||
|
|
||||||
Synchronisation
|
Synchronisation
|
||||||
---------------
|
~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
As we have seen, we can get a running_time:
|
As we have seen, we can get a running_time:
|
||||||
|
|
||||||
|
|
@ -161,7 +161,7 @@ synchronized buffers have the same timestamps.
|
||||||
|
|
||||||
|
|
||||||
Stream time
|
Stream time
|
||||||
-----------
|
~~~~~~~~~~~
|
||||||
|
|
||||||
The stream time is also known as the position in the stream and is a value
|
The stream time is also known as the position in the stream and is a value
|
||||||
between 0 and the total duration of the media file.
|
between 0 and the total duration of the media file.
|
||||||
|
|
|
||||||
|
|
@ -1,5 +1,5 @@
|
||||||
Trickmodes
|
Trickmodes
|
||||||
==========
|
----------
|
||||||
|
|
||||||
GStreamer provides API for performing various trickmode playback. This includes:
|
GStreamer provides API for performing various trickmode playback. This includes:
|
||||||
|
|
||||||
|
|
@ -18,7 +18,7 @@ Seeking can both be done in a playback pipeline and a transcoding pipeline.
|
||||||
|
|
||||||
|
|
||||||
General seeking overview
|
General seeking overview
|
||||||
------------------------
|
~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
Consider a typical playback pipeline:
|
Consider a typical playback pipeline:
|
||||||
|
|
||||||
|
|
@ -35,56 +35,60 @@ position 0 and stopping at the total duration of the file.
|
||||||
|
|
||||||
When performing a seek, the following steps have to be taken by the application:
|
When performing a seek, the following steps have to be taken by the application:
|
||||||
|
|
||||||
1) Create a seek event:
|
Create a seek event
|
||||||
|
^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
The seek event contains:
|
The seek event contains:
|
||||||
|
|
||||||
- various flags describing:
|
- various flags describing:
|
||||||
- where to seek to (KEY_UNIT)
|
- where to seek to (KEY_UNIT)
|
||||||
- how accurate the seek should be (ACCURATE)
|
- how accurate the seek should be (ACCURATE)
|
||||||
- how to perform the seek (FLUSH)
|
- how to perform the seek (FLUSH)
|
||||||
- what to do when the stop position is reached (SEGMENT).
|
- what to do when the stop position is reached (SEGMENT).
|
||||||
- extra playback options (SKIP)
|
- extra playback options (SKIP)
|
||||||
- a format to seek in, this can be time, bytes, units (frames, samples), ...
|
- a format to seek in, this can be time, bytes, units (frames, samples), ...
|
||||||
- a playback rate, 1.0 is normal playback speed, positive values bigger than 1.0
|
- a playback rate, 1.0 is normal playback speed, positive values bigger than 1.0
|
||||||
mean fast playback. negative values mean reverse playback. A playback speed of
|
mean fast playback. negative values mean reverse playback. A playback speed of
|
||||||
0.0 is not allowed (but is equivalent to PAUSING the pipeline).
|
0.0 is not allowed (but is equivalent to PAUSING the pipeline).
|
||||||
- a start position, this value has to be between 0 and the total duration of the
|
- a start position, this value has to be between 0 and the total duration of the
|
||||||
file. It can also be relative to the previously configured start value.
|
file. It can also be relative to the previously configured start value.
|
||||||
- a stop position, this value has to be between 0 and the total duration. It can
|
- a stop position, this value has to be between 0 and the total duration. It can
|
||||||
also be relative to the previously configured stop value.
|
also be relative to the previously configured stop value.
|
||||||
|
|
||||||
See also gst_event_new_seek().
|
See also gst_event_new_seek().
|
||||||
|
|
||||||
|
|
||||||
2) Send the seek event to the pipeline with gst_element_send_event()
|
Send the seek event
|
||||||
|
^^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
By default the pipeline will send the event to all sink elements.
|
Send the new seek event to the pipeline with gst_element_send_event().
|
||||||
By default an element will forward the event upstream on all sinkpads.
|
|
||||||
Elements can modify the format of the seek event. The most common format is
|
|
||||||
GST_FORMAT_TIME.
|
|
||||||
|
|
||||||
One element will actually perform the seek, this is usually the demuxer or
|
By default the pipeline will send the event to all sink elements.
|
||||||
source element. For more information on how to perform the different seek
|
By default an element will forward the event upstream on all sinkpads.
|
||||||
types see part-seeking.txt.
|
Elements can modify the format of the seek event. The most common format is
|
||||||
|
GST_FORMAT_TIME.
|
||||||
|
|
||||||
For client side trickmode a NEW_SEGMENT event will be sent downstream with
|
One element will actually perform the seek, this is usually the demuxer or
|
||||||
the new rate and start/stop positions. All elements prepare themselves to
|
source element. For more information on how to perform the different seek
|
||||||
handle the rate (see below). The applied rate of the NEW_SEGMENT event will
|
types see part-seeking.txt.
|
||||||
be set to 1.0 to indicate that no rate adjustment has been done.
|
|
||||||
|
|
||||||
for server side trick mode a NEW_SEGMENT event is sent downstream with a
|
For client side trickmode a NEW_SEGMENT event will be sent downstream with
|
||||||
rate of 1.0 and the start/stop positions. The elements will configure themselves
|
the new rate and start/stop positions. All elements prepare themselves to
|
||||||
for normal playback speed since the server will perform the rate conversions.
|
handle the rate (see below). The applied rate of the NEW_SEGMENT event will
|
||||||
The applied rate will be set to the rate that will be applied by the server. This
|
be set to 1.0 to indicate that no rate adjustment has been done.
|
||||||
is done to insure that the position reporting performed in the sink is aware
|
|
||||||
of the trick mode.
|
|
||||||
|
|
||||||
When the seek succeeds, the _send_event() function will return TRUE.
|
for server side trick mode a NEW_SEGMENT event is sent downstream with a
|
||||||
|
rate of 1.0 and the start/stop positions. The elements will configure themselves
|
||||||
|
for normal playback speed since the server will perform the rate conversions.
|
||||||
|
The applied rate will be set to the rate that will be applied by the server. This
|
||||||
|
is done to insure that the position reporting performed in the sink is aware
|
||||||
|
of the trick mode.
|
||||||
|
|
||||||
|
When the seek succeeds, the _send_event() function will return TRUE.
|
||||||
|
|
||||||
|
|
||||||
Server side trickmode
|
Server side trickmode
|
||||||
---------------------
|
~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
The source element operates in push mode. It can reopen a server connection requesting
|
The source element operates in push mode. It can reopen a server connection requesting
|
||||||
a new byte or time position and a new playback speed. The capabilities can be queried
|
a new byte or time position and a new playback speed. The capabilities can be queried
|
||||||
|
|
@ -131,7 +135,7 @@ playback speed or direction.
|
||||||
|
|
||||||
|
|
||||||
client side forward trickmodes
|
client side forward trickmodes
|
||||||
------------------------------
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
The seek happens as stated above. a NEW_SEGMENT event is sent downstream with a rate
|
The seek happens as stated above. a NEW_SEGMENT event is sent downstream with a rate
|
||||||
different from 1.0. Plugins receiving the NEW_SEGMENT can decide to perform the
|
different from 1.0. Plugins receiving the NEW_SEGMENT can decide to perform the
|
||||||
|
|
@ -154,7 +158,7 @@ case, the demuxer would scale the timestamps and would set an applied rate of S.
|
||||||
|
|
||||||
|
|
||||||
client side backwards trickmode
|
client side backwards trickmode
|
||||||
-------------------------------
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
For backwards playback the following rules apply:
|
For backwards playback the following rules apply:
|
||||||
|
|
||||||
|
|
@ -202,7 +206,8 @@ For plugins the following rules apply:
|
||||||
In SKIP mode, the same algorithm as for forward SKIP mode can be used.
|
In SKIP mode, the same algorithm as for forward SKIP mode can be used.
|
||||||
|
|
||||||
|
|
||||||
Notes:
|
Notes
|
||||||
|
~~~~~
|
||||||
|
|
||||||
- The clock/running_time keeps running forward.
|
- The clock/running_time keeps running forward.
|
||||||
- backwards playback potentially uses a lot of memory as frames and undecoded
|
- backwards playback potentially uses a lot of memory as frames and undecoded
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue