mirror of
https://gitlab.freedesktop.org/gstreamer/gstreamer.git
synced 2024-12-27 18:50:48 +00:00
321 lines
10 KiB
Text
321 lines
10 KiB
Text
purpose
|
|
-------
|
|
|
|
GStreamer has a very flexible mechanism to describe media types using
|
|
mime types and key/value pairs (GstCaps). The definition of media types
|
|
is entirely done by the plugins which can set the media type to one or
|
|
more of the plugins GstPads.
|
|
|
|
Pads with the same mime type and 'compatible' properties are allowed to
|
|
connect. It is possible that a pad can accept or produce many different
|
|
media types.
|
|
|
|
The purpose of caps negotiation is to provide a framework for plugins so
|
|
that they can agree on a common media type for their pads.
|
|
|
|
|
|
Capabilities (GstCaps)
|
|
----------------------
|
|
|
|
The core component in the caps negotiation system are GstCaps. They consist
|
|
of:
|
|
|
|
- a name (ex. "my_audio_capabilities")
|
|
- a mime-type (ex. audio/raw, audio/mp3, ...)
|
|
- a list of key/value pairs (ex. channels=2, rate=44100, ...)
|
|
|
|
The list of key/value pairs is maintained by the GstProps object.
|
|
|
|
The GstProps object maintains a GList of GstPropsEntries. An entry has
|
|
a key, which is always a string constant (internally converted to a GQuark)
|
|
and a value, which can be one of the following:
|
|
|
|
- an integer constant (ex. 5)
|
|
- a float constant (ex. 1.0)
|
|
- a string constant (ex. "int")
|
|
- a boolean constant (ex. FALSE)
|
|
- a fourcc constant (ex. I420)
|
|
* fourcc codes are usually used to describe a video format
|
|
|
|
In addition to these constant values, the following variable values are
|
|
supported too:
|
|
|
|
- an integer range (ex. 0-200)
|
|
- a float range (ex. 1.0-3.0)
|
|
- a list of values (ex. 1, 2, 5).
|
|
* A List cannot contain another list and the
|
|
entries in the list should all have the
|
|
same type (int, float, string, fourcc). It is
|
|
allowed to mix integers/floats and
|
|
integer/float ranges in a list.
|
|
|
|
A capability is usually described as follows:
|
|
|
|
GST_CAPS_NEW (
|
|
capability name ---> "my_audio_capabilities",
|
|
mime-type ---------> "audio/raw",
|
|
( "format", GST_PROPS_STRING ("int"),
|
|
GstProps ---------> ( "channels", GST_PROPS_INT_RANGE (1, 2),
|
|
(list of entries) ( "rate", GST_PROPS_INT (44100)
|
|
)
|
|
|
|
(-----------) (--------------------------)
|
|
entry key entry value
|
|
|
|
Two capabilities can be chained together to form a larger capability:
|
|
|
|
( GST_CAPS_NEW (
|
|
( "my_mp3_capability",
|
|
( "audio/mp3",
|
|
one capability ----> ( NULL
|
|
created by chaining ( ),
|
|
two capabilities. ( GST_CAPS_NEW (
|
|
( "my_vorbis_capability",
|
|
( "audio/vorbis",
|
|
( NULL
|
|
( )
|
|
|
|
Capabilities always work as constraints, this means that they constrain the
|
|
media type to the given mime-type and properties. By this definition a NULL
|
|
GstCaps or a NULL GstProps means: no constraints.
|
|
|
|
|
|
Variable capabilities vs. fixed capabilities
|
|
--------------------------------------------
|
|
|
|
Definition:
|
|
|
|
A GstProps structure is said to be fixed if it doesn't contain lists or
|
|
ranges, otherwise it is a variable GstProps. A variable GstProps, by definitin
|
|
does not constrain the media type to a set of fixed properties.
|
|
|
|
A GstCaps is said to be fixed if it is not chained and it doesn't contain
|
|
a variable GstProps component.
|
|
|
|
|
|
GstCaps compatibility
|
|
---------------------
|
|
|
|
<write me>
|
|
|
|
GstCaps intersection
|
|
--------------------
|
|
|
|
<write me>
|
|
|
|
|
|
GstCaps usage
|
|
-------------
|
|
|
|
GstCaps are used in the following data structures:
|
|
|
|
- padtemplates.
|
|
* padtemplates are added to elementfactory to describe the possible
|
|
pads that an element created from said factory can have.
|
|
* padtemplates contain the name, direction and presence of the pads.
|
|
* padtemplates also describe the media types that this element can
|
|
accept/produce using GstCaps.
|
|
* padtemplates can provide fixed or variable GstCaps for the pads.
|
|
* padtemplates can be used by the element to create its pads and is
|
|
highly recommended.
|
|
* the padtemplate GstCaps are saved into the registry so that the
|
|
media types an element can operate on, are known without having to
|
|
bring the element into memory.
|
|
|
|
- pad caps
|
|
* pad caps are _fixed_ caps attached to a pad to describe the exact media
|
|
type this pad is handling. A pad with caps is said to be a "tainted" pad.
|
|
|
|
- connection filters
|
|
* a connection filter is created when two pads are connected. It describes
|
|
the media type(s) that _can_ flow through this connection.
|
|
|
|
- application connection filters
|
|
* When the application connects two pads, it can specify an application
|
|
connection filter that will act as additional constraints on the media types
|
|
that can flow through the connection.
|
|
|
|
Connection filters and application filters are cleared when two connected pads
|
|
are disconnected. Pad caps are not cleared. Padtemplates are immutable and
|
|
never cleared.
|
|
|
|
|
|
The GstPad get_caps function
|
|
----------------------------
|
|
|
|
the gst_pad_get_caps function returns the caps of a given pad. The pad caps are
|
|
calculated as:
|
|
|
|
- if the pad has pad caps, return those
|
|
- else if the pad has a getcaps function, call the function and return the result
|
|
- else if the pad has a padtemplate, return the padtemplate caps
|
|
- else return NULL
|
|
|
|
|
|
Purpose of caps negotiation
|
|
---------------------------
|
|
|
|
The purpose of the caps negotiation procedure is to set "pad caps" on a pad
|
|
so that it is compatible with the "connection filter". This has to be done
|
|
_before_ any data passing happens through the connection. Data passing between two
|
|
pads is _not_ allowed when the pad caps are not compatible with the connection
|
|
filter or when the pad caps of two pads participating in the connection are not
|
|
equal.
|
|
|
|
Real caps negotiation starts as soon as an element is in the READY state or higher.
|
|
This means that the connect functions of the pads are only called when the element
|
|
is at least READY. The intersection between two pads is made at connect time,
|
|
regardless of element state.
|
|
|
|
|
|
GstPad connection
|
|
-----------------
|
|
|
|
When two pads are connected get_caps is called on both pads to get their caps.
|
|
Then the intersection between those caps is made, this will give us all the
|
|
possible media types that can flow through this pad connection. Optionally the
|
|
application can provide additional caps, the pad intersection is also made with
|
|
the application caps.
|
|
|
|
The intersection and the optional application caps are stored in the two pads.
|
|
|
|
If the intersection is NULL, the two pads have no common types and the connection
|
|
is refused.
|
|
|
|
If the intersection is a fixed caps, this means there is only one possible media type
|
|
that can be used for this connection.
|
|
|
|
For all not NULL intersections the pad connect functions are called with the
|
|
intersection. Depending on the result of the connect function the connection is
|
|
allowed or refused.
|
|
|
|
If the intersection is fixed and the pad connect functions agreed to the caps,
|
|
the caps are set on the pads.
|
|
|
|
Note that pad caps are never set for non fixed caps.
|
|
|
|
!Example 1:
|
|
!
|
|
! 1. before connecting the pads, they both have a set of possible media types,
|
|
! on the pad, through the getcaps function or on the padtemplate (here
|
|
! represented with capital letters)
|
|
!
|
|
! srcpad sinkpad
|
|
! A B
|
|
! B F
|
|
! C A
|
|
! G
|
|
!
|
|
! 2. when performing the connection, the intersection between the two sets of caps
|
|
! is made.
|
|
!
|
|
! srcpad sinkpad
|
|
! A ) ( B
|
|
! B )--------> A <------( F
|
|
! C ) B ( A
|
|
! ( G
|
|
!
|
|
! 3. In this case the intersection is not a fixed caps (it's a chained caps).
|
|
! the connect function of the two pads are called (if any), the connect
|
|
! function can accept of refuse the caps.
|
|
!
|
|
! 4. if the caps are accepted, the intersection caps are set on both pads.
|
|
!
|
|
! 5. plugins will typically not configure themselves if they get variable caps.
|
|
! It is possible though for a plugin to select one of the caps, fixate
|
|
! some properties and refine the filter to fixed caps (see later)
|
|
!
|
|
|
|
|
|
!Example 2:
|
|
!
|
|
! 1. we take two pads that intersect to fixed caps (B):
|
|
!
|
|
! srcpad sinkpad
|
|
! A ) ( B
|
|
! B )-------> B <-------( F
|
|
!
|
|
! 2. both pad connect functions are called.
|
|
!
|
|
! 3. assuming the connect functions did not refuse the connection, the caps
|
|
! are set on both pads (because they are fixed).
|
|
!
|
|
! srcpad (B)--------------(B) sinkpad
|
|
! A B
|
|
! B F
|
|
!
|
|
|
|
!Example 3:
|
|
!
|
|
! 1. we take two pads, one with pad caps, another one with a padtemplate.
|
|
!
|
|
! srcpad (B) sinkpad
|
|
! ! ( B
|
|
! ------> B <-------( F
|
|
!
|
|
! 2. the pad get_caps function will return the pad caps of the srcpad and
|
|
! the padtemplate caps of the sinkpad. The intersection is made, yielding
|
|
! a fixed caps (B) which is sent to both connect functions (if any)
|
|
!
|
|
! 3. If the connect function(s) didn't refuse the connection the fixed caps
|
|
! are set on both pads. The intersection is also kept in the pad object.
|
|
!
|
|
! srcpad (B)--------------(B) sinkpad
|
|
! B
|
|
! F
|
|
!
|
|
! 4. when fixed caps are received on the sinkpad the plugin will typically
|
|
! configure itself to deal with the format.
|
|
!
|
|
|
|
|
|
Pad connect functions
|
|
---------------------
|
|
|
|
A Pad can be notified when another pad is connected or reconnected to it with
|
|
fixed or variable caps. This notification will be done with the optional
|
|
GstPadConnectFunction callback that an element can provide for a pad.
|
|
Remember that this connection is _always_ called, not only with fixed caps
|
|
but also with variable caps (if any).
|
|
|
|
We will explain some of the common situations with some examples.
|
|
|
|
!Example 4:
|
|
!
|
|
! 1. We have a simple mpeg audio decoder connected to a simple audio sink.
|
|
! The mpeg decoder doesn't have a connect function on
|
|
!
|
|
!
|
|
!
|
|
!
|
|
!
|
|
!
|
|
!
|
|
!
|
|
!
|
|
!
|
|
!
|
|
!
|
|
!
|
|
!
|
|
!
|
|
!
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|