mirror of
https://gitlab.freedesktop.org/gstreamer/gstreamer.git
synced 2024-12-23 16:50:47 +00:00
ff31df763e
Original commit message from CVS: Added .dia UML of GstCaps Added the autoplugger factory to plugins.dia Added a first draft for caps-negotiation
248 lines
7.5 KiB
Text
248 lines
7.5 KiB
Text
caps negotiation
|
|
================
|
|
|
|
1) purpose
|
|
----------
|
|
|
|
The pads expose the media types they can handle using a mime
|
|
type and a set of properties. Before the pad is created or
|
|
used to pass buffers, we only know the global 'range' of media
|
|
data this pad can accept. When the element has had a chance to
|
|
look at the media data, only then it knows the exact values of the
|
|
properties.
|
|
|
|
example1:
|
|
!
|
|
! The mp3 decoder exposes the capabilities of its src pad
|
|
! with the following caps:
|
|
!
|
|
! 'mpg123_src':
|
|
! MIME type: 'audio/raw':
|
|
! format: Integer: 16
|
|
! depth: Integer: 16
|
|
! rate: Integer range: 11025 - 48000
|
|
! channels: Integer range: 1 - 2
|
|
|
|
as you can see in example1, the padtemplate has both a range
|
|
(for the audio rate) and a list (for the number of channels)
|
|
for its properties.
|
|
|
|
only when the mpg123 element has decoded the first mpeg audio
|
|
header, it knows the exact values of the rate and channels
|
|
properties.
|
|
|
|
suppose that we want to connect this src pad to the sink pad
|
|
of an audiosink with the properties given in example2:
|
|
|
|
example2:
|
|
!
|
|
! 'audiosink_sink':
|
|
! MIME type: 'audio/raw':
|
|
! format: Integer: 16
|
|
! depth: List:
|
|
! Integer: 8
|
|
! Integer: 16
|
|
! rate: Integer range: 8000 - 44000
|
|
! channels: Integer range: 1 - 2
|
|
|
|
we can see that connecting the mpg123 src pad with the
|
|
audiosinks sink pad can cause a potential problem with the
|
|
rate property.
|
|
|
|
When the mpg123 decoder decides to output raw audio with a
|
|
48000Hz samplerate, the audiosink will not be able to handle
|
|
it. The conservative approach would be to disallow the connection
|
|
between the two incompatible pads. This rules out any potential
|
|
problems but severely limits the amount of possible connections
|
|
between the elements.
|
|
|
|
Another approach would be to allow the connection (and mark it
|
|
as dangerous) and let the two elements figure out a suitable
|
|
media type at runtime. This procedure is called caps negotiation.
|
|
|
|
|
|
2) a bit of history
|
|
-------------------
|
|
|
|
The typing of the data that was attached to a buffer used to be
|
|
done using GstMeta* (and it still is as of 11 feb 2001). With
|
|
the new GstCaps and GstProps system this typing is gradually moved
|
|
to the pads and to the padtemplates. This has several advantages:
|
|
|
|
- the typing of the data tends to be static. The type of media
|
|
doesn't change for every buffer.
|
|
|
|
- Moving the typing up to the pad(templates) allows us to save
|
|
them into the registry and allows us to figure out what pads
|
|
are compatible.
|
|
|
|
- the current metadata implementation needs header files. this may
|
|
change when we also use properties for metadata.
|
|
|
|
example3:
|
|
!
|
|
! This is the current GstMeta structure that travels with audio buffers
|
|
!
|
|
! struct _MetaAudioRaw {
|
|
! GstMeta meta;
|
|
!
|
|
! /* formatting information */
|
|
! gint format;
|
|
! gint channels;
|
|
! gint frequency;
|
|
! gint bps;
|
|
! };
|
|
|
|
|
|
The question still remains what purpose the metadata will serve
|
|
now that we expose the media type in the pads. Some possibilities:
|
|
|
|
- interesting information, not describing the data itself but the
|
|
context in which the data was generated (suggested buffer size,
|
|
timestamps, etc...)
|
|
|
|
- user app metadata.
|
|
|
|
In this proposal we also assume that the current use of metadata using
|
|
GstMeta is deprecated and that we move this information to the properties
|
|
of the pads.
|
|
|
|
|
|
3) the pad/padtemplates caps
|
|
----------------------------
|
|
|
|
All elements have to provide a padtemplate for their pads.
|
|
|
|
The padtemplates provide a range of possible media types this pad can
|
|
src/sink. the main purpose for the padtemplates is to allow a
|
|
rough guess at which pads are compatible before even a single buffer
|
|
has been processed by the element.
|
|
|
|
pads are usually created from the templates. When the pad is created
|
|
it has no GstCaps* attached to it yet. The possible caps this pad
|
|
can have is exposed in the padtemplate.
|
|
|
|
|
|
4) the connect function
|
|
-----------------------
|
|
|
|
when two pads are connected the following steps will take
|
|
place:
|
|
|
|
- if both pads have caps, the caps are checked. If the caps
|
|
are incompatible, the padtemplates are checked, if they
|
|
are compatible, caps negotiation is performed.
|
|
|
|
- if one of the pads has caps, the caps is checked against
|
|
the padtemplate of the peer pad. If they are incompatible,
|
|
the padtemplates are compared, if they are incompatible,
|
|
caps negotiation is performed.
|
|
|
|
- if none of the pads have caps, the padtemplates are checked,
|
|
if they are incompatible, a warning is issued.
|
|
|
|
|
|
5) when the element knows the media type it is handling
|
|
-------------------------------------------------------
|
|
|
|
When the element has received its first buffer it will know
|
|
the media type it is handling by inspecting the buffer.
|
|
|
|
before pushing the data out to its peer element(s), the element
|
|
will set its src pad with the appropriate caps and properties.
|
|
These caps must follow the following rules:
|
|
|
|
- the caps must be compatible with the padtemplates of this
|
|
pad.
|
|
|
|
- the caps cannot contain ranges or lists.
|
|
|
|
by setting the caps of the src pad, the following procedure
|
|
happens:
|
|
|
|
- if the peer pad has a negotiate function, it is called and
|
|
negotiation is performed (see later)
|
|
|
|
- if the peer pad has no negotiate function, the padtemplate
|
|
is checked:
|
|
|
|
- if the caps are compatible with the template, the peer pad
|
|
receives the same caps as the src pad.
|
|
|
|
- if the caps are not compatible, the negotiation fails and
|
|
the elements are disconnected. A signal is emitted to inform
|
|
the user app or the manager of the element to find a solution.
|
|
|
|
the caps can be set with the gst_pad_set_caps function, which accepts
|
|
the following parameters:
|
|
|
|
- the pad to set the caps to
|
|
- whether the peer pad *must* accept the caps (AUTHORITATIVE) or
|
|
whether if can reject the caps (TRY)
|
|
- a GstCaps* structure
|
|
|
|
example4:
|
|
!
|
|
! an audio element setting its src pad caps (need something easier):
|
|
!
|
|
! gst_pad_set_caps (
|
|
! pad,
|
|
! GST_CAPS_NEGOTIATE_AUTHORITATIVE,
|
|
! gst_caps_new_with_props (
|
|
! "src_caps", /* name */
|
|
! "audio/raw", /* mime */
|
|
! gst_props_new (
|
|
! "format", GST_PROPS_INT (AFMT_S16_LE),
|
|
! "depth", GST_PROPS_INT (16),
|
|
! "rate", GST_PROPS_INT (44100),
|
|
! "channels", GST_PROPS_INT (2),
|
|
! NULL
|
|
! )
|
|
! )
|
|
! );
|
|
|
|
the _set_caps function returns a gboolean, indicating that the
|
|
new caps are accepted by the peer pad.
|
|
|
|
when the negotiation type is set to TRY, and the negotiation
|
|
fails, the pad has to perform another _set_caps call. The last
|
|
call to _set_caps has to be of type AUTHORITATIVE.
|
|
|
|
when the negotiation fails and the type is set to AUTHORITATIVE,
|
|
the pads are disconnected and a signal is emitted to inform any
|
|
interested listeners.
|
|
|
|
|
|
6) caps negotiation
|
|
-------------------
|
|
|
|
the negotiate function of a pad is called whenever the peer pad
|
|
modifies the caps using the gst_pad_set_caps function.
|
|
|
|
The negotiate function has to return a gboolean indicating the
|
|
new caps are acceptable. When it accepts the caps, both pads will
|
|
be set to the negotiated caps.
|
|
|
|
example5:
|
|
! this is the caps negotiation function implemented by an element on
|
|
! one of its sink pads.
|
|
!
|
|
! static gboolean
|
|
! gst_pad_caps_negotiate (GstPad *pad, GstCaps *caps)
|
|
! {
|
|
! /* we don't accept anything else than audio/raw */
|
|
! if (strcmp (gst_caps_get_mime (caps), "audio/raw"))
|
|
! return FALSE;
|
|
!
|
|
! if (gst_caps_get_int_prop (caps, "format") != AFMT_S16_LE)
|
|
! return FALSE;
|
|
!
|
|
! /* we accept everything else */
|
|
! return TRUE;
|
|
! }
|
|
|
|
|
|
|
|
|
|
|
|
|