mirror of
https://gitlab.freedesktop.org/gstreamer/gstreamer.git
synced 2024-11-29 13:11:06 +00:00
Added .dia UML of GstCaps
Original commit message from CVS: Added .dia UML of GstCaps Added the autoplugger factory to plugins.dia Added a first draft for caps-negotiation
This commit is contained in:
Wim Taymans
parent
1a66e83975
commit
ff31df763e
3 changed files with 1380 additions and 0 deletions
1132
docs/random/caps.dia
Normal file
1132
docs/random/caps.dia
Normal file
File diff suppressed because it is too large
Load diff
Binary file not shown.
248
docs/random/wtay/caps-negociation
Normal file
248
docs/random/wtay/caps-negociation
Normal file
|
|
@ -0,0 +1,248 @@
|
|||
caps negotiation
|
||||
================
|
||||
|
||||
1) purpose
|
||||
----------
|
||||
|
||||
The pads expose the media types they can handle using a mime
|
||||
type and a set of properties. Before the pad is created or
|
||||
used to pass buffers, we only know the global 'range' of media
|
||||
data this pad can accept. When the element has had a chance to
|
||||
look at the media data, only then it knows the exact values of the
|
||||
properties.
|
||||
|
||||
example1:
|
||||
!
|
||||
! The mp3 decoder exposes the capabilities of its src pad
|
||||
! with the following caps:
|
||||
!
|
||||
! 'mpg123_src':
|
||||
! MIME type: 'audio/raw':
|
||||
! format: Integer: 16
|
||||
! depth: Integer: 16
|
||||
! rate: Integer range: 11025 - 48000
|
||||
! channels: Integer range: 1 - 2
|
||||
|
||||
as you can see in example1, the padtemplate has both a range
|
||||
(for the audio rate) and a list (for the number of channels)
|
||||
for its properties.
|
||||
|
||||
only when the mpg123 element has decoded the first mpeg audio
|
||||
header, it knows the exact values of the rate and channels
|
||||
properties.
|
||||
|
||||
suppose that we want to connect this src pad to the sink pad
|
||||
of an audiosink with the properties given in example2:
|
||||
|
||||
example2:
|
||||
!
|
||||
! 'audiosink_sink':
|
||||
! MIME type: 'audio/raw':
|
||||
! format: Integer: 16
|
||||
! depth: List:
|
||||
! Integer: 8
|
||||
! Integer: 16
|
||||
! rate: Integer range: 8000 - 44000
|
||||
! channels: Integer range: 1 - 2
|
||||
|
||||
we can see that connecting the mpg123 src pad with the
|
||||
audiosinks sink pad can cause a potential problem with the
|
||||
rate property.
|
||||
|
||||
When the mpg123 decoder decides to output raw audio with a
|
||||
48000Hz samplerate, the audiosink will not be able to handle
|
||||
it. The conservative approach would be to disallow the connection
|
||||
between the two incompatible pads. This rules out any potential
|
||||
problems but severely limits the amount of possible connections
|
||||
between the elements.
|
||||
|
||||
Another approach would be to allow the connection (and mark it
|
||||
as dangerous) and let the two elements figure out a suitable
|
||||
media type at runtime. This procedure is called caps negotiation.
|
||||
|
||||
|
||||
2) a bit of history
|
||||
-------------------
|
||||
|
||||
The typing of the data that was attached to a buffer used to be
|
||||
done using GstMeta* (and it still is as of 11 feb 2001). With
|
||||
the new GstCaps and GstProps system this typing is gradually moved
|
||||
to the pads and to the padtemplates. This has several advantages:
|
||||
|
||||
- the typing of the data tends to be static. The type of media
|
||||
doesn't change for every buffer.
|
||||
|
||||
- Moving the typing up to the pad(templates) allows us to save
|
||||
them into the registry and allows us to figure out what pads
|
||||
are compatible.
|
||||
|
||||
- the current metadata implementation needs header files. this may
|
||||
change when we also use properties for metadata.
|
||||
|
||||
example3:
|
||||
!
|
||||
! This is the current GstMeta structure that travels with audio buffers
|
||||
!
|
||||
! struct _MetaAudioRaw {
|
||||
! GstMeta meta;
|
||||
!
|
||||
! /* formatting information */
|
||||
! gint format;
|
||||
! gint channels;
|
||||
! gint frequency;
|
||||
! gint bps;
|
||||
! };
|
||||
|
||||
|
||||
The question still remains what purpose the metadata will serve
|
||||
now that we expose the media type in the pads. Some possibilities:
|
||||
|
||||
- interesting information, not describing the data itself but the
|
||||
context in which the data was generated (suggested buffer size,
|
||||
timestamps, etc...)
|
||||
|
||||
- user app metadata.
|
||||
|
||||
In this proposal we also assume that the current use of metadata using
|
||||
GstMeta is deprecated and that we move this information to the properties
|
||||
of the pads.
|
||||
|
||||
|
||||
3) the pad/padtemplates caps
|
||||
----------------------------
|
||||
|
||||
All elements have to provide a padtemplate for their pads.
|
||||
|
||||
The padtemplates provide a range of possible media types this pad can
|
||||
src/sink. the main purpose for the padtemplates is to allow a
|
||||
rough guess at which pads are compatible before even a single buffer
|
||||
has been processed by the element.
|
||||
|
||||
pads are usually created from the templates. When the pad is created
|
||||
it has no GstCaps* attached to it yet. The possible caps this pad
|
||||
can have is exposed in the padtemplate.
|
||||
|
||||
|
||||
4) the connect function
|
||||
-----------------------
|
||||
|
||||
when two pads are connected the following steps will take
|
||||
place:
|
||||
|
||||
- if both pads have caps, the caps are checked. If the caps
|
||||
are incompatible, the padtemplates are checked, if they
|
||||
are compatible, caps negotiation is performed.
|
||||
|
||||
- if one of the pads has caps, the caps is checked against
|
||||
the padtemplate of the peer pad. If they are incompatible,
|
||||
the padtemplates are compared, if they are incompatible,
|
||||
caps negotiation is performed.
|
||||
|
||||
- if none of the pads have caps, the padtemplates are checked,
|
||||
if they are incompatible, a warning is issued.
|
||||
|
||||
|
||||
5) when the element knows the media type it is handling
|
||||
-------------------------------------------------------
|
||||
|
||||
When the element has received its first buffer it will know
|
||||
the media type it is handling by inspecting the buffer.
|
||||
|
||||
before pushing the data out to its peer element(s), the element
|
||||
will set its src pad with the appropriate caps and properties.
|
||||
These caps must follow the following rules:
|
||||
|
||||
- the caps must be compatible with the padtemplates of this
|
||||
pad.
|
||||
|
||||
- the caps cannot contain ranges or lists.
|
||||
|
||||
by setting the caps of the src pad, the following procedure
|
||||
happens:
|
||||
|
||||
- if the peer pad has a negotiate function, it is called and
|
||||
negotiation is performed (see later)
|
||||
|
||||
- if the peer pad has no negotiate function, the padtemplate
|
||||
is checked:
|
||||
|
||||
- if the caps are compatible with the template, the peer pad
|
||||
receives the same caps as the src pad.
|
||||
|
||||
- if the caps are not compatible, the negotiation fails and
|
||||
the elements are disconnected. A signal is emitted to inform
|
||||
the user app or the manager of the element to find a solution.
|
||||
|
||||
the caps can be set with the gst_pad_set_caps function, which accepts
|
||||
the following parameters:
|
||||
|
||||
- the pad to set the caps to
|
||||
- whether the peer pad *must* accept the caps (AUTHORITATIVE) or
|
||||
whether if can reject the caps (TRY)
|
||||
- a GstCaps* structure
|
||||
|
||||
example4:
|
||||
!
|
||||
! an audio element setting its src pad caps (need something easier):
|
||||
!
|
||||
! gst_pad_set_caps (
|
||||
! pad,
|
||||
! GST_CAPS_NEGOTIATE_AUTHORITATIVE,
|
||||
! gst_caps_new_with_props (
|
||||
! "src_caps", /* name */
|
||||
! "audio/raw", /* mime */
|
||||
! gst_props_new (
|
||||
! "format", GST_PROPS_INT (AFMT_S16_LE),
|
||||
! "depth", GST_PROPS_INT (16),
|
||||
! "rate", GST_PROPS_INT (44100),
|
||||
! "channels", GST_PROPS_INT (2),
|
||||
! NULL
|
||||
! )
|
||||
! )
|
||||
! );
|
||||
|
||||
the _set_caps function returns a gboolean, indicating that the
|
||||
new caps are accepted by the peer pad.
|
||||
|
||||
when the negotiation type is set to TRY, and the negotiation
|
||||
fails, the pad has to perform another _set_caps call. The last
|
||||
call to _set_caps has to be of type AUTHORITATIVE.
|
||||
|
||||
when the negotiation fails and the type is set to AUTHORITATIVE,
|
||||
the pads are disconnected and a signal is emitted to inform any
|
||||
interested listeners.
|
||||
|
||||
|
||||
6) caps negotiation
|
||||
-------------------
|
||||
|
||||
the negotiate function of a pad is called whenever the peer pad
|
||||
modifies the caps using the gst_pad_set_caps function.
|
||||
|
||||
The negotiate function has to return a gboolean indicating the
|
||||
new caps are acceptable. When it accepts the caps, both pads will
|
||||
be set to the negotiated caps.
|
||||
|
||||
example5:
|
||||
! this is the caps negotiation function implemented by an element on
|
||||
! one of its sink pads.
|
||||
!
|
||||
! static gboolean
|
||||
! gst_pad_caps_negotiate (GstPad *pad, GstCaps *caps)
|
||||
! {
|
||||
! /* we don't accept anything else than audio/raw */
|
||||
! if (strcmp (gst_caps_get_mime (caps), "audio/raw"))
|
||||
! return FALSE;
|
||||
!
|
||||
! if (gst_caps_get_int_prop (caps, "format") != AFMT_S16_LE)
|
||||
! return FALSE;
|
||||
!
|
||||
! /* we accept everything else */
|
||||
! return TRUE;
|
||||
! }
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Loading…
Reference in a new issue