mirror of
https://gitlab.freedesktop.org/gstreamer/gstreamer.git
synced 2024-11-15 22:01:27 +00:00
209 lines
7.4 KiB
Text
209 lines
7.4 KiB
Text
SOME OF THE FIRST IDEAS, PRETTY OUTDATED
|
|
----------------------------------------
|
|
|
|
|
|
During the course of a discussion on IRC, it turned out
|
|
that there are many possible ways to handle the capabilities.
|
|
|
|
A capability is basically a set of properties attached to a
|
|
mimetype in order to more closely describe the mimetype.
|
|
Capabilities are supposed to be attached to pads so that the
|
|
autoplugging algorithm has more specific information to connect
|
|
compatible pads.
|
|
|
|
We present 3 possible implementation for the capabilities. we need
|
|
to pick one of them.
|
|
|
|
1. static capabilities
|
|
----------------------
|
|
|
|
When an element is created, it creates its pads like:
|
|
|
|
mpg123->sinkpad = gst_pad_new ("sink", GST_PAD_SINK);
|
|
gst_element_add_pad (GST_ELEMENT (mpg123), mpg123->sinkpad);
|
|
|
|
mpg123->srcpad = gst_pad_new ("src", GST_PAD_SRC);
|
|
gst_element_add_pad (GST_ELEMENT (mpg123), mpg123->srcpad);
|
|
|
|
In the static capabilities case, it will attach a GstCaps* structure
|
|
to the pad. The GstCaps structure in the above example might look like:
|
|
|
|
static GstCapsFactory mpg123_sink_caps = {
|
|
"audio/mp3",
|
|
"layer", GST_CAPS_INT_RANGE (1, 3),
|
|
"bitrate", GST_CAPS_INT_RANGE (8, 320),
|
|
NULL
|
|
};
|
|
|
|
with
|
|
|
|
mpg123sinkcaps = gst_caps_register (mpg123_sink_caps);
|
|
|
|
the factory can be converted into a GstCaps* structure. The
|
|
GstCaps* structure is attached to the pad with:
|
|
|
|
gst_pad_add_caps (mpg123->sinkpad, mpg123sinkcaps);
|
|
|
|
The GstElement would then have a sinkpad with the given
|
|
mimetype (audio/mp3) and with the capabilitities of accepting
|
|
mpeg layer 1 to 3 and a bitrate from 8 up to 320 Kbps.
|
|
|
|
Likewise, the src pad could be set up in the same way. An
|
|
example capability factory could look like:
|
|
|
|
static GstCapsFactory mpg123_src_caps = {
|
|
"audio/raw",
|
|
"format", GST_CAPS_BITFIELD (...),
|
|
"depth", GST_CAPS_INT (16),
|
|
"rate", GST_CAPS_INT_RANGE (4000, 96000),
|
|
"channels", GST_CAPS_INT_RANGE (1, 2),
|
|
NULL
|
|
};
|
|
|
|
All GstElements would present their pads with the appropriate
|
|
capabilities structure.
|
|
|
|
The autoplugger would then proceed (once the source media type
|
|
is known with a typefind function) in finding all the elements
|
|
with compatible pads and connecting them into a pipeline.
|
|
|
|
All elements of the complete pipeline could then be constructed
|
|
with one single pass. No new elements should be added to the
|
|
pipeline because we can figure out all the possibilities using the
|
|
pad capabilities.
|
|
|
|
We call this the static case because the capabilities of the pads
|
|
are supposed to stay the same after creating the element.
|
|
|
|
While the ability to completely setup the pipeline before actually
|
|
starting playback is an advantage regarding performance, one obvious
|
|
problem with this setup is that the static case may be too static in
|
|
some cases. We can illustrate this with the following setup:
|
|
|
|
----------) (------------
|
|
mpg123 ! ! audiosink
|
|
src sink
|
|
! !
|
|
----------) (------------
|
|
|
|
The mpg123 element has its src capabilities set up as mpg123_src_caps
|
|
in the above example.
|
|
|
|
The audio renderer has its capabilities set up with the following
|
|
factory:
|
|
|
|
static GstCapsFactory audio_sink_caps = {
|
|
"audio/raw",
|
|
"format", GST_CAPS_BITFIELD (...),
|
|
"depth", GST_CAPS_INT (16),
|
|
"rate", GST_CAPS_INT_RANGE (22000, 44000),
|
|
"channels", GST_CAPS_INT_RANGE (1, 2),
|
|
NULL
|
|
};
|
|
|
|
The static autoplugger has to be careful when connecting the mpg123
|
|
element with the audiosink because it is theoretically possible that
|
|
the mpg123 element outputs raw audio with a rate that cannot be
|
|
handled by the audiosink (ex. 4000KHz). In the absence of another
|
|
audiosink with more capabilities, the autoplugging of this simple
|
|
pipeline will not be possible and would fail.
|
|
|
|
the autoplugging algorithm would probably select another element to
|
|
insert between the mpg123 element and the audiosink in order to handle
|
|
the (uncommon) case of a rate conversion (audioscaler).
|
|
|
|
It is clear that this static setup might even fail or work suboptimal
|
|
for even the common case and should therefore be considered as too
|
|
restrictive.
|
|
|
|
|
|
2. dynamic capabilities
|
|
-----------------------
|
|
|
|
The idea of dynamic capabilities is that the capabilities are not set
|
|
at element create time but rather while the pipeline is running.
|
|
|
|
An element would still list its mime type using:
|
|
|
|
gst_pad_add_type_id(mpg123->sinkpad, mp3type);
|
|
|
|
The idea would then be that a rough draft of the pipeline would be
|
|
built after the media type of the stream has been detected with the
|
|
typefind functions. The rough draft would consist of laying out a
|
|
global plan to reach the renderer(s). this plan would basically list
|
|
the set of conversions that have to be performed. (mime-type to
|
|
mime-type conversion).
|
|
|
|
Elements that accept the src mime-type are tried by giving it a buffer.
|
|
If the element accepts the buffer, it will set its capabilities for
|
|
both the sink pad and the src pad. At that time other elements can be
|
|
tried and added to the src pad, until we reach the renderer. As usual
|
|
one has to be careful to add just the minimum amount of elements to
|
|
reach the renderer. The global plan will help with that.
|
|
|
|
Since we basically do not use the capabilities of the sink pad one has
|
|
to question the need for sink pad capabilities in the first place.
|
|
|
|
We might also have a hard time trying different elements until we find
|
|
a compatible one that does not cause a dead end at some point.
|
|
|
|
|
|
3. combined setup
|
|
-----------------
|
|
|
|
This combined setup will minimise the effort needed to try different
|
|
elements encountered by option 2 while still allowing a more dynamic
|
|
setup based on the actual media stream we are handling.
|
|
|
|
The combined setup will list/add the sink capabilities at create time.
|
|
It will only set the mime-type of its src pads.
|
|
|
|
As with option2, a global plan will be built. At runtime the src pads
|
|
will actually specify the capabilities they need for any element that
|
|
wants to be connected to its source pads.
|
|
|
|
In this case we specify the capabilities for all the sink pads of an
|
|
element at create time. The capabilities of the src pads would only
|
|
become available when data has been processed by the element.
|
|
|
|
The autoplugger would then be able to choose an element that can handle
|
|
the capability listed by the src pad.
|
|
|
|
in our previous example:
|
|
|
|
----------) (------------
|
|
mpg123 ! ! audiosink
|
|
src sink
|
|
! !
|
|
----------) (------------
|
|
|
|
the audiosink element would specify its sink pad capabilities at create
|
|
time, while the mpg123 elements src pad would not yet have any capabilities
|
|
set.
|
|
|
|
When data is handled by the mpg123 element, a capability would be added to
|
|
the mpg123 src pad. This capability might be:
|
|
|
|
static GstCapsFactory mpg123_src_caps = {
|
|
"audio/raw",
|
|
"format", GST_CAPS_INT (S16),
|
|
"depth", GST_CAPS_INT (16),
|
|
"rate", GST_CAPS_INT (44000),
|
|
"channels", GST_CAPS_INT (2),
|
|
NULL
|
|
};
|
|
|
|
This capability would be compatible with the audiosinks sinkpad capabilities
|
|
and the autoplugger would therefore be able to connect the two elements.
|
|
|
|
While allowing a more flexible setup with option3, compared to option1, we
|
|
introduce a slightly higher overhead when we need to dynamically connect
|
|
elements. This overhead will not be as big as option2 because we don't
|
|
have to 'try' elements.
|
|
|
|
so:
|
|
|
|
src caps: added at runtime to list the caps needed for an element that
|
|
wants to connect to this pad.
|
|
sink caps: the (static) capabilities that this sinkpad has.
|
|
|