mirror of
https://gitlab.freedesktop.org/gstreamer/gstreamer.git
synced 2024-12-05 16:06:43 +00:00
4a402c1c7d
Found via `codespell` https://bugzilla.gnome.org/show_bug.cgi?id=795610
320 lines
11 KiB
Text
320 lines
11 KiB
Text
The elementfactory for a given element will contain some information about the capabilities of element's
|
|
pads or potential pads. An indication will be provided as to whether the pad always exists, always
|
|
exists once data is present, or *might* exist once data is present (the latter case is for things like
|
|
the MPEG system parsers, where an audio stream might or might not exist).
|
|
|
|
|
|
First, an entirely normal example:
|
|
----------------------------------
|
|
|
|
(-----------) (----------) (-------------)
|
|
! disksrc ! ! mpg123 ! ! audiosink !
|
|
! src sink src sink !
|
|
! ! ! ! ! !
|
|
(-----------) (----------) (-------------)
|
|
|
|
We start with only the disksrc. The typefind filter is attached to the disksrc, and via typefind magic
|
|
the properties of the disksrc are found to be:
|
|
|
|
disksrc->src->caps = {
|
|
"audio/mp3",
|
|
"layer", GST_CAPS_INT (3),
|
|
"bitrate", GST_CAPS_INT (128),
|
|
NULL
|
|
};
|
|
|
|
A look through the plugin registry shows that we have an element called mpg123 that has the following
|
|
caps:
|
|
|
|
static GstCapsFactory mpg123_sink_caps = {
|
|
"audio/mp3",
|
|
"layer", GST_CAPS_INT_RANGE (1, 3),
|
|
"bitrate", GST_CAPS_INT_RANGE (8, 320),
|
|
NULL
|
|
};
|
|
|
|
The caps of the disksrc fit within those parameters, so we instantiate an mpg123 and attach it to the
|
|
disksrc. The connection succeeds negotiation and as a result the mpg123 specifies its output caps as:
|
|
|
|
mpg123->src->caps = {
|
|
"audio/raw",
|
|
"format", GST_CAPS_BITFIELD (S16),
|
|
"depth", GST_CAPS_INT (16),
|
|
"rate", GST_CAPS_INT (44100),
|
|
"channels", GST_CAPS_INT (2),
|
|
NULL
|
|
};
|
|
|
|
Again from the plugin registry we find an element audiosink that has appropriate caps:
|
|
|
|
static GstCapsFactory audiosink_src_caps = {
|
|
"audio/raw",
|
|
"format", GST_CAPS_BITFIELD (S16,....),
|
|
"depth", GST_CAPS_INT (16),
|
|
"rate", GST_CAPS_INT_RANGE (4000, 96000),
|
|
"channels", GST_CAPS_INT_RANGE (1, 2),
|
|
NULL
|
|
};
|
|
|
|
A copy of the audiosink is instantiated and attached, negotiation goes smoothly, and we're done. No
|
|
dataflow has occurred, no failure found, etc. An ideal autoplug.
|
|
|
|
|
|
Now, a slightly more convoluted example:
|
|
----------------------------------------
|
|
|
|
Start with the same graph:
|
|
|
|
(-----------) (----------) (-------------)
|
|
! disksrc ! ! mpg123 ! ! audiosink !
|
|
! src sink src sink !
|
|
! ! ! ! ! !
|
|
(-----------) (----------) (-------------)
|
|
|
|
Run typefind on the disksrc's output, get the same output caps:
|
|
|
|
disksrc->src->caps = {
|
|
"audio/mp3",
|
|
"layer", GST_CAPS_INT (3),
|
|
"bitrate", GST_CAPS_INT (128),
|
|
NULL
|
|
};
|
|
|
|
Find and attach mpg123, get the following output caps this time:
|
|
|
|
mpg123->src->caps = {
|
|
"audio/raw",
|
|
"format", GST_CAPS_BITFIELD (S16),
|
|
"depth", GST_CAPS_INT (16),
|
|
"rate", GST_CAPS_INT (44100),
|
|
"channels", GST_CAPS_INT (1),
|
|
NULL
|
|
};
|
|
|
|
Note that this time we have a mono output. A look into the audiosink caps shows that we have a match.
|
|
So we instantiate a copy. Oops. We now find that the caps for the input pad on our audiosink have
|
|
changed:
|
|
|
|
mpg123->src->caps = {
|
|
"audio/raw",
|
|
"format", GST_CAPS_BITFIELD (S16,...),
|
|
"depth", GST_CAPS_INT (16),
|
|
"rate", GST_CAPS_INT (11025, 48000),
|
|
"channels", GST_CAPS_INT (2),
|
|
NULL
|
|
};
|
|
|
|
Whoops. It seems that the sound card we've got in this machine (FIXME how on earth to deal with
|
|
multiple sound cards???) doesn't support mono output *at all*. This is a problem. We now find that we
|
|
have no options as far as directly matching the mpg123 to the audiosink.
|
|
|
|
A look through our (fictitious) plugin registry shows at least one element that at least has audio/raw
|
|
on both input and output (since both mpg123 and audiosink have open pads with this mime type). A closerlook shows that its caps are:
|
|
|
|
static GstCapsFactory mono2stereo_sink_caps = {
|
|
"audio/raw",
|
|
"channels", GST_CAPS_INT (1),
|
|
NULL
|
|
};
|
|
static GstCapsFactory mono2stereo_src_caps = {
|
|
"audio/raw",
|
|
"channels", GST_CAPS_INT (2),
|
|
NULL
|
|
};
|
|
|
|
Wow, that's a perfect match. Instantiate, attach to mpg123, no problems. Attach to audiosink, no
|
|
problems. Done. When we start up the pipeline, we should get absolutely no callbacks from pads saying
|
|
"help me, I've fallen and..., er, I don't like this buffer!".
|
|
|
|
|
|
A really messy case:
|
|
--------------------
|
|
|
|
Start with a disksrc, typefind it, get the following:
|
|
|
|
disksrc->src->caps = {
|
|
"audio/mp3",
|
|
"layer", GST_CAPS_INT (3),
|
|
"bitrate", GST_CAPS_INT (128),
|
|
NULL
|
|
};
|
|
|
|
Look through the plugin registry, find mpg123. Instantiate it, attach it. It spits out audio
|
|
parameters as usual:
|
|
|
|
mpg123->src->caps = {
|
|
"audio/raw",
|
|
"format", GST_CAPS_BITFIELD (S16),
|
|
"depth", GST_CAPS_INT (16),
|
|
"rate", GST_CAPS_INT (44100),
|
|
"channels", GST_CAPS_INT (2),
|
|
NULL
|
|
};
|
|
|
|
Now we instantiate an audiosink plugin. This time, we're sunk:
|
|
|
|
mpg123->src->caps = {
|
|
"audio/raw",
|
|
"format", GST_CAPS_BITFIELD (S8,U8),
|
|
"depth", GST_CAPS_INT (8),
|
|
"rate", GST_CAPS_INT_RANGE (11025, 22050),
|
|
"channels", GST_CAPS_INT (1),
|
|
NULL
|
|
};
|
|
|
|
ACK! It's one of those Disney Sound Source things. We've got a problem here that isn't obviously
|
|
solvable. However, there happens to be another mp3 decoder sitting around. It's got the same
|
|
properties as mpg123, but a lower merit value. Let's instantiate one and attach it. We get the
|
|
following output pad caps:
|
|
|
|
mp3decoder->src->caps = {
|
|
"audio/raw",
|
|
"format", GST_CAPS_BITFIELD (S8,S16),
|
|
"depth", GST_CAPS_INT_RANGE (8,16),
|
|
"rate", GST_CAPS_INT_RANGE (8000, 44100),
|
|
"channels", GST_CAPS_INT (1,2),
|
|
NULL
|
|
};
|
|
|
|
Well, that matches the audiosink. We try attaching it, and during negotiation the mp3decoder finds
|
|
sufficient common ground with the castrated audiosink and sets its output pad to match the best of the
|
|
options: S8 at 22050 KHz.
|
|
|
|
|
|
|
|
Next to impossible scenario: DVD
|
|
--------------------------------
|
|
|
|
Start with a dvdsrc. It's output pad caps are:
|
|
|
|
static GstCapsFactory dvdsrc_src_caps = {
|
|
"video/mpeg",
|
|
"mpegversion", GST_CAPS_INT (2),
|
|
"systemstream", GST_CAPS_BOOLEAN (TRUE),
|
|
NULL
|
|
};
|
|
|
|
The type would be classified as incomplete via some mechanism. This might cause the autoplug code to go
|
|
and run the typefind function. It would flesh the type out to the following:
|
|
|
|
dvdsrc->src->caps = {
|
|
"video/mpeg",
|
|
"mpegversion", GST_CAPS_INT (2),
|
|
"systemstream", GST_CAPS_BOOLEAN (TRUE),
|
|
"videostreams", GST_CAPS_INT (1),
|
|
"audiostreams", GST_CAPS_INT (3),
|
|
"bitrate", GST_CAPS_INT (40960),
|
|
NULL,
|
|
};
|
|
|
|
Wow, that helped a lot. A check through the plugin registry shows that the mpeg2parse will match those
|
|
properties:
|
|
|
|
static GstCapsFactory mpeg2parse_sink_caps = {
|
|
"video/mpeg",
|
|
"mpegversion", GST_CAPS_INT (2),
|
|
"systemstream", GST_CAPS_BOOLEAN (TRUE),
|
|
NULL
|
|
};
|
|
|
|
(In retrospect, it may not be necessary to run typefind if there's match this good right away. Only run
|
|
typefind when there's no exact match.)
|
|
Since there are no output pads yet, we have to actually push data through the pipeline. The moment a
|
|
buffer or two get to the mpeg2parse element, it promptly goes and creates an output pad, probably of the
|
|
following caps:
|
|
|
|
mpeg2parse_video_src_caps = {
|
|
"video/mpeg",
|
|
"mpegversion", GST_CAPS_RANGE (1,2),
|
|
"systemstream", GST_CAPS_BOOLEAN (FALSE),
|
|
NULL
|
|
};
|
|
|
|
This seems to be a task for typefind again. But since data is flowing, we have to be careful with the
|
|
buffers. (This is the case in any typefind maneuver, but more so when one really can't rewind the
|
|
source without consequences) The autoplug system attaches a special pseudo-element to mpeg2parse's new
|
|
output pad, and attaches the typefind element to the end of that. The pseudo-element takes the buffer,
|
|
stores it, and passes a copy off to the attached element, in this case typefind. This repeats until
|
|
typefind has determined the type, at which point the typefind is removed, and the newly found element is
|
|
attached instead.
|
|
|
|
The pseudo-element is 'rewound' and the stored buffers flow out and into the newly attached element.
|
|
When the cache of buffers is gone, a signal fires and the autoplug system removes the pseudo-element and
|
|
reconnects the pipeline.
|
|
|
|
In this case, the typefind function will find the following:
|
|
|
|
mpeg2parse_video_src_caps = {
|
|
"video/mpeg",
|
|
"mpegversion", GST_CAPS_INT (2),
|
|
"systemstream", GST_CAPS_BOOLEAN (FALSE),
|
|
"bitrate", GST_CAPS_INT (36864),
|
|
"width", GST_CAPS_INT (720),
|
|
"height", GST_CAPS_INT (480),
|
|
"framerate", GST_CAPS_FLOAT (29.97002997),
|
|
"chromaformat", GST_CAPS_INT (1), [GST_CAPS_STRING ("4:2:0") ?]
|
|
NULL
|
|
};
|
|
|
|
Back to the plugin registry, we find our only choice is mpeg2dec, which has input caps of:
|
|
|
|
static GstCapsFactory mpeg2dec_sink_caps = {
|
|
"video/mpeg",
|
|
"mpegversion", GST_CAPS_RANGE (1,2),
|
|
"systemstream", GST_CAPS_BOOLEAN (FALSE),
|
|
NULL
|
|
};
|
|
|
|
Once again it just so happens that we really didn't need to do the typefind at all. But it can't hurt
|
|
unless the typefind is slow and painful, which we can guess won't be the case since the choices are
|
|
rather limited by the fact that there's already a MIME type attached, meaning we can drastically reduce
|
|
the number of typefind functions we try (down to one, actually).
|
|
|
|
However, since we *have* run the typefind, upon attachment of the input pad of mpeg2dec, the output pad
|
|
looks like the following:
|
|
|
|
mpeg2dec_src_caps = {
|
|
"video/raw",
|
|
"fourcc", GST_CAPS_LIST (
|
|
GST_CAPS_FOURCC ("YV12"), [identical...]
|
|
GST_CAPS_FOURCC ("IYUV"),
|
|
GST_CAPS_FOURCC ("I420"),
|
|
),
|
|
"width", GST_CAPS_INT (720),
|
|
"height", GST_CAPS_INT (480),
|
|
"framerate", GST_CAPS_FLOAT (29.97002997),
|
|
NULL
|
|
};
|
|
|
|
Currently only videosink supports the output of video/raw. It claims a list of FOURCCs but nothing
|
|
more:
|
|
|
|
static GstCapsFactory videosink_sink_caps = {
|
|
"video/raw",
|
|
"fourcc", GST_CAP_LIST ( GST_CAPS_FOURCC ("YV12"),
|
|
GST_CAPS_FOURCC ("IYUV"), GST_CAPS_FOURCC ("I420"),
|
|
GST_CAPS_FOURCC ("YUY2"), GST_CAPS_FOURCC ("UYVY"),
|
|
[ etc... ],
|
|
),
|
|
NULL
|
|
};
|
|
|
|
When instantiated, we potentially have the same problem as with the audiosink: we don't necessarily know
|
|
which hardware output to use. Somehow we have to solve the problem of setting some element arguments
|
|
before we can get useful information out of them as to the properties. In this case anyway, if the
|
|
videosink were to find only one output possibility, it would trim the list of FOURCCs it can deal with
|
|
to what the hardware can handle, as well as add further properties:
|
|
|
|
videosink_sink_caps = {
|
|
"video/raw",
|
|
"fourcc", GST_CAPS_LIST (GST_CAPS_FOURCC ("YV12"),
|
|
GST_CAPS_FOURCC ("YUY2"),
|
|
),
|
|
"width", GST_CAPS_INT_RANGE (4,1020),
|
|
"height", GST_CAPS_INT_RANGE (4,1020),
|
|
NULL
|
|
};
|
|
|
|
We can now connect the mpeg2dec output to the videosink, and we now have displaying video.
|
|
|
|
. . . .
|