mirror of
https://gitlab.freedesktop.org/gstreamer/gstreamer.git
synced 2024-11-18 15:51:11 +00:00
567 lines
24 KiB
XML
567 lines
24 KiB
XML
<chapter id="chapter-elements" xreflabel="Elements">
|
|
<title>Elements</title>
|
|
<para>
|
|
The most important object in &GStreamer; for the application programmer
|
|
is the <ulink type="http"
|
|
url="&URLAPI;GstElement.html"><classname>GstElement</classname></ulink>
|
|
object. An element is the basic building block for a media pipeline. All
|
|
the different high-level components you will use are derived from
|
|
<classname>GstElement</classname>. Every decoder, encoder, demuxer, video
|
|
or audio output is in fact a <classname>GstElement</classname>
|
|
</para>
|
|
|
|
<sect1 id="section-elements-design" xreflabel="What are elements?">
|
|
<title>What are elements?</title>
|
|
<para>
|
|
For the application programmer, elements are best visualized as black
|
|
boxes. On the one end, you might put something in, the element does
|
|
something with it and something else comes out at the other side. For
|
|
a decoder element, for example, you'd put in encoded data, and the
|
|
element would output decoded data. In the next chapter (see <xref
|
|
linkend="chapter-pads"/>), you will learn more about data input and
|
|
output in elements, and how you can set that up in your application.
|
|
</para>
|
|
|
|
<sect2 id="section-elements-src">
|
|
<title>Source elements</title>
|
|
<para>
|
|
Source elements generate data for use by a pipeline, for example
|
|
reading from disk or from a sound card. <xref
|
|
linkend="section-element-srcimg"/> shows how we will visualise
|
|
a source element. We always draw a source pad to the right of
|
|
the element.
|
|
</para>
|
|
<figure float="1" id="section-element-srcimg">
|
|
<title>Visualisation of a source element</title>
|
|
<mediaobject>
|
|
<imageobject>
|
|
<imagedata scale="75" fileref="images/src-element.ℑ"
|
|
format="&IMAGE;"/>
|
|
</imageobject>
|
|
</mediaobject>
|
|
</figure>
|
|
<para>
|
|
Source elements do not accept data, they only generate data. You can
|
|
see this in the figure because it only has a source pad (on the
|
|
right). A source pad can only generate data.
|
|
</para>
|
|
</sect2>
|
|
|
|
<sect2 id="section-elements-filter">
|
|
<title>Filters, convertors, demuxers, muxers and codecs</title>
|
|
<para>
|
|
Filters and filter-like elements have both input and outputs pads.
|
|
They operate on data that they receive on their input (sink) pads,
|
|
and will provide data on their output (source) pads. Examples of
|
|
such elements are a volume element (filter), a video scaler
|
|
(convertor), an Ogg demuxer or a Vorbis decoder.
|
|
</para>
|
|
<para>
|
|
Filter-like elements can have any number of source or sink pads. A
|
|
video demuxer, for example, would have one sink pad and several
|
|
(1-N) source pads, one for each elementary stream contained in the
|
|
container format. Decoders, on the other hand, will only have one
|
|
source and sink pads.
|
|
</para>
|
|
<figure float="1" id="section-element-filterimg">
|
|
<title>Visualisation of a filter element</title>
|
|
<mediaobject>
|
|
<imageobject>
|
|
<imagedata scale="75" fileref="images/filter-element.ℑ"
|
|
format="&IMAGE;"/>
|
|
</imageobject>
|
|
</mediaobject>
|
|
</figure>
|
|
<para>
|
|
<xref linkend="section-element-filterimg"/> shows how we will
|
|
visualise a filter-like element. This specific element has one source
|
|
and one sink element. Sink pads, receiving input data, are depicted
|
|
at the left of the element; source pads are still on the right.
|
|
</para>
|
|
<figure float="1" id="section-element-multifilterimg">
|
|
<title>Visualisation of a filter element with
|
|
more than one output pad</title>
|
|
<mediaobject>
|
|
<imageobject>
|
|
<imagedata scale="75" fileref="images/filter-element-multi.ℑ"
|
|
format="&IMAGE;" />
|
|
</imageobject>
|
|
</mediaobject>
|
|
</figure>
|
|
<para>
|
|
<xref linkend="section-element-multifilterimg"/> shows another
|
|
filter-like element, this one having more than one output (source)
|
|
pad. An example of one such element could, for example, be an Ogg
|
|
demuxer for an Ogg stream containing both audio and video. One
|
|
source pad will contain the elementary video stream, another will
|
|
contain the elementary audio stream. Demuxers will generally fire
|
|
signals when a new pad is created. The application programmer can
|
|
then handle the new elementary stream in the signal handler.
|
|
</para>
|
|
</sect2>
|
|
|
|
<sect2 id="section-elements-sink">
|
|
<title>Sink elements</title>
|
|
<para>
|
|
Sink elements are end points in a media pipeline. They accept
|
|
data but do not produce anything. Disk writing, soundcard playback,
|
|
and video output would all be implemented by sink elements.
|
|
<xref linkend="section-element-sinkimg"/> shows a sink element.
|
|
</para>
|
|
<figure float="1" id="section-element-sinkimg">
|
|
<title>Visualisation of a sink element</title>
|
|
<mediaobject>
|
|
<imageobject>
|
|
<imagedata scale="75" fileref="images/sink-element.ℑ"
|
|
format="&IMAGE;" />
|
|
</imageobject>
|
|
</mediaobject>
|
|
</figure>
|
|
</sect2>
|
|
</sect1>
|
|
|
|
<sect1 id="section-elements-create">
|
|
<title>Creating a <classname>GstElement</classname></title>
|
|
<para>
|
|
The simplest way to create an element is to use <ulink type="http"
|
|
url="&URLAPI;GstElementFactory.html#gst-element-factory-make"><function>gst_element_factory_make
|
|
()</function></ulink>. This function takes a factory name and an
|
|
element name for the newly created element. The name of the element
|
|
is something you can use later on to look up the element in a bin,
|
|
for example. The name will also be used in debug output. You can
|
|
pass <symbol>NULL</symbol> as the name argument to get a unique,
|
|
default name.
|
|
</para>
|
|
<para>
|
|
When you don't need the element anymore, you need to unref it using
|
|
<ulink type="http"
|
|
url="&URLAPI;GstObject.html#gst-object-unref"><function>gst_object_unref
|
|
()</function></ulink>. This decreases the reference count for the
|
|
element by 1. An element has a refcount of 1 when it gets created.
|
|
An element gets destroyed completely when the refcount is decreased
|
|
to 0.
|
|
</para>
|
|
<para>
|
|
The following example &EXAFOOT; shows how to create an element named
|
|
<emphasis>source</emphasis> from the element factory named
|
|
<emphasis>fakesrc</emphasis>. It checks if the creation succeeded.
|
|
After checking, it unrefs the element.
|
|
</para>
|
|
<programlisting><!-- example-begin elementmake.c --><![CDATA[
|
|
#include <gst/gst.h>
|
|
|
|
int
|
|
main (int argc,
|
|
char *argv[])
|
|
{
|
|
GstElement *element;
|
|
|
|
/* init GStreamer */
|
|
gst_init (&argc, &argv);
|
|
|
|
/* create element */
|
|
element = gst_element_factory_make ("fakesrc", "source");
|
|
if (!element) {
|
|
g_print ("Failed to create element of type 'fakesrc'\n");
|
|
return -1;
|
|
}
|
|
|
|
gst_object_unref (GST_OBJECT (element));
|
|
|
|
return 0;
|
|
}
|
|
]]><!-- example-end elementmake.c --></programlisting>
|
|
<para>
|
|
<function>gst_element_factory_make</function> is actually a shorthand
|
|
for a combination of two functions. A <ulink type="http"
|
|
url="&URLAPI;GstElement.html"><classname>GstElement</classname></ulink>
|
|
object is created from a factory. To create the element, you have to
|
|
get access to a <ulink type="http"
|
|
url="&URLAPI;GstElementFactory.html"><classname>GstElementFactory</classname></ulink>
|
|
object using a unique factory name. This is done with <ulink type="http"
|
|
url="&URLAPI;GstElementFactory.html#gst-element-factory-find"><function>gst_element_factory_find
|
|
()</function></ulink>.
|
|
</para>
|
|
<para>
|
|
The following code fragment is used to get a factory that can be used
|
|
to create the <emphasis>fakesrc</emphasis> element, a fake data source.
|
|
The function <ulink type="http"
|
|
url="&URLAPI;GstElementFactory.html#gst-element-factory-create"><function>gst_element_factory_create
|
|
()</function></ulink> will use the element factory to create an
|
|
element with the given name.
|
|
</para>
|
|
<programlisting><!-- example-begin elementcreate.c --><![CDATA[
|
|
#include <gst/gst.h>
|
|
|
|
int
|
|
main (int argc,
|
|
char *argv[])
|
|
{
|
|
GstElementFactory *factory;
|
|
GstElement * element;
|
|
|
|
/* init GStreamer */
|
|
gst_init (&argc, &argv);
|
|
|
|
/* create element, method #2 */
|
|
factory = gst_element_factory_find ("fakesrc");
|
|
if (!factory) {
|
|
g_print ("Failed to find factory of type 'fakesrc'\n");
|
|
return -1;
|
|
}
|
|
element = gst_element_factory_create (factory, "source");
|
|
if (!element) {
|
|
g_print ("Failed to create element, even though its factory exists!\n");
|
|
return -1;
|
|
}
|
|
|
|
gst_object_unref (GST_OBJECT (element));
|
|
|
|
return 0;
|
|
}
|
|
]]><!-- example-end elementcreate.c --></programlisting>
|
|
</sect1>
|
|
|
|
<sect1 id="section-elements-properties">
|
|
<title>Using an element as a <classname>GObject</classname></title>
|
|
<para>
|
|
A <ulink type="http"
|
|
url="&URLAPI;GstElement.html"><classname>GstElement</classname></ulink>
|
|
can have several properties which are implemented using standard
|
|
<classname>GObject</classname> properties. The usual
|
|
<classname>GObject</classname> methods to query, set and get
|
|
property values and <classname>GParamSpecs</classname> are
|
|
therefore supported.
|
|
</para>
|
|
<para>
|
|
Every <classname>GstElement</classname> inherits at least one
|
|
property from its parent <classname>GstObject</classname>: the
|
|
"name" property. This is the name you provide to the functions
|
|
<function>gst_element_factory_make ()</function> or
|
|
<function>gst_element_factory_create ()</function>. You can get
|
|
and set this property using the functions
|
|
<function>gst_object_set_name</function> and
|
|
<function>gst_object_get_name</function> or use the
|
|
<classname>GObject</classname> property mechanism as shown below.
|
|
</para>
|
|
<programlisting><!-- example-begin elementget.c --><![CDATA[
|
|
#include <gst/gst.h>
|
|
|
|
int
|
|
main (int argc,
|
|
char *argv[])
|
|
{
|
|
GstElement *element;
|
|
gchar *name;
|
|
|
|
/* init GStreamer */
|
|
gst_init (&argc, &argv);
|
|
|
|
/* create element */
|
|
element = gst_element_factory_make ("fakesrc", "source");
|
|
|
|
/* get name */
|
|
g_object_get (G_OBJECT (element), "name", &name, NULL);
|
|
g_print ("The name of the element is '%s'.\n", name);
|
|
g_free (name);
|
|
|
|
gst_object_unref (GST_OBJECT (element));
|
|
|
|
return 0;
|
|
}
|
|
]]><!-- example-end elementget.c --></programlisting>
|
|
<para>
|
|
Most plugins provide additional properties to provide more information
|
|
about their configuration or to configure the element.
|
|
<command>gst-inspect</command> is a useful tool to query the properties
|
|
of a particular element, it will also use property introspection to give
|
|
a short explanation about the function of the property and about the
|
|
parameter types and ranges it supports. See
|
|
<xref linkend="section-applications-inspect"/>
|
|
in the appendix for details about <command>gst-inspect</command>.
|
|
</para>
|
|
<para>
|
|
For more information about <classname>GObject</classname>
|
|
properties we recommend you read the <ulink
|
|
url="http://developer.gnome.org/gobject/stable/rn01.html"
|
|
type="http">GObject manual</ulink> and an introduction to <ulink
|
|
url="http://developer.gnome.org/gobject/stable/pt01.html" type="http">
|
|
The Glib Object system</ulink>.
|
|
</para>
|
|
<para>
|
|
A <ulink type="http" url="&URLAPI;GstElement.html">
|
|
<classname>GstElement</classname></ulink> also provides various
|
|
<classname>GObject</classname> signals that can be used as a flexible
|
|
callback mechanism. Here, too, you can use <command>gst-inspect</command>
|
|
to see which signals a specific element supports. Together, signals
|
|
and properties are the most basic way in which elements and
|
|
applications interact.
|
|
</para>
|
|
</sect1>
|
|
|
|
<sect1 id="section-elements-factories">
|
|
<title>More about element factories</title>
|
|
<para>
|
|
In the previous section, we briefly introduced the <ulink type="http"
|
|
url="&URLAPI;GstElementFactory.html"><classname>GstElementFactory</classname></ulink>
|
|
object already as a way to create instances of an element. Element
|
|
factories, however, are much more than just that. Element factories
|
|
are the basic types retrieved from the &GStreamer; registry, they
|
|
describe all plugins and elements that &GStreamer; can create. This
|
|
means that element factories are useful for automated element
|
|
instancing, such as what autopluggers do, and for creating lists
|
|
of available elements.
|
|
</para>
|
|
|
|
<sect2 id="section-elements-factories-details">
|
|
<title>Getting information about an element using a factory</title>
|
|
<para>
|
|
Tools like <command>gst-inspect</command> will provide some generic
|
|
information about an element, such as the person that wrote the
|
|
plugin, a descriptive name (and a shortname), a rank and a category.
|
|
The category can be used to get the type of the element that can
|
|
be created using this element factory. Examples of categories include
|
|
<classname>Codec/Decoder/Video</classname> (video decoder),
|
|
<classname>Codec/Encoder/Video</classname> (video encoder),
|
|
<classname>Source/Video</classname> (a video generator),
|
|
<classname>Sink/Video</classname> (a video output), and all these
|
|
exist for audio as well, of course. Then, there's also
|
|
<classname>Codec/Demuxer</classname> and
|
|
<classname>Codec/Muxer</classname> and a whole lot more.
|
|
<command>gst-inspect</command> will give a list of all factories, and
|
|
<command>gst-inspect <factory-name></command> will list all
|
|
of the above information, and a lot more.
|
|
</para>
|
|
<programlisting><!-- example-begin elementfactory.c --><![CDATA[
|
|
#include <gst/gst.h>
|
|
|
|
int
|
|
main (int argc,
|
|
char *argv[])
|
|
{
|
|
GstElementFactory *factory;
|
|
|
|
/* init GStreamer */
|
|
gst_init (&argc, &argv);
|
|
|
|
/* get factory */
|
|
factory = gst_element_factory_find ("fakesrc");
|
|
if (!factory) {
|
|
g_print ("You don't have the 'fakesrc' element installed!\n");
|
|
return -1;
|
|
}
|
|
|
|
/* display information */
|
|
g_print ("The '%s' element is a member of the category %s.\n"
|
|
"Description: %s\n",
|
|
gst_plugin_feature_get_name (GST_PLUGIN_FEATURE (factory)),
|
|
gst_element_factory_get_metadata (factory, GST_ELEMENT_METADATA_KLASS),
|
|
gst_element_factory_get_metadata (factory, GST_ELEMENT_METADATA_DESCRIPTION));
|
|
|
|
return 0;
|
|
}
|
|
]]><!-- example-end elementfactory.c --></programlisting>
|
|
<para>
|
|
You can use <function>gst_registry_pool_feature_list (GST_TYPE_ELEMENT_FACTORY)</function>
|
|
to get a list of all the element factories that &GStreamer; knows
|
|
about.
|
|
</para>
|
|
</sect2>
|
|
|
|
<sect2 id="section-elements-factories-padtemplates">
|
|
<title>Finding out what pads an element can contain</title>
|
|
<para>
|
|
Perhaps the most powerful feature of element factories is that
|
|
they contain a full description of the pads that the element
|
|
can generate, and the capabilities of those pads (in layman words:
|
|
what types of media can stream over those pads), without actually
|
|
having to load those plugins into memory. This can be used
|
|
to provide a codec selection list for encoders, or it can be used
|
|
for autoplugging purposes for media players. All current
|
|
&GStreamer;-based media players and autopluggers work this way.
|
|
We'll look closer at these features as we learn about
|
|
<classname>GstPad</classname> and <classname>GstCaps</classname>
|
|
in the next chapter: <xref linkend="chapter-pads"/>
|
|
</para>
|
|
</sect2>
|
|
</sect1>
|
|
|
|
<sect1 id="section-elements-link" xreflabel="Linking elements">
|
|
<title>Linking elements</title>
|
|
<para>
|
|
By linking a source element with zero or more filter-like
|
|
elements and finally a sink element, you set up a media
|
|
pipeline. Data will flow through the elements. This is the
|
|
basic concept of media handling in &GStreamer;.
|
|
</para>
|
|
<figure float="1" id="section-link">
|
|
<title>Visualisation of three linked elements</title>
|
|
<mediaobject>
|
|
<imageobject>
|
|
<imagedata scale="75" fileref="images/linked-elements.ℑ"
|
|
format="&IMAGE;"/>
|
|
</imageobject>
|
|
</mediaobject>
|
|
</figure>
|
|
<para>
|
|
By linking these three elements, we have created a very simple
|
|
chain of elements. The effect of this will be that the output of
|
|
the source element (<quote>element1</quote>) will be used as input
|
|
for the filter-like element (<quote>element2</quote>). The
|
|
filter-like element will do something with the data and send the
|
|
result to the final sink element (<quote>element3</quote>).
|
|
</para>
|
|
<para>
|
|
Imagine the above graph as a simple Ogg/Vorbis audio decoder. The
|
|
source is a disk source which reads the file from disc. The second
|
|
element is a Ogg/Vorbis audio decoder. The sink element is your
|
|
soundcard, playing back the decoded audio data. We will use this
|
|
simple graph to construct an Ogg/Vorbis player later in this manual.
|
|
</para>
|
|
<para>
|
|
In code, the above graph is written like this:
|
|
</para>
|
|
<programlisting><!-- example-begin elementlink.c a -->
|
|
#include <gst/gst.h>
|
|
|
|
int
|
|
main (int argc,
|
|
char *argv[])
|
|
{
|
|
GstElement *pipeline;
|
|
GstElement *source, *filter, *sink;
|
|
|
|
/* init */
|
|
gst_init (&argc, &argv);
|
|
|
|
/* create pipeline */
|
|
pipeline = gst_pipeline_new ("my-pipeline");
|
|
|
|
/* create elements */
|
|
source = gst_element_factory_make ("fakesrc", "source");
|
|
filter = gst_element_factory_make ("identity", "filter");
|
|
sink = gst_element_factory_make ("fakesink", "sink");
|
|
|
|
/* must add elements to pipeline before linking them */
|
|
gst_bin_add_many (GST_BIN (pipeline), source, filter, sink, NULL);
|
|
|
|
/* link */
|
|
if (!gst_element_link_many (source, filter, sink, NULL)) {
|
|
g_warning ("Failed to link elements!");
|
|
}
|
|
<!-- example-end elementlink.c a -->
|
|
[..]<!-- example-begin elementlink.c b --><!--
|
|
return 0;
|
|
--><!-- example-end elementlink.c b -->
|
|
<!-- example-begin elementlink.c c -->
|
|
}
|
|
<!-- example-end elementlink.c c --></programlisting>
|
|
<para>
|
|
For more specific behaviour, there are also the functions
|
|
<function>gst_element_link ()</function> and
|
|
<function>gst_element_link_pads ()</function>. You can also obtain
|
|
references to individual pads and link those using various
|
|
<function>gst_pad_link_* ()</function> functions. See the API
|
|
references for more details.
|
|
</para>
|
|
<para>
|
|
Important: you must add elements to a bin or pipeline
|
|
<emphasis>before</emphasis> linking them, since adding an element to
|
|
a bin will disconnect any already existing links. Also, you cannot
|
|
directly link elements that are not in the same bin or pipeline; if
|
|
you want to link elements or pads at different hierarchy levels, you
|
|
will need to use ghost pads (more about ghost pads later,
|
|
see <xref linkend="section-pads-ghost"/>).
|
|
</para>
|
|
</sect1>
|
|
|
|
<sect1 id="section-elements-states">
|
|
<title>Element States</title>
|
|
<para>
|
|
After being created, an element will not actually perform any actions
|
|
yet. You need to change elements state to make it do something.
|
|
&GStreamer; knows four element states, each with a very specific
|
|
meaning. Those four states are:
|
|
</para>
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>
|
|
<classname>GST_STATE_NULL</classname>: this is the default state.
|
|
No resources are allocated in this state, so, transitioning to it
|
|
will free all resources. The element must be in this state when
|
|
its refcount reaches 0 and it is freed.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
<classname>GST_STATE_READY</classname>: in the ready state, an
|
|
element has allocated all of its global resources, that is,
|
|
resources that can be kept within streams. You can think about
|
|
opening devices, allocating buffers and so on. However, the
|
|
stream is not opened in this state, so the stream positions is
|
|
automatically zero. If a stream was previously opened, it should
|
|
be closed in this state, and position, properties and such should
|
|
be reset.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
<classname>GST_STATE_PAUSED</classname>: in this state, an
|
|
element has opened the stream, but is not actively processing
|
|
it. An element is allowed to modify a stream's position, read
|
|
and process data and such to prepare for playback as soon as
|
|
state is changed to PLAYING, but it is <emphasis>not</emphasis>
|
|
allowed to play the data which would make the clock run.
|
|
In summary, PAUSED is the same as PLAYING but without a running
|
|
clock.
|
|
</para>
|
|
<para>
|
|
Elements going into the PAUSED state should prepare themselves
|
|
for moving over to the PLAYING state as soon as possible. Video
|
|
or audio outputs would, for example, wait for data to arrive and
|
|
queue it so they can play it right after the state change. Also,
|
|
video sinks can already play the first frame (since this does
|
|
not affect the clock yet). Autopluggers could use this same
|
|
state transition to already plug together a pipeline. Most other
|
|
elements, such as codecs or filters, do not need to explicitly
|
|
do anything in this state, however.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
<classname>GST_STATE_PLAYING</classname>: in the PLAYING state,
|
|
an element does exactly the same as in the PAUSED state, except
|
|
that the clock now runs.
|
|
</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
<para>
|
|
You can change the state of an element using the function
|
|
<function>gst_element_set_state ()</function>. If you set an element
|
|
to another state, &GStreamer; will internally traverse all intermediate
|
|
states. So if you set an element from NULL to PLAYING, &GStreamer;
|
|
will internally set the element to READY and PAUSED in between.
|
|
</para>
|
|
<para>
|
|
When moved to <classname>GST_STATE_PLAYING</classname>, pipelines
|
|
will process data automatically. They do not need to be iterated in
|
|
any form. Internally, &GStreamer; will start threads that take this
|
|
task on to them. &GStreamer; will also take care of switching
|
|
messages from the pipeline's thread into the application's own
|
|
thread, by using a <ulink type="http"
|
|
url="&URLAPI;GstBus.html"><classname>GstBus</classname></ulink>. See
|
|
<xref linkend="chapter-bus"/> for details.
|
|
</para>
|
|
<para>
|
|
When you set a bin or pipeline to a certain target state, it will usually
|
|
propagate the state change to all elements within the bin or pipeline
|
|
automatically, so it's usually only necessary to set the state of the
|
|
top-level pipeline to start up the pipeline or shut it down. However,
|
|
when adding elements dynamically to an already-running pipeline, e.g.
|
|
from within a "pad-added" signal callback, you
|
|
need to set it to the desired target state yourself using
|
|
<function>gst_element_set_state ()</function> or
|
|
<function>gst_element_sync_state_with_parent ()</function>.
|
|
</para>
|
|
</sect1>
|
|
</chapter>
|