<chapter id="chapter-elements" xreflabel="Elements">
<title>Elements</title>
<para>
The most important object in &GStreamer; for the application programmer
is the <ulink type="http"
url="../../gstreamer/html/GstElement.html"><classname>GstElement</classname></ulink>
object. An element is the basic building block for a media pipeline. All
the different high-level components you will use are derived from
<classname>GstElement</classname>. Every decoder, encoder, demuxer, video
or audio output is in fact a <classname>GstElement</classname>
</para>
<sect1 id="section-elements-design" xreflabel="What are elements?">
<title>What are elements?</title>
<para>
For the application programmer, elements are best visualized as black
boxes. On the one end, you might put something in, the element does
something with it and something else comes out at the other side. For
a decoder element, ifor example, you'd put in encoded data, and the
element would output decoded data. In the next chapter (see <xref
linkend="chapter-pads"/>), you will learn more about data input and
output in elements, and how you can set that up in your application.
</para>
<sect2 id="section-elements-src">
<title>Source elements</title>
<para>
Source elements generate data for use by a pipeline, for example
reading from disk or from a sound card. <xref
linkend="section-element-srcimg"/> shows how we will visualise
a source element. We always draw a source pad to the right of
the element.
</para>
<figure float="1" id="section-element-srcimg">
<title>Visualisation of a source element</title>
<mediaobject>
<imageobject>
<imagedata fileref="images/src-element.ℑ"
format="&IMAGE;"/>
</imageobject>
</mediaobject>
</figure>
<para>
Source elements do not accept data, they only generate data. You can
see this in the figure because it only has a source pad (on the
right). A source pad can only generate data.
</para>
</sect2>
<sect2 id="section-elements-filter">
<title>Filters, convertors, demuxers, muxers and codecs</title>
<para>
Filters and filter-like elements have both input and outputs pads.
They operate on data that they receive on their input (sink) pads,
and will provide data on their output (source) pads. Examples of
such elements are a volume element (filter), a video scaler
(convertor), an Ogg demuxer or a Vorbis decoder.
</para>
<para>
Filter-like elements can have any number of source or sink pads. A
video demuxer, for example, would have one sink pad and several
(1-N) source pads, one for each elementary stream contained in the
container format. Decoders, on the other hand, will only have one
source and sink pads.
</para>
<figure float="1" id="section-element-filterimg">
<title>Visualisation of a filter element</title>
<mediaobject>
<imageobject>
<imagedata fileref="images/filter-element.ℑ"
format="&IMAGE;"/>
</imageobject>
</mediaobject>
</figure>
<para>
<xref linkend="section-element-filterimg"/> shows how we will
visualise a filter-like element. This specific element has one source
and one sink element. Sink pads, receiving input data, are depicted
at the left of the element; source pads are still on the right.
</para>
<figure float="1" id="section-element-multifilterimg">
<title>Visualisation of a filter element with
more than one output pad</title>
<mediaobject>
<imageobject>
<imagedata fileref="images/filter-element-multi.ℑ"
format="&IMAGE;" />
</imageobject>
</mediaobject>
</figure>
<para>
<xref linkend="section-element-multifilterimg"/> shows another
filter-like element, this one having more than one output (source)
pad. An example of one such element could, for example, be an Ogg
demuxer for an Ogg stream containing both audio and video. One
source pad will contain the elementary video stream, another will
contain the elementary audio stream. Demuxers will generally fire
signals when a new pad is created. The application programmer can
then handle the new elementary stream in the signal handler.
</para>
</sect2>
<sect2 id="section-elements-sink">
<title>Sink elements</title>
<para>
Sink elements are end points in a media pipeline. They accept
data but do not produce anything. Disk writing, soundcard playback,
and video output would all be implemented by sink elements.
<xref linkend="section-element-sinkimg"/> shows a sink element.
</para>
<figure float="1" id="section-element-sinkimg">
<title>Visualisation of a sink element</title>
<mediaobject>
<imageobject>
<imagedata fileref="images/sink-element.ℑ"
format="&IMAGE;" />
</imageobject>
</mediaobject>
</figure>
</sect2>
</sect1>
<sect1 id="section-elements-create">
<title>Creating a <classname>GstElement</classname></title>
<para>
The simplest way to create an element is to use <ulink type="http"
url="&URLAPI;GstElementFactory.html#gst-element-factory-make"><function>gst_element_factory_make
()</function></ulink>. This function takes a factory name and an
element name for the newly created element. The name of the element
is something you can use later on to look up the element in a bin,
for example. The name will also be used in debug output. You can
pass <symbol>NULL</symbol> as the name argument to get a unique,
default name.
</para>
<para>
When you don't need the element anymore, you need to unref it using
<ulink type="http"
url="&URLAPI;GstObject.html#gst-object-unref"><function>gst_object_unref
()</function></ulink>. This decreases the reference count for the
element by 1. An element has a refcount of 1 when it gets created.
An element gets destroyed completely when the refcount is decreased
to 0.
</para>
<para>
The following example &EXAFOOT; shows how to create an element named
<emphasis>source</emphasis> from the element factory named
<emphasis>fakesrc</emphasis>. It checks if the creation succeeded.
After checking, it unrefs the element.
</para>
<programlisting><!-- example-begin elementmake.c --><![CDATA[
#include <gst/gst.h>
int
main (int argc,
char *argv[])
{
GstElement *element;
/* init GStreamer */
gst_init (&argc, &argv);
/* create element */
element = gst_element_factory_make ("fakesrc", "source");
if (!element) {
g_print ("Failed to create element of type 'fakesrc'\n");
return -1;
}
gst_object_unref (GST_OBJECT (element));
return 0;
}
]]><!-- example-end elementmake.c --></programlisting>
<para>
<function>gst_element_factory_make</function> is actually a shorthand
for a combination of two functions. A <ulink type="http"
url="&URLAPI;GstElement.html"><classname>GstElement</classname></ulink>
object is created from a factory. To create the element, you have to
get access to a <ulink type="http"
url="&URLAPI;GstElementFactory.html"><classname>GstElementFactory</classname></ulink>
object using a unique factory name. This is done with <ulink type="http"
url="&URLAPI;GstElementFactory.html#gst-element-factory-find"><function>gst_element_factory_find
()</function></ulink>.
</para>
<para>
The following code fragment is used to get a factory that can be used
to create the <emphasis>fakesrc</emphasis> element, a fake data source.
The function <ulink type="http"
url="&URLAPI;GstElementFactory.html#gst-element-factory-create"><function>gst_element_factory_create
()</function></ulink> will use the element factory to create an
element with the given name.
</para>
<programlisting><!-- example-begin elementcreate.c --><![CDATA[
#include <gst/gst.h>
int
main (int argc,
char *argv[])
{
GstElementFactory *factory;
GstElement * element;
/* init GStreamer */
gst_init (&argc, &argv);
/* create element, method #2 */
factory = gst_element_factory_find ("fakesrc");
if (!factory) {
g_print ("Failed to find factory of type 'fakesrc'\n");
return -1;
}
element = gst_element_factory_create (factory, "source");
if (!element) {
g_print ("Failed to create element, even though its factory exists!\n");
return -1;
}
gst_object_unref (GST_OBJECT (element));
return 0;
}
]]><!-- example-end elementcreate.c --></programlisting>
</sect1>
<sect1 id="section-elements-properties">
<title>Using an element as a <classname>GObject</classname></title>
<para>
A <ulink type="http"
url="&URLAPI;GstElement.html"><classname>GstElement</classname></ulink>
can have several properties which are implemented using standard
<classname>GObject</classname> properties. The usual
<classname>GObject</classname> methods to query, set and get
property values and <classname>GParamSpecs</classname> are
therefore supported.
</para>
<para>
Every <classname>GstElement</classname> inherits at least one
property from its parent <classname>GstObject</classname>: the
"name" property. This is the name you provide to the functions
<function>gst_element_factory_make ()</function> or
<function>gst_element_factory_create ()</function>. You can get
and set this property using the functions
<function>gst_object_set_name</function> and
<function>gst_object_get_name</function> or use the
<classname>GObject</classname> property mechanism as shown below.
</para>
<programlisting><!-- example-begin elementget.c --><![CDATA[
#include <gst/gst.h>
int
main (int argc,
char *argv[])
{
GstElement *element;
gchar *name;
/* init GStreamer */
gst_init (&argc, &argv);
/* create element */
element = gst_element_factory_make ("fakesrc", "source");
/* get name */
g_object_get (G_OBJECT (element), "name", &name, NULL);
g_print ("The name of the element is '%s'.\n", name);
g_free (name);
gst_object_unref (GST_OBJECT (element));
return 0;
}
]]><!-- example-end elementget.c --></programlisting>
<para>
Most plugins provide additional properties to provide more information
about their configuration or to configure the element.
<command>gst-inspect</command> is a useful tool to query the properties
of a particular element, it will also use property introspection to give
a short explanation about the function of the property and about the
parameter types and ranges it supports. See the appendix for details
about <command>gst-inspect</command>.
</para>
<para>
For more information about <classname>GObject</classname>
properties we recommend you read the <ulink
url="http://developer.gnome.org/doc/API/2.0/gobject/index.html"
type="http">GObject manual</ulink> and an introduction to <ulink
url="http://developer.gnome.org/doc/API/2.0/gobject/pr01.html" type="http">
The Glib Object system</ulink>.
</para>
<para>
A <ulink type="http" url="&URLAPI;gstreamer/html/GstElementFactory.html">
<classname>GstElement</classname></ulink> also provides various
<classname>GObject</classname> signals that can be used as a flexible
callback mechanism. Here, too, you can use <command>gst-inspect</command>
to see which signals a specific elements supports. Together, signals
and properties are the most basic way in which elements and
applications interact.
</para>
</sect1>
<sect1 id="section-elements-factories">
<title>More about element factories</title>
<para>
In the previous section, we briefly introduced the <ulink type="http"
url="&URLAPI;GstElement.html"><classname>GstElementFactory</classname></ulink>
object already as a way to create instances of an element. Element
factories, however, are much more than just that. Element factories
are the basic types retrieved from the &GStreamer; registry, they
describe all plugins and elements that &GStreamer; can create. This
means that element factories are useful for automated element
instancing, such as what autopluggers do, and for creating lists
of available elements, such as what pipeline editing applications
(e.g. <ulink type="http"
url="http://gstreamer.freedesktop.org/modules/gst-editor.html">&GStreamer;
Editor</ulink>) do.
</para>
<sect2 id="section-elements-factories-details">
<title>Getting information about an element using a factory</title>
<para>
Tools like <command>gst-inspect</command> will provide some generic
information about an element, such as the person that wrote the
plugin, a descriptive name (and a shortname), a rank and a category.
The category can be used to get the type of the element that can
be created using this element factory. Examples of categories include
<classname>Codec/Decoder/Video</classname> (video decoder),
<classname>Codec/Encoder/Video</classname> (video encoder),
<classname>Source/Video</classname> (a video generator),
<classname>Sink/Video</classname> (a video output), and all these
exist for audio as well, of course. Then, there's also
<classname>Codec/Demuxer</classname> and
<classname>Codec/Muxer</classname> and a whole lot more.
<command>gst-inspect</command> will give a list of all factories, and
<command>gst-inspect <factory-name></command> will list all
of the above information, and a lot more.
</para>
<programlisting><!-- example-begin elementfactory.c --><![CDATA[
#include <gst/gst.h>
int
main (int argc,
char *argv[])
{
GstElementFactory *factory;
/* init GStreamer */
gst_init (&argc, &argv);
/* get factory */
factory = gst_element_factory_find ("fakesrc");
if (!factory) {
g_print ("You don't have the 'fakesrc' element installed!\n");
return -1;
}
/* display information */
g_print ("The '%s' element is a member of the category %s.\n"
"Description: %s\n",
gst_plugin_feature_get_name (GST_PLUGIN_FEATURE (factory)),
gst_element_factory_get_klass (factory),
gst_element_factory_get_description (factory));
return 0;
}
]]><!-- example-end elementfactory.c --></programlisting>
<para>
You can use <function>gst_registry_pool_feature_list (GST_TYPE_ELEMENT_FACTORY)</function>
to get a list of all the element factories that &GStreamer; knows
about.
</para>
</sect2>
<sect2 id="section-elements-factories-padtemplates">
<title>Finding out what pads an element can contain</title>
<para>
Perhaps the most powerful feature of element factories is that
they contain a full description of the pads that the element
can generate, and the capabilities of those pads (in layman words:
what types of media can stream over those pads), without actually
having to load those plugins into memory. This can be used
to provide a codec selection list for encoders, or it can be used
for autoplugging purposes for media players. All current
&GStreamer;-based media players and autopluggers work this way.
We'll look closer at these features as we learn about
<classname>GstPad</classname> and <classname>GstCaps</classname>
in the next chapter: <xref linkend="chapter-pads"/>
</para>
</sect2>
</sect1>
<sect1 id="section-elements-link" xreflabel="Linking elements">
<title>Linking elements</title>
<para>
By linking a source element with zero or more filter-like
elements and finally a sink element, you set up a media
pipeline. Data will flow through the elements. This is the
basic concept of media handling in &GStreamer;.
</para>
<figure float="1" id="section-link">
<title>Visualisation of three linked elements</title>
<mediaobject>
<imageobject>
<imagedata fileref="images/linked-elements.ℑ"
format="&IMAGE;"/>
</imageobject>
</mediaobject>
</figure>
<para>
By linking these three elements, we have created a very simple
chain of elements. The effect of this will be that the output of
the source element (<quote>element1</quote>) will be used as input
for the filter-like element (<quote>element2</quote>). The
filter-like element will do something with the data and send the
result to the final sink element (<quote>element3</quote>).
</para>
<para>
Imagine the above graph as a simple Ogg/Vorbis audio decoder. The
source is a disk source which reads the file from disc. The second
element is a Ogg/Vorbis audio decoder. The sink element is your
soundcard, playing back the decoded audio data. We will use this
simple graph to construct an Ogg/Vorbis player later in this manual.
</para>
<para>
In code, the above graph is written like this:
</para>
<programlisting><!-- example-begin elementlink.c a -->
#include <gst/gst.h>
int
main (int argc,
char *argv[])
{
GstElement *pipeline;
GstElement *source, *filter, *sink;
/* init */
gst_init (&argc, &argv);
/* create pipeline */
pipeline = gst_pipeline_new ("my-pipeline");
/* create elements */
source = gst_element_factory_make ("fakesrc", "source");
filter = gst_element_factory_make ("identity", "filter");
sink = gst_element_factory_make ("fakesink", "sink");
/* must add elements to pipeline before linking them */
gst_bin_add_many (GST_BIN (pipeline), source, filter, sink, NULL);
/* link */
if (!gst_element_link_many (source, filter, sink, NULL)) {
g_warning ("Failed to link elements!");
}
<!-- example-end elementlink.c a -->
[..]<!-- example-begin elementlink.c b --><!--
return 0;
--><!-- example-end elementlink.c b -->
<!-- example-begin elementlink.c c -->
}
<!-- example-end elementlink.c c --></programlisting>
<para>
For more specific behaviour, there are also the functions
<function>gst_element_link ()</function> and
<function>gst_element_link_pads ()</function>. You can also obtain
references to individual pads and link those using various
<function>gst_pad_link_* ()</function> functions. See the API
references for more details.
</para>
<para>
Important: you must add elements to a bin or pipeline
<emphasis>before</emphasis> linking them, since adding an element to
a bin will disconnect any already existing links. Also, you cannot
directly link elements that are not in the same bin or pipeline; if
you want to link elements or pads at different hierarchy levels, you
will need to use ghost pads (more about ghost pads later).
</para>
</sect1>
<sect1 id="section-elements-states">
<title>Element States</title>
<para>
After being created, an element will not actually perform any actions
yet. You need to change elements state to make it do something.
&GStreamer; knows four element states, each with a very specific
meaning. Those four states are:
</para>
<itemizedlist>
<listitem>
<para>
<classname>GST_STATE_NULL</classname>: this is the default state.
This state will deallocate all resources held by the element.
</para>
</listitem>
<listitem>
<para>
<classname>GST_STATE_READY</classname>: in the ready state, an
element has allocated all of its global resources, that is,
resources that can be kept within streams. You can think about
opening devices, allocating buffers and so on. However, the
stream is not opened in this state, so the stream positions is
automatically zero. If a stream was previously opened, it should
be closed in this state, and position, properties and such should
be reset.
</para>
</listitem>
<listitem>
<para>
<classname>GST_STATE_PAUSED</classname>: in this state, an
element has opened the stream, but is not actively processing
it. An element is allowed to modify a stream's position, read
and process data and such to prepare for playback as soon as
state is changed to PLAYING, but it is <emphasis>not</emphasis>
allowed to play the data which would make the clock run.
In summary, PAUSED is the same as PLAYING but without a running
clock.
</para>
<para>
Elements going into the PAUSED state should prepare themselves
for moving over to the PLAYING state as soon as possible. Video
or audio outputs would, for example, wait for data to arrive and
queue it so they can play it right after the state change. Also,
video sinks can already play the first frame (since this does
not affect the clock yet). Autopluggers could use this same
state transition to already plug together a pipeline. Most other
elements, such as codecs or filters, do not need to explicitely
do anything in this state, however.
</para>
</listitem>
<listitem>
<para>
<classname>GST_STATE_PLAYING</classname>: in the PLAYING state,
an element does exactly the same as in the PAUSED state, except
that the clock now runs.
</para>
</listitem>
</itemizedlist>
<para>
You can change the state of an element using the function
<function>gst_element_set_state ()</function>. If you set an element
to another state, &GStreamer; will internally traverse all intermediate
states. So if you set an element from NULL to PLAYING, &GStreamer;
will internally set the element to READY and PAUSED in between.
</para>
<para>
When moved to <classname>GST_STATE_PLAYING</classname>, pipelines
will process data automatically. They do not need to be iterated in
any form. Internally, &GStreamer; will start threads that take this
task on to them. &GStreamer; will also take care of switching
messages from the pipeline's thread into the application's own
thread, by using a <ulink type="http"
url="&URLAPI;GstBus.html"><classname>GstBus</classname></ulink>. See
<xref linkend="chapter-bus"/> for details.
</para>
</sect1>
</chapter>