mirror of
https://gitlab.freedesktop.org/gstreamer/gstreamer.git
synced 2024-12-23 08:46:40 +00:00
f0e1d945a4
Original commit message from CVS: 2004-01-26 Ronald Bultje <rbultje@ronald.bitfreak.net> * docs/pwg/advanced_types.xml: * docs/pwg/intro_basics.xml: * docs/pwg/intro_preface.xml: * docs/pwg/pwg.xml: * docs/pwg/titlepage.xml: First try to resurrect the PWG. I'm halfway integrating the mimetypes in here (docs/random/mimetypes), and will from there on work on both updating outdated parts and adding missing parts. That doesn't mean I'll fix it completely, but I'll try at least. ;).
408 lines
17 KiB
XML
408 lines
17 KiB
XML
<!-- ############ chapter ############# -->
|
|
|
|
<chapter id="cha-intro-basics" xreflabel="Basic Concepts">
|
|
<title>Basic Concepts</title>
|
|
<para>
|
|
This chapter of the guide introduces the basic concepts of &GStreamer;.
|
|
Understanding these concepts will help you grok the issues involved in
|
|
extending &GStreamer;. Many of these concepts are explained in greater
|
|
detail in the &GstAppDevMan;; the basic concepts presented here serve mainly
|
|
to refresh your memory.
|
|
</para>
|
|
|
|
<!-- ############ sect1 ############# -->
|
|
|
|
<sect1 id="sect1-basics-elements" xreflabel="Elements and Plugins">
|
|
<title>Elements and Plugins</title>
|
|
<para>
|
|
Elements are at the core of &GStreamer;. In the context of plugin
|
|
development, an <emphasis>element</emphasis> is an object derived from the
|
|
<classname>GstElement</classname> class. Elements provide some sort of
|
|
functionality when linked with other elements: For example, a source
|
|
element provides data to a stream, and a filter element acts on the data
|
|
in a stream. Without elements, &GStreamer; is just a bunch of conceptual
|
|
pipe fittings with nothing to link. A large number of elements ship
|
|
with &GStreamer;, but extra elements can also be written.
|
|
</para>
|
|
<para>
|
|
Just writing a new element is not entirely enough, however: You will need
|
|
to encapsulate your element in a <emphasis>plugin</emphasis> to enable
|
|
&GStreamer; to use it. A plugin is essentially a loadable block of code,
|
|
usually called a shared object file or a dynamically linked library. A
|
|
single plugin may contain the implementation of several elements, or just
|
|
a single one. For simplicity, this guide concentrates primarily on plugins
|
|
containing one element.
|
|
</para>
|
|
<para>
|
|
A <emphasis>filter</emphasis> is an important type of element that
|
|
processes a stream of data. Producers and consumers of data are called
|
|
<emphasis>source</emphasis> and <emphasis>sink</emphasis> elements,
|
|
respectively. <emphasis>Bin</emphasis> elements contain other elements.
|
|
One type of bin is responsible for scheduling the elements that they
|
|
contain so that data flows smoothly. Another type of bin, called
|
|
<emphasis>autoplugger</emphasis> elements, automatically add other
|
|
elements to the bin and link them together so that they act as a
|
|
filter between two arbitary stream types.
|
|
</para>
|
|
<para>
|
|
The plugin mechanism is used everywhere in &GStreamer;, even if only the
|
|
standard packages are being used. A few very basic functions reside in the
|
|
core library, and all others are implemented in plugins. A plugin registry
|
|
is used to store the details of the plugins in an XML file. This way, a
|
|
program using &GStreamer; does not have to load all plugins to determine
|
|
which are needed. Plugins are only loaded when their provided elements are
|
|
requested.
|
|
</para>
|
|
<para>
|
|
See the &GstLibRef; for the current implementation details of <ulink
|
|
type="http"
|
|
url="../gstreamer/gstelement.html"><classname>GstElement</classname></ulink>
|
|
and <ulink type="http"
|
|
url="../gstreamer/gstreamer-gstplugin.html"><classname>GstPlugin</classname></ulink>.
|
|
</para>
|
|
</sect1>
|
|
|
|
<!-- ############ sect1 ############# -->
|
|
|
|
<sect1 id="sect1-basics-pads" xreflabel="Pads">
|
|
<title>Pads</title>
|
|
<para>
|
|
<emphasis>Pads</emphasis> are used to negotiate links and data flow
|
|
between elements in &GStreamer;. A pad can be viewed as a
|
|
<quote>place</quote> or <quote>port</quote> on an element where
|
|
links may be made with other elements, and through which data can
|
|
flow to or from those elements. Pads have specific data handling
|
|
capabilities: A pad can restrict the type of data that flows
|
|
through it. Links are only allowed between two pads when the
|
|
allowed data types of the two pads are compatible.
|
|
</para>
|
|
<para>
|
|
An analogy may be helpful here. A pad is similar to a plug or jack on a
|
|
physical device. Consider, for example, a home theater system consisting
|
|
of an amplifier, a DVD player, and a (silent) video projector. Linking
|
|
the DVD player to the amplifier is allowed because both devices have audio
|
|
jacks, and linking the projector to the DVD player is allowed because
|
|
both devices have compatible video jacks. Links between the
|
|
projector and the amplifier may not be made because the projector and
|
|
amplifier have different types of jacks. Pads in &GStreamer; serve the
|
|
same purpose as the jacks in the home theater system.
|
|
</para>
|
|
<para>
|
|
For the most part, all data in &GStreamer; flows one way through a link
|
|
between elements. Data flows out of one element through one or more
|
|
<emphasis>source pads</emphasis>, and elements accept incoming data through
|
|
one or more <emphasis>sink pads</emphasis>. Source and sink elements have
|
|
only source and sink pads, respectively.
|
|
</para>
|
|
<para>
|
|
See the &GstLibRef; for the current implementation details of a <ulink
|
|
type="http"
|
|
url="../gstreamer/gstreamer-gstpad.html"><classname>GstPad</classname></ulink>.
|
|
</para>
|
|
</sect1>
|
|
|
|
<!-- ############ sect1 ############# -->
|
|
|
|
<sect1 id="sect1-basics-data" xreflabel="Data, Buffers and Events">
|
|
<title>Data, Buffers and Events</title>
|
|
<para>
|
|
All streams of data in &GStreamer; are chopped up into chunks that are
|
|
passed from a source pad on one element to a sink pad on another element.
|
|
<emphasis>Data</emphasis> are structures used to hold these chunks of
|
|
data.
|
|
</para>
|
|
<para>
|
|
Data contains the following important types:
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>
|
|
An exact type indicating what type of data (control, content, ...)
|
|
this Data is.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
A reference count indicating the number of elements currently
|
|
holding a reference to the buffer. When the buffer reference count
|
|
falls to zero, the buffer will be unlinked, and its memory will be
|
|
freed in some sense (see below for more details).
|
|
</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
</para>
|
|
<para>
|
|
There are two types of data defined: events (control) and buffers
|
|
(content).
|
|
</para>
|
|
<para>
|
|
Buffers may contain any sort of data that the two linked pads
|
|
know how to handle. Normally, a buffer contains a chunk of some sort of
|
|
audio or video data that flows from one element to another.
|
|
</para>
|
|
<para>
|
|
Buffers also contain metadata describing the buffer's contents. Some of
|
|
the important types of metadata are:
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>
|
|
A pointer to the buffer's data.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
An integer indicating the size of the buffer's data.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
A timestamp indicating the preferred display timestamp of the
|
|
content in the buffer.
|
|
</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
</para>
|
|
<para>
|
|
Events
|
|
contain information on the state of the stream flowing between the two
|
|
linked pads. Events will only be sent if the element explicitely supports
|
|
them, else the core will (try to) handle the events automatically. Events
|
|
are used to indicate, for example, a clock discontinuity, the end of a
|
|
media stream or that the cache should be flushed.
|
|
</para>
|
|
<para>
|
|
Events may contain several of the following items:
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>
|
|
A subtype indicating the type of the contained event.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
The other contents of the event depend on the specific event type.
|
|
</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
</para>
|
|
<para>
|
|
See the &GstLibRef; for the current implementation details of a <ulink
|
|
type="http"
|
|
url="../gstreamer/gstreamer-gstdata.html"><classname>GstData</classname></ulink>, <ulink type="http"
|
|
url="../gstreamer/gstreamer-gstbuffer.html"><classname>GstBuffer</classname></ulink> and <ulink type="http"
|
|
url="../gstreamer/gstreamer-gstevent.html"><classname>GstEvent</classname></ulink>.
|
|
</para>
|
|
|
|
<sect2 id="sect2-buffers-bufferpools" xreflabel="Buffer Allocation and
|
|
Buffer Pools">
|
|
<title>Buffer Allocation and Buffer Pools</title>
|
|
<para>
|
|
Buffers can be allocated using various schemes, and they may either be
|
|
passed on by an element or unreferenced, thus freeing the memory used by
|
|
the buffer. Buffer allocation and unlinking are important concepts when
|
|
dealing with real time media processing, since memory allocation is
|
|
relatively slow on most systems.
|
|
</para>
|
|
<para>
|
|
To improve the latency in a media pipeline, many &GStreamer; elements
|
|
use a <emphasis>buffer pool</emphasis> to handle buffer allocation and
|
|
releasing. A buffer pool is a virtual representation of one or more
|
|
buffers of which the data is not actually allocated by &GStreamer;
|
|
itself. Examples of these include hardware framebuffer memory in video
|
|
output elements or kernel-allocated DMA memory for video capture. The
|
|
huge advantage of using these buffers instead of creating our own is
|
|
that we do not have to copy memory from one place to another, thereby
|
|
saving a noticeable number of CPU cycles. Elements should not provide
|
|
a bufferpool to decrease the number of memory allocations: the kernel
|
|
will generally take care of that - and will probably do that much more
|
|
efficiently than we ever could. Using bufferpools in this way is highly
|
|
discouraged.
|
|
</para>
|
|
<para>
|
|
Normally in a media pipeline, most filter elements in &GStreamer; deal
|
|
with a buffer in place, meaning that they do not create or destroy
|
|
buffers. Sometimes, however, elements might need to alter the reference
|
|
count of a buffer, either by copying or destroying the buffer, or by
|
|
creating a new buffer. These topics are generally reserved for
|
|
non-filter elements, so they will be addressed at that point.
|
|
</para>
|
|
</sect2>
|
|
</sect1>
|
|
|
|
<!-- ############ sect1 ############# -->
|
|
|
|
<sect1 id="sect1-basics-types" xreflabel="Types and Properties">
|
|
<title>Mimetypes and Properties</title>
|
|
<para>
|
|
&GStreamer; uses a type system to ensure that the data passed between
|
|
elements is in a recognized format. The type system is also important
|
|
for ensuring that the parameters required to fully specify a format match
|
|
up correctly when linking pads between elements. Each link that is
|
|
made between elements has a specified type and optionally a set of
|
|
properties.
|
|
</para>
|
|
|
|
<!-- ############ sect2 ############# -->
|
|
|
|
<sect2 id="sect2-types-basictypes" xreflabel="Basic Types">
|
|
<title>The Basic Types</title>
|
|
<para>
|
|
&GStreamer; already supports many basic media types. Following is a
|
|
table of a few of the the basic types used for buffers in
|
|
&GStreamer;. The table contains the name ("mime type") and a
|
|
description of the type, the properties associated with the type, and
|
|
the meaning of each property. A full list of supported types is
|
|
included in <xref linkend="sect1-types-definitions"/>.
|
|
</para>
|
|
|
|
<table frame="all" id="table-basictypes" xreflabel="Table of Basic Types">
|
|
<title>Table of Basic Types</title>
|
|
<tgroup cols="6" align="left" colsep="1" rowsep="1">
|
|
|
|
<thead>
|
|
<row>
|
|
<entry>Mime Type</entry>
|
|
<entry>Description</entry>
|
|
<entry>Property</entry>
|
|
<entry>Property Type</entry>
|
|
<entry>Property Values</entry>
|
|
<entry>Property Description</entry>
|
|
</row>
|
|
</thead>
|
|
|
|
<tbody valign="top">
|
|
|
|
<!-- ############ type ############# -->
|
|
|
|
<row>
|
|
<entry morerows="1">audio/*</entry>
|
|
<entry morerows="1">
|
|
<emphasis>All audio types</emphasis>
|
|
</entry>
|
|
<entry>rate</entry>
|
|
<entry>integer</entry>
|
|
<entry>greater than 0</entry>
|
|
<entry>
|
|
The sample rate of the data, in samples (per channel) per second.
|
|
</entry>
|
|
</row>
|
|
<row>
|
|
<entry>channels</entry>
|
|
<entry>integer</entry>
|
|
<entry>greater than 0</entry>
|
|
<entry>
|
|
The number of channels of audio data.
|
|
</entry>
|
|
</row>
|
|
|
|
<!-- ############ type ############# -->
|
|
|
|
<row>
|
|
<entry morerows="3">audio/x-raw-int</entry>
|
|
<entry morerows="3">
|
|
Unstructured and uncompressed raw integer audio data.
|
|
</entry>
|
|
<entry>endianness</entry>
|
|
<entry>integer</entry>
|
|
<entry>G_BIG_ENDIAN (1234) or G_LITTLE_ENDIAN (4321)</entry>
|
|
<entry>
|
|
The order of bytes in a sample. The value G_LITTLE_ENDIAN (4321)
|
|
means <quote>little-endian</quote> (byte-order is <quote>least
|
|
significant byte first</quote>). The value G_BIG_ENDIAN (1234)
|
|
means <quote>big-endian</quote> (byte order is <quote>most
|
|
significant byte first</quote>).
|
|
</entry>
|
|
</row>
|
|
<row>
|
|
<entry>signed</entry>
|
|
<entry>boolean</entry>
|
|
<entry>TRUE or FALSE</entry>
|
|
<entry>
|
|
Whether the values of the integer samples are signed or not.
|
|
Signed samples use one bit to indicate sign (negative or
|
|
positive) of the value. Unsigned samples are always positive.
|
|
</entry>
|
|
</row>
|
|
<row>
|
|
<entry>width</entry>
|
|
<entry>integer</entry>
|
|
<entry>greater than 0</entry>
|
|
<entry>
|
|
Number of bits allocated per sample.
|
|
</entry>
|
|
</row>
|
|
<row>
|
|
<entry>depth</entry>
|
|
<entry>integer</entry>
|
|
<entry>greater than 0</entry>
|
|
<entry>
|
|
The number of bits used per sample. This must be less than or
|
|
equal to the width: If the depth is less than the width, the
|
|
low bits are assumed to be the ones used. For example, a width
|
|
of 32 and a depth of 24 means that each sample is stored in a
|
|
32 bit word, but only the low 24 bits are actually used.
|
|
</entry>
|
|
</row>
|
|
|
|
<!-- ############ type ############# -->
|
|
|
|
<row>
|
|
<entry morerows="3">audio/mpeg</entry>
|
|
<entry morerows="3">
|
|
Audio data compressed using the MPEG audio encoding scheme.
|
|
</entry>
|
|
<entry>mpegversion</entry>
|
|
<entry>integer</entry>
|
|
<entry>1, 2 or 4</entry>
|
|
<entry>
|
|
The MPEG-version used for encoding the data. The value 1 refers
|
|
to MPEG-1, -2 and -2.5 layer 1, 2 or 3. The values 2 and 4 refer
|
|
to the MPEG-AAC audio encoding schemes.
|
|
</entry>
|
|
</row>
|
|
<row>
|
|
<entry>framed</entry>
|
|
<entry>boolean</entry>
|
|
<entry>0 or 1</entry>
|
|
<entry>
|
|
A true value indicates that each buffer contains exactly one
|
|
frame. A false value indicates that frames and buffers do not
|
|
necessarily match up.
|
|
</entry>
|
|
</row>
|
|
<row>
|
|
<entry>layer</entry>
|
|
<entry>integer</entry>
|
|
<entry>1, 2, or 3</entry>
|
|
<entry>
|
|
The compression scheme layer used to compress the data
|
|
<emphasis>(only if mpegversion=1)</emphasis>.
|
|
</entry>
|
|
</row>
|
|
<row>
|
|
<entry>bitrate</entry>
|
|
<entry>integer</entry>
|
|
<entry>greater than 0</entry>
|
|
<entry>
|
|
The bitrate, in bits per second. For VBR (variable bitrate)
|
|
MPEG data, this is the average bitrate.
|
|
</entry>
|
|
</row>
|
|
|
|
<!-- ############ type ############# -->
|
|
|
|
<row>
|
|
<entry>audio/x-vorbis</entry>
|
|
<entry>Vorbis audio data</entry>
|
|
<entry></entry>
|
|
<entry></entry>
|
|
<entry></entry>
|
|
<entry>
|
|
There are currently no specific properties defined for this type.
|
|
</entry>
|
|
</row>
|
|
</tbody>
|
|
</tgroup>
|
|
</table>
|
|
</sect2>
|
|
</sect1>
|
|
</chapter>
|