mirrors/gstreamer
mirrors/gstreamer
1
1
Fork 0
mirror of https://gitlab.freedesktop.org/gstreamer/gstreamer.git synced 2025-01-27 09:38:17 +00:00
Code Issues 1 Projects Releases Wiki Activity

docs/pwg/: Document base classes, update sections of n-to-1 and 1-to-n (muxer, fix some code examples and links and u...

Browse source 
Original commit message from CVS:
* docs/pwg/advanced-clock.xml:
* docs/pwg/appendix-porting.xml:
* docs/pwg/intro-preface.xml:
* docs/pwg/other-base.xml:
* docs/pwg/other-manager.xml:
* docs/pwg/other-nton.xml:
* docs/pwg/other-ntoone.xml:
* docs/pwg/other-oneton.xml:
* docs/pwg/pwg.xml:
Document base classes, update sections of n-to-1 and 1-to-n (muxer,
demuxer), remove n-to-n (was never written), fix some code examples
and links and update the porting section to include all this.
This commit is contained in:
Ronald S. Bultje 2005-07-20 08:29:06 +00:00
parent f529a0bafc
commit 835e826907
10 changed files with 464 additions and 306 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

15
ChangeLog
View file

@ -1,3 +1,18 @@
2005-07-20 Ronald S. Bultje <rbultje@ronald.bitfreak.net>
* docs/pwg/advanced-clock.xml:
* docs/pwg/appendix-porting.xml:
* docs/pwg/intro-preface.xml:
* docs/pwg/other-base.xml:
* docs/pwg/other-manager.xml:
* docs/pwg/other-nton.xml:
* docs/pwg/other-ntoone.xml:
* docs/pwg/other-oneton.xml:
* docs/pwg/pwg.xml:
Document base classes, update sections of n-to-1 and 1-to-n (muxer,
demuxer), remove n-to-n (was never written), fix some code examples
and links and update the porting section to include all this.
2005-07-19 Wim Taymans <wim@fluendo.com> 2005-07-19 Wim Taymans <wim@fluendo.com>
* gst/gstqueue.c: (gst_queue_init), (gst_queue_handle_sink_event), * gst/gstqueue.c: (gst_queue_init), (gst_queue_handle_sink_event),

1
docs/pwg/advanced-clock.xml
View file

@ -135,7 +135,6 @@
under discussion, and might change in a future release. under discussion, and might change in a future release.
</para> </para>
</footnote> </footnote>
See an example in <xref linkend="other-sink-processing"/>
</para> </para>
</sect2> </sect2>
</sect1> </sect1>

38
docs/pwg/appendix-porting.xml
View file

@ -8,7 +8,12 @@
With this list, it should be possible to port most plugins to With this list, it should be possible to port most plugins to
&GStreamer;-0.9 in less than a day. Exceptions are elements that will &GStreamer;-0.9 in less than a day. Exceptions are elements that will
require a base class in 0.9 (sources, sinks), in which case it may take require a base class in 0.9 (sources, sinks), in which case it may take
a lot longer, depending on the coder's skills. a lot longer, depending on the coder's skills (however, when using the
<classname>GstBaseSink</classname> and <classname>GstBaseSrc</classname>
base-classes, it shouldn't be all too bad), and elements requiring
the deprecated bytestream interface, which should take 1-2 days with
random access. The scheduling parts of muxers will also need a rewrite,
which will take about the same amount of time.
</para> </para>
<sect1 id="section-porting-objects"> <sect1 id="section-porting-objects">
@ -54,7 +59,36 @@
</listitem> </listitem>
<listitem> <listitem>
<para> <para>
base classes, async state changes. For as far as possible, elements should try to use existing base
classes in 0.9. Sink and source elements, for example, could derive
from <classname>GstBaseSrc</classname> and
<classname>GstBaseSink</classname>. Audio sinks or sources could even
derive from audio-specific base classes. All existing base classes
have been discussed in <xref linkend="chapter-other-base"/> and the
next few chapters.
</para>
</listitem>
<listitem>
<para>
In 0.9, event handling and buffers are separated once again. This
means that in order to receive events, one no longer has to set the
<classname>GST_FLAG_EVENT_AWARE</classname> flag, but can simply
set an event handling function on its sinkpad(s), using the function
<function>gst_pad_set_event_function ()</function>. The
<function>_chain ()</function>-function will only receive buffers.
</para>
</listitem>
<listitem>
<para>
Although core will wrap most threading-related locking for you (e.g.
it takes the stream lock before calling your data handling
functions), you are still responsible for locking around certain
functions, e.g. object properties. Be sure to lock properly here,
since applications will change those properties in a different thread
than the thread which does the actual data passing! You can use the
<function>GST_LOCK ()</function> and <function>GST_UNLOCK
()</function> helpers in most cases, fortunately, which grabs the
default property lock of the element.
</para> </para>
</listitem> </listitem>
</itemizedlist> </itemizedlist>

7
docs/pwg/intro-preface.xml
View file

@ -197,9 +197,10 @@
the concepts apply equally to other plugin types, including sources, the concepts apply equally to other plugin types, including sources,
sinks, and autopluggers. This part of the guide presents the issues sinks, and autopluggers. This part of the guide presents the issues
that arise when working on these more specialized plugin types. The that arise when working on these more specialized plugin types. The
part includes chapters on <xref linkend="chapter-other-source"/>, chapter starts with a special focus on elements that can be written
<xref linkend="chapter-other-sink"/>, <xref using a base-class (<xref linkend="chapter-other-base"/>), and
linkend="chapter-other-oneton"/>, <xref later also goes into writing special types of elements in
<xref linkend="chapter-other-oneton"/>, <xref
linkend="chapter-other-ntoone"/> and <xref linkend="chapter-other-ntoone"/> and <xref
linkend="chapter-other-manager"/>. linkend="chapter-other-manager"/>.
</para> </para>

327
docs/pwg/other-base.xml Normal file
View file

@ -0,0 +1,327 @@
<chapter id="chapter-other-base" xreflabel="Pre-made base classes">
<title>Pre-made base classes</title>
<para>
So far, we've been looking at low-level concepts of creating any type of
&GStreamer; element. Now, let's assume that all you want is to create an
simple audiosink that works exactly the same as, say,
<quote>esdsink</quote>, or a filter that simply normalizes audio volume.
Such elements are very general in concept and since they do nothing
special, they should be easier to code than to provide your own scheduler
activation functions and doing complex caps negotiation. For this purpose,
&GStreamer; provides base classes that simplify some types of elements.
Those base classes will be discussed in this chapter.
</para>
<sect1 id="section-base-sink" xreflabel="Writing a sink">
<title>Writing a sink</title>
<para>
Sinks are special elements in &GStreamer;. This is because sink elements
have to take care of <emphasis>preroll</emphasis>, which is the process
that takes care that elements going into the
<classname>GST_STATE_PAUSED</classname> state will have buffers ready
after the state change. The result of this is that such elements can
start processing data immediately after going into the
<classname>GST_STATE_PLAYING</classname> state, without requiring to
take some time to initialize outputs or set up decoders; all that is done
already before the state-change to <classname>GST_STATE_PAUSED</classname>
successfully completes.
</para>
<para>
Preroll, however, is a complex process that would require the same
code in many elements. Therefore, sink elements can derive from the
<classname>GstBaseSink</classname> base-class, which does preroll and
a few other utility functions automatically. The derived class only
needs to implement a bunch of virtual functions and will work
automatically.
</para>
<para>
The <classname>GstBaseSink</classname> base-class specifies some
limitations on elements, though:
</para>
<itemizedlist>
<listitem>
<para>
It requires that the sink only has one sinkpad. Sink elements that
need more than one sinkpad, cannot use this base-class.
</para>
</listitem>
<listitem>
<para>
The base-class owns the pad, and specifies caps negotiation, data
handling, pad allocation and such functions. If you need more than
the ones provided as virtual functions, then you cannot use this
base-class.
</para>
</listitem>
<listitem>
<para>
By implementing the <function>pad_allocate ()</function> function,
it is possible for upstream elements to use special memory, such
as memory on the X server side that only the sink can allocate, or
even hardware memory <function>mmap ()</function>'ed from the kernel.
Note that in almost all cases, you will want to subclass the
<classname>GstBuffer</classname> object, so that your own set of
functions will be called when the buffer loses its last reference.
</para>
</listitem>
</itemizedlist>
<para>
Sink elements can derive from <classname>GstBaseSink</classname> using
the usual <classname>GObject</classname> type creation voodoo, or by
using the convenience macro <function>GST_BOILERPLATE ()</function>:
</para>
<programlisting>
GST_BOILERPLATE_FULL (GstMySink, gst_my_sink, GstBaseSink, GST_TYPE_BASE_SINK);
[..]
static void
gst_my_sink_class_init (GstMySinkClass * klass)
{
klass->set_caps = [..];
klass->render = [..];
[..]
}
</programlisting>
<para>
The advantages of deriving from <classname>GstBaseSink</classname> are
numerous:
</para>
<itemizedlist>
<listitem>
<para>
Derived implementations barely need to be aware of preroll, and do
not need to know anything about the technical implementation
requirements of preroll. The base-class does all the hard work.
</para>
<para>
Less code to write in the derived class, shared code (and thus
shared bugfixes).
</para>
</listitem>
</itemizedlist>
<para>
There are also specialized base classes for audio and video, let's look
at those a bit.
</para>
<sect2 id="section-base-audiosink" xreflabel="Writing an audio sink">
<title>Writing an audio sink</title>
<para>
Essentially, audio sink implementations are just a special case of a
general sink. There are two audio base classes that you can choose to
derive from, depending on your needs:
<classname>GstBaseAudiosink</classname> and
<classname>GstAudioSink</classname>. The baseaudiosink provides full
control over how synchronization and scheduling is handled, by using
a ringbuffer that the derived class controls and provides. The
audiosink base-class is a derived class of the baseaudiosink,
implementing a standard ringbuffer implementing default
synchronization and providing a standard audio-sample clock. Derived
classes of this base class merely need to provide a <function>_open
()</function>, <function>_close ()</function> and a <function>_write
()</function> function implementation, and some optional functions.
This should suffice for many sound-server output elements and even
most interfaces. More demanding audio systems, such as Jack, would
want to implement the <classname>GstBaseAudioSink</classname>
base-class.
</para>
<para>
The <classname>GstBaseAusioSink</classname> has little to no
limitations and should fit virtually every implementation, but is
hard to implement. The <classname>GstAudioSink</classname>, on the
other hand, only fits those systems with a simple <function>open
()</function> / <function>close ()</function> / <function>write
()</function> API (which practically means pretty much all of them),
but has the advantage that it is a lot easier to implement. The
benefits of this second base class are large:
</para>
<itemizedlist>
<listitem>
<para>
Automatic synchronization, without any code in the derived class.
</para>
</listitem>
<listitem>
<para>
Also automatically provides a clock, so that other sinks (e.g. in
case of audio/video playback) are synchronized.
</para>
</listitem>
<listitem>
<para>
Features can be added to all audiosinks by making a change in the
base class, which makes maintainance easy.
</para>
</listitem>
<listitem>
<para>
Derived classes require only three small functions, plus some
<classname>GObject</classname> boilerplate code.
</para>
</listitem>
</itemizedlist>
<para>
In addition to implementing the audio base-class virtual functions,
derived classes can (should) also implement the
<classname>GstBaseSink</classname> <function>set_caps ()</function> and
<function>get_caps ()</function> virtual functions for negotiation.
</para>
</sect2>
<sect2 id="section-base-videosink" xreflabel="Writing a general sink">
<title>Writing a video sink</title>
<para>
Writing a videosink can be done using the
<classname>GstVideoSink</classname> base-class, which derives from
<classname>GstBaseSink</classname> internally. Currently, it does
nothing yet but add another compile dependency, so derived classes
will need to implement all base-sink virtual functions. When they do
this correctly, this will have some positive effects on the end user
experience with the videosink:
</para>
<itemizedlist>
<listitem>
<para>
Because of preroll (and the <function>preroll ()</function> virtual
function), it is possible to display a video frame already when
going into the <classname>GST_STATE_PAUSED</classname> state.
</para>
</listitem>
<listitem>
<para>
By adding new features to <classname>GstVideoSink</classname>, it
will be possible to add extensions to videosinks that affect all of
them, but only need to be coded once, which is a huge maintainance
benefit.
</para>
</listitem>
</itemizedlist>
</sect2>
</sect1>
<sect1 id="section-base-src" xreflabel="Writing a source">
<title>Writing a source</title>
<para>
In the previous part, particularly <xref
linkend="section-scheduling-randomxs"/>, we have learned that some types
of elements can provide random access. This applies most definitely to
source elements reading from a randomly seekable location, such as file
sources. However, other source elements may be better described as a
live source element, such as a camera source, an audio card source and
such; those are not seekable and do not provide byte-exact access. For
all such use cases, &GStreamer; provides two base classes:
<classname>GstBaseSrc</classname> for the basic source functionality, and
<classname>GstPushSrc</classname>, which is a non-byte exact source
base-class. The pushsource base class itself derives from basesource as
well, and thus all statements about the basesource apply to the
pushsource, too.
</para>
<para>
The basesrc class does several things automatically for derived classes,
so they no longer have to worry about it:
</para>
<itemizedlist>
<listitem>
<para>
Fixes to <classname>GstBaseSrc</classname> apply to all derived
classes automatically.
</para>
</listitem>
<listitem>
<para>
Automatic pad activation handling, and task-wrapping in case we get
assigned to start a task ourselves.
</para>
</listitem>
</itemizedlist>
<para>
The <classname>GstBaseSrc</classname> may not be suitable for all cases,
though; it has limitations:
</para>
<itemizedlist>
<listitem>
<para>
There is one and only one sourcepad. Source elements requiring
multiple sourcepads cannot use this base-class.
</para>
</listitem>
<listitem>
<para>
Since the base-class owns the pad and derived classes can only
control it as far as the virtual functions allow, you are limited
to the functionality provided by the virtual functions. If you need
more, you cannot use this base-class.
</para>
</listitem>
</itemizedlist>
<para>
It is possible to use special memory, such as X server memory pointers
or <function>mmap ()</function>'ed memory areas, as data pointers in
buffers returned from the <function>create()</function> virtual function.
In almost all cases, you will want to subclass
<classname>GstBuffer</classname> so that your own set of functions can
be called when the buffer is destroyed.
</para>
<sect2 id="section-base-audiosrc" xreflabel="Writing an audio source">
<title>Writing an audio source</title>
<para>
An audio source is nothing more but a special case of a pushsource.
Audio sources would be anything that reads audio, such as a source
reading from a soundserver, a kernel interface (such as ALSA) or a
test sound / signal generator. &GStreamer; provides two base classes,
similar to the two audiosinks described in <xref
linkend="section-base-audiosink"/>; one is ringbuffer-based, and
requires the derived class to take care of its own scheduling,
synchronization and such. The other is based on this
<classname>GstBaseAudioSrc</classname> and is called
<classname>GstAudioSrc</classname>, and provides a simple
<function>open ()</function>, <function>close ()</function> and
<function>read ()</function> interface, which is rather simple to
implement and will suffice for most soundserver sources and audio
interfaces (e.g. ALSA or OSS) out there.
</para>
<para>
The <classname>GstAudioSrc</classname> base-class has several benefits
for derived classes, on top of the benefits of the
<classname>GstPushSrc</classname> base-class that it is based on:
</para>
<itemizedlist>
<listitem>
<para>
Does syncronization and provides a clock.
</para>
</listitem>
<listitem>
<para>
New features can be added to it and will apply to all derived
classes automatically.
</para>
</listitem>
</itemizedlist>
</sect2>
</sect1>
<sect1 id="section-base-transform"
xreflabel="Writing a transformation element">
<title>Writing a transformation element</title>
<para>
A third base-class that &GStreamer; provides is the
<classname>GstBaseTransform</classname>. This is a base class for
elements with one sourcepad and one sinkpad which act as a filter
of some sort, such as volume changing, audio resampling, audio format
conversion, and so on and so on. There is quite a lot of bookkeeping
that such elements need to do in order for things such as buffer
allocation forwarding, passthrough, in-place processing and such to all
work correctly. This base class does all that for you, so that you just
need to do the actual processing.
</para>
<para>
Since the <classname>GstBaseTransform</classname> is based on the 1-to-1
model for filters, it may not apply well to elements such as decoders,
which may have to parse properties from the stream. Also, it will not
work for elements requiring more than one sourcepad or sinkpad.
</para>
</sect1>
</chapter>

32
docs/pwg/other-manager.xml
View file

@ -1,6 +1,3 @@
<!-- ############ chapter ############# -->
<chapter id="chapter-other-manager" xreflabel="Writing a Manager"> <chapter id="chapter-other-manager" xreflabel="Writing a Manager">
<title>Writing a Manager</title> <title>Writing a Manager</title>
<para> <para>
@ -30,25 +27,18 @@
function). function).
</para> </para>
</listitem> </listitem>
<listitem>
<para>
To embed an element, or a series of elements, into something that
looks and works like a simple element to the outside world.
</para>
</listitem>
</itemizedlist> </itemizedlist>
<para> <para>
This chapter will explain the setup of managers. As a specific example, Making a manager is about as simple as it gets. You can derive from a
we will try to add EOS event support to source elements. This can be <classname>GstBin</classname>, and in most cases, you can embed the
used to finish capturing an audio stream to a file. Source elements required elements in the <function>_init ()</function> already, including
normally don't do any EOS handling at all, so a manager is perfect to setup of ghostpads. If you need any custom data handlers, you can connect
extend those element's functionalities. signals or embed a second element which you control.
</para> </para>
<para>
Specifically, this element will contain two child elements: the actual
source element and a <quote>helper element</quote> that implement an
event handler on its source pad. This event handler will respond to
EOS events by storing them internally and returning the event (rather
than data) on the next call to the <function>_get ()</function>
function. After that, it will go into EOS and set the parent (and
thereby the contained source element) to EOS as well. Other events will
be forwarded to the source element, which will handle them as usual.
</para>
<programlisting>
..
</programlisting>
</chapter> </chapter>

8
docs/pwg/other-nton.xml
View file

@ -1,8 +0,0 @@
<!-- ############ chapter ############# -->
<chapter id="chapter-other-nton" xreflabel="Writing a N-to-N element">
<title>Writing a N-to-N element</title>
<para>
FIXME: write.
</para>
</chapter>

136
docs/pwg/other-ntoone.xml
View file

@ -1,82 +1,31 @@
<chapter id="chapter-other-ntoone"
<!-- ############ chapter ############# --> xreflabel="Writing a N-to-1 Element or Muxer">
<chapter id="chapter-other-ntoone" xreflabel="Writing a N-to-1 Element or Muxer">
<title>Writing a N-to-1 Element or Muxer</title> <title>Writing a N-to-1 Element or Muxer</title>
<para> <para>
N-to-1 elements have been previously mentioned and discussed in both N-to-1 elements have been previously mentioned and discussed in both
<xref linkend="chapter-advanced-request"/> and in <xref linkend="chapter-advanced-request"/> and in
<xref linkend="chapter-scheduling"/>. The main noteworthy thing <xref linkend="chapter-scheduling"/>. The main noteworthy thing
about N-to-1 elements is that they should <emphasis>always</emphasis>, about N-to-1 elements is that each pad is push-based in its own thread,
without any single exception, be <function>_loop ()</function>-based. and the N-to-1 element synchronizes those streams by
Apart from that, there is not much general that you need to know. We expected-timestamp-based logic. This means it lets all streams wait
will discuss one special type of N-to-1 elements here, these being except for the one that provides the earliest next-expected timestamp.
muxers. The first two of these sections apply to N-to-1 elements in When that stream has passwed one buffer, the next
general, though. earliest-expected-timestamp is calculated, and we start back where we
were, until all streams have reached EOS. There is a helper base class,
called <classname>GstCollectPads</classname>, that will help you to do
this.
</para>
<para>
Note, however, that this helper class will only help you with grabbing
a buffer from each input and giving you the one with earliest timestamp.
If you need anything more difficult, such as "don't-grab-a-new-buffer
until a given timestamp" or something like that, you'll need to do this
yourself.
</para> </para>
<sect1 id="section-muxer-dataloop" xreflabel="The Data Loop Function"> <!--
<title>The Data Loop Function</title> Note: I'd like to have something like this in the final text, but since
<para> the code below doesn't work and this is all 0.8'y, I commented it for now.
As previously mentioned in <xref linkend="chapter-scheduling"/>,
N-to-1 elements generally try to have one buffer from each sink pad
and then handle the one with the earliest timestamp. There's some
exceptions to this rule, we will come to those later. This only works
if all streams actually continuously provide input. There might be
cases where this is not true, for example subtitles (there might be
no subtitle for a while), overlay images and so forth. For this
purpose, there is a <function>_select ()</function> function in
&GStreamer;. It checks whether input is available on a (list of)
pad(s). In this way, you can skip over the pads that are 'non-
continuous'.
</para>
<programlisting>
/* Pad selection is currently broken, FIXME some day */
</programlisting>
</sect1>
<sect1 id="section-muxer-events" xreflabel="Events in the Loop Function">
<title>Events in the Loop Function</title>
<para>
N-to-1 elements using a cache will sometimes receive events, and it
is often unclear how to handle those. For example, how do you seek
to a frame in an <emphasis>output</emphasis> file (and what's the
point of it anyway)? So, do discontinuity or seek events make sense,
and should you use them?
</para>
<sect2 id="section-muxevents-discont" xreflabel="Discontinuities and flushes">
<title>Discontinuities and flushes</title>
<para>
Don't do anything. They specify a discontinuity in the output, and
you should continue to playback as you would otherwise. You
generally do not need to put a discontinuity in the output stream
in muxers; you would have to manually start adapting timestamps of
output frames (if appliccable) to match the previous timescale,
though. Note that the output data stream should be continuous. For
other types of N-to-1-elements, it is generally fine to forward
the discontinuity once it has been received from all pads. This
depends on the specific element.
</para>
</sect2>
<sect2 id="section-muxevents-seek" xreflabel="Seeks">
<title>Seeks</title>
<para>
Depends on the element. Muxers would generally not implement this,
because the concept of seeking in an <emphasis>output</emphasis>
stream at frame level is not very useful. Seeking at byte level
can be useful, but that is more generally done
<emphasis>by</emphasis> muxers <emphasis>on</emphasis> sink
elements.
</para>
</sect2>
<sect2 id="section-muxevents-eos" xreflabel="End-of-Stream">
<title>End-of-Stream</title>
<para>
Speaks for itself.
</para>
</sect2>
</sect1>
<sect1 id="section-muxer-negotiation" xreflabel="Negotiation"> <sect1 id="section-muxer-negotiation" xreflabel="Negotiation">
<title>Negotiation</title> <title>Negotiation</title>
<para> <para>
@ -202,46 +151,5 @@ gst_avi_mux_loop (GstElement *element)
muxers, it is not a complete solution, nor is it a pretty one. muxers, it is not a complete solution, nor is it a pretty one.
</para> </para>
</sect1> </sect1>
-->
<sect1 id="section-muxer-markup" xreflabel="Markup vs. data processing">
<title>Markup vs. data processing</title>
<para>
As we noted on demuxers before, we love common programming paradigms
such as clean, lean and mean code. To achieve that in muxers, it's
generally a good idea to separate the actual data stream markup from
the data processing. To illustrate, here's how AVI muxers should
write out RIFF tag chunks:
</para>
<programlisting>
static void
gst_avi_mux_write_chunk (GstAviMux *mux,
guint32 id,
GstBuffer *data)
{
GstBuffer *hdr;
hdr = gst_buffer_new_and_alloc (8);
((guint32 *) GST_BUFFER_DATA (buf))[0] = GUINT32_TO_LE (id);
((guint32 *) GST_BUFFER_DATA (buf))[1] = GUINT32_TO_LE (GST_BUFFER_SIZE (data));
gst_pad_push (mux->srcpad, hdr);
gst_pad_push (mux->srcpad, data);
}
static void
gst_avi_mux_loop (GstElement *element)
{
GstAviMux *mux = GST_AVI_MUX (element);
GstBuffer *buf;
[..]
buf = gst_pad_pull (mux->sinkpad[0]);
[..]
gst_avi_mux_write_chunk (GST_MAKE_FOURCC ('0','0','d','b'), buf);
}
</programlisting>
<para>
In general, try to program clean code, that should cover pretty
much everything.
</para>
</sect1>
</chapter> </chapter>

178
docs/pwg/other-oneton.xml
View file

@ -1,18 +1,7 @@
<chapter id="chapter-other-oneton" xreflabel="Writing a Demuxer or Parser">
<!-- ############ chapter ############# --> <title>Writing a Demuxer or Parser</title>
<chapter id="chapter-other-oneton" xreflabel="Writing a 1-to-N Element, Demuxer or Parser">
<title>Writing a 1-to-N Element, Demuxer or Parser</title>
<para> <para>
1-to-N elements don't have much special needs or requirements that Demuxers are the 1-to-N elements that need very special care.
haven't been discussed already. The most important thing to take care
of in 1-to-N elements (things like <classname>tee</classname>-elements
or so) is to use proper buffer refcounting and caps negotiation. If
those two are taken care of (see the <classname>tee</classname> element
if you need example code), there's little that can go wrong.
</para>
<para>
Demuxers are the 1-to-N elements that need very special care, though.
They are responsible for timestamping raw, unparsed data into They are responsible for timestamping raw, unparsed data into
elementary video or audio streams, and there are many things that you elementary video or audio streams, and there are many things that you
can optimize or do wrong. Here, several culprits will be mentioned can optimize or do wrong. Here, several culprits will be mentioned
@ -20,132 +9,37 @@
one source pad. Also, they only cut the stream into buffers, they one source pad. Also, they only cut the stream into buffers, they
don't touch the data otherwise. don't touch the data otherwise.
</para> </para>
<para>
<sect1 id="section-oneton-capsnego" xreflabel="Demuxer Caps Negotiation"> As mentioned previously in <xref linkend="chapter-negotiation"/>,
<title>Demuxer Caps Negotiation</title> demuxers should use fixed caps, since their data type will not change.
<para> </para>
Demuxers will usually contain several elementary streams, and each <para>
of those streams' properties will be defined in a stream header at As discussed in <xref linkend="chapter-scheduling"/>, demuxer elements
the start of the file (or, rather, stream) that you're parsing. can be written in multiple ways:
Since those are fixed and there is no possibility to negotiate </para>
stream properties with elements earlier in the pipeline, you should <itemizedlist>
always use explicit caps on demuxer source pads. This prevents a <listitem>
whole lot of caps negotiation or re-negotiation errors. <para>
</para> They can be the driving force of the pipeline, by running their own
</sect1> task. This works particularly well for elements that need random
access, for example an AVI demuxer.
<sect1 id="section-oneton-data" xreflabel="Data processing and downstream events"> </para>
<title>Data processing and downstream events</title> </listitem>
<para> <listitem>
Data parsing, pulling this into subbuffers and sending that to the <para>
source pads of the elementary streams is the one single most They can also run in push-based mode, which means that an upstream
important task of demuxers and parsers. Usually, an element will element drives the pipeline. This works particularly well for streams
have a <function>_loop ()</function> function using the that may come from network, such as Ogg.
<classname>bytestream</classname> object to read data. Try to have </para>
a single point of data reading from the bytestream object. In this </listitem>
single point, do <emphasis>proper</emphasis> event handling (in </itemizedlist>
case there is any) and <emphasis>proper</emphasis> error handling <para>
in case that's needed. Make your element as fault-tolerant as In addition, audio parsers with one output can, in theory, also be written
possible, but do not go further than possible. in random access mode. Although simple playback will mostly work if your
</para> element only accepts one mode, it may be required to implement multiple
</sect1> modes to work in combination with all sorts of applications, such as
editing. Also, performance may become better if you implement multiple
<sect1 id="section-oneton-parsing" xreflabel="Parsing versus interpreting"> modes. See <xref linkend="chapter-scheduling"/> to see how an element
<title>Parsing versus interpreting</title> can accept multiple scheduling modes.
<para> </para>
One particular convention that &GStreamer; demuxers follow is that
of separation of parsing and interpreting. The reason for this is
maintainability, clarity and code reuse. An easy example of this
is something like RIFF, which has a chunk header of 4 bytes, then
a length indicator of 4 bytes and then the actual data. We write
special functions to read one chunk, to peek a chunk ID and all
those; that's the <emphasis>parsing</emphasis> part of the demuxer.
Then, somewhere else, we like to write the main data processing
function, which calls this parse function, reads one chunk and
then does with the data whatever it needs to do.
</para>
<para>
Some example code for RIFF-reading to illustrate the above two points:
</para>
<programlisting>
static gboolean
gst_my_demuxer_peek (GstMyDemuxer *demux,
guint32 *id,
guint32 *size)
{
guint8 *data;
while (gst_bytestream_peek_bytes (demux->bs, &amp;data, 4) != 4) {
guint32 remaining;
GstEvent *event;
gst_bytestream_get_status (demux->bs, &amp;remaining, &amp;event);
if (event) {
GstEventType type = GST_EVENT_TYPE (event);
/* or maybe custom event handling, up to you - we lose reference! */
gst_pad_event_default (demux->sinkpad, event);
if (type == GST_EVENT_EOS)
return FALSE;
} else {
GST_ELEMENT_ERROR (demux, STREAM, READ, (NULL), (NULL));
return FALSE;
}
}
*id = GUINT32_FROM_LE (((guint32 *) data)[0]);
*size = GUINT32_FROM_LE (((guint32 *) data)[0]);
return TRUE;
}
static void
gst_my_demuxer_loop (GstElement *element)
{
GstMyDemuxer *demux = GST_MY_DEMUXER (element);
guint32 id, size;
if (!gst_my_demuxer_peek (demux, &amp;id, &amp;size))
return;
switch (id) {
[.. normal chunk handling ..]
}
}
</programlisting>
<para>
Reason for this is that event handling is now centralized in one
place and the <function>_loop ()</function> function is a lot
cleaner and more readable. Those are common code practices, but
since the mistake of <emphasis>not</emphasis> using such common
code practices has been made too often, we explicitely mention
this here.
</para>
</sect1>
<sect1 id="section-oneton-seeking" xreflabel="Simple seeking and indexes">
<title>Simple seeking and indexes</title>
<para>
Sources will generally receive a seek event in the exact supported
format by the element. Demuxers, however, can not seek in
themselves directly, but need to convert from one unit (e.g.
time) to the other (e.g. bytes) and send a new event to its sink
pad. Given this, the <function>_convert ()</function>-function (or,
more general: unit conversion) is the most important function in a
demuxer. Some demuxers (AVI, Matroska) and parsers will keep an
index of all chunks in a stream, firstly to improve seeking
precision and secondly so they won't lose sync. Some other demuxers
will seek the stream directly without index (e.g. MPEG, Ogg) -
usually based on something like a cumulative bitrate - and then
find the closest next chunk from their new position. The best
choice depends on the format.
</para>
<para>
Note that it is recommended for demuxers to implement event,
conversion and query handling functions (using time units or so),
in addition to the ones (usually in byte units) provided by the
pipeline source element.
</para>
</sect1>
</chapter> </chapter>

28
docs/pwg/pwg.xml
View file

@ -31,11 +31,9 @@
<!ENTITY ADVANCED_TAGGING SYSTEM "advanced-tagging.xml"> <!ENTITY ADVANCED_TAGGING SYSTEM "advanced-tagging.xml">
<!ENTITY ADVANCED_EVENTS SYSTEM "advanced-events.xml"> <!ENTITY ADVANCED_EVENTS SYSTEM "advanced-events.xml">
<!ENTITY OTHER_SOURCE SYSTEM "other-source.xml"> <!ENTITY OTHER_BASE SYSTEM "other-base.xml">
<!ENTITY OTHER_SINK SYSTEM "other-sink.xml">
<!ENTITY OTHER_ONETON SYSTEM "other-oneton.xml"> <!ENTITY OTHER_ONETON SYSTEM "other-oneton.xml">
<!ENTITY OTHER_NTOONE SYSTEM "other-ntoone.xml"> <!ENTITY OTHER_NTOONE SYSTEM "other-ntoone.xml">
<!ENTITY OTHER_NTON SYSTEM "other-nton.xml">
<!ENTITY OTHER_MANAGER SYSTEM "other-manager.xml"> <!ENTITY OTHER_MANAGER SYSTEM "other-manager.xml">
<!ENTITY APPENDIX_CHECKLIST SYSTEM "appendix-checklist.xml"> <!ENTITY APPENDIX_CHECKLIST SYSTEM "appendix-checklist.xml">
@ -152,26 +150,26 @@
<!-- ############ part ############# --> <!-- ############ part ############# -->
<part id="part-other" xreflabel="Other Element Types"> <part id="part-other" xreflabel="Creating special element types">
<title>Other Element Types</title> <title>Creating special element types</title>
<partintro> <partintro>
<para> <para>
By now, we have looked at pretty much any feature that can be embedded By now, we have looked at pretty much any feature that can be embedded
into a &GStreamer; element. However, we have limited ourselves to the into a &GStreamer; element. Most of this has been fairly low-level and
simple model of a filter element. In this chapter, we will look at the given deep insights in how &GStreamer; works internally. Fortunately,
specific difficulties and things to keep in mind when writing specific &GStreamer; contains some easier-to-use interfaces to create such
types of elements. We will discuss output elements (sinks), input elements. In order to do that, we will look closer at the element
elements (sources), 1-to-N elements, N-to-1 elements, N-to-N elements, types for which &GStreamer; provides base classes (sources, sinks and
autopluggers and managers. Some of these represent elements that don't transformation elements). We will also look closer at some types of
actually exist. Rather, they represent a general concept. elements that require no specific coding such as scheduling-interaction
or data passing, but rather require specific pipeline control (e.g.
N-to-1 elements and managers).
</para> </para>
</partintro> </partintro>
&OTHER_SOURCE; &OTHER_BASE;
&OTHER_SINK;
&OTHER_ONETON; &OTHER_ONETON;
&OTHER_NTOONE; &OTHER_NTOONE;
&OTHER_NTON;
&OTHER_MANAGER; &OTHER_MANAGER;
</part> </part>