mirrors/gstreamer
mirrors/gstreamer
1
1
Fork 0
mirror of https://gitlab.freedesktop.org/gstreamer/gstreamer.git synced 2024-06-16 13:00:41 +00:00
Code Issues 1 Projects Releases Wiki Activity

docs: random: clean up outdated documents

Browse source 
Most of these are only of historical interest, and for that it's
fine if they're maintained in the git history. They're confusing
for anyone stumbling across them expecting documentation relating
to current versions of GStreamer.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/824>
This commit is contained in:
Tim-Philipp Müller 2021-05-16 02:10:55 +01:00
parent 2a710a484c
commit caa608e5c0
95 changed files with 0 additions and 9718 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
docs/random/API
View file

@ -1,2 +0,0 @@
* signals should use dashes in their names, not underscores, so ::notify
works correctly

18
docs/random/LICENSE
View file

@ -1,18 +0,0 @@
/* GStreamer
* Copyright (C) <1999> Erik Walthinsen <omega@cse.ogi.edu>
*
* This library is free software; you can redistribute it and/or
* modify it under the terms of the GNU Library General Public
* License as published by the Free Software Foundation; either
* version 2 of the License, or (at your option) any later version.
*
* This library is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Library General Public License for more details.
*
* You should have received a copy of the GNU Library General Public
* License along with this library; if not, write to the
* Free Software Foundation, Inc., 51 Franklin St, Fifth Floor,
* Boston, MA 02110-1301, USA.
*/

170
docs/random/TODO-pre-0.9
View file

@ -1,170 +0,0 @@
TODO:
-----
short term core API stability
-----------------------------
Changes that probably impact the API, careful discussion (IRC) + design doc is required
before changes are accepted.
target release ! description
!
0.4.1 ! expose and API to query the supported seek formats/flags on
(done) ! pads, something like an extra arg to gst_pad_set_convert_function
! and gst_pad_set_event_function with some function to query the
! flags and formats. more ideas in docs/random/wtay/query_events
! (API: medium difficulty)
!
0.4.1 ! add event for segment playback/looping and seeking (docs/random/wtay/segments)
(done) ! (API: medium difficulty, plugins: HARD to very HARD)
!
? ! add event to adjust rate (reverse playback, slow motion, frame skipping)
! (docs/random/wtay/rate_event)
! (API: medium difficulty, plugins: HARD to very HARD)
!
? ! add method in the scheduler to set the entry point (frame stepping?)
! (docs/random/wtay/scheduler_entry)
! (API: moderatly EASY, scheduler implementation MEDIUM)
!
0.6.0 ! create a design doc for a timecache implementation,
(done) ! (docs/wtay/random/timecache)
! (API: MEDIUM, needs lots of discussion, plugin implementation MEDIUM to HARD)
! (done: implemented using GstIndex base class + subclasses)
!
? ! implement a QoS event and a policy for handling the event.
! (API: kindof EASY, plugins MEDIUM to very HARD)
!
0.4.1 ! implement user defined GstFormat values, make a format factory etc..
(done) ! (API: MEDIUM, plugins MEDIUM)
!
? ! strip the API to a bare bones minimal set of functions, leave the automatic
! stuff to the app instead of forcing a policy in the core.
! create a library with useful higher level function for people who don't want
! to deal with the lowlevel stuff.
! (HARD, need to negotiate with people :))
!
? ! use GMarkup to load/save objects, remove dependency on libxml2
! (MEDIUM) breaks API/ABI compatibility
!
shortterm core feature enhancements
-----------------------------------
0.4.1 ! Implement PAD_DISABLED. This requires simple checks in the scheduler so that
! it doesn't try to pull/push data from/to a disabled pad.
! When an element goes to the PAUSED state, all of its pads should be disabled.
! This should also work for ghostpads.
! (API: MEDIUM to moderatly HARD, requires some scheduler understanding)
short term usability
--------------------
Writing docs is NOT boring, you learn a lot, you get insight in stuff, you help a lot
of people, hey! you might even find YOUR book on a shelf in a bookstore!!
? ! plugin writers guide
! (we have almost nothing, so any start is welcomed)
! (MEDIUM)
!
? ! app writers guide needs to cover common tips and tricks and HOWTOs
! (MEDIUM)
short to midterm policy definition
----------------------------------
Policy definition is closely related to a HOWTO but sometimes needs some thinking.
? ! document thread safety guidelines, what stuff needs locking in the app, what
! is done in the core.
! most of this stuff is in the heads of various people but needs to be written
! down so that people get more insights into the design and vision of GStreamer.
! (MEDIUM, some research and discussion needed)
!
? ! a step by step guide to the implementation of various events in a plugin, what can you
! do, when is data passing forbidden etc..
! (MEDIUM, some research needed)
!
? ! figure out a policy for the NEW_MEDIA event
! (MEDIUM to HARD)
!
? ! figure out how to better handle clock resync
! (MEDIUM to HARD)
!
midterm feature enhancement
---------------------------
0.6.0 | Define and implement a syntax in gst_parse to handle filtered pad connections.
(done) | (MEDIUM)
|
? | Figure out a way to set properties on schedulers (and bins?) from gst_parse.
| (MEDIUM)
|
? | Make gst-inspect do inspection of plugins, schedulers, autopluggers, types.
| An idea would be to make -inspect output an XML representation of the objects
| and use XSLT to transform this into text, HTML, docbook, ...
| (MEDIUM to EASY)
|
midterm (longterm) optimisations
--------------------------------
These optimisations can be done without changing the existing API.
(in progress) ! implement an optimal scheduler that uses chaining when possible
! (HARD, requires detailed knowledge of element scheduling)
!
? ! alternatively optimisations to the current scheduler can be done such
! as: do nothing when the pipeline structure (or chain) has not changed
! (MEDIUM)
!
? ! GstQueue is a little mess. implement a better queue (lockfree?), refactor
! queueing policy (max buffer, max time, levels etc..)
! (MEDIUM to farily EASY)
longterm feature enhancements
-----------------------------
Various features that are not critical yet.
? ! factory aliases. map a generic name like "videosink" to and actual
! user configurable plugin (aasink, sdlsink, xvideosink, ...)
! (MEDIUM)
!
? ! property proxy in compound elements. not sure if it's possible at all.
! what with elements with the same property?
! (MEDIUM, needs some thinking)
!
? ! Make _pad_select work for muxers
! (MEDIUM to HARD)
needs consensus
---------------
Some stuff that needs to be figured out based on a pro/con comparison.
? ! can we decide on the fact that downstream events are traveling using the
! scheduler? do we need to reevaluate that design decision?
! (MEDIUM, needs pros vs cons document)
benchmarks
----------
Benchmarks are always good to get acceptance in a wider comunity or to identify performance
problems that need fixing.
? ! do a latency comparison with popular other frameworks, document GStreamer
! overhead.
! (MEDIUM to somewhat EASY)
!

206
docs/random/autoplug1
View file

@ -1,206 +0,0 @@
COMPLETELY OUTDATED
-------------------
A little explanation of the first autoplugger in GStreamer:
Autoplugging is implemented in the following places:
gstpipeline.c : construction of the pipeline
gstautoplug.c : selection of the elementfactories needed for autoplugging
1) pipeline setup
-----------------
before any autoplugging will take place, a new GstPipeline has to be created.
The autoplugger needs to have a src element and one or more sink elements. the
autoplugger will try to find the elements needed to connect the src element
to the sinks.
using:
gst_pipeline_add_src (GstPipeline *pipeline, GstElement *element);
a source element is added to the pipeline. only one src element can be added
for now.
using:
gst_pipeline_add_sink (GstPipeline *pipeline, GstElement *element);
a sink element can be added to the pipeline.
2) starting autoplug
--------------------
when the pipeline has been set up as above, you will call
gst_pipeline_autoplug (GstPipeline *pipeline);
to start the autoplugger. this will be done in four phases
ex. we are going to autoplug an mpeg1 system stream.
2a) phase1: figure out the type (GstCaps) of the src element.
-------------------------------------------------------------
the gsttypefind element is connected to the "src" pad of the source
element. gst_bin_iterate is called in a loop until gsttypefind
signals "have_type". the gst_bin_iterate is stopped and the GstCaps
is retrieved from the gsttypefind element.
gsttypefind is disconnected from the src element and removed from the
bin.
the GstCaps of the source element is called src_caps later on.
ex. all typefind functions are tried and the one in mpeg1types will
return a GstCaps:
video/mpeg,
"systemstream", GST_PROPS_BOOLEAN (TRUE),
"mpegversion", GST_PROPS_INT (1),
NULL
2b) phase2: create lists of factories.
---------------------------------------
for each sink:
{
sinkpad = take the first sinkpad of the sink (HACK)
call
list[i] = gst_autoplug_caps (src_caps, sinkpad->caps);
I++;
}
gst_autoplug_caps will figure out (based on the padtemplates)
which elementfactories are needed to connect src_caps to sinkpad->caps
and will return them in a list.
ex. we have two sinks with following caps:
video/raw audio/raw
"...." "...."
gst_autoplug_caps will figure out that for the first sink the following
elements are needed:
mpeg1parse, mp1videoparse, mpeg_play
for the second sink the following is needed:
mpeg1parse, mp3parse, mpg123
We now have two lists of elementfactories.
2c) phase3: collect common elements from the lists.
---------------------------------------------------
the rationale is that from the lists we have created in phase2, there
must be some element that is a splitter and that it has to come first (HACK)
We try to find that element by comparing the lists until an element differs.
we add the common elements to the bin and run gst_pipeline_pads_autoplug. this
function will loop over the pads of the previous element and the one we
just added, and tries to connect src to sink if possible.
If a connection between the two elements could not be made, a signal "new_pad"
is connected to the element so that pad connection can occur later on when
the pad is actually created.
ex. when we compare the two lists we see that we have common element: mpeg1parse.
we add this element to the bin and try to connect it to the previous element in
the bin, the disksrc.
we see that the src pad of the disksrc and the sinkpad of the mpeg1parse element
can be connected because they are compatible. We have a pipeline like:
---------) (--------
disksrc ! ! mpeg1parse
src --- sink
---------) (--------
2d) phase4: add remaining elements
----------------------------------
now we loop over all the list and try to add the remaining elements
(HACK) we always use a new thread for the elements when there is a common
element found.
if a new thread is needed (either bacuase the previous element is a common
element or the object flag of the next element is set to GST_SUGGEST_THREAD)
we add a queue to the bin and we add a new thread. We add the elements to
the bin and connect them using gst_pipeline_pads_autoplug.
If we add a queue, we have to copy the caps of the sink element of the queue
to the src pad of the queue (else they won't connect)
we finally arrive at the sink element and we're done.
ex.
we have just found our mpeg1parse common element, so we start a thread.
We add a queue to the bin and a new thread, we add the elements
mp1videoparse and mpeg_play to the thread. We arrive at the videosink, we
see that the SUGGEST_THREAD flag is set, we add a queue and a thread and
add the videosink in the thread.
the same procedure happens for the audio part. We are now left with the
following pipeline:
We will also have set a signal "new_pad" on the mpeg1parse element because
the element mp1videoparse could not be connected to the element just yet.
(------------------------------------) (----------
!thread ! ! thread
! (-------------) (---------) ! ! (---------)
! !mp1videoparse! !mpeg_play! ! ! !videosink!
videoqueue--sink src -- sink src -- queue --- sink !
---------) (-----------) ! (-------------) (---------) ! ! (---------)
disksrc ! ! mpeg1parse! (------------------------------------) (-------------
src --- sink !
---------) (-----------)
queue----- same for audio
then we play, create_plan happens, data is flowing and the "new_pad" signal is called
from mpeg1parse, gst_pipeline_pad_autoplug is called and the connection between
mpeg1parse and the videoqueue is made. same for audio.
voila. smame procedure for mp3/vorbis/avi/qt/mpeg2 etc...
Problems:
---------
this is obviously a very naive solution. the creation of the elements actually happens
beforehand. MPEG2, for one, fails because there are multiple possibilities to go
from the mpeg demuxer to audio/raw (ac3, mp3)
Also any intermedia elements like mixers (subtitles) are not possible because we
assume that after the common elements, the streams to not converge anymore.

289
docs/random/autoplug2
View file

@ -1,289 +0,0 @@
1) The Autoplugger API
----------------------
We'll first describe how to use the autoplugger. We will provide
a use case: autoplug an mpeg1 system stream for audio/video playback.
a) creating an autoplugger
--------------------------
Before any autoplugging can be done, you'll have to create an
autoplugger object. Autoplugger objects (autopluggers) are
provided by plugins and are created with gst_autoplugfactor_make().
GStreamer has provisions for two types of autopluggers:
- regular autopluggers, which act as a complex element construction
mechanism. They usually don't create threads and operate solely on
GstCaps* for the source and destination. The complex elements
created by regular autopluggers have src and sink pad compatible
with the requested GstCaps*.
- renderer autopluggers, which are designed to create a complex
object that can be used to playback media. Renderer autoplugged
complex elements have no src pads, only one sink pad.
We'll create a renderer autoplugger like this:
!
! GstAutoplug *autoplug;
!
! autoplug = gst_autoplugfactory_make ("staticrender");
!
b) finding out the source media type.
-------------------------------------
Before we can start the autoplugger, we have to find out the
source media type. This can be done using the typefind functions
provided by various plugins.
We will create a little pipeline to detect the media type by connecting
a disksrc element to a typefind element. The typefind element will
repeatedly call all registered typefind functions with the buffer it
receives on its sink pad. when a typefind function returns a non NULL
GstCaps*, that caps is set to the sink pad of the typefind element and
a signal is emitted to notify the app.
Due to caps negotiation, the disksrc will have the detected GstCaps*
set on its src pad.
We typically use a function like below to detect the type of a media stream
on an element (typically a disksrc). The function accepts a pipeline and the
element inside the pipeline on which the typefind should be performed (passing
a GstPad* is probably a better option FIXME).
!
! static GstCaps*
! gst_play_typefind (GstBin *bin, GstElement *element)
! {
! GstElement *typefind;
! GstCaps *caps = NULL;
!
! typefind = gst_elementfactory_make ("typefind", "typefind");
! g_return_val_if_fail (typefind != NULL, FALSE);
!
! gst_pad_connect (gst_element_get_pad (element, "src"),
! gst_element_get_pad (typefind, "sink"));
!
! gst_bin_add (bin, typefind);
!
! gst_element_set_state (GST_ELEMENT (bin), GST_STATE_PLAYING);
!
! // push a buffer... the have_type signal handler will set the found flag
! gst_bin_iterate (bin);
!
! gst_element_set_state (GST_ELEMENT (bin), GST_STATE_NULL);
!
! caps = gst_pad_get_caps (gst_element_get_pad (element, "src"));
!
! gst_pad_disconnect (gst_element_get_pad (element, "src"),
! gst_element_get_pad (typefind, "sink"));
! gst_bin_remove (bin, typefind);
! gst_object_unref (GST_OBJECT (typefind));
!
! return caps;
! }
!
Also note that the disksrc was added to the pipeline before calling this
typefind function.
When the function returns a non-NULL pointer, the media type has been
determined and autoplugging can commence.
Assume that in our mpeg1 use case the above function returns a GstCaps*
like:
!
! srccaps = GST_CAPS_NEW ("mpeg1system_typefind",
! "video/mpeg",
! "mpegversion", GST_PROPS_INT (1),
! "systemstream", GST_PROPS_BOOLEAN (TRUE)
! );
!
c) Performing the autoplugging
------------------------------
Since we use the renderer API, we have to create the output elements
that are going to be used as the final sink elements.
!
! osssink = gst_elementfactory_make("osssink", "play_audio");
! videosink = gst_elementfactory_make("xvideosink", "play_video");
!
We then create a complex element using the following code.
!
! new_element = gst_autoplug_to_renderers (autoplug,
! srccaps,
! videosink,
! osssink,
! NULL);
!
! if (!new_element) {
! g_print ("could not autoplug, no suitable codecs found...\n");
! exit (-1);
! }
!
2) Autoplugging internals
-------------------------
We will now describe the internals of the above gst_autoplug_to_renderers()
function call. This code is implemented in a plugin found in:
gst/autoplug/gststaticautoplugrender.c
a) phase1: create lists of factories.
---------------------------------------
The autoplugger will start with executing the following piece of
code:
!
! i = 0;
!
! for each sink:
! {
! sinkpad = take the first sinkpad of the sink (HACK)
!
! list[i] = gst_autoplug_caps (srccaps, sinkpad->caps);
!
! i++;
! }
!
gst_autoplug_caps will figure out (based on the padtemplates)
which elementfactories are needed to connect srccaps to sinkpad->caps
and will return them in a list.
The element list is created by using a modified shortest path algorithm
by Dijkstra (http://www.orie.cornell.edu/~or115/handouts/handout3/handout3.html).
The nodes of the graph are the elementfactories and the weight of the
arcs is based on the pad compatibility of the padtemplates of the
elementfactory. For incompatible elementfactories, we use a weight of
MAX_COST (999999) and for compatible padtemplates we use 1.
ex. we have two sinks with following caps:
!
! video/raw audio/raw
! "...." "...."
!
gst_autoplug_caps will figure out that for the first sink the following
elements are needed:
!
! mpeg1parse, mp1videoparse, mpeg_play
!
for the second sink the following is needed:
!
! mpeg1parse, mad
!
Note that for the audio connection the element list "mpeg1parse, mp3parse,
mpg123" would also connect the srccaps to the audiosink caps. Since the
"mpeg1parse, mad" list is shorter, it it always preferred by the autoplugger.
We now have two lists of elementfactories.
b) phase2: collect common elements from the lists and add them to a bin.
------------------------------------------------------------------------
The rationale is that from the lists we have created in phase1, there
must be some element that is a splitter and that it has to come first (HACK)
We try to find that element by comparing the lists until an element differs.
We start by creating a toplevel bin that is going to be our complex element.
In our use-case we find that mpeg1parse is an element common to both lists,
so we add it to the bin. We then try to find a good ghostpad for the resulting
complex element. This is done by looping over the sink pads of the first common
element and taking the pad that is compatible with the srcaps.
We end up with a bin like this:
!
! (----------------------)
! ! autoplug_bin !
! ! !
! ! (------------) !
! ! ! mpeg1parse ! !
! ! - sink ! !
! ! / (------------) !
! sink !
! (----------------------)
!
c) phase3: add remaining elements
---------------------------------
now we loop over all the list and try to add the remaining elements
(HACK) we always use a new thread for the elements when there is a common
element found.
if a new thread is needed (either because the previous element is a common
element or the object flag of the next element is set to GST_SUGGEST_THREAD)
we add a queue to the bin and we add a new thread. We add the elements to
the bin and connect them using gst_pipeline_pads_autoplug.
we finally arrive at the sink element and we're done.
ex.
we have just found our mpeg1parse common element, so we start a thread.
We add a queue to the bin and a new thread, we add the elements
mp1videoparse and mpeg_play to the thread. We arrive at the videosink, we
see that the SUGGEST_THREAD flag is set, we add a queue and a thread and
add the videosink in the thread.
the same procedure happens for the audio part. We are now left with the
following pipeline:
We will also have set a signal "new_pad" on the mpeg1parse element because
the element mp1videoparse could not be connected to the element just yet.
(---------------------------------------------------------------------------------------------)
!autoplug_bin !
! !
! (----------------------------------------) (------------) !
! !thread ! ! thread ! !
! (-----) ! (-------------) (---------) (-----) ! ! (---------)! !
! !queue! ! !mp1videoparse! !mpeg_play! !queue! ! ! !videosink!! !
! sink src-sink src-sink src-sink src-sink !! !
! (-----------) (-----) ! (-------------) (---------) (-----) ! ! (---------)! !
! ! mpeg1parse! (----------------------------------------) (------------) !
! - sink ! !
! / (-----------) !
sink (----------------------------------------) (------------) !
! !thread ! ! thread ! !
! (-----) ! (-------------) (-----) ! ! (---------)! !
! !queue! ! !mad ! !queue! ! ! !videosink!! !
! sink src-sink src ------------ sink src-sink !! !
! (-----) ! (-------------) (-----) ! ! (---------)! !
! (----------------------------------------) (------------) !
(---------------------------------------------------------------------------------------------)
The autoplugger will return the autoplug_bin. the app will then connect the
disksrc to the sinkpad of the autoplugged bin.
Then we play, create_plan happens, data is flowing and the "new_pad" signal is called
from mpeg1parse, gst_pipeline_pad_autoplug is called and the connection between
mpeg1parse and the videoqueue is made. same for audio.
Et voila. same procedure for mp3/vorbis/avi/qt/mpeg2 etc...

85
docs/random/bbb/streamselection
View file

@ -1,85 +0,0 @@
Stream selection
=
1. Problem
URIs (that being either a media stream or a media stream plus subtitle) can
contain multiple streams of a type (audio, subtitle). A user has to be given
the option of selecting one of those streams (or none altogether).
2. Implementation ideas
Stream selection, in GStreamer, has to be integrated at the player plugging
level, which is (in the case of Totem) playbin. Playbin offers a feature to
'mute' a stream (which means that no processing is done on that stream
altogether, saving the decoding step). Currently, playbin will select the
first occurrence of a stream type and mute all others. A queue is connected
(for pre-roll) to the active stream. What is missing here is a way to change
the active stream.
Playbin interface - one possible interface could simply consist of a bunch of
GObject properties: 'textstream' and 'audiostream', both integer. The number
of available streams can be retrieved using the 'stream-info' GObject property.
Similar to the 'nstreams' property, we could add utility GObject properties
for getting the number of available audio/text streams ('naudiostreams' and
'ntextstreams'). Names of these streams (like language name or so) can be
added as an additional GObject property to streaminfo. Some container
formats contain such names internally. Alternatively, we could allow those
to be user-settable as well (for .sub files).
On a set of either of these properties, playbasebin would mute the old
selected stream (if any), unmute the newly selected stream (if any) and
replug the preroll queue. The queue itself is disabled as well if no new
stream was linked. Alternatively, a switch-like element is used, which
requires no replugging. Pad disabling/enabling is then enough. This also
makes relinking less painful. The switch-like element needs to proxy the
active pads' caps. However, since those caps are (in practice) always the
same across streams, caps setting will (inside the core) immediately
return success.
The switch-like element simply works like this:
=
static void
loop_func (GstElement * element)
{
GList *inpads;
GstPad *pad;
GstData *data;
for (inpads = ..; inpads != NULL; inpads = inpads->next) {
pad = inpads->data;
if (!GST_PAD_IS_USABLE (pad))
continue;
/* you'd also want some way to synchronize the inputs... */
data = gst_pad_pull (pad);
if (is_active_pad (pad))
gst_pad_push (srcpad, data);
else
gst_data_unref (data);
}
}
=
It'd require an active-stream property itself, which (when set) takes
care of doing renegotiation and so on. Using internal pad linkage is
extremely useful here, and requires little code in the switch-like
element itself. Note that there is a slight bit of duplication in the
playbin interface and the switch-like element interface, but that's "just
the way it is".
The implementation of the switch-like element could initially be local to
playbin, until it has been cleaned up and confirmed to be useful to a
wider audience. This allows a lot of experimenting with interfaces because
we won't be forced to maintain a stable interface.
The current 'switch' element (gst-plugins/gst/switch/) already does a few
of those operations, but stream synchronization, re-negotiation on stream
changes, internal pad linkage and some other things are completely missing.
If we're gonna use this element, it'll need a large overhaul. The choice of
writing a new element or using an existing element as basis, and also the
choice of whether or not to make this element local to playbin, should be
based on technical merits and cost/effect analysis and not on personal
pride.
Notes:
* seamless has the same switch-like element, but it's chain-based. Apart
from scheduler considerations, this is a good idea, but limits its use
(making either good docs and abuse-prevention [see multifilesrc] or
private-to-playbin a prerequisite).
* maybe text-* properties need to be renamed to subtitle-*.
3. Written by
Ronald S. Bultje <rbultje@ronald.bitfreak.net> - Jan. 2nd, 2005.

105
docs/random/bbb/subtitles
View file

@ -1,105 +0,0 @@
Subtitles
=========
1. Problem
GStreamer currently does not support subtitles.
2. Proposed solution
- Elements
- Text-overlay
- Autoplugging
- Scheduling
- Stream selection
The first thing we'll need is subtitle awareness. I'll focus on AVI/MKV/OGM
here, because I know how that works. The same methods apply to DVD subtitles
as well. The matroska demuxer (and Ogg) will need subtitle awareness. For
AVI, this is not needed. Secondly, we'll need subtitle stream parsers (for
all popular subtitle formats), that can deal both with parsed streams (MKV,
OGM) as well as .sub file chunks (AVI). Sample code is available in
gst-sandbox/textoverlay/.
Secondly, we'll need a textoverlay filter that can take text and video and
blits text on video. We have several such elements (e.g. the cairo-based
element) in gst-plugins already. Those might need some updates to work
exactly as expected.
Thirdly, playbin will need to handle all that. We expect subtitle streams
to end up as subimages or plain text (or xhtml text). Note that playbin
should also allow access to the unblitted subtitle as text (if available)
for accessibility purposes.
A problem popping up is that subtitles are no continuous streams. This is
especially noticeable in the MKV/OGM case, because there the input of data
depends on the other streams, so we'll only notice delays inside an element
when we've received the next data chunk. There are two possible solutions:
using timestamped filler events or using decoupled subtitle overlay elements
(bins, probably). The first has as a difficulty that it only works well in
the AVI/.sub case, where we will notice discontinuities before they become
problematic. The second is more difficult to implement, but works for both
cases.
A) fillers
Imagine that two subtitles come after each other, with 10 seconds of no-data
in between. By parsing a .sub file, we would notice immediately and we could
send a filler event (or empty data) with a timestamp and duration in between.
B) decoupled
Imagine this text element:
------------------------------
video ----- | actual element |out
| / -----------------|
text - - |
------------------------------
where the text pad is decoupled, like a queue. When no text data is available,
the pad will have received no data, and the element will render no subtitles.
The actual element can be a bin here, containing another subtitle rendering
element. Disadvantage: it requires threading, and the element itself is (in
concept) kinda gross. The element can be embedded in playbin to hide this
fact (i.e. not be available outside the scope of playbin).
Whichever solution we take, it'll require effort from the implementer.
Scheduling (process, not implementation) knowledge is assumed.
Stream selection is a problem that audio has, too. We'll need a solution for
this at the playback bin level, e.g. playbin. By muting all unused streams
and dynamically unmuting the selected stream, this is easily solved. Note
that synchronization needs to be checked in this case. The solution is not
hard, but someone has to do it.
3. Written by
Ronald S. Bultje <rbultje@ronald.bitfreak.net>, Dec. 25th, 2004
Appendix A: random IRC addition
<Company> intersting question: would it be a good idea to have a "max-buffer-length" property?
<Company> that way demuxewrs would now how often they'd need to generate filler events
<Company> s/now/know/
<BBB> hm...
<BBB> I don't think it's good to make that variable
<Company> dunno
<Company> (i'm btw always looking at this from the midi perspective, too)
<Company> (because both subtitles and midi are basically the same in this regard)
<BBB> and do you mean 'after the stream has advanced <time> and we didn't read a new subtitle in this mkv stream, we should send a filler'?
<Company> yeah
<BBB> it goes for avi with large init_delay values, too
<Company> so you don't need to send fillers every frame
<BBB> right
<BBB> cant' we just set that to, for example, 1s?
<BBB> it's fairly random, but still
<Company> that's another option, too
<Company> though you could write all file parsers with max-delay=MAXINT
<Company> would make them a lot easier
<BBB> it's true that queue size, for example, depends on this value
<BBB> e.g. if you make this 5s and set queue size to 1s, it'll hang
<Company> right
<BBB> whereas if you set it to 1s and queue size to 5s, you waste space
<BBB> :)
<BBB> you ought to set it to max-delay * (n_streams + 1)
<BBB> or so
<BBB> or -1
<BBB> I forgot
<BBB> ohwell
<Company> if you'd use filtercaps and queue sizes in your app, you could at least work around deadlocks
<BBB> yeah
<Company> though ideally it should just work of course...
<BBB> good point...

19
docs/random/buffers
View file

@ -1,19 +0,0 @@
METADATA ON BUFFERS IS OUTDATED
-------------------------------
Buffer mutability properties are the most important part of gst, and
similarly are the most complex.
The simple case is that a buffer is created, memory allocated, data put
in it, and passed to the next filter. That filter reads the data, does
something (like creating a new buffer and decoding into it), and
unreferences the buffer. This causes the data to be freed and the buffer
to be destroyed.
A more complex case is when the filter modifies the data in place. It
does so and simply passes on the buffer to the next element. This is just
as easy to deal with.
If the second filter adds metadata to the buffer, it just has to add the
pointer to the list. The next element simply traverses the list and

209
docs/random/caps
View file

@ -1,209 +0,0 @@
SOME OF THE FIRST IDEAS, PRETTY OUTDATED
----------------------------------------
During the course of a discussion on IRC, it turned out
that there are many possible ways to handle the capabilities.
A capability is basically a set of properties attached to a
mimetype in order to more closely describe the mimetype.
Capabilities are supposed to be attached to pads so that the
autoplugging algorithm has more specific information to connect
compatible pads.
We present 3 possible implementation for the capabilities. we need
to pick one of them.
1. static capabilities
----------------------
When an element is created, it creates its pads like:
mpg123->sinkpad = gst_pad_new ("sink", GST_PAD_SINK);
gst_element_add_pad (GST_ELEMENT (mpg123), mpg123->sinkpad);
mpg123->srcpad = gst_pad_new ("src", GST_PAD_SRC);
gst_element_add_pad (GST_ELEMENT (mpg123), mpg123->srcpad);
In the static capabilities case, it will attach a GstCaps* structure
to the pad. The GstCaps structure in the above example might look like:
static GstCapsFactory mpg123_sink_caps = {
"audio/mp3",
"layer", GST_CAPS_INT_RANGE (1, 3),
"bitrate", GST_CAPS_INT_RANGE (8, 320),
NULL
};
with
mpg123sinkcaps = gst_caps_register (mpg123_sink_caps);
the factory can be converted into a GstCaps* structure. The
GstCaps* structure is attached to the pad with:
gst_pad_add_caps (mpg123->sinkpad, mpg123sinkcaps);
The GstElement would then have a sinkpad with the given
mimetype (audio/mp3) and with the capabilitities of accepting
mpeg layer 1 to 3 and a bitrate from 8 up to 320 Kbps.
Likewise, the src pad could be set up in the same way. An
example capability factory could look like:
static GstCapsFactory mpg123_src_caps = {
"audio/raw",
"format", GST_CAPS_BITFIELD (...),
"depth", GST_CAPS_INT (16),
"rate", GST_CAPS_INT_RANGE (4000, 96000),
"channels", GST_CAPS_INT_RANGE (1, 2),
NULL
};
All GstElements would present their pads with the appropriate
capabilities structure.
The autoplugger would then proceed (once the source media type
is known with a typefind function) in finding all the elements
with compatible pads and connecting them into a pipeline.
All elements of the complete pipeline could then be constructed
with one single pass. No new elements should be added to the
pipeline because we can figure out all the possibilities using the
pad capabilities.
We call this the static case because the capabilities of the pads
are supposed to stay the same after creating the element.
While the ability to completely setup the pipeline before actually
starting playback is an advantage regarding performance, one obvious
problem with this setup is that the static case may be too static in
some cases. We can illustrate this with the following setup:
----------) (------------
mpg123 ! ! audiosink
src sink
! !
----------) (------------
The mpg123 element has its src capabilities set up as mpg123_src_caps
in the above example.
The audio renderer has its capabilities set up with the following
factory:
static GstCapsFactory audio_sink_caps = {
"audio/raw",
"format", GST_CAPS_BITFIELD (...),
"depth", GST_CAPS_INT (16),
"rate", GST_CAPS_INT_RANGE (22000, 44000),
"channels", GST_CAPS_INT_RANGE (1, 2),
NULL
};
The static autoplugger has to be careful when connecting the mpg123
element with the audiosink because it is theoretically possible that
the mpg123 element outputs raw audio with a rate that cannot be
handled by the audiosink (ex. 4000KHz). In the absence of another
audiosink with more capabilities, the autoplugging of this simple
pipeline will not be possible and would fail.
the autoplugging algorithm would probably select another element to
insert between the mpg123 element and the audiosink in order to handle
the (uncommon) case of a rate conversion (audioscaler).
It is clear that this static setup might even fail or work suboptimal
for even the common case and should therefore be considered as too
restrictive.
2. dynamic capabilities
-----------------------
The idea of dynamic capabilities is that the capabilities are not set
at element create time but rather while the pipeline is running.
An element would still list its mime type using:
gst_pad_add_type_id(mpg123->sinkpad, mp3type);
The idea would then be that a rough draft of the pipeline would be
built after the media type of the stream has been detected with the
typefind functions. The rough draft would consist of laying out a
global plan to reach the renderer(s). this plan would basically list
the set of conversions that have to be performed. (mime-type to
mime-type conversion).
Elements that accept the src mime-type are tried by giving it a buffer.
If the element accepts the buffer, it will set its capabilities for
both the sink pad and the src pad. At that time other elements can be
tried and added to the src pad, until we reach the renderer. As usual
one has to be careful to add just the minimum amount of elements to
reach the renderer. The global plan will help with that.
Since we basically do not use the capabilities of the sink pad one has
to question the need for sink pad capabilities in the first place.
We might also have a hard time trying different elements until we find
a compatible one that does not cause a dead end at some point.
3. combined setup
-----------------
This combined setup will minimise the effort needed to try different
elements encountered by option 2 while still allowing a more dynamic
setup based on the actual media stream we are handling.
The combined setup will list/add the sink capabilities at create time.
It will only set the mime-type of its src pads.
As with option2, a global plan will be built. At runtime the src pads
will actually specify the capabilities they need for any element that
wants to be connected to its source pads.
In this case we specify the capabilities for all the sink pads of an
element at create time. The capabilities of the src pads would only
become available when data has been processed by the element.
The autoplugger would then be able to choose an element that can handle
the capability listed by the src pad.
in our previous example:
----------) (------------
mpg123 ! ! audiosink
src sink
! !
----------) (------------
the audiosink element would specify its sink pad capabilities at create
time, while the mpg123 elements src pad would not yet have any capabilities
set.
When data is handled by the mpg123 element, a capability would be added to
the mpg123 src pad. This capability might be:
static GstCapsFactory mpg123_src_caps = {
"audio/raw",
"format", GST_CAPS_INT (S16),
"depth", GST_CAPS_INT (16),
"rate", GST_CAPS_INT (44000),
"channels", GST_CAPS_INT (2),
NULL
};
This capability would be compatible with the audiosinks sinkpad capabilities
and the autoplugger would therefore be able to connect the two elements.
While allowing a more flexible setup with option3, compared to option1, we
introduce a slightly higher overhead when we need to dynamically connect
elements. This overhead will not be as big as option2 because we don't
have to 'try' elements.
so:
src caps: added at runtime to list the caps needed for an element that
wants to connect to this pad.
sink caps: the (static) capabilities that this sinkpad has.

1132
docs/random/caps.dia
View file

File diff suppressed because it is too large Load diff

231
docs/random/caps2
View file

@ -1,231 +0,0 @@
The new caps code uses the type name GstCaps2 and the function
names gst_caps2_*(). Before the CAPS branch is merged, there
will be a global change from caps2 to caps. Since GstCaps is
no longer defined, it no longer compiles, thus highlighting
exactly what needs to be changed in an element.
Pad Templates:
Old style:
GST_PAD_TEMPLATE_FACTORY (fakesrc_src_factory,
"src%d",
GST_PAD_SRC,
GST_PAD_REQUEST,
GST_CAPS_ANY
);
New style:
GstStaticPadTemplate fakesrc_src_template = GST_STATIC_PAD_TEMPLATE (
"src%d",
GST_PAD_SRC,
GST_PAD_REQUEST,
GST_STATIC_CAPS2_ANY
);
The old style defined a function called fakesrc_src_factory(), which,
when called, returns a pad template. The new style defines a
GstStaticPadTemplate, which can be converted to a GstPadTemplate
by the function gst_static_pad_template_get(). The 4th argument
is also different -- previously it would call the GST_CAPS_NEW()
function. Now it is a GstStaticCaps.
Not every pad template can be converted to a GstStaticPadTemplate,
particularly those which create caps from another source at runtime,
such as videotestsrc.
Caps:
Old style:
GST_CAPS_NEW (
"sinesrc_src",
"audio/x-raw-int",
"endianness", GST_PROPS_INT (G_BYTE_ORDER),
"signed", GST_PROPS_BOOLEAN (TRUE),
"width", GST_PROPS_INT (16),
"depth", GST_PROPS_INT (16),
"rate", GST_PROPS_INT_RANGE (8000, 48000),
"channels", GST_PROPS_INT (1)
)
New style:
GST_STATIC_CAPS2 ( "audio/x-raw-int, "
"endianness = (int) " G_STRINGIFY (G_BYTE_ORDER) ", "
"signed = (boolean) true, "
"width = (int) 16, "
"depth = (int) 16, "
"rate = (int) [ 8000, 48000 ], "
"channels = (int) 1"
)
The old style calls a function that creates a GstCaps. The new style
stores a string in a GstStaticCaps2, and this string is converted to
a caps in the function gst_static_caps2_get().
Note that the old caps name is no longer used.
Old style:
caps = GST_CAPS_NEW ("videotestsrc_filter",
"video/x-raw-rgb",
"bpp", GST_PROPS_INT(format->bitspp),
"endianness", GST_PROPS_INT(endianness),
"depth", GST_PROPS_INT(format->depth),
"red_mask", GST_PROPS_INT(format->red_mask),
"green_mask", GST_PROPS_INT(format->green_mask),
"blue_mask", GST_PROPS_INT(format->blue_mask));
New style:
caps = gst_caps2_new_simple("video/x-raw-rgb",
"bpp", G_TYPE_INT, format->bitspp,
"endianness", G_TYPE_INT, endianness,
"depth", G_TYPE_INT, format->depth,
"red_mask", G_TYPE_INT, format->red_mask,
"green_mask", G_TYPE_INT, format->green_mask,
"blue_mask", G_TYPE_INT, format->blue_mask);
Not everything can be converted in this way, especially lists and
ranges.
IMPLEMENTATION
Pad Capabilities (caps) are mathematical sets that represent all the
possible stream types that a pad can use. These general sets are
represented by unions of simpler sets known as caps structures. Each
caps structure has a media type (e.g., "audio/mpeg") and a number of
properties. Each property has a name and a GValue. In normal
circumstances, the GValue will have the types int, boolean, string,
fourcc, and double. Simple sets are constructed by using GValues
that are lists of other GValues, or the special types that represent
int ranges and double ranges.
A "fixed" caps represents exactly one media format. This means that
the caps is a union of exactly one caps structure, and each property
in the caps structure is a simple type, i.e., no ranges or lists.
There are two special caps values, "ANY" which represents the union
of all stream types, and "EMPTY", which represents the set of no
stream types. The ANY caps is often used on generic elements that
handle any type of data (e.g., filesrc and filesink). The EMPTY
caps is the return value of gst_caps_intersect(), when the two
given caps do not intersect. In many cases, using EMPTY is invalid.
CAPS NEGOTIATION
Elements provide information to the core about what stream formats
they understand in four ways: the caps in the pad templates, the
caps returned by a pad's getcaps function, accepting/denying
a given caps in the pad link function, and a new fixate function.
The pad template caps should be the union of caps a pad supports
in any potential situation. Simultaneously, these caps should be
as specific as possible, since it is used to decide which elements
to attempt for autoplugging, without having to load the element.
The pad template caps are generally determined at compile time, but
might be actually computed at run-time from other information.
The getcaps() function returns the caps supported by a given pad,
in the context of the element's state, its link to other elements,
and the devices or files it has opened. These caps must be a
subset of the pad template caps. In the NULL state with no links,
the getcaps function should ideally return the same caps as the
pad template. In rare circumstances, an object property can affect
the caps returned by getcaps, but this is discouraged. For most
filters, the caps returned by getcaps is directly affected by the
allowed caps on other pads. For demuxers and decoders, the caps
returned by the srcpad's getcaps function is directly related to
the stream data. Again, getcaps should return the most specific
caps it reasonably can, since this helps with autoplugging.
The pad link function is the last step in negotiating caps. The
core calls the pad link function with a fixed caps, meaning that
the stream format is precisely defined, with the caps having one
structure, with no fields that are ranges or lists.
There is also a new pad function "fixate", which is used to help
choose a fixed caps from a non-fixed caps. This is called in
situations where normal negotiation cannot decide on a fixed caps.
You should almost never implement a fixate function, please ask
me if it is appropriate for your case. Fixate functions are called
iteratively on the pads until a fixed caps is found. Fixate functions
are called with a const caps, and should return a caps that is a
strict subset of the given caps. That is, the function should
create a caps that is "more fixed" than previously, but does not
have to return fixed caps. If the fixate function can't provide
more fixed caps, it should return NULL.
Checklist for getcaps:
- The getcaps function prototype no longer has the caps parameter.
Remove it.
- The returned caps is owned by the caller. Make sure you don't
keep a pointer to the caps.
- Make sure that the getcaps function can be called safely in each
element state (NULL, READY, PAUSED, PLAYING), and for any element
configuration (properties, links, devices/files opened or not,
error state, etc.)
- Make sure that the returned caps do not depend on the caps that
indicate the stream type that the pad is currently using.
Checklist for pad_link:
- The pad link function prototypes uses a const GstCaps *.
- Pad link functions are called with fixed caps. There's no need
to check for this. This means that you can assume that the caps
is not ANY or EMPTY, and that there is exactly one structure in
the caps, and that all the fields in the structure are fixed.
- Pad link functions are called with caps that are a subset of the
most recent return value of the pad's getcaps function. Generally,
the getcaps function was called immediately prior to calling the
src_link function. For 0.8, you can assume that nothing has changed
in your element that would cause a change to the return value of
getcaps.
- the return value GST_PAD_LINK_OK should be used when the caps are
acceptable, and you've extracted all the necessary information from
the caps and set the element's internal state appropriately.
- the return value GST_PAD_LINK_REFUSED should be used when the caps
are unacceptable for whatever reason.
- the return value GST_PAD_LINK_DELAYED should be used when the
element is in a state where it can't determine whether the caps
are acceptable or not. This is often used if the element needs
to open a device or process data before determining acceptable
caps.
- the pad_link function must not call gst_caps_try_set_caps() on
the pad that was specified as a parameter.
- the pad_link function may (and often should) call
gst_caps_try_set_caps() on pads that are not specified as the
pad parameter.
Checklist for fixate:
- Make sure you actually should be using a fixate function. Fixate
functions are reasonable for non-fixed primary sources, such as
videotestsrc, v4lsrc, and osssrc.
- The user_data parameter is mainly used for user-provided fixate
function. It should be ignored in element fixate functions.

BIN
docs/random/classes.dia
View file

Binary file not shown.

165
docs/random/ds/0.9-planning
View file

@ -1,165 +0,0 @@
Scheduling:
- remove loop/get/chain from GstElement and add a "iterate" method.
The iterate method is called with the event (or events) that
triggered it, performs some action, and resets the events (file
descriptors becoming readable, semaphores, pads becoming readable
or writable, or a time occurs).
- Add GstLoopElement, GstChainElement, etc. for compatibility.
- Remove existing state handling and create 2 states, "playing" and
"stopped". "playing" means that the iterate() method of the
element may be called, that is, the element is allowed to move
buffers, negotiate, etc. "stopped" means that no gstreamer-ish
things happen to an element, only gobject-ish. A separate
reset() method will handle the difference between READY and NULL.
- Add a flag "ready" to GstElement that is under the control of the
element. If the element is ready to stream, it sets this flag,
and the entire pipeline starts streaming. (This is basically
the difference between PAUSED and PLAYING.) For example, osssink
won't set the ready flag until the device is opened and there is
a buffer available to write to the device.
- Scheduling of elements and movement of buffers will be timed by
clocks.
Example #1:
Pipeline: sinesrc ! osssink
- The application creates the pipeline and sets it to "playing".
- The clock is created and set to "paused".
- sinesrc.iterate() decides to watch for the event "src pad
negotiation" and sets the available caps on the pad.
- osssink.iterate() opens device, determines available caps, and
sets the available caps on the pad. Then it decides to wait for
"sink pad negotiation".
- The scheduler realizes that the two elements are waiting for
negotiation, so it negotiates the link.
- sinesrc.iterate() sets the "ready" flag (because it needs no more
preparation to stream) and decides to watch for the event "src
pad ready to accept buffer".
- osssink.iterate() decides to watch for the event "sink pad has
available buffer".
- The scheduler realizes that sinesrc.srcpad is now ready, so it
calls sinesrc.iterate()
- sinesrc.iterate() creates a buffer and pushes it, and decides to
wait for the same event.
- The scheduler realizes that osssink.sinkpad now has a buffer, so
it calls osssink.iterate().
- osssink.iterate() is now ready to stream, so it sets the "ready"
flag and waits for "time 0".
- The pipeline is now completely ready, so the clock may be
started. A signal is fired to let the application know this
(and possibly change the default behavior).
- The clock starts with the time 0. The scheduler realizes this,
and decides to schedule osssink.
- osssink.iterate() is called, and writes the buffer to the device.
This starts the clock counting. (Actually, the buffer could be
written by the clock code, since presumably the clock is related
to osssink.) iterate() then waits for "sink pad has available
buffer".
We're now basically in streaming mode. A streaming cycle:
- osssink.iterate() decides the audio output buffer is full enough,
so it waits for "time X", where X is the time when the output
buffer will be below some threshold.
- osssink.iterate() waits for "sink pad has available buffer"
- sinesrc.iterate() creates and pushes a buffer, then waits for
"src pad ready".
Further ideas:
- osssink can set a hard deadline time, which means that if it is
not scheduled before that time, you'll get a skip. Skipping
involves setting osssink to "not ready" and pauses the clock.
Then the scheduler needs to go through the same process as above
to start the clock.
- As a shortcut, osssink can say "I need a buffer on the sinkpad
at time X". This information can be passed upstream, and be used
in filters -- filter.sinkpad says "I need a buffer at time X-N",
where N is the latency of the filter.
Example #2:
Pipeline: osssrc ! osssink
- The application creates the pipeline and sets it to "playing".
- The clock is created and set to "paused".
- negotiation happens roughly as in example #1, although osssrc
additionally opens and prepares the device.
- osssrc.iterate() sets the "ready" flag (because it needs no more
preparation to stream) and waits for "time 0", since it presumably
can't wait for the file descriptor (audio input hasn't been
enabled on the device yet.)
- osssink.iterate() decides to watch for the event "sink pad has
available buffer".
- The scheduler realizes the deadlock and (somehow) tells osssink
that it can't pre-roll. (This needs more work) In other words,
osssink can't be the clock master, but only a clock slave.
- osssink.iterates() agrees to start at time SOME_LATENCY, sets the
"ready" flag, and waits for a buffer on its sink pad.
- The pipeline is now completely ready, so the clock may be
started. A signal is fired to let the application know this
(and possibly change the default behavior).
- The clock starting causes two things to happen: osssrc starts
the recording of data, and osssink starts the outputting of data.
The data being output is a chunk of silence equal to SOME_LATENCY.
- osssrc.iterate() is called for "time 0", does nothing, and waits
on the file descriptor (via the scheduler, of course). All waiting
on file descriptors should have an associated timeout.
- osssrc.iterate() is called when the file descriptor is ready,
reads a chunk of data, and pushes the buffer. It then waits for
its file descriptor to be ready.
- osssink.iterate() is called
Evil:
fakesrc ! tee ! fakesink tee0. ! never_accept_a_buffer_sink
sinesrc ! osssink videotestsrc ! ximagesink
fakesrc ! fakesink (pausing)
sinesrc ! identity ! osssink

30
docs/random/ds/buffer_locking
View file

@ -1,30 +0,0 @@
Buffers should have readlocks and writelocks to enforce
GST_BUFFER_DONTKEEP and relax the restriction that buffers with
multiple refcounts are read-only.
Example:
videotestsrc ! ximagesink
videotestsrc requests a buffer from its src pad
ximagesink creates a buffer (refcount:1 readlock:0 writelock:0)
videotestsrc writelocks it (refcount:1 readlock:0 writelock:1)
videotestsrc writes to the buffer
videotestsrc un-writelocks it (refcount:1 readlock:0 writelock:0)
ximagesink readlocks it (refcount:1 readlock:1 writelock:0)
ximagesink writes it to the screen
ximagesink un-readlocks it (refcount:1 readlock:0 writelock:0)

57
docs/random/ds/bufferpools
View file

@ -1,57 +0,0 @@
On Thu, Jan 08, 2004 at 04:10:00PM +0100, Julien MOUTTE wrote:
>
> Hi David,
>
> I'd like to implement bufferpools again in x[v]imagesink asap.. Could
> you please point me to a template/doc on how to do that ?
>
> The best for me would be a simple testcase showing how to use the buffer
> free methods to replace bufferpools.
x[v]imagesink should call gst_pad_set_bufferalloc_function() on
their sink pads with a bufferalloc implementation. This bufferalloc
function is to allocate buffers that _peers_ will send _to_ that pad.
A trivial version of a bufferalloc function, i.e., one that just
allocates normal buffers:
static GstBuffer *
gst_ximagesink_sink_bufferalloc (GstPad *pad, guint64 offset, guint
size)
{
GstBuffer *buffer;
buffer = gst_buffer_new_and_alloc (size);
GST_DATA_FREE_FUNC (data) = gst_ximagesink_buffer_free;
GST_BUFFER_POOL_PRIVATE (data) = ximagesink; /* whatever */
return buffer;
}
static void
gst_ximagesink_buffer_free (GstData *data)
{
g_free (GST_BUFFER_DATA (data));
}
The hard part is going through each element, and every time it
allocates a buffer using gst_buffer_new_and_alloc () that is then
sent to a sink pad, the call should be replaced with
gst_pad_alloc_buffer (sinkpad, offset, size).
dave...
-------------------------------------------------------
This SF.net email is sponsored by: Perforce Software.
Perforce is the Fast Software Configuration Management System offering
advanced branching capabilities and atomic changes on 50+ platforms.
Free Eval! http://www.perforce.com/perforce/loadprog.html
_______________________________________________
gstreamer-devel mailing list
gstreamer-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

50
docs/random/ds/categories
View file

@ -1,50 +0,0 @@
Element Categories:
-------------------
Decoder:
Encoder:
Converter:
A converter has one or more source pads and one or more sink pads.
Converter pads may have different caps templates. Converters are
expected
Filter:
A filter has one source and one sink pad. These pads have the same
caps and (thus) the same caps template.
Filters generally do not handle events.
Filters may have interfaces.
Filters are generally not autoplugged unless they have interfaces.
Source:
A source has one source pad and no sink pads.
Sources usually handle events.
Sources may have interfaces.
Sources are not autoplugged.
Sink:
A sink has one sink pad and no source pads.
Sources usually handle events.
Sources may have interfaces.
Sinks are not autoplugged.
Converter/Colorspace

43
docs/random/ds/element-checklist
View file

@ -1,43 +0,0 @@
This is a list of things to check in elements.
1. Add a "Element-Checklist-Version: 1" comment
This comment is to indicate which items on this checklist have been
checked, so that future checkers can skip things that you've already
carefully checked. This number should obviously be updated whenever
you do additional checks. Ideally, you should not do checks out of
order, but using "1, also 4-6" is acceptable.
2. Each pad should have a pad template
Create and register a pad template to the element class.
3. config.h
Make sure that each .c file includes config.h (conditionally on
HAVE_CONFIG_H). Make sure that each .h file does _not_ include
config.h.
4. src event handling
Every element having multiple sink pads or having a specific need to handle
src events in a different manner than the default event handler should do that
correctly. Correctly means that :
- It should not leak the event if it is handled (gst_event_unref)
- It should call return gst_pad_event_default (pad, event) for all non handled
localy events. Otherwise it breaks navigation, seeking, etc...
- it should return a TRUE/FALSE boolean if the localy handled event was handled.
other ideas:
- plugins should avoid using gst_caps_to_string() in debug statement.
They should use %"GST_PTR_FORMAT" instead. Check all usage for leaks.

15
docs/random/ds/registry
View file

@ -1,15 +0,0 @@
Ideas for a new registry system:
- each plugin has an associated registry XML file. Each xml file in
a plugin directory is loaded to create the in-core registry.
- needed for avidemux:
<Company> plugin x create some registry information that says stuff like:
<Company> "caps x are fourcc FOOO in avi"
<Company> or "caps y may be contained in id3 tagged files"
<Company> or "caps z is a valid format to put in an ogg container"

79
docs/random/ds/roadmap
View file

@ -1,79 +0,0 @@
After some discussion on IRC, I think it's time to propose a
formal plan for the next 6 months or so to the list.
I think most people are in agreement that we need a period of time
in which we can focus on improving non-core bits (like elements,
schedulers, autopluggers, applications, etc.) In the last cycle,
there was a lot of time spent with a significant number of plugins
broken, and it's realistic to assume that this could happen again
in a 0.9 unstable series.
What I propose is:
- We continue to develop the 0.8.x series as HEAD, with the obvious
requirement that all changes be ABI/API compatible.
- API additions are encouraged, as long as they are well-thought-out.
- Significant API additions should be developed on a separate branch
(not HEAD) to test out any bugs.
In mid-August (or so, in order to coordinate with GNOME-2.8), we have
two options:
- Continue with 0.8.x releases, obviously ABI compatible with 0.8.0.
- or, remove deprecated functions, readjust the padding on structures,
perhaps make a few additional ABI changes [1], and quickly go to
0.10.0.
I prefer the latter, although we don't have to decide that until
later. In either case, there should be no API changes that affect
more than a bare minimum of elements or applications.
A few things that we won't be able to do without a true unstable
branch are:
- using GstStructure for all GstEvents.
- significant clock changes
- significant scheduling changes
- separation of headers into application and plugin headers
- anything that requires modification of every plugin
There are perils with having HEAD being the stable branch,
specifically that bugs can creep in and accidentally cause regressions
in releases. I'm hoping that the introduction of media regression
testing and also the development of new testsuites will keep this
to a minimum. Don't forget that accidental bugs that get into
releases typically cause rude IRC conversations, which we really
don't need. Please keep the bugs (and the rudeness) to a minimum.
Also, keep in mind that we will have to live with 0.8's unfixable
bugs for an entire year.[2]
dave...
--
[1] I'm thinking about making GstData a subclass of GTypeInstance or
GObject.
[2] But then, 0.6 was 1 year ago, and felt a lot more buggy when it
was released.
-------------------------------------------------------
This SF.Net email is sponsored by: IBM Linux Tutorials
Free Linux tutorial presented by Daniel Robbins, President and CEO of
GenToo technologies. Learn everything from fundamentals to system
administration.http://ads.osdn.com/?ad_id=1470&alloc_id=3638&op=click
_______________________________________________
gstreamer-devel mailing list
gstreamer-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/gstreamer-devel

18
docs/random/ensonic/audiobaseclasses.txt
View file

@ -1,18 +0,0 @@
$Id$
= audio base classes =
We have gst-plugins-base/gst-libs/gst/audio with some audio helper and base
classes.
audiofilter: for src_caps=sink_caps
basetransform: *
= new stuff =
= todo =
* mv gstaudiofilterexample -> gst-template
* make more elements using the baseclass
* base: audiorate, audioresample, volume
* good: audiopanorama

14
docs/random/ensonic/dparams.txt
View file

@ -1,14 +0,0 @@
$Id$
* controller changes
* handling of G_TYPE_XXX is distributed over
gst-controller.c::gst_controlled_property_set_interpolation_mode()
gst-controller.c::gst_controlled_property_new
gst-interpolation.c
* it would be better to put handlers for each type into a struct
and register it into a hashmap with the G_TYPE as the key
* functions in gst-controller.c could try the key,
if no handler is registered the try the type_parent and so on
* implement quadric/cubic interpolation

56
docs/random/ensonic/draft-registry-change-hooks.txt
View file

@ -1,56 +0,0 @@
Registry Change Hooks
----------------------
This document proposes a mechanism to register on registry updates.
Last edited: 2009-11-09 Stefan Kost
Current Behaviour
-----------------
When new plugins are installed or some are removed, the next application that
runs rebuiilds the registry. Any application that runs later on won't know about
it.
Problem
-------
Some usecases need to cache gstreamer capabilities. They need to know when the
cache needs to be rebuilt. We could either export the registry mtime as api or
do the following.
This gets more useful once we can have more plugin/element metadata in the
registry (e.g. mime-types for demuxers and codecs, see
http://www.rfc-editor.org/rfc/rfc4281.txt).
Proposal
--------
We add a ~/.gstreamer-0.10/registry-hooks directory. Applications can install
scripts there. When the registry is rebuild, those scripts are run async in no
specific order. There is no feedback channel (return values, stdout/err).
Examples
---------
gst-inspector
-------------
it would install a script that simply deletes its cachefile.
gupnp
-----
it would install a script that runs gst-inspect-0.10 --print-plugin-auto-install-info
and post process it.
totem,banshee,rythmbox
-----------------------
they could tune the "MimeType=" lists in their desktop files
Open Items
----------
Should we have a gst-registry-hook script with --install/--remove/--list actions
to manage hooks?

89
docs/random/ensonic/dynlink.txt
View file

@ -1,89 +0,0 @@
$Id$
Currently its only save to link/unlink elements/pad when pipeline is in READY.
Belowe some thoughts. See http://bugzilla.gnome.org/show_bug.cgi?id=435487
for patches.
= current api =
gboolean gst_element_link (GstElement *src, GstElement *dest);
void gst_element_unlink (GstElement *src, GstElement *dest);
gst_element_link_many, gst_element_unlink_many, gst_element_link_filtered,
gst_element_link_pads, gst_element_unlink_pads, gst_element_link_pads_filtered
GstPadLinkReturn gst_pad_link (GstPad *srcpad, GstPad *sinkpad);
gboolean gst_pad_unlink (GstPad *srcpad, GstPad *sinkpad);
= use cases =
== inserting an element ==
* we have: e1 ! e4
* we want: e1 ! e2 ! e3 ! e4
* we want: e1 ! e2 ! e4
gst_element_insert_linked(e1, e2, e3, e4); // e2 == e3 allowed
gst_pads_insert_link (e1.src, e2.sink, e3.src, e4.sink);
disconnect e1.src, e1.src.peer
disconnect e4.sink, e4.sink.peer
connect e1.src, e2.sink
connect e3.src, e4.sink
== removing an element ==
* we have: e1 ! e2 ! e3
* we want: e1 ! e3
gst_element_remove_linked(e2);
gst_pads_remove_link (e1.src, e3.sink);
disconnect e1.src, e1.src.peer
disconnect e3.sink, e3.sink.peer
connect e1.src, e3.sink
== swapping out an elelment ==
* we have: e1 ! e2 ! e6
* we have: e1 ! e2 ! e3 ! e6
* we want: e1 ! e4 ! e5 ! e6
* we want: e1 ! e3 ! e6
gst_element_swap_linked(e1, e4, e5, e6);
gst_pads_insert_link (e1.src, e4.sink, e5.src, e6.sink);
disconnect e1.src, e1.src.peer (=e2.sink)
disconnect e6.sink, e6.sink.peer
connect e1.src, e4.sink
connect e5.src, e6.sink
= thoughts =
* I don't think we need api for pads
* Should current api check for the state?
* do we want to swapp multiple elements at once
== events ==
* tee and adder need special treatment
* both would need to cache an accumulated segment
* tee
* would also cache tags
* when linkfunc is called, it can send out the segment and the tags
* when all pads got unlinked it could clear the segment
* adder
* when linkfunc gets called it sends a seek-event
= ideas =
== dynlinkpoint ==
* use cases
* its meant to be used with one side disconnected to allow to connect elements
at runtime
* it can be used in a pipeline to remove/insert elements at runtime
* element with 1 source- and 1 sinkpad
* when both connected it passes data through
* if src is not connected it drops received buffers
* if sink is not connected
* it does not push
* it creates silence on pull
* events
* it caches events
* down: newsegment, tags, buffersize
* up: seek (needs to be modified)
* when other-pad get connected it pushes events depending on direction

85
docs/random/ensonic/interfaces.txt
View file

@ -1,85 +0,0 @@
$Id$
* existing gstreamer interfaces
./gstreamer/gst/gsttaginterface.c
./gstreamer/gst/gstinterface.c
./gstreamer/gst/gsturi.c
./gst-plugins/gst-libs/gst/propertyprobe/propertyprobe.c
./gst-plugins/gst-libs/gst/mixer/mixer.c
./gst-plugins/gst-libs/gst/tuner/tuner.c
./gst-plugins/gst-libs/gst/xoverlay/xoverlay.c
./gst-plugins/gst-libs/gst/colorbalance/colorbalance.c
./gst-plugins/gst-libs/gst/navigation/navigation.c
* new general interfaces
* GST_TYPE_UI_HINT
- add hints to generate 'good' looking interfaces to elements
- API:
GList *get_group_list();
struct ui_hint_group {
gchar *label;
gachr *role;
GList *entries;
}
struct ui_hint_group_entry {
enum UiHintGroupEntryType type={PARAM,DPARAM};
gchar *name;
}
roles {
/* graphics */
"color_selection/rgb",
"color_selection/hsv",
"aspect_ratio",
/* audio */
"envelope/adsr",
}
- features
- grouping of parameters, each group has:
- a label: giving a title to the group
- a role:
- this can give the UI a hint about the purpose of the controls
- this only makes sense, if we don't make this a thousand templates
- a list of dparams or properties
- question
- should this be aware of instruments (voice-groups)
- no, instruments should auto-generate those
* GST_TYPE_QUALITY_VS_SPEED
- get the name of a property that can be used to switch between
- a fast version for e.g. realtime usage
- a slower version with higher precision that can be used for off-line
rendering
* new interfaces for audio applications
* GST_TYPE_MULTI_VOICE
- control interface for elements that support multiple voices (in one output-pad)
- API:
gulong number_of_voices;
void add_voice();
void remove_last_voice();
gulong get_number_of_voices();
- features
- plugin will initially have one voice and that one can not be deleted
* GST_TYPE_MUSIC_GENERATOR
- add hints so that application can use a element as an instrument
- API:
// param types
DParam *get_note_dparam();
GList *get_trigger_dparams();
// -- or
DParamType get_dparam_type(DParam *);
dparamtype = { NOTE, TRIGGER, OTHER }
// voices
char *get_number_of_voices_property();
GList *get_global_param_names();
GList *get_voice_param_names();
- features
- find out which params to use to play notes/trigger sounds
- these params will not appear in a control-ui
- notes involve a key to frequency translation
- find out if the element has a number_of_voices property
- if yes, we can find out about the max by looking at the gparamspec
- setting the property, adds/removes voices
- if the element supports it, it needs to:
- register voice-dparams as e.g. note_XXX, where XXX is the voice-number
- run the voice-loop in the chain/loop function
each voice processes the same input, if at all
the outputs of all voices are mixed together

72
docs/random/ensonic/lazycaps.txt
View file

@ -1,72 +0,0 @@
Lazy Caps
=========
The idea is to have caps where we evaluate the structures as needed. We can have
these variants of lazy caps:
- construction (e.g. from_string)
- need user_data (the string/string-array)
- need a free_func for the user_data
- need a get_structure_func to fill structure slots as needed
- we can more easily make this iterator based too
- operations (e.g. intersect) (see [1]).
- need user_data (the iterator)
- need a free_func for the user_data
- need a get_structure_func to fill structure slots as needed
We should add PERFORMANCE category logging to methods that cause lazy caps to be
fully evaluated. To get maximum speed benefit we need to inspect places that
loop over structures of a caps and turn them into while() loops where possible.
We can use GST_CAPS_FLAGS_LAZY to indicate that caps are not fully constructed.
Once caps are fully evaluated, we can remove the flag (and call the free_func).
Lazy caps might need a lock to protect multiple threads requesting a structure
(get_structure_func).
Accessors
---------
guint gst_caps_get_size (const GstCaps *caps);
- needs to fully evaluate the caps if iterator based :/
GstStructure *gst_caps_get_structure (const GstCaps *caps, guint index);- needs to fully evaluate the caps if iterator based :/
- needs to evaluate all entries up to index if iterator based
- needs to only evaluate requested index if e.g. parse based
gchar *gst_caps_to_string (const GstCaps * caps)
- needs to fully evaluate the caps
- it is used in debugging and serialisation
Construction
------------
Manipulation
------------
void gst_caps_append (GstCaps *caps1, GstCaps *caps2);
- use an iterator
void gst_caps_merge (GstCaps *caps1, GstCaps *caps2);
- use an iterator
void gst_caps_append_structure (GstCaps *caps, GstStructure *structure);
void gst_caps_remove_structure (GstCaps *caps, guint idx);
void gst_caps_merge_structure (GstCaps *caps,
- fully evaluate caps?
GstCaps * gst_caps_copy_nth (const GstCaps *caps, guint nth);
- eval structure and copy
void gst_caps_truncate (GstCaps *caps);
- eval first structure as needed and remove GST_CAPS_FLAGS_LAZY + call free_func
void gst_caps_set_value (GstCaps *caps, const char *field, const GValue *value);
void gst_caps_set_simple (GstCaps *caps, const char *field, ...);
void gst_caps_set_simple_valist (GstCaps *caps, const char *field, va_list varargs);
- fully evaluate caps
Operations
----------
[1] https://bugzilla.gnome.org/show_bug.cgi?id=618853

56
docs/random/ensonic/logging.txt
View file

@ -1,56 +0,0 @@
$Id$
rethink log format. current format:
* is not easy to parse/process by commandline tools
* cannot be easily diffed (timestamps, pid)
gst_debug_log_default() is default gst-log handler.
try new via:
gst_debug_remove_log_function(gst_debug_log_default)
gst_debug_add_log_function(func,data)
== reorder fields ==
format of default handler is:
DEBUG (0x8134bc0 - 0:00:00.848191000) GST_QOS( 3340) gstbasesink.c(1431):gst_base_sink_do_render_stats:<xvimagesink0> avg_render: 0:00:00.009044000
log_level_name --^^^^^ | | | | | | | | | | | | |
thread-id --------------^^^^^^^^^ | | | | | | | | | | |
timestamp --------------------------^^^^^^^^^^^^^^^^^ | | | | | | | | |
log_category -------------------------------------------------------^^^^^^^ | | | | | | |
process-id -----------------------------------------------------------------^^^^^ | | | | |
file:line -------------------------------------------------------------------------^^^^^^^^^^^^^^^^^^^ | | |
function ----------------------------------------------------------------------------------------------^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
message -----------------------------------------------------------------------------------------------------------------------------^^^^^^^^^^^^^^^
new proposed format:
0:00:00.848191000 3340 0x8134bc0 DEBUG GST_QOS gstbasesink.c:1431:gst_base_sink_do_render_stats:<xvimagesink0> avg_render: 0:00:00.009044000
timestamp -------^^^^^^^^^^^^^^^^^ | | | | | | | | | | | | |
process-id ------------------------^^^^^ | | | | | | | | | | |
thread-id -------------------------------^^^^^^^^^ | | | | | | | | |
log_level_name ------------------------------------^^^^^ | | | | | | |
log_category ----------------------------------------------------------^^^^^^^ | | | | |
file:line ---------------------------------------------------------------------^^^^^^^^^^^^^^^^^^ | | |
function -----------------------------------------------------------------------------------------^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ |
message ------------------------------------------------------------------------------------------------------------------------^^^^^^^^^^^^^^^
something like:
cut -c35- | sed -e "s/0x[0-9a-f]\{7,8\}//g"
should make logs easily 'diffable'.
== color code level ==
I suggedt to color-code loglevels if COLOR is allowed:
ERROR : red
WARNING : yellow
INFO : green
DEBUG : turkis
LOG : gray
NONE : gray
== log sources ==
What about differentating between log sources:
core
elements
application

42
docs/random/ensonic/media-device-daemon.txt
View file

@ -1,42 +0,0 @@
$Id$
components
================================================================================
- daemon process
- is a gstreamer application
- open physical sink, src elements
- prepends an adder to sinks
- appends an tee to sources
- listens to dbus, to get notified by virtual-endpoints of init/finalize
(the dbus notify, would also be useful for gst-editor to hook on running
apps)
- 4 new elements
- virtual-audiosink, virtual-videosink
virtual-audiosrc, virtual-videosrc
- virtual sinks establish a connection to the daemon
- they link to request_pads of the adder/tee elements
- on init and finalize they send a dbus-message
- gui app
- lists instances as mixing-desk like channelstrips
- channelstrips would contain
- audio
- volume, panorama, 3-band eq
- video
- brightness, contrast, alpha-level
- user can
- add insert-fx
- route channel to targets, where targets can be real sinks or more
virtual-sinks (sub-groups)
- virtual sinks need queues to decouple application processes
- interfaces
- expose child-elements via child-proxy
- then e.g. the applications volume-control could directly access the
channelstrip
- state-control (play, pause/mute)
- it would be useful if one app could pause/mute others
- think of a voip-client, if there is an incoming call, if pauses your
media-player, or mutes the monitoring of your recording app

51
docs/random/ensonic/plugindocs.txt
View file

@ -1,51 +0,0 @@
Plugin Docs
===========
In contrast to normal gtk-doc workflow modules will include
common/gtk-doc-plugins.mak. This provides an additional 'make udpate' target.
the "make update" target calls
1.) common/gstdoc-scangobj --type-init-func="gst_init(NULL,NULL)" --module=$(DOC_MODULE) --source=$(PACKAGE)
common/scangobj-merge.py $(DOC_MODULE)
the "tmpl" target is modified to run this extra step after "gtkdoc-mktmpl":
2.) common/mangle-tmpl.py $(srcdir)/inspect tmpl
and the "sgml" target is modified to run this step before "gtkdoc-mkdb":
3.) for a in $(srcdir)/inspect/*.xml; do \
xsltproc --stringparam module $(MODULE) $(top_srcdir)/common/plugins.xsl $$a > xml/`basename $$a`;
done
Details
=======
1a.) gstdoc-scangobj
-------------------
- get types from registry instead of .types file
- outputs inspect/plugin-<pluginname>.xml files
- outputs signal and args files
1b.) scangobj-merge.py
---------------------
- parse, update and write .signals and .args files
2.) mangle-tmpl.py
------------------
- read data from inspect/plugin-<pluginname>.xml
- insert/overwrite "Short Description" and "Long Description" in tmpl/
- the "Long Description" contains a <xi:include> for xml/element-<name>-details.xml
3.) common/plugins.xsl
----------------------
- creates xml/plugin-<name>.xml and xml/element-<name>-details.xml
TODO:
-----
- scangobj-merge.py -> gstdoc-scangobj
- need a way to skip the tmpl step
- gtkdoc-mkdb needs a way to insert extra content
- maybe patch generated xml/*.xml files
- could common/plugins.xsl patch the files?
<refsect1 id="gstreamer-plugins-capsfilter.description" role="desc">

37
docs/random/ensonic/receipies.txt
View file

@ -1,37 +0,0 @@
$Id$
= receipies =
The idea is to collect some recommendations for common, but not so trivial
tasks. docs/design/part-block.txt has something like that already. Ideally these
would go to the application developer manual and there would be sample code.
== initial seeking ==
=== question ===
How to I configure the initial playback segment?
=== idea ===
1) set pipeline to PAUSED
2) send seek event
3) set pipeline to PLAYING
=== problems ===
1) would preroll the pipeline only, to flush it when the seek comes
== async state changes ==
=== question ===
what to do when gst_element_set_state() returns ASYNC?
=== idea ===
1) listen to the STATE_CHANGED message on the bus
2) trigger next action
=== problems ===
This scatters logic over multiple functions (callbacks).
== topic ==
=== question ===
=== idea ===
=== problems ===

209
docs/random/eos
View file

@ -1,209 +0,0 @@
OUTDATED
--------
case 1)
(--------) (--------) (--------)
! fakesrc! !identity! !fakesink!
! src ----- sink src ---- sink !
(--------) (--------) (--------)
fakesrc detects the end of stream. It just sent the last buffer
and sets the srcpad to EOS with gst_pad_eos ().
gst_pad_eos() will notify the parent about the plugins attempt to
signal eos. the parent adds the element to its possible EOS
providers.
gst_pad_eos() will by default propagate to identy and to fakesink.
none of these plugins override the default behaviour so gst_pad_eos
returns TRUE and fakesrc signals EOS with the value TRUE.
The parent looks in the list of EOS providers and finds the faksrc
element that is now signaling EOS. all EOS providers are now in EOS
and so the bin fires EOS.
case 2)
(---------------)
!thread !
(--------) (--------) (--------) ! (--------)!
! fakesrc! !identity! ! queue ! ! !fakesink!!
! src ----- sink src ---- sink src ---- sink !!
(--------) (--------) (--------) ! (--------)!
(---------------)
fakesrc detects the end of stream. It just sent the last buffer
and sets the srcpad to EOS with gst_pad_eos ().
gst_pad_eos() will notify the parent about the plugins attempt to
signal eos. the parent adds the element to its possible EOS
providers.
gst_pad_eos() will by default propagate to identy and to queue.
queue overrides the eos handler and returns false on the eos
request. fakesrc signals EOS with a value of false and the parent
bin removes the EOS provider from its list.
after the queue has sent out the last buffer, its calls eos on its
src pad. queue is added to the top level bin as an eos provider and
the default eos handler signals EOS with a value of TRUE to the parent.
the parent sees that all the eos providers are in eos now and signals
EOS.
case 3)
(---------------)
!thread !
(--------) (--------) (--------) ! (--------)!
! fakesrc! ! tee ! ! queue1 ! ! !fakesink!!
! src ----- sink src ---- sink src ---- sink !!
(--------) ! ! (--------) ! (--------)!
! ! (---------------)
! !
! ! (---------------)
! ! !thread !
! ! (--------) ! (--------)!
! ! ! queue2 ! ! !fakesink!!
! src ---- sink src ---- sink !!
! ! (--------) ! (--------)!
(--------) (---------------)
fakesrc detects the end of stream. It just sent the last buffer
and sets the srcpad to EOS with gst_pad_eos ().
the eos handler returns false because both queues return false on the
eos request. the parent removes fakesrc as an EOS provider.
queue1 and queue2 were responsible for the EOS delay and so they get
added to the bin as possible EOS providers.
after the queues have sent out their last buffer, they calls eos on their
src pads.
the parent already has the two queues in the EOS provider list so they don't
get added twice.
the two queues perform gst_pad_eos () on their pads when the queue is empty,
the parent removes the EOS providers from its list, when the list is empty,
the parent fires EOS.
case 4)
(---------------)
!thread !
(--------) (----------) (--------) ! (--------)!
! fakesrc! !mpeg1parse! ! queue1 ! ! !fakesink!!
! src -- sink src ---- sink src ---- sink !!
(--------) ! ! (--------) ! (--------)!
! ! (---------------)
! !
! ! (---------------)
! ! !thread !
! ! (--------) ! (--------)!
! ! ! queue2 ! ! !fakesink!!
! src ---- sink src ---- sink !!
! ! (--------) ! (--------)!
(----------) (---------------)
this case differs from case3 in that one of the queues can be empty
while the other isn't. we assume queue1 is empty while queue2 isn't yet.
fakesrc detects the end of stream. It just sent the last buffer
and sets the srcpad to EOS with gst_pad_eos ().
the eos handler returns false because queue2 returns false on the
eos request. the parent removes fakesrc as an EOS provider.
queue2 was responsible for the EOS delay and so it gets added to the bin
as a possible EOS provider.
after the queue2 has sent its last buffer, it performs gst_pad_eos on its
src pad.
the parent already has the queue2 in the list of EOS providers so it does not
get added twice.
queue2 finally fires the EOS signal and the parent removes the EOS provider
from its list, when the list is empty, the parent fires EOS.
case 5)
(--------) (--------) (--------)
! disksrc! ! mad ! !filesink!
! src ----- sink src ---- sink !
(--------) (--------) (--------)
disksrc detects the end of stream. It just sent the last buffer
and sets the srcpad to EOS with gst_pad_eos ().
the eos handler returns false because mad returns false on the
eos request. the parent removes mad as an EOS provider.
mad was responsible for the EOS delay and so it gets added to the bin
as a possible EOS provider.
After mad has sent its last buffer, it performs gst_pad_eos on its
src pad.
the parent already has mad in the list of EOS providers so it does not
get added twice.
mad finally fires the EOS signal. This time, filesink returns false on
the eos request. the parent removes mad as an EOS provider.
filesink was responsible for the EOS delay and gets added to the bin
as a possible EOS provider.
When filesink has written all of it's data and closed the output file,
it fires EOS.
The parent already has filesink in the list of EOS providers so it does
not get added twice.
The parent removes the EOS provider
from its list, and since the list is empty, the parent fires EOS.
case 6)
(--------) (--------) (--------)
!disksrc1! ! mad1 ! ! mixer !
! src ----- sink src ---- sink1 ! (--------)
(--------) (--------) ! ! !filesink!
! src ---- sink !
(--------) (--------) ! ! (--------)
!disksrc2! ! mad2 ! ! !
! src ----- sink src ---- sink2 !
(--------) (--------) (--------)
In this case, we want to make sure the pipeline keeps running after one
of the two sources reaches eos. Suppose in this case that disksrc1 will
reach eos first.
disksrc1 detects the end of stream. It sets eos, mad1 will return false,
and mad1 will be responsible for eos. When mad1 had sent out the last
buffer, it sends out eos.
The mixer intercepts eos and returns false. mad1 is removed from the
eos providers and mixer is added.
(At this point, the mixer might choose to disconnect mad1->src and
mixer->sink1 pads, since it's received eos on mad1->src)
mixer will not send out eos since it hasn't received eos from
mad2->src.
After a while, disksrc2 will detect end of stream, and eos will finally
propagate to mixer. mixer might disconnect mad->src2, and after
realizing all of it's sources have reached eos, it sends out the final
buffer and fires EOS.
At this point, filesink will return false, mixer will be removed as an
eos provider, and filesink will write out it's final buffer and close
the file on disk. At this point, it fires eos, and since it's the last
eos provider, the parent can fire eos.

59
docs/random/error
View file

@ -1,59 +0,0 @@
Some notes on the error handling introduced after 0.7.3
- there are four domains for errors:
- CORE: core GStreamer errors
- LIBRARY: supporting library errors
- RESOURCE: errors accessing resources (files, network, memory, ...)
- STREAM: errors in the data stream
- Each error in GStreamer or plug-ins can be put under one of these four.
- the enum is called Gst(Domain)Error and has GST_(DOMAIN)_ERROR_(CODE) members
(see GError API docs for rationale)
- each of the enums starts with _FAILED at 1 (with FAILED being the generic
error if none of the others applies)
- second in the enum is TOO_LAZY, which allows us to keep track of those errors
that we haven't categorized yet. This means they really should either move
to another one, or a new one ought to be created.
- each enum ends with NUM_ERRORS
- elements call gst_element_error to signal an error:
gst_element_error (element, domain, code, message, debug);
With :
- element being the one signalling the error
- domain one of CORE, LIBRARY, RESOURCE, STREAM
- code one of the suffixes in the Gst(Domain)Error enum
- message is either NULL (to signal the standard error message for the
given domain and code), or a printf-like set with the format being
marked for translation.
The string should start with a capital and end in a period so it forms
a complete sentence.
The string can/should be shown to the user of an application.
- debug is either NULL or a non-translated debug string, which can be used
to diagnose errors after they've happened.
The string can be shown to the user when he asks for additional debug info.
A useful macro is GST_ERROR_SYSTEM, which prints out the system error
that explains the failure by using g_strerror.
- Some example calls:
gst_element_error (src, RESOURCE, OPEN_READ,
(_("Could not open file \"%s\" for reading"), src->filename),
GST_ERROR_SYSTEM);
The message is specified since we have more information:
- the resource is a file
- we know the file name
gst_element_error (element, CORE, NEGOTIATION, NULL, NULL);
This is a simple negotiation error. The default message will be
signaled, telling the user that GStreamer had an internal error.
gst_element_error (ebml, RESOURCE, READ, NULL,
("Read error at position %llu (0x%llx)",
pos, pos));
The plugin asked to read on the underlying resource (using bytestream),
but failed. The user will get a generic read error. The debug info
will contain the exact position in the stream at which the read error
occurred.

108
docs/random/example
View file

@ -1,108 +0,0 @@
OUTDATED, methods have different names now
------------------------------------------
Here's a pipeline that does audio/video MPEG streams with a queue on
either side of each decompressor, for a total of 5 threads (read/split,
decode audio, decode video, play audio, play video):
NOTES: mpegsplit is the intelligence in this pipeline, providing an IDL
that allows one to connect things to a GUI.
Pipeline(mpegplay)
Thread(reader)
Element(:disk_async_src) [StreamerAsyncSrc]
OutPad(disk1)
Element(:mpegsplit)
InPad(disk1)
OutPad(audio1)
OutPad(video1)
Queue(audioQ1)
InPad(audio1)
OutPad(audio2)
Thread(audiodecode)
Element(:mpeg_audio_decode) [StreamerVideoFilter]
InPad(audio2)
OutPad(audio3)
Queue(audioQ2)
InPad(audio3)
OutPad(audio4)
Thread(audioplay)
Element(:audio_play) [StreamerAudioSink]
InPad(audio4)
Queue(videoQ1)
InPad(video1)
OutPad(video2)
Thread(videodecode)
Element(:mpeg_video_decode) [StreamerVideoFilter]
InPad(video2)
OutPad(video3)
Queue(videoQ2)
InPad(video3)
OutPad(video4)
Thread(videoplay)
Element(:video_play) [StreamerVideoSink]
InPad(video4)
A simpler pipeline that just does MPEG videos:
Pipeline(mpegplay)
Thread(reader)
Element(:disk_async_src) [GstAsyncSrc]
OutPad(disk1)
Element(:mpeg_control)
InPad(disk1)
OutPad(video1)
Element(:mpeg_video_decode) [GstVideoFilter]
InPad(video1)
InPad(video2)
Queue(queue)
InPad(video2)
OutPad(video3)
Thread(play)
Element(:video_play) [GstVideoSink]
InPad(video3)
The code for the above looks roughly like:
/* all the objects we're worried about */
GstPipeline *mpegplay;
GstThread *reader;
GstSrc *disk_src;
GstControl *mpeg_control;
GstFilter *mpeg_video_decode;
GstQueue *queue;
GstThread *play;
GstSink *video_play;
/*** first we create all of the objects ***/
mpegplay = gst_pipeline_new();
reader = gst_thread_new();
disk_src = gst_async_disk_src_new("filename.mpg");
mpeg_control = gst_mpeg_control_new();
mpeg_video_decode = gst_mpeg_video_decode_new();
queue = gst_queue_new();
play = gst_thread_new();
video_play = gst_video_sink_new();
/*** now we start to create the pipeline ***/
/* first set up the reader thread */
gst_bin_add(reader,disk_src);
gst_object_connect(disk_src,"out",mpeg_control,"in");
gst_object_connect(mpeg_control,"out",mpeg_audio_decode,"in");
gst_bin_ghost_pad(reader,mpeg_audio_decode,"out");
/* then set up the player thread */
gst_bin_add(play,audio_play);
gst_bin_ghost_pad(play,audio_play,"in");
/* now plug them all into the main pipeline */
gst_bin_add(mp3play,reader);
gst_object_connect(reader,"out",queue,"in");
gst_object_connect(queue,"out",play,"in");

153
docs/random/hierarchy
View file

@ -1,153 +0,0 @@
OUTDATED
--------
Face it, the plugins/ directory hierarchy is crap. We want to propose a
better layout for it now. Some things to consider:
- Elements have a klass member in the factory that is used to
denote the functional type of the element. For example, the
mp3 encoder has a klass of Codec/Encoder/Audio
- The plugins can be grouped together by the media type they
operate on or by the way they work (decoder/encoder)
In GStreamer all plugins are technically filters, the only way they
can be considered sources or sinks (input/output) elements is
by the absence of src/sink pads. At first sight the source/filter/
sink distinction is quite useless because most of the plugins
will go into the filters category anyway.
We don't want to make the hierarchy too deep, yet provide a
clean way to ask for a mp3 decoder element..
Anyway this is a rough proposal to fire off the discussions...
Wim
Source
Disk
disksrc
fdsrc
multifilesrc
Network
HTTPsrc
RTPsrc
CDDA
cdparanoia
XMMS
..
DVD
dvdsrc
Audio
ASLA
OSS
Capture
v4lsrc
firewire
Demuxer
AVI
MPEG1
MPEG2
QT
Muxer
AVI
MPEG1
QT
Aggregator
Tee
gsttee
Connection
queue
CORBA
Parser
MPEG1
MPEG2
AC3
Mixer
Audio
Merge
Video
Subtitles
Merge
Filters
Audio
ladspa
resample
Video
colorspace
Effect
Audio
stereo
volume
delay
chorus
Video
median
smooth
XMMS
Decoder
MPEG1
MPEG2
MP3
mpg123
xing
win32
AU
WAV
JPEG
AC3
ac3dec
RTJPEG
vorbis
Encoder
MPEG1
MPEG2
MP3
lame
mpegaudio
win32
JPEG
AU
WAV
RTJPEG
Vorbis
Visualisation
Video
histogram
Audio
smoothwave
spectrum
synaesthesia
vumeter
XMMS
Sink
Disk
filesink
multifilesink
Network
ICECASTsink
FTPsink
RTPsink
XMMS
ESD
Video
videosink
SDLsink
Audio
OSSsink
ALSAsink

337
docs/random/interfaces
View file

@ -1,337 +0,0 @@
INTERFACES & ELEMENTS
---------------------
1) Introduction
===============
Interfaces are descriptions on how to handle an object, without actually
implementing the object. This allows for multiple objects to be instantiated
based on this interface. Each of them can then be handled equally by an
application.
Glib, apparently (unchecked), has a way of creating interfaces, probably
by means of a class struct without actually defining the object. The object,
then, does not define a class and these two add up. Benjamin knows more
about interfaces, I didn't study interfaces & glib too deeply, yet. I know
them just from Java.
Interfaces are cool! It allows for some sort of random element creation
without needing to link to the implementation. This is similar to how
GStreamer currently handles media plugins. GStreamer itself could be seen
as an interface too, in that respect.
2) So why do we need interfaces?
================================
Because GStreamer doesn't handle it all. GStreamer in itself is a media
framework for streams of data from one element to the next. There's lots
of things that's media-related, but not handled in this description.
Several examples will probably clarify this: think of the Xvideo output
plugin. We can create an overlay here (Xv-based), and we currently control
this X-connection using glib properties. However, what property name is
associated with what control? And does it work the same as v4lsrc's
overlay image control?
The same goes for a mixer, for image control, audio control, and probably
a lot more. The general idea is simple: *this needs to be documented*.
But properties aren't all - they simply cannot do this all. Some things
cannot be described in a simple one-argument property thing. Of course,
we could give a pointer to a struct as argument, but that's merely a hack
and requires both plugin and app to know the ABI of the struct. This
kills the whole idea of making the plugin independent of the app.
In short: we want interfaces for this.
3) How to integrate an interface in GStreamer
=============================================
Let us start with some starting point: an interface is associated
with an element. It is a feature exported by that specific element,
not by a pipeline or anything more complex. Pipelines are already
handled just fine by GStreamer (or you wouldn't be reading all
this).
Obviously, a pipeline can be a fallback for an interface. Imagine
that we're looking for an audio sink that exposes a mixer, but our
fakesink audio output doesn't ("I wonder why"). We could then create
a pipeline with the volume element in it to "fake" a mixer. Ideally,
the volume element would implement a mixer interface itself.
How are we going to do that in programmatic way? We currently use
properties. Their huge advantage is that we do not need to care
about adding new functions or whatever. Their disadvantage is that
they're limited to one argument. Anything more complex requires
app/plugin knowledge about the shared data, and that defeats the
point of them: to have no dependency on each other. This could be
solved partially by using action signals, but that makes the whole
picture quite complex (since you use multiple methods for doing one
simple thing). Also, they are quite slow compared to functions
because of the table lookups. In short: it'd work, but I'm not in
facour of it...
OK, so an element exposes interfaces. This allows us to think of
the idea of embedding interfaces (dynamically, of course) in the
GstElement object. Think of an object being able to register an
indefinite number of interfaces per object instance, and a client
application could then enumerate interfaces and instantiate one.
Glib gives us GInterface for this purpose. The disadvantage of
this is that it's on a per-class basis, not a per-instance basis.
This is a problem in case of elements where it depends on several
properties whether it supports an interface or not. This can be
solved by simply making one generic virtual function "supported ()"
in a generic derived object of GInterface (GstInterface?).
GstInterface is then a generic thing that is inherited by specific
interfaces (see examples). Obviously, the client will need to know
about the ABI/API of this struct, but that'll happen either way.
Surely, there needs to binary linkage, but I don't consider that a
bad thing. It does improve performance compared to action signals!
So an element contains interfaces. But where are these interfaces
described? And who creates them? I suggest that we do that just as
we handle gstvideo and gstaudio right now (these libs do *nothing*
useful currently, so this would make them a lot more interesting).
These interfaces inherit from GstInterface. The functions that
are needed, can be provided through a class object. The element is
then responsible for storing variables and so on. gstvideo/audio
provides wrapper functions for the class functions. That's also how
glib suggest us to use GInterfaces.
Plugin and application then handle and retrieve interfaces as
documented in the glib documentation, which is available at:
http://www.gnome.org/~mathieu/gobject/main.html
So the most important part left is to document the interfaces
and make sure all elements exporting them work equally. For this,
I'll give two examples.
4) Examples
===========
/* This small extra virtual function is here to provide an
* interface functionality on a per-instance basis rather
* than a per-class basis, which is the case for glib.
*/
typedef struct _GstInterfaceClass {
GTypeInterface parent;
/* virtual functions */
gboolean (* supported) (GstInterface *iface);
} GstInterfaceClass;
There would probably be a convenience function that checks
a specific interface's implementation (glib allows for this)
and checks for ->supported () to be set and to return TRUE:
gboolean
gst_element_implements_interface (GstElement *element,
GType iface_type)
{
if (G_TYPE_CHECK_INSTANCE_TYPE (G_OBJECT (element),
type)) {
GstInterface *iface;
GstInterfaceClass *ifclass;
iface = G_TYPE_CHECK_INSTANCE_CAST (G_OBJECT (element),
type, GstInterface)
ifclass = GST_INTERFACE_GET_CLASS (iface);
if (ifclass->supported != NULL &&
ifclass->supported (iface) == TRUE) {
return TRUE;
}
}
return FALSE;
}
Let's now add some functions so we can abuse this in case/check
functions.
GstInterface *
gst_interface_cast (gpointer from,
GType type)
{
GstInterface *iface;
/* check cast, give warning+fail if it's invalid */
if (!(iface = G_TYPE_CHECK_INSTANCE_CAST (G_OBJECT (element),
type, GstInterface))) {
return NULL;
}
/* if we're an element, take care that this interface
* is actually implemented */
if (GST_IS_ELEMENT (from)) {
gboolean interface_is_implemented =
gst_element_implements_interface (GST_ELEMENT (from),
type);
g_return_val_if_fail (interface_is_implemented == TRUE, NULL);
}
return iface;
}
gboolean
gst_interface_check (gpointer from,
GType type)
{
GstInterface *iface;
/* check cast, return FALSE if it fails, don't give a warning... */
if (!G_TYPE_CHECK_INSTANCE_CAST (from, type,
GstInterface)) {
return FALSE;
}
iface = G_TYPE_CHECK_INSTANCE_CAST (G_OBJECT (element),
type, GstInterface);
/* now, if we're an element (or derivative), is this thing
* actually implemented for real? */
if (GST_IS_ELEMENT (from)) {
if (!gst_element_implements_interface (GST_ELEMENT (from),
type)) {
return FALSE;
}
}
return TRUE;
}
#define GST_INTERFACE_CHECK_INSTANCE_CAST(obj, type, cast_t) \
((cast_t *) gst_interface_cast ((obj), (type))
#define GST_INTERFACE_CHECK_INSTANCE_TYPE(obj, type) \
(gst_interface_check ((obj), (type))
We could use this in the GST_IS_... () macros. For example, the
macros GST_IS_MIXER () and GST_MIXER () would then look like this:
/* Note that this is a non-standard macro, and with a reason! */
#define GST_MIXER(obj) \
(GST_INTERFACE_CHECK_INSTANCE_CAST ((obj), \
GST_TYPE_MIXER,
GstMixer))
#define GST_IS_MIXER(obj) \
(GST_INTERFACE_CHECK_INSTANCE_TYPE ((obj), \
GST_TYPE_MIXER))
So the application would just tread it with the known macro, and
everything would look extremely simple to the end user.
4a) mixer
---------
A mixer is a way of controlling volume and input/output channels.
This doesn't mean that you control which channel is the subwoofer,
all that is supposed to be done automatically. It is really meant
as a way of representing system-level volumes and such. It could
also be used to turn on/off certain outputs or inputs.
As you've noticed, I'm not only talking about output, but also
input. Indeed, I want both osssrc *and* osssink to export the
same mixer interface! Or at least a very similar one. Volume
control works the same for both. You could say that osssrc should
enumerate the input channels (such as microphone, line-in). Of
course, osssink should not. Or maybe it should, not sure... Maybe,
we'd need a parent osselement which implements all mixer channels.
And alsa* would surely implement the same interface.
/* This is confusing naming... (i.e. FIXME)
* A channel is referred to both as the number of simultaneous
* sound streams the input can handle as well as the in-/output
* itself
*/
#define GST_MIXER_CHANNEL_INPUT (1<<0)
#define GST_MIXER_CHANNEL_OUTPUT (1<<1)
#define GST_MIXER_CHANNEL_MUTE (1<<2)
#define GST_MIXER_CHANNEL_RECORD (1<<3)
typedef struct _GstMixerChannel {
gchar *label;
gint current_num_channels,
max_num_channels,
flags;
} GstMixerChannel;
typedef struct _GstMixerClass {
GTypeInterface klass;
/* virtual functions */
GList * (* list_channels) (GstMixer *mixer);
void (* set_volume) (GstMixer *mixer,
GstMixerChannel *channel,
gint *volumes);
void (* get_volume) (GstMixer *mixer,
GstMixerChannel *channel,
gint *volumes);
void (* set_mute) (GstMixer *mixer,
GstMixerChannel *channel,
gboolean mute);
void (* set_record) (GstMixer *mixer,
GstMixerChannel *channel,
gboolean record);
} GstMixerClass;
libgstmixer.la/so provides wrapper functions for each of the
class' virtual functions. Possibly also some macros for
GST_MIXER_CHANNEL_HAS_FLAG () or _get_channel ().
The rest is done automatically, as described in the already-
mentioned glib documentation for GInterface. This includes
things like the base_init () function of the GstMixerClass,
which fills all the virtual functions for the mixer, and the
actual function implementations. The mixer, basically, operates
as an element on its own. It gets the file descriptor from
the interface->element (every oss... is a osscommon, etc.).
4b) overlay
-----------
Overlay is used in both in- and output, too. Think of v4lsrc,
v4l2src, v4lmjpegsrc, xvideosink - all overlays. But where do
we position the overlay window? Control of this can be done at
various levels: locational control (over the server, asynchronous)
or XID control (but that makes you depend on X and limits the
ability to broaden it over to non-X elements such as fbsink).
However, simplicity *is* an issue here. Do we really care about
overlay? In the end, users will have to link against either FB
or X anyway, so we might want to create separate interfaces for
both. On the other hand, we want to be general too... This is a
decision that we need to make as early as possible in this process.
For now, I propose making X- and FB-based interfaces.
Let's assume that we take X as a basis. Then, overlay becomes as
simple as one function. Possible extendible by providing inputs
(like in the mixer) and norms, although that only applies to
input-to-analog, not to-digital... Others simply return NULL.
typedef struct _GstOverlayClass {
GTypeInterface klass;
/* virtual functions */
void (* set_xwindowid) (GstOverlay *overlay,
XID xid);
} GstOverlayClass;
That's all! It would look similar for FB & co.
4c) user input
--------------
And yes, user input could be an interface too. Even better, it
should definitely be. And wasn't this one of our key issues for
0.8.0?
No code here. Go implement it, lazy ass!
General ways of thinking: input can come from a plugin, or from
the application (we don't have modules for joystick input et all).
However, plugins handling input (such as dvdsrc) need to be able
to handle each. So we get both input-to-application as well as
input-from-application APIs.
5) Status of this document
==========================
The interfaces are implemented, more (for metadata, framebuffer-
overlay, video balancing (brightness), user input etc. are all
pending.
6) Copyright and blabla
=======================
(c) Ronald Bultje, 2003 <rbultje@ronald.bitfreak.net> under the
terms of the GNU Free Documentation License. See http://www.gnu.org/
for details.
And no, I'm not for hire. ;).

80
docs/random/metadata
View file

@ -1,80 +0,0 @@
OUTDATED
--------
The point of the metadata is to provide some context for each buffer. In
the case of audio data, for instance, it would provide the samplerate, bit
depth, and channel count.
The trick is that there may be multiple types of metadata ganged onto a
single buffer. This is why they're going to be a GList. This does mean
extra overhead in all cases, but I think it's minimal. The GList type
uses a chunk allocater so we're not wasting too much memory or time when
adding to the list.
The trick is dealing with these structs as they pass through a pipeline,
since they have potentially different mutability properties. For
instance, if you've got a mp3 decoder connected to a tee, which sends the
buffers off to both the decoder and a spectrum analyzer (and then a
visualization element). The preferred setup would be where every time a
audio/raw metadata comes down the pipe (indicating a potential change in
audio format), the audiosink and spectrum would just save off pointers.
So when exactly does this metadata go away (deallocated)? Well, that
means metadata has to be refcounted. But that gets rather hairy. OK, in
the simple case you create a metadata struct, it comes with refcount set
to 1. You pass it through, it stays one, eventually someone drops the
last reference on the buffer it's tied to, you free the metadata too.
Easy. What if you tee? You could go through and for every metadata in
the buffer, increment the refcount by the same as the buffer. So in the
above case (tee'd), the audiosink and spectrum would get the buffer with a
refcount of 2, and it'd have a metadata with refcount 2. Do they ref it
each themselves, then unref the buffer? Or do they remove the metadata?
Removing the metadata would require a buffer CoW, which would suck, so
yes, they'd just ref the metadata.
But.... what if they're all in different threads? Then we're off into
the magical world of mutexes. Everything with a refcount in a threaded
world must be mutexed, else you can do atomic increment and atomic
dec&test. Can this be done from C easily? Perhaps it needs to be found
from kernel includes via autoconf?
The goal in designing the way metadata will be defined and used is to keep
it as simple as possible. The basis for accomplishing this is the fact
that in order to actually use (rather than just pass) the metadata, you
have to know what the fields are, which means you have to have compiled in
support for that metadata at build time. Therefore, if you're using
metadata, you must have build-time access to the necessary include file
that defines it.
So, given that you've got an include file, it would be nice if the whole
thing could be contained there. This would limit the need to be linked
against something, or have load-time requirements as to that has to be
loaded before you are.
Given that really all metadata is is a region of memory of a given size
with a certain signature, this isn't all that hard. First you lay out the
struct that defines the metadata. Then you set up #defines that expand to
the size of the struct in question, as well as the four-cc code that
defines the type.
The work is done by a few #defines, a la the #defines used in all Gtk
objects. The first is a NEW() method that allocates the memory for the
metadata and fills in all the normal fields (type, size, utility
functions). Because of the way it's defined (as a #define, no less),
you'll have to invoke it as META_NEW(meta), since it can't return()
anything.
Another #define will check to make sure a meta is indeed that type by
verifying the type code and size. Theoretically, meta types can overlap
with the same fourcc code, as long as they have different sizes. But I
probably ought to have a global public registry so people writing things
don't conflict. MSFT got that right, at least.
So, a hairy problem is what to do when there are utility functions
associated with one of these things. One option is to not bother with
them. This is very likely a possible solution, since metadata is supposed
to be flat memory of a given size. Not much to do to either free or copy
it, is there?

30
docs/random/mutability
View file

@ -1,30 +0,0 @@
Mutability is the property of an object that defines whether or not you
are allowed to modify it. In the context of GST, that means that if you
want to mutilate a buffer, say to do an audio effect, you may have to do
this on a copy of the buffer, if someone else has a reference on it.
The simplest sequence of events in a decoder pipeline is as follows:
1) create buffer
2) allocate and fill data region, attach to buffer
3) pass to next element
4) decode the data into new buffer, free original buffer
5) pass to next element
6) buffer gets copied to output device (sound, video, whatever)
Both of these buffers are created from malloc()'d memory, are referenced
by one and only one element at a time, and are never modified in place.
They have no special flags, and when ref==0, they're simply free()'d.
An optimization in the case of the sound card or video double buffering,
where the output buffer actually comes from the output device. In that
case the element will be aware of such things.
A more complex example is where the data is teed after being decoded, sent
to an effects or visualization object.
1) create buffer, fill from source
2) hand to decoder
3) create new buffer, decode into it, free old buffer
4) hand to tee
5) ref++, hand off to

286
docs/random/negotiation
View file

@ -1,286 +0,0 @@
Some notes on pad negotiation
A "pad link" is a connection between two pads. It can be in one of
two states, "negotiated" or "not negotiated". Pad links are created
by gst_pad_link().
A "pad link" is created when two pads are linked using gst_pad_link().
When initially created, the link only specifies a src pad, a sink pad,
and optionally a filter caps provided by the application.
In order to pass data through a link, the peer pads must decide on
what data format to use. This is called negotiation. Pads
describe acceptable data formats by using a combination of pad
template caps and (optionally) a pad->getcaps function.
Negotiation can happen in one of two forms, directed or undirected.
Directed negotiation happens when one element has decided (usually
during negotiation on another pad) to ask for a specific format on
a pad. This happens when a pad calls gst_pad_try_set_caps().
Undirected negotiation happens when the core decides to negotiate
a link, either due to a state change or a specific application
request.
Steps in undirected negotiation (core view):
- core checks that both pad's parent elements are in the READY or
higher state.
- core calls gst_pad_get_caps() on each pad, intersects the two caps,
and intersects again with the filter caps. If the intersection is
empty, then the pads have no formats in common, and the link fails.
- If the intersection caps is not fixed, there are multiple possible
formats that the link could use. If this is the case, fixate
functions are called until the caps are fixed. The fixate functions
are called by priority -- src application fixate function, sink
application fixate function, src and sink fixate functions, and
the default core fixate function. The application fixate functions
are implemented by the "fixate" signal on each pad. The core
loops through the fixate functions until a fixed caps is decided
on.
- Each pad may have a pad_link function, which is called with the
fixed caps. The pad_link function has the option of accepting,
rejecting, or delaying the negotiation.
- If both pads accept the caps, the link is then negotiated.
Steps in directed negotiation (gst_pad_try_set_caps):
- the originator is the pad that gst_pad_try_set_caps() is called
on.
- the elements owning both pads are assumed to be in a non-NULL state
- the caps argument of try_set_caps() must be fixed.
- gst_pad_get_caps() is called on the peer pad, and intersected with
the originating pad's pad template caps and the filter caps. The
caps argument is checked to be a subset of the intersection. (It's
important that this intersection uses the pad _template_ caps.)
- Fixate functions don't need to be called, since the caps are
already fixed.
- The peer's pad_link function is called.
- If the peer's pad_link function accepts the caps, the link is then
negotiated.
- If the peer's pad_link function refuses the caps, and the link had
already been negotiated, the peer's pad_link function is called
with the caps of the old negotiation.
- Note: the originator's pad_link function is _not_ called. The
originator must take appropriate alternative steps.
Notes about renegotiation:
- same as negotiation. Note that get_caps() functions should always
ignore the currently negotiated caps of a link.
- if renegotiation fails, the previous negotiation is still in effect.
If the renegotiation fails in the last pad_link step, the pad_link
functions are called with the previously negotiated caps.
Notes for sources and sinks:
- sources and sinks that talk to hardware may not be able to fully
describe their available formats, and thus need to rely on pad_link
functions to test a particular format. FIXME: currently, the core
completely fails negotiation if a pad_link function refuses a caps,
instead of attempting with an alternate caps.
Example: Assume osssink advertises rate=(int)[8000,48000], but
the device cannot actually handle rate=44100 (unknown to osssink).
Assume that the pad_link function is called with rate=44100 --
ideally, the pad_link function should return GST_PAD_LINK_DELAYED,
and future calls to getcaps should return {[8000,44099],[44101,
48000]}. I don't know how to make this easy and/or work well.
Notes for decoders/demuxers:
- Decoders will typically negotiate a sink pad, receive some data,
determine the output format, and call try_set_caps() with the given
format. If the output format is non-fixed, gst_pad_renegotiate()
may be used instead, in order to have the fixate functions choose
the optimal format. Note that this requires communication so that
the pad's getcaps function returns the correct caps.
Notes for converters:
- Converters change one or more properties of the format of a data
stream. A typical converter's getcaps function will call
gst_pad_get_allowed_caps() for the opposite pad in the element,
change one or more fields in the caps, and return the result.
- getcaps function:
- call gst_pad_get_allowed_caps() on the other pad in the element
- for each possible format ("A") in the allowed caps, determine all
the formats ("B") that your converter could convert the original
format (A) to. The union of all these formats (all the B's) is
the caps that should be returned. (This is how to do it
_theoretically_, but an optimal implementation will probably be
quite different.)
For example, a simple way to do this for an element that can convert
a given field of the caps is to remove the field(s) from the structure,
then intersect with the pad template.
- As an example, videoscale can convert any sized video to any other
sized video. Its getcaps function iterates over each structure in
the caps, and replaces the width and height with the range
[1,MAXINT].
- pad_link function:
- the "otherpad" is the opposite pad in the element.
- extract fields from the caps that are relevant for your converter
handling the format. Store these in _local_ variables. (E.g,
things like video size, sample rate, etc.)
- If it's possible to pass buffers through without modifying them
(passthrough), you should call gst_try_set_caps() with the caps
that was specified as the parameter to the pad_link function. If
this is successful, save the local variables to the element
structure, perform whatever other setup is necessary for your
element, and return GST_PAD_LINK_OK.
- Otherwise, you're not using passthrough, and may need to
change the caps on the otherpad to match the given format.
- If the otherpad is not negotiated (!gst_pad_is_negotiated()),
you shouldn't attempt to set a format on it. It will eventually
be negotiated. Save the local variables to the element structure,
perform whatever other setup is necessary, and return
GST_PAD_LINK_OK.
- At this point, the other pad is already negotiated, but won't
accept the passthrough format, so you should combine the existing
negotiated caps on the otherpad and the caps that was the pad link
argument. This can either be done using existing information in the
element that was saved during a previous pad_link call, or you can
get the information from the negotiated caps
(gst_pad_get_negotiated_caps()).
As an example, consider the videoscale element. Assume that
videoscale.src has already negotiated "video/x-raw-yuv,
format=(fourcc)I420, width=320, height=240", and that the sink
pad's link function is called with "video/x-raw-yuv,
format=(fourcc)YUY2, width=640, height=480". Since it's the
videoscale element, we can have different width and height
fields on the pads, but the format must be the same. So we'll
use the existing negotiated size (640x480), and the new format,
and call gst_pad_try_set_caps() with "video/x-raw-yuv,
format=(fourcc)I420, width=640, height=480".
This may seem overkill, but most of the time, you'll end up
calling try_set_caps() with the same caps that are currently
negotiated -- try_set_caps() just returns GST_PAD_LINK_OK in
this case.
- If gst_pad_try_set_caps() returns GST_PAD_LINK_OK, save the
local variables to the element structure. In any case, return
the return value of gst_pad_try_set_caps().
Notes for filters:
- Filters can almost always use gst_pad_proxy_getcaps() as the
getcaps function. This just returns gst_pad_get_allowed_caps()
on the otherpad.
- You may be able to use gst_pad_proxy_pad_link() as the pad link
function, but only if you don't need to extract parameters from
the caps.
Notes for encoders/muxers:
- Encoders and muxers should roughly work like converters. Many
converters are symmetric; encoders and muxers obvious are not,
thus it may make the code clearer to have separate src and sink
getcaps and pad_link functions.
- Encoders and muxers should handle multiple negotiations until
the first buffer has been passed. After this point, it's unlikely
that additional negotiations will happen in well-constructed
pipelines, but it may be wise to "lock" the caps after the
muxer has committed to a format. (FIXME: it's still unclear to
me when the caps should get "unlocked". Obviously at EOS or
PAUSED->READY transitions. Any others?)
- Locking caps can be done by adding (near the top) of the getcaps
function:
if (my_element->lock_caps) {
return gst_pad_get_negotiated_caps (pad);
}
Explicit caps:
- There's a hack in the core to make the code for decoder elements
a lot simpler. This hack can be used only for src pads of elements
that get their srcpad capabilities directly from the data stream,
i.e., decoders, demuxers, and typefind. This hack overrides the
pad's getcaps() and pad_link() function, so that they work correctly
in all decoder states.
- To enable this hack on a pad, call gst_pad_use_explicit_caps().
- To indicate that a decoder has found the format of the stream, call
gst_pad_set_explicit_caps(pad,caps) with the caps of the stream.
This caps must be fixed.
- To indicate that a decoder has lost the format of the stream, i.e.,
there's been a NEW_MEDIA event, call gst_pad_set_explicit_caps(pad,
NULL).
- If the explicit caps are set, the getcaps function will return that
caps, and the pad_link function will return GST_PAD_LINK_OK. If
the explicit caps are not set, the getcaps function returns the pad
template caps, and the pad_link function returns GST_PAD_LINK_DELAYED.
Other junk:
- negotiation can happen at any time
- negotiation can happen multiple times/often happens multiple times
- initial negotiation can lead to strange caps
- directed negotiation can happen in either direction (src to sink or
sink to src)
- Other considerations ignored, every pad should have a getcaps function.
- If a pad's getcaps function returns the same caps in every
circumstance, the getcaps function can be omitted.
- If you use gst_pad_use_explicit_caps(), the getcaps function must
be omitted.
- fixate functions are a method for applications to exert influence
on how a format is chosen from a caps. It's also used as a hack to
allow elements to do the same. Element fixate functions are _not_
intended to give good results for applications -- they're intended
to give non-disgusting results in gst-launch. Don't attempt to
make them do more than they're capable of.
- Fixate functions should not be implemented on anything except source
and sink elements.

19
docs/random/omega/EOS/chain-walkthrough
View file

@ -1,19 +0,0 @@
disksrc -> mp3parse -> mpg123 -> audiosink
disksrc reads last 4K chunk from disk. Sets EOS on the buffer, pushes buffer out to peer.
disksrc:src sets itself to EOS and chains to mp3parse.
mp3parse takes the buffer and slices it up into several.
Each buffer gets chained off to mpg123.
mpg123 decoders each frame and sends it to audiosink.
audiosink dumps the audio data to the sound card.
mpg123 returns from the chain, and returns.
mp3parse returns from each of the several chains,a nd returns.
disksrc:src returns from the chain, signals EOS, and sets disksrc's EOS flag
At this point the pipeline state code takes over and starts to set everyone to READY

21
docs/random/omega/IDEAS
View file

@ -1,21 +0,0 @@
Plugin Registry:
Have a web-accessible database of plugins from everywhere, including their merit values. gstplugin.c
and the autoplug code (should we abstract out autoplug into a special file? I think so) can make use of
this database via another module (gstwebregistry.c?) to look up stuff. A copy of the registry (gzip'd
XML) could even be cached. System and user options would determine whether this registry is checked
and/or updated automatically. The registry could simply be merged with the local machine and user
registries, with the state bit set to "don't even have it". Autodownload/install code should be
provided, though designed such that it's not toolkit/OS specific.
Merit:
Plugins and even type definition should carry merit values, allowing the system to determine which
plugin or type definition is better. This can be tied into the web registry setup, where merit values
can be updated without requiring an update of the plugins themselves. They can also be guaranteed
unique if there's a range reserved for registered objects. Unfortunately, that might get us into some
political wars. We could leave that up to the users via some voting system.
Scheduling state:
All the scheduling info for a Bin should be contained in a single object of some form. This can be
saved off and restored seamlessly at some point in the future. This would be ideal for some autoplug
cases, where the rest of the pipeline simply ceases to exist temporarily, and scheduling entry points
may need to be modified until the autoplug stage is finished.

54
docs/random/omega/TODO-0.1.0
View file

@ -1,54 +0,0 @@
mandatory:
-----------
(done) Fix compile warnings in gst/*
Make sure all common media types work with autoplug and gstmediaplay
Add interface to control the level of output, both compile- and run-time
(done) Make sure the build is capable of being run without any debugging noise
Make sure that the absence of any of the optional libraries will not be fatal
Make sure all the old plugins don't build normally, via some configure option?
Try to get more of the INFO calls worked out, removing gst_info
Decide whether DEBUG should use the category system, and if so, implement it
Icon for gstplay, .desktop file, etc.
Build tarballs
Build RPMs
Figure out which docs we're going to make 'offical'
UPDATE THE WEB SITE
optional:
---------
Fix gstreamer-launch to do named pads
Need to fix EOS subsystem so -launch can play out
gstreamer-launch should play out till EOS
Finish LADSPA plugin to data-moving stage
things to remember for the announcement:
---------------------------------------
Build requirements list:
libtool 1.3.5 or patched to "pass_all"
optional:
libcdparanoia
libmp3lame
libxaudio
libXv
libgdk-pixbuf
libglade

3
docs/random/omega/TYPE_FOURCC
View file

@ -1,3 +0,0 @@
#define GST_TYPE_FOURCC(f) \
(((f)[0]) & ((f)[1] << 8) & ((f)[2] << 16) & ((f)[3] << 24))

7
docs/random/omega/build/TODO
View file

@ -1,7 +0,0 @@
Convert all the STREAMER_ version stuff to GSTREAMER_
Convert all the --version-info lines to use GSTREAMER_LIBVERSION
Fix all the masses of Makefile.am's that override CFLAGS
Makefile.am's should use the library_la_CFLAGS and _LIBADD always

320
docs/random/omega/caps2
View file

@ -1,320 +0,0 @@
The elementfactory for a given element will contain some information about the capabilities of element's
pads or potential pads. An indication will be provided as to whether the pad always exists, always
exists once data is present, or *might* exist once data is present (the latter case is for things like
the MPEG system parsers, where an audio stream might or might not exist).
First, an entirely normal example:
----------------------------------
(-----------) (----------) (-------------)
! disksrc ! ! mpg123 ! ! audiosink !
! src sink src sink !
! ! ! ! ! !
(-----------) (----------) (-------------)
We start with only the disksrc. The typefind filter is attached to the disksrc, and via typefind magic
the properties of the disksrc are found to be:
disksrc->src->caps = {
"audio/mp3",
"layer", GST_CAPS_INT (3),
"bitrate", GST_CAPS_INT (128),
NULL
};
A look through the plugin registry shows that we have an element called mpg123 that has the following
caps:
static GstCapsFactory mpg123_sink_caps = {
"audio/mp3",
"layer", GST_CAPS_INT_RANGE (1, 3),
"bitrate", GST_CAPS_INT_RANGE (8, 320),
NULL
};
The caps of the disksrc fit within those parameters, so we instantiate an mpg123 and attach it to the
disksrc. The connection succeeds negotiation and as a result the mpg123 specifies its output caps as:
mpg123->src->caps = {
"audio/raw",
"format", GST_CAPS_BITFIELD (S16),
"depth", GST_CAPS_INT (16),
"rate", GST_CAPS_INT (44100),
"channels", GST_CAPS_INT (2),
NULL
};
Again from the plugin registry we find an element audiosink that has appropriate caps:
static GstCapsFactory audiosink_src_caps = {
"audio/raw",
"format", GST_CAPS_BITFIELD (S16,....),
"depth", GST_CAPS_INT (16),
"rate", GST_CAPS_INT_RANGE (4000, 96000),
"channels", GST_CAPS_INT_RANGE (1, 2),
NULL
};
A copy of the audiosink is instantiated and attached, negotiation goes smoothly, and we're done. No
dataflow has occurred, no failure found, etc. An ideal autoplug.
Now, a slightly more convoluted example:
----------------------------------------
Start with the same graph:
(-----------) (----------) (-------------)
! disksrc ! ! mpg123 ! ! audiosink !
! src sink src sink !
! ! ! ! ! !
(-----------) (----------) (-------------)
Run typefind on the disksrc's output, get the same output caps:
disksrc->src->caps = {
"audio/mp3",
"layer", GST_CAPS_INT (3),
"bitrate", GST_CAPS_INT (128),
NULL
};
Find and attach mpg123, get the following output caps this time:
mpg123->src->caps = {
"audio/raw",
"format", GST_CAPS_BITFIELD (S16),
"depth", GST_CAPS_INT (16),
"rate", GST_CAPS_INT (44100),
"channels", GST_CAPS_INT (1),
NULL
};
Note that this time we have a mono output. A look into the audiosink caps shows that we have a match.
So we instantiate a copy. Oops. We now find that the caps for the input pad on our audiosink have
changed:
mpg123->src->caps = {
"audio/raw",
"format", GST_CAPS_BITFIELD (S16,...),
"depth", GST_CAPS_INT (16),
"rate", GST_CAPS_INT (11025, 48000),
"channels", GST_CAPS_INT (2),
NULL
};
Whoops. It seems that the sound card we've got in this machine (FIXME how on earth to deal with
multiple sound cards???) doesn't support mono output *at all*. This is a problem. We now find that we
have no options as far as directly matching the mpg123 to the audiosink.
A look through our (fictitious) plugin registry shows at least one element that at least has audio/raw
on both input and output (since both mpg123 and audiosink have open pads with this mime type). A closerlook shows that its caps are:
static GstCapsFactory mono2stereo_sink_caps = {
"audio/raw",
"channels", GST_CAPS_INT (1),
NULL
};
static GstCapsFactory mono2stereo_src_caps = {
"audio/raw",
"channels", GST_CAPS_INT (2),
NULL
};
Wow, that's a perfect match. Instantiate, attach to mpg123, no problems. Attach to audiosink, no
problems. Done. When we start up the pipeline, we should get absolutely no callbacks from pads saying
"help me, I've fallen and..., er, I don't like this buffer!".
A really messy case:
--------------------
Start with a disksrc, typefind it, get the following:
disksrc->src->caps = {
"audio/mp3",
"layer", GST_CAPS_INT (3),
"bitrate", GST_CAPS_INT (128),
NULL
};
Look through the plugin registry, find mpg123. Instantiate it, attach it. It spits out audio
parameters as usual:
mpg123->src->caps = {
"audio/raw",
"format", GST_CAPS_BITFIELD (S16),
"depth", GST_CAPS_INT (16),
"rate", GST_CAPS_INT (44100),
"channels", GST_CAPS_INT (2),
NULL
};
Now we instantiate an audiosink plugin. This time, we're sunk:
mpg123->src->caps = {
"audio/raw",
"format", GST_CAPS_BITFIELD (S8,U8),
"depth", GST_CAPS_INT (8),
"rate", GST_CAPS_INT_RANGE (11025, 22050),
"channels", GST_CAPS_INT (1),
NULL
};
ACK! It's one of those Disney Sound Source things. We've got a problem here that isn't obviously
solvable. However, there happens to be another mp3 decoder sitting around. It's got the same
properties as mpg123, but a lower merit value. Let's instantiate one and attach it. We get the
following output pad caps:
mp3decoder->src->caps = {
"audio/raw",
"format", GST_CAPS_BITFIELD (S8,S16),
"depth", GST_CAPS_INT_RANGE (8,16),
"rate", GST_CAPS_INT_RANGE (8000, 44100),
"channels", GST_CAPS_INT (1,2),
NULL
};
Well, that matches the audiosink. We try attaching it, and during negotiation the mp3decoder finds
sufficient common ground with the castrated audiosink and sets its output pad to match the best of the
options: S8 at 22050 KHz.
Next to impossible scenario: DVD
--------------------------------
Start with a dvdsrc. It's output pad caps are:
static GstCapsFactory dvdsrc_src_caps = {
"video/mpeg",
"mpegversion", GST_CAPS_INT (2),
"systemstream", GST_CAPS_BOOLEAN (TRUE),
NULL
};
The type would be classified as incomplete via some mechanism. This might cause the autoplug code to go
and run the typefind function. It would flesh the type out to the following:
dvdsrc->src->caps = {
"video/mpeg",
"mpegversion", GST_CAPS_INT (2),
"systemstream", GST_CAPS_BOOLEAN (TRUE),
"videostreams", GST_CAPS_INT (1),
"audiostreams", GST_CAPS_INT (3),
"bitrate", GST_CAPS_INT (40960),
NULL,
};
Wow, that helped a lot. A check through the plugin registry shows that the mpeg2parse will match those
properties:
static GstCapsFactory mpeg2parse_sink_caps = {
"video/mpeg",
"mpegversion", GST_CAPS_INT (2),
"systemstream", GST_CAPS_BOOLEAN (TRUE),
NULL
};
(In retrospect, it may not be necessary to run typefind if there's match this good right away. Only run
typefind when there's no exact match.)
Since there are no output pads yet, we have to actually push data through the pipeline. The moment a
buffer or two get to the mpeg2parse element, it promptly goes and creates an output pad, probably of the
following caps:
mpeg2parse_video_src_caps = {
"video/mpeg",
"mpegversion", GST_CAPS_RANGE (1,2),
"systemstream", GST_CAPS_BOOLEAN (FALSE),
NULL
};
This seems to be a task for typefind again. But since data is flowing, we have to be careful with the
buffers. (This is the case in any typefind maneuver, but more so when one really can't rewind the
source without consequences) The autoplug system attaches a special pseudo-element to mpeg2parse's new
output pad, and attaches the typefind element to the end of that. The pseudo-element takes the buffer,
stores it, and passes a copy off to the attached element, in this case typefind. This repeats until
typefind has determined the type, at which point the typefind is removed, and the newly found element is
attached instead.
The pseudo-element is 'rewound' and the stored buffers flow out and into the newly attached element.
When the cache of buffers is gone, a signal fires and the autoplug system removes the pseudo-element and
reconnects the pipeline.
In this case, the typefind function will find the following:
mpeg2parse_video_src_caps = {
"video/mpeg",
"mpegversion", GST_CAPS_INT (2),
"systemstream", GST_CAPS_BOOLEAN (FALSE),
"bitrate", GST_CAPS_INT (36864),
"width", GST_CAPS_INT (720),
"height", GST_CAPS_INT (480),
"framerate", GST_CAPS_FLOAT (29.97002997),
"chromaformat", GST_CAPS_INT (1), [GST_CAPS_STRING ("4:2:0") ?]
NULL
};
Back to the plugin registry, we find our only choice is mpeg2dec, which has input caps of:
static GstCapsFactory mpeg2dec_sink_caps = {
"video/mpeg",
"mpegversion", GST_CAPS_RANGE (1,2),
"systemstream", GST_CAPS_BOOLEAN (FALSE),
NULL
};
Once again it just so happens that we really didn't need to do the typefind at all. But it can't hurt
unless the typefind is slow and painful, which we can guess won't be the case since the choices are
rather limited by the fact that there's already a MIME type attached, meaning we can drastically reduce
the number of typefind functions we try (down to one, actually).
However, since we *have* run the typefind, upon attachment of the input pad of mpeg2dec, the output pad
looks like the following:
mpeg2dec_src_caps = {
"video/raw",
"fourcc", GST_CAPS_LIST (
GST_CAPS_FOURCC ("YV12"), [identical...]
GST_CAPS_FOURCC ("IYUV"),
GST_CAPS_FOURCC ("I420"),
),
"width", GST_CAPS_INT (720),
"height", GST_CAPS_INT (480),
"framerate", GST_CAPS_FLOAT (29.97002997),
NULL
};
Currently only videosink supports the output of video/raw. It claims a list of FOURCCs but nothing
more:
static GstCapsFactory videosink_sink_caps = {
"video/raw",
"fourcc", GST_CAP_LIST ( GST_CAPS_FOURCC ("YV12"),
GST_CAPS_FOURCC ("IYUV"), GST_CAPS_FOURCC ("I420"),
GST_CAPS_FOURCC ("YUY2"), GST_CAPS_FOURCC ("UYVY"),
[ etc... ],
),
NULL
};
When instantiated, we potentially have the same problem as with the audiosink: we don't necessarily know
which hardware output to use. Somehow we have to solve the problem of setting some element arguments
before we can get useful information out of them as to the properties. In this case anyway, if the
videosink were to find only one output possibility, it would trim the list of FOURCCs it can deal with
to what the hardware can handle, as well as add further properties:
videosink_sink_caps = {
"video/raw",
"fourcc", GST_CAPS_LIST (GST_CAPS_FOURCC ("YV12"),
GST_CAPS_FOURCC ("YUY2"),
),
"width", GST_CAPS_INT_RANGE (4,1020),
"height", GST_CAPS_INT_RANGE (4,1020),
NULL
};
We can now connect the mpeg2dec output to the videosink, and we now have displaying video.
. . . .

20
docs/random/omega/caps3
View file

@ -1,20 +0,0 @@
/* fake mp3 where bitrates 8 through 32 can't be stereo */
static GstCapsListFactory mpg123_sink_caps = {
{
"audio/mp3",
"layer", GST_CAPS_INT_RANGE (1, 3),
"bitrate", GST_CAPS_INT_RANGE (32, 320),
"channels", GST_CAPS_INT_RANGE (1, 2),
"framed", GST_CAPS_BOOLEAN (TRUE),
NULL
},
{
"audio/mp3",
"layer", GST_CAPS_INT_RANGE (1, 3),
"bitrate", GST_CAPS_INT_RANGE (8, 32),
"channels", GST_CAPS_INT_RANGE (1),
"framed", GST_CAPS_BOOLEAN (TRUE),
NULL
},
NULL
};

30
docs/random/omega/debug-commit
View file

@ -1,30 +0,0 @@
Changes made to the DEBUG system. New header file gstdebug.h holds the stuff to keep it out of gst.h's
hair. DEBUG prints out the process id, cothread id, source filename and line number. Two new macros
DEBUG_ENTER and DEBUG_LEAVE are used to show the entry and exit of a given function. This eventually
might be used to construct call trace graphs, even taking cothreads into account. This would be quite
useful in visualizing the scheduling mechanism.
Minor changes to various debug messages.
Also sitting in gstdebug.h is a prototypical DEBUG_ENTER that's capable of performing DEBUG_LEAVE
automatically. It does this by utilizing a little-known GCC extension that allows one to call a
function with the same parameters as the current function. The macro uses this to basically call
itself. A boolean is used to ensure that when it calls itself it actually runs the body of the
function. In the meantime it prints stuff out before and after the real function, as well as
constructing a debugging string. This can be used eventually to provide call-wide data on the DEBUG
lines, instead of having to replicate data on each call to DEBUG. More research is needed into how this
would most cleanly be fit into some other chunk of code, like GStreamer (I think of this DEBUG trick as
a separate project, sorta).
Unfortunately, the aforementioned DEBUG trick interacts quite poorly with cothreads. Almost any time
it's used in a function that has anything remotely to do with a cothread context (as in, it runs in
one), a segfault results from the __builtin_apply call, which is the heart of the whole thing. If
someone who really knows assembly could analyze the resulting code to see what's really going on, we
might find a way to fix either the macro or the cothreads (I'm thinking that there's something we missed
in constructing the cothreads themselves) so this works in all cases.
In the meantime, please insert both DEBUG_ENTER and DEBUG_LEAVE in your functions. Be sure to put
DEBUG_ENTER after your variable declarations and before any functional code, not to put the function
name in any DEBUG strings (it's already there, trust me), and put a DEBUG_LEAVE if you care enough.
Changes are going to happen in the way DEBUGs and other printouts occur, so stay tuned.

52
docs/random/omega/eos.old
View file

@ -1,52 +0,0 @@
OUTDATED
--------
What I propose for EOS condition is th following: As is stands, EOS is a
flag in a buffer. However, it's also a signal from an element, which is
odd. What I propose is that EOS become a pad thing. Here's how it would
work:
When a source has a final buffer, it marks it as EOS. It passes this on
through the pad, and the pads set a 'pending' EOS state on the way
through. When another push or pull happens, only then does the pad signal
EOS. This means that EOS doesn't happen until the buffer has passed all
the way to the end, and a request for another buffer is starting to work
its way back. It gets stopped almost immediately.
This gets a bit messy in complex cases, haven't thought them all out.
Assuming everything is reasonably symmetric, it should work out cleanly.
We may have to add the ability to force buffers to pass even post EOS, but
that sounds like a hack.
Example: DVD
============
------------ |-------|
-| mpeg2dec |-----| |
------------------------ / ------------ | |
| video0|------/ ------------- | | -------------
| | -| subtitle0 |-----| merge |-------| videosink |
| subtitle0|------/ ------------- | | -------------
---------- | | ------------- | |
| dvdsrc |---| mpeg2parse subtitle1|--------| subtitle1 |-----| |
---------- | | ------------- |-------|
| audio0|--------\ ----------- |----------|
| | -| ac3dec0 |-----| |
| audio1|-------\ ----------- | | -------------
------------------------ \ ----------- | switcher |----| audiosink |
-| ac3dec1 |-----| | -------------
----------- |----------|
When dvdsrc reads its last buffer, it sets EOS on the buffer and passes it. The pad (probably audio1,
since video and audio are interleaved and subtitles don't usually end up at the end of a movie) would
set EOS pending, and pass it to mpeg2dec which would pass the EOS to the merge. Merge would then ask
for a buffer from subtitle1, since that's the one we're showing subtitles on right now. It would then
pull from the mpeg2parse. This would cause a switch back the parser, who's next task is to go and set
all the output pads to EOS. It may have to accomplish this by sending a zero-byte buffer.
Well, this example isn't actually very good. I'll go ahead and implement it, and we can see what
happens. The current setup is so lacking that anything is better....

3
docs/random/omega/filterfactory
View file

@ -1,3 +0,0 @@
A very useful feature would be a FilterFactory, i.e. some system whereby filter writers can avoid
getting their hands messy with the details of building and operating a G[tk]Object. A couple of
structures and a couple of functions should do it.

61
docs/random/omega/output_policies
View file

@ -1,61 +0,0 @@
GStreamer polices for various forms of library output, including error cases.
g_return_...
============
The parameters of a function are checked for validity (non-NULL, correct type, etc.) with g_return[_val]_if_fail, i.e.:
gst_element_connect (GstElement *src, gchar *srcpadname,
GstElement *dest, gchar *destpadname) {
g_return_if_fail (src != NULL);
g_return_if_fail (GST_IS_ELEMENT(src));
g_return_if_fail (srcpadname != NULL);
g_return_if_fail (dest != NULL);
g_return_if_fail (GST_IS_ELEMENT(dest));
g_return_if_fail (destpadname != NULL);
This will inform the user of the library of any basic problems with the arguments they pass, such as a NULL pointer.
ERROR
=====
The ERROR macro will be used whenever there is some other type of flaw in the data passed, at a GStreamer-specific
level:
srcpad = gst_element_get_pad (src, srcpadname);
if (srcpad == NULL) {
ERROR(src,"source element has no pad \"%s\"",srcpadname);
return;
}
An ERROR will generally cause ceasation of the application, and ideally launch gdb.
INFO
====
The INFO macro will be used to output any interesting state from the library, such as plugin loading and various events
of interest. They will be separated into categories that can be individually enabled and disabled.
Categories (first draft of list, unordered):
cothreads [cothreads.c]
gst initialization [gst.c]
autoplug results [autoplug.c]
autoplug attempts [autoplug.c]
bin parentage issues [gstbin.c]
plan generation [gstbin.c]
schedule generation [gstschedule.c]
schedule implementation [gstpad.c, gstbin.c, etc.]
buffer operations [gstbuffer.h]
caps matching [gstcaps.c]
clock stuff [gstclock.c]
element pad operations [gstelement.c]
elementfactory operations [gstelementfactory.c]
pad creation/connection [gstpad.c]
pipeline stuff [gstpipeline.c]
plugin loading [gstplugin.c]
plugin loading errors [gstplugin.c]
properties operations [gstprops.c]
thread creation/management [gstthread.c]
type operations [gsttype.c]
XML load/save [gstxml.c]

40
docs/random/omega/pad-negotiation
View file

@ -1,40 +0,0 @@
OUTDATED, EARLY IDEA
--------------------
When two pads are connected, a negotiation phase is going to have to
happen. Ideally, the caps of the two pads will both be fully-specified,
and match. That's the ideal case, but may rarely happen in practice.
It'll work the following way:
1) gst_pad_connect(pad1,pad2) is called by something
2) pad1's negotiate() method is called, with pad2 as argument
3) negotiate() repeatedly calls pad2's set_caps() method
At some point, the two pads will agree on a set of caps, and proceed by
returning TRUE from negotiate(), at which point gst_pad_connect()
finishes. If it returns FALSE, gst_pad_connect() is forced to fail.
Now, obviously the algorithm used to find matching caps can get
complicated. But in some cases it'll be simple. Ideally, if there is no
negotiate() function for pad1, there'll be a function that will go through
the options and try to make pad1 and pad2 meet in the middle, with no
specific knowledge of what the caps actually mean.
Another detail is deciding which pads are pad1 and pad2. In the autoplug
case, the code will sometimes know which of the pads have the more
specific caps. In others, you may not. Either you can guess, and
possibly lose to having the slower of the two pad's negotiate() methods do
the work, or you might be able to actually guess at which is the most
specific set of caps:
For any given typeid in the union of both pads, look at all properties
For each property, find the smallest range, assign this a 1.0
For all other instances of this property, assign relative to 1.0
For each pad1,pad2
Take the assigned value of every property, and multiply together
Whichever value is lower between pad1 and pad2 is most likely to be the
most specific set of caps. The trick is implementing the above
efficiently, but on the surface there appear to be more than enough
potential optimizations.

46
docs/random/omega/padtemplates
View file

@ -1,46 +0,0 @@
OUTDATED
--------
typedef gpointer GstCapsFactoryEntry;
typedef GstCapsFactoryEntry GstCapsFactory[];
typedef GstCapsFactory *GstCapsListFactory[];
typedef gpointer GstPadFactoryEntry;
typedef GstPadFactoryEntry GstPadFactory[];
typedef GstPadFactory *GstPadListFactory[];
#define GST_PADFACTORY_SRC GINT_TO_POINTER (GST_PAD_SRC)
#define GST_PADFACTORY_ALWAYS GINT_TO_POINTER (GST_PAD_ALWAYS)
typedef struct GstPadTemplate {
gchar *name_template;
gint direction;
gint presence;
GList *caps;
};
GstPadFactory mpg123_src_padfactory = {
"src"
GST_PADFACTORY_SRC,
GST_PADFACTORY_ALWAYS,
"audio/raw"
"samplerate", GST_PROPS_INT (44100),
. . .
NULL
};
static GstPadTemplate *srcpadtemplate;
mpg123_new(GstMpg123 *mpg123) {
mpg123->srcpad = gst_pad_new_template("src", srcpadtemplate);
. . .
}
plugin_init() {
GstElementFactory *factory;
factory = gst_elementfactory_new("mpg123",. . .);
srcpadtemplate = gst_padfactory_new(mpg123_src_padfactory);
get_elementfactory_add_padtemplate (srcpadtemplate);
}

87
docs/random/omega/plan-generation
View file

@ -1,87 +0,0 @@
OUTDATED
--------
Plan generation happens at transition from NULL to READY (and PLAYING to READY right now, need to fix
that). By way of some logic in gst_bin_change_state(), gst_bin_create_plan() is only called for the
outer Bin, usually a Pipeline. This keeps things from getting nasty later on.
A major new concept in plan generation is that of the 'manager'. This is the element that is responsible
for running a given element. In general, Pipelines and Threads are the only managing-capable elements
(have the MANAGER flag set), since they are the only ones with real scheduling authority (because they
have a process context to play with, basically).
gst_bin_set_manager() is called to set the manager element of the bin and all it's children and their
children. However, there's one important trick: it won't recurse into child Bins that have the MANAGER
flag set. This avoids some highly redundant recursion.
When create_plan() is called on the outside Pipeline, the first thing it does is call
set_manager(self,self). As noted above, this recursion will not proceed into child Bins that have the
MANAGER flag set.
The next step is to recursively generate the plan (yes, head-recursive). This gives child Bins the
opportunity to generate their plan first, causing a inside-to-outside sequence. This matches the way
the scheduling is arranged now, where the plan for a Src/Connection outside a Bin is handled by that
Bin, not it's parent. But we must be very careful not to stomp on that plan in the parent Bin.
Because create_plan() is called on all Bins, but we can only set up scheduling state in MANAGER bins,
create_plan() must perform create_plan() recursion, but not do anything else *unless* the MANAGER bit is
set. It shouldn't even call set_manager() unless it's a MANAGER itself, because calling it otherwise
would waste time doing the work again. Basically, from the standpoing of setting the manager,
create_plan() recursion starts it when the current Bin is a MANAGER, and set_manager() stops when it
finds the next one. create_plan()'s further recursion eventually starts the process back up again
furtuer down the hierarchy, until everything is covered.
For all MANAGER Bins, the last step is to actually create the scheduling plan. This is still one of the
nastiest chunks of code in the whole project, and probably will do nothing but get worse from now on (it
got better recently, but only because I took a chainsaw to the code and broke everything...). It will
remain similar to what it is now, but with some definite differences.
First task is now to find all the elements that we're responsible for. This is normally a recursive
process, because the structure is an arbitrary tree. However, something like the following should work
(bin is self):
GSList *elements = NULL;
GList *children;
GSList *waiting_bins = NULL;
GstBin *waiting_bin;
waiting_bins = g_slist_prepend (waiting_bins,bin);
while (waiting_bins) {
// retrieve the top of the stack and pop it
waiting_bin = GST_BIN (waiting_bins->data);
waiting_bins = g_slist_remove (waiting_bins,waiting_bin);
// walk the list of elements, and find bins
children = waiting_bin->children;
while (children) {
// add it to the list of elements
elements = g_slist_prepend (elements, children->data);
// if it's a bin and it's not a managing bin,
// shove it on the list of bins to recurse into
if (GST_IS_BIN (children->data) &&
!GST_FLAG_IS_SET (GST_ELEMENT (children->data)))
waiting_bins = g_slist_prepend (waiting_bins,children->data);
children = g_list_next (children);
}
}
The code makes the assumption that the manager of every element is the same until such time as a
different managing parent appears in the hierarchy. This is the result of the aforementioned nested
recursion of create_plan() and set_manager(), but may not remain the case forever. The above loop
should probably be slightly re-written to work solely on whether or not the Bin in question is the
element's manager. This means that the child Bins are *always* recursed into, in case there's a rogue
element inside of one of them that's supposed to be managed.
At the same time all the elements managed by this bin are found (i.e. in the inner loop), we can
determine some useful bits of information, such as testing for several cases that require the use of
cothreads. The availability of manager information at this point may aid significantly in this
decision.
Finally, the scheduling plan is generated, based on all the elements to be managed by the Bin (the list
of which may span several 'generations' of Bins and elements). Elements which have peers in child
self-managed Bins are left alone on for the pad in that makes that connection. This should keep the
parent Bins from stepping all over state set up by the child Bins, by establishing clear implicit
ownership on the pad level, based on the managing Bins' relationship to the pad.

20
docs/random/omega/sched-case
View file

@ -1,20 +0,0 @@
Case 1:
---------------------------------------------------
| pipeline |
| --------------- ---------------- |
| | bin | | thread | |
| | ----------- | ------------ | ------------ | |
| | | fakesrc | | | queue | | | fakesink | | |
| | | src>|-|--|<sink src>|--|-|<sink | | |
| | ----------- | ------------ | ------------ | |
| --------------- ---------------- |
---------------------------------------------------
Pipeline manages: fakesrc, queue
Thread manages: fakesink
Both forced to use cothreads.
First thing the thread does is try to pull from the queue. Because it's a chain function, it runs in
_chain_wrapper, which calls gst_pad_pull().

103
docs/random/omega/sched-commit1
View file

@ -1,103 +0,0 @@
STATUS: pushregion/pullregion is gone
-------------------------------------
Changed the way things are scheduled, especially sources. A Src used to
have a push() function, and optionally a pushregion() to deal with async
reads, etc. That whole thing has gone away, in favor of providing a
pull() function for the output (Src) pad instead, ala chain functions.
This makes constructing cothreaded schedules out of non-loop elements
somewhat easier. Basically there was always a question as to which pad
was being dealt with. In the pullregion case, cothread-specific data was
used to try to pass the region struct to the right place, which is a slow
hack. And in general, the push function severely limited the kind of
tricks that could be played when there's more than one output pad, such as
a multi-out file reader with async capabilities on each pad independently.
This changes the way cothread scheduling occurs. Instead of the hack to
deal with Src's by calling their push() function (or optionally the
pushregion(), in certain cases), we now are working towards a general
mechanism where pads are the only thing that are dealt with directly.
An optimization was made in the process of doing this: the loopfunction
actually run as the outer [stack] frame of the cothread is now set more
intelligently in create_plan() based on what kind of element it is. We
now have:
loopfunc_wrapper: used for loop-based elements, it simply calls the
loopfunc in a loop, paying attention to COTHREAD_STOPPING (see
below). It currently does other, soon to be deprecated, stuff.
pullsrc_wrapper: wraps a Src that's not loop-based (since your options
are now loop- or pull-based)
There will be a couple more to deal with other cases, such as Connections
and chain-based elements. The general idea is that it's a lot more
efficient to make the decisions once in create_plan than to keep doing
this huge if/else chain in the wrapper. Just choose the right wrapper up
front. It'll be most apparent performance-wise in the case of whichever
element context is switched to first for each iteration, since the whole
wrapper setup is done for every iteration.
The tricky part is that there is now a bit of overloading of the function
pointers in a pad. The current meanings (possibly to change a bit more
soon) are:
chainfunc: as always, chainfunc pointer is mirrored between peer pads
(this may change, and the chain func may end up in pushfunc)
pushfunc: SrcPad: gst_pad_pushfunc_proxy, cothread_switch to peer
SinkPad: none (may take over chainfunc, see below) pullfunc:
SrcPad: Src or Connection's function to construct buffers
SinkPad: gst_pad_pullfunc_proxy, cothread_switch to peer
There are a number of issues remaining with the scheduling, not the least
of which is the fact that Connections are still dealt with the old way,
with _push() functions and such. I'm trying to figure out a way to unify
the system so it makes sense. Following the scheduling system is hard
enough, trying to change it is murder.
Another useful scheduling addition, mentioned above, is COTHREAD_STOPPING.
It's an element flag that's used to signal whatever code is running in
cothread context that it should be finishing up and exiting soon. An
example of this is in plugins/cobin/spindentity.c. All the loops should
now be composed of do/while loops, rather than while(1) loops:
do {
buf = gst_pad_pull(spindentity->sinkpad);
gst_pad_push(spindentity->srcpad,buf);
} while (!GST_ELEMENT_IS_COTHREAD_STOPPING(element));
The reason for this is that COTHREAD_STOPPING may be set before the above
loop ever gets started. It wouldn't do for the body of the loop to never
once get called, that would simply stall the pipeline. Note that only the
core library code is ever responsible for setting and unsetting this flag.
All elements have to do is respond to it by cleanly exiting the loop and
the function holding it.
This is needed primarily to allow iterations to occur properly.
Basically, there's a single entry point in the cothread scheduling loop,
gst_bin_iterate_func() simply switches to this cothread. If the element
in this context is allowed to loop infinitely, nothing would even switch
back to the context from which the iterate() was originally called. This
is a bit of a problem. The solution is for there to be an implicit switch
back to the originating context. Now, even I'm not sure exactly how this
works, but if the cothread that's switched to actually returns, execution
returns back to the calling context, i.e. iterate_func().
COTHREAD_STOPPING is therefore set just before switching into this
(currently randomly chosen) context, on the assumption that it will return
promptly after finishing its duties. The burden of clearing the flag
falls to the various wrapper functions provided by the Bin code, thus
element writers don't have to worry about doing that at all (and simply
shouldn't).
Related changes:
All the sources in elements/ have been changed to reflect the new system.
FIXMEs:
1) gstpipeline.c calls gst_src_push at some point, dunno why, it's
commented out now.
2) any other sources, including vcdsrc, dvdsrc, and v4lsrc will break
badly and need to be modified to work as pull-based sources.

89
docs/random/omega/sched/chains
View file

@ -1,89 +0,0 @@
A new concept in scheduling is that of chains of elements that need to be schedule separately, even in the same set of
managed elements (which is the set of elements that the Bin in question [a pipeline or thread] is responsible for).
An example would by anywhere you have a non-blocking queue in place for buffering. This kind of element might be
useful in cases where the scheduling on a buffer level is tight enough that deadlocks might occur.
The scheduler will find chains by traversing the pipeline through the list of managed elements. A chain boundary is
anywhere you have a 'DECOUPLED' element. A DECOUPLED element is one where there is no direct correlation between the
activities of the various pads. A source fits this description, although the normal single-pad source is the
degenerate case. A queue more properly fits the bill, since pushing a buffer at the sink pad doesn't trigger anything
on the src pad, and vice versa. A multi-src async source is probably the best example, since you want to leave the
scheduling up to the elements connected to it.
Anyway, first the simple case:
fakesrc -> fakesink
Both of them should probably have the DECOUPLED bit set, at least to be true to the nature of the actual fake elements.
These two end up being a chain, and scheduling has to be set up for the chain. There are no cothreaded elements in the
chain, which means it's relatively easy. The goal is to find a single entry into the chain, which can be called in a
loop to get things done. Since the fakesrc is DECOUPLED, and we'd be messing with the source pad, it has lower
priority than a DECOUPLED sink pad, so the fakesrc's sink pad is the ideal entry into the chain. This can be
simplified into saying that the fakesink is the entry.
In the end, the code to do this boils down to:
buf = gst_pad_pull (fakesink->sinkpad);
gst_pad_push (fakesink->sinkpad, buf);
Because of the way things are no implemented for scheduling, turning it around and making the source the entry has no
effect as far as the efficiency. That's because _get no longer directly calls gst_pad_push(), so we have to do it
outside. No big deal, it boils down to the same thing I think, modulo a cache-line of stack (i.e. one or two fewer
this way).
If we put an identity in the middle:
fakesrc -> identity -> fakesink
then we have the same thing, except that there's now an element that isn't DECOUPLED, so it gets higher priority. That
means the identity is now the entry, and when we push the buffer into its chain function, the fakesink gets called.
Now, we can make this much more complex, with the following elementary echo meta-filter:
|=====| -> delay1 -> |=====|
| | | |
-> queue -> | tee | -> delay2 -> | mix | -> queue ->
| | | |
|=====| -> delay3 -> |=====|
The tee takes a buffer in and spits three out, delay shifts the timestamps around and possibly reframes things to be
friendly. mix takes the three buffers and simply sums them (they're all audio). The tee element takes one buffer in
and promptly spits three out, one after another. Delay takes an element and immediately spits out a buffer (it
zero-pads at the beginning [the duration of the delay] for the sake of argument). Mix in this case is chained, but
assumes that buffer will arrive in order. On the last chain, it does a push of the newly mixed audio buffer.
The queues are both DECOUPLED, so they have lower weight. That leaves a bunch of other elements sitting there ripe for
entry. But if we were to take delay1, what would happen? Well we can't, since there's no _get function on the tee's
src pads.
This just re-enforces the idea that the left-most (closest to the source, for you right-to-left people) element should
get to be the entry. But what if we have multiple left-most elements?:
-> queue -> eq1 -> |=====|
| mix | -> queue
-> queue -> eq2 -> |=====|
If eq1 is the only entry, we call pull on the queue, then chain over to mix. Mix then doesn't do anything with it,
since it's waiting for another buffer before doing anything. That means we have to do the same with eq2, and have it
chain to mix, at which point mix will do its magic and chain out to the right-hand side. Figure out to actually use
both entries is hard, because the idea at this point is that there's only a single entry to a chain.
Does this mean that we should make mix a DECOUPLED element? That would fix it to some extent, giving us three chains
in the above case. Each eq chain would be driven by the eq element, pulling from the queue and pushing into the mixer.
The mixer -> queue chain is problematic, because there is no possibly entry. The mixer side has no _get function
(since the push always happens upon the receipt of a buffer from the second sink pad), which means that those two
pads have no possible entrance.
Cothreads make this case much easier, since the mix element would drive things, forcing the eq elements to pull and
process buffers in order as needed. It may be that the best option in the case where there are any multi-sinkpad
elements is to turn them into cothreads.
Now, on to cothreaded cases. The simplest possible is to turn all the elements into cothreads. I may punt on this and
do just that for the moment, but there's still the question of what to do at the ends of the chain, where the DECOUPLED
elements are. The easiest is to simply always make then chained, so there's never any worry about who owns the
cothread context for the element, simply because there never will be one.
fakesrc -> queue -> @identity -> fakesink
We just set it up so both ends of the queue are chained, and all is well.

39
docs/random/omega/sched/walkthrough-72
View file

@ -1,39 +0,0 @@
72)
[-pipeline---------------------------------------------------------------------------------------------]
! [-bin-----------------------------] [-thread---------] !
! ! [--------] [---------] ! [------] [---------] [------] ! [--------] ! !
! ! !faksesrc! !identity1! ! !queue1! !identity2! !queue2! ! !fakesink! ! !
! ! ! src --- sink * src --- sink n src -- sink src -- sink src -- sink ! ! !
! ! [--------] [---------] ! [------] [---------] [------] ! [--------] ! !
! [---------------------------------] [----------------] !
[------------------------------------------------------------------------------------------------------]
-----
Ideally, you'd end up with the following sub-pipelines
pipeline:
fakesrc -> identity1 -> queue1
queue1 -> identity2 -> queue2
thread:
queue2 -> fakesink
They'd be scheduled as following:
fakesrc: passive chained, pulled by identity1
identity1: loopfunc cothreaded, ENTRY
queue1:sink: passive chained, pushed by identity1
queue1:src: _get-based iteration, ENTRY
identity2: chained by queue1
queue2:src: passively chained, pushed by identity2
queue2:sink: passively chained, pulled by fakesink
fakesink: loopfunc cothreaded, ENTRY
-----
Most likely, we'd end up with the following

13
docs/random/omega/sched2
View file

@ -1,13 +0,0 @@
Currently, when an element wants to push or pull, they call gst_pad_*. These functions then decide what
to do based on the selection of pointers in the pad. There are at least 2 options for each, plus some
header stuff to do various checking of flags. Unfortunately, because of the selection of pointers in
the pad, there is at least one case where scheduling has to be tricked, by providing a pointer to a
function with no body.
What I propose is that these functions be replaced with macros that call a function pointer directly.
The default functions (provided by GstPad) would be capable of chaining, that's about it. When a
schedule is formed, these get replaced with more specific functions, provided by GstBin or a subclass.
In the chain case, the pad_push_func might even be replaced brute force with the chain function, since
they have the same prototype. In the cothreaded case, various functions would provide the ability to
switch, deal with bufpens, etc.

66
docs/random/omega/scheduling
View file

@ -1,66 +0,0 @@
STATUS: pull on srcpads is outdated, they use _get
--------------------------------------------------
0) definitions:
All pads without further specifiers are assumed to belong to the element
in question. The pad's peer is always denoted with ->peer. If there's
question, pads will be prefixed with self-> and other-> as necessary.
All elements in this document have at most one source and one sink pad,
called srcpad and sinkpad. Multi-pad cases are supposed to be simple
extrapolations except in a couple strange cases, to be covered elsewhere.
1) loop functions:
A loop function will call gst_pad_pull(sinkpad), do something, and call
gst_pad_push(srcpad).
gst_pad_pull first checks to see if there's a buffer in the pen. If not,
it calls that pad's pullfunc handler, passing it the peer pad. When that
finishes, we check again and return the buffer. If no buffer, we squawk.
The pullfunc handler simply causes a cothread switch to the peer pad's
context;
gst_pad_push places the buffer in the peer pad's pen, and calls the local
pad's pushfunc. The pushfunc simply causes a switch to the peer pad's
context.
2) chain functions
The loopfunc constructed around a chain function starts by finding all
sink pads. For each sink pad, it calls gst_pad_pull. This causes a
switch to the peer pad's context and a buffer to appear, which is
returned. The pad is the handed off to the chain function, which
presumably does some processing and calls gst_pad_push(), which causes a
context switch to that pad's peer.
3) source
The loopfunc must repeatedly call the srcpad's pull
--------------------------------- ---------------------------------
srcpad | | sinkpad
| |
GstPad *peer; | <-p-> | GstPad *peer;
pointer to peer pad | | pointer to peer pad
| |
funcptr *pushfunc; | --s-> | funcptr *pushfunc;
causes switch to peer ctx | | element uses buffer in pen,
| |
funcptr *pullfunc; | <-s-- | funcptr *pullfunc;
element puts buffer in pen, | | causes switch to peer ctx
calls gst_pad_push() | |
| |
funcptr *pullregion func | <-s-- | funcptr *pullregionfunc;
element puts region in pen | | causes switch to peer ctx
calls gst_pad_push() | |
--------------------------------- ---------------------------------

3
docs/random/omega/testing/Makefile
View file

@ -1,3 +0,0 @@
gstobject: gstobject.c
libtool gcc -o gstobject gstobject.c -I../../../.. ../../../../gst/libgst.la \
`gtk-config --cflags --libs` `xml-config --cflags --libs`

144
docs/random/omega/testing/framework
View file

@ -1,144 +0,0 @@
Construction
Validation
Testing
Construction will generate some state
Validation will ensure that everything is kosher
Tests use combinations of the above to check things
##### Example:
parent = gst_object_get_parent(object);
setup:
create: new object
action: object = gst_object_new();
validation: object != NULL, object->parent == NULL, etc
[cleanup: gst_object_destroy(object);]
create: new parent
action: parent = gst_object_new();
validation: parent != NULL, parent->parent == NULL, etc
[cleanup: gst_object_destroy(parent);]
create: set object's parent
precondition: object->parent == NULL
action: gst_object_set_parent(object,parent);
validation: object->parent = parent
preconditions:
nothing
action:
curparent = gst_element_get_parent(object);
validation:
curparent == object->parent
curparent == parent
cleanup:
nothing
##### Resulting code:
///// setup
// new object
object = gst_object_new();
ASSERT(object != NULL);
ASSERT(object->parent == NULL);
// new object
parent = gst_object_new();
ASSERT(parent != NULL);
ASSERT(parent->parent == NULL);
// set object parent
ASSERT(object->parent == NULL);
gst_object_set_parent(object,parent);
ASSERT(object->parent != NULL);
///// preconditions
///// action
curparent = gst_element_get_parent(object);
///// validation
ASSERT(object->parent == parent);
///// cleanup
gst_object_destroy(parent);
gst_object_destroy(object);
##### XML descriptions
<construct name="new object">
<variable>
GstObject *object;
</variabls>
<action>
object = gst_object_new();
</action>
<validation>
<assert>
object != NULL
</assert>
<assert>
GST_IS_OBJECT(object)
<assert>
object->parent == NULL
</assert>
</validation>
<cleanup>
<validation>
<assert>
object != NULL
</assert>
<assert>
GST_IS_OBJECT(object)
</assert>
</validation>
<action>
gst_object_destroy(object);
</action>
</cleanup>
</construct>
<construct name="set object parent">
<variable>
GstObject *object;
</variable>
<variable>
GstObject *object;
</variable>
<precondition>
<assert>
object->parent == NULL
</assert>
</precondition>
<action>
gst_object_set_parent(object,parent);
</action>
<validation>
<assert>
object->parent == parent
</assert>
</validation>
</construct>
<test name="set object parent">
<variable>
GstObject *object;
<variable>
</variable>
GstObject *parent;
<variable>
</variable>
GstObject *curparent;
</variable>
<setup>
object = gst_object_new();
parent = gst_object_new();
gst_object_set_parent(object,parent);
</setup>
<action>
curparent = gst_element_get_parent(object);
</action>
<validation>
curparent == object->parent
curparent == parent
</validation>
</test>

314
docs/random/omega/testing/gstobject.c
View file

@ -1,314 +0,0 @@
#include <gst/gst.h>
static gchar *_subject, *_category;
static gint _testnum = 0;
static gboolean _passed;
static gint _total_tests = 0, _passed_tests = 0;
static gint _random_size;
void
tabpad (gchar * str, gint width)
{
int i;
for (i = 0; i < width - strlen (str); i++)
fprintf (stderr, " ");
}
#define TEST_SUBJECT(subject) fprintf(stderr,"Subject: %s\n",subject),_subject = subject
#define TEST_CATEGORY(category) fprintf(stderr,"\n\nCategory: %s\n",category)
#define TEST(test) fprintf(stderr,"Test %d: %s...\n",_testnum,test),_passed = TRUE
#define ASSERT(expr) G_STMT_START{ \
fprintf(stderr,"\t%s:",#expr);tabpad(#expr,50); \
if (!(expr)) { \
fprintf(stderr,"FAILED\n"); \
_passed = FALSE; \
} else { \
fprintf(stderr,"passed\n"); \
} \
}G_STMT_END;
#define ENDTEST() G_STMT_START{ \
_testnum++; \
if (_passed) { \
fprintf(stderr,"\tpassed.\n"); \
_passed_tests++; \
} else { \
fprintf(stderr,"\tFAILED.\n"); \
} \
_total_tests++; \
}G_STMT_END;
void
SETUP_RANDOM_SIZE (void *random, gint size)
{
int i;
if (random)
g_free (random);
_random_size = size;
random = g_malloc (_random_size);
for (i = 0; i < _random_size; i++)
((unsigned char *) random)[i] = i;
}
#define SETUP_RANDOM(random,type) SETUP_RANDOM_SIZE(random,sizeof(type))
gboolean
RANDOM_OK (void *random)
{
int i;
for (i = 0; i < _random_size; i++) {
if (((unsigned char *) random)[i] != i) {
SETUP_RANDOM_SIZE (random, _random_size);
return FALSE;
}
}
return TRUE;
}
int
main (int argc, char *argv[])
{
GstObject *object;
GstObject *parent;
GstObject *newparent;
GtkObject *gtkobject;
GstObject *curparent;
gst_init (&argc, &argv);
TEST_SUBJECT ("GstObject");
TEST_CATEGORY ("Creation");
TEST ("create object");
/* setup */
/* action */
object = gst_object_new ();
/* assertions */
ASSERT (object != NULL);
ASSERT (GST_IS_OBJECT (object));
/* cleanup */
g_free (object);
ENDTEST ();
/* new category */
TEST_CATEGORY ("Refcounting");
/* category setup */
object = gst_object_new ();
TEST ("new object");
/* setup */
/* action */
/* assertions */
ASSERT (object->refcount == 1);
ASSERT (GTK_OBJECT_FLOATING (object) == TRUE);
/* cleanup */
ENDTEST ();
TEST ("increment refcount");
/* setup */
/* action */
gst_object_ref (object);
/* assertions */
ASSERT (object->refcount == 2);
ASSERT (GTK_OBJECT_FLOATING (object) == TRUE);
/* cleanup */
ENDTEST ();
TEST ("sink object");
/* setup */
/* action */
gst_object_sink (object);
/* assertions */
ASSERT (object->refcount == 1);
ASSERT (GTK_OBJECT_FLOATING (object) == FALSE);
/* cleanup */
ENDTEST ();
TEST ("increment refcount after sink");
/* setup */
/* action */
gst_object_ref (object);
/* assertions */
ASSERT (object->refcount == 2);
ASSERT (GTK_OBJECT_FLOATING (object) == FALSE);
/* cleanup */
ENDTEST ();
TEST ("decrement refcount after sink");
/* setup */
/* action */
gst_object_unref (object);
/* assertions */
ASSERT (object->refcount == 1);
ASSERT (GTK_OBJECT_FLOATING (object) == FALSE);
/* cleanup */
ENDTEST ();
/* category cleanup */
g_free (object);
/* new category */
TEST_CATEGORY ("Parentage");
/* category setup */
object = gst_object_new ();
parent = gst_object_new ();
newparent = gst_object_new ();
gtkobject = gtk_type_new (gtk_object_get_type ());
/* category assertions */
ASSERT (object != NULL);
ASSERT (object->refcount == 1);
ASSERT (object->parent == NULL);
ASSERT (parent != NULL);
ASSERT (newparent != NULL);
ASSERT (gtkobject != NULL);
ASSERT (!GST_IS_OBJECT (gtkobject));
TEST ("gst_object_set_parent: null object");
/* setup */
/* action */
gst_object_set_parent (NULL, NULL);
/* assertions */
ASSERT (object->parent == NULL);
/* cleanup */
ENDTEST ();
TEST ("gst_object_set_parent: invalid object");
/* setup */
/* action */
gst_object_set_parent ((GstObject *) gtkobject, NULL);
/* assertions */
ASSERT (object->parent == NULL);
/* cleanup */
ENDTEST ();
TEST ("gst_object_set_parent: null parent");
/* setup */
/* action */
gst_object_set_parent (object, NULL);
/* assertions */
ASSERT (object->parent == NULL);
/* cleanup */
ENDTEST ();
TEST ("gst_object_set_parent: invalid parent");
/* setup */
/* action */
gst_object_set_parent (object, (GstObject *) gtkobject);
/* assertions */
ASSERT (object->parent == NULL);
/* cleanup */
ENDTEST ();
TEST ("gst_object_set_parent: valid object, parent is object");
/* setup */
/* action */
gst_object_set_parent (object, object);
/* assertions */
ASSERT (object->parent == NULL);
/* cleanup */
ENDTEST ();
TEST ("gst_object_set_parent: valid object and parent");
/* setup */
/* action */
gst_object_set_parent (object, parent);
/* assertions */
ASSERT (object->parent == parent);
/* cleanup */
ENDTEST ();
TEST ("gst_object_set_parent: parent already set");
/* setup */
/* action */
gst_object_set_parent (object, newparent);
/* assertions */
ASSERT (object->parent != newparent);
ASSERT (object->parent == parent);
/* cleanup */
g_free (object);
ENDTEST ();
TEST ("gst_object_get_parent: null object");
/* setup */
/* action */
curparent = gst_object_get_parent (NULL);
/* assertions */
ASSERT (curparent == NULL);
/* cleanup */
ENDTEST ();
TEST ("gst_object_get_parent: invalid object");
/* setup */
/* action */
curparent = gst_object_get_parent ((GstObject *) gtkobject);
/* assertions */
ASSERT (curparent == NULL);
/* cleanup */
ENDTEST ();
TEST ("gst_object_get_parent: no parent");
/* setup */
object = gst_object_new ();
/* action */
curparent = gst_object_get_parent (object);
/* assertions */
ASSERT (curparent == NULL);
/* cleanup */
ENDTEST ();
TEST ("gst_object_get_parent: valid parent");
/* setup */
gst_object_set_parent (object, parent);
/* action */
curparent = gst_object_get_parent (object);
/* assertions */
ASSERT (curparent == parent);
/* cleanup */
g_free (object);
ENDTEST ();
TEST ("gst_object_unparent: null object");
/* setup */
/* action */
gst_object_unparent (NULL);
/* assertions */
/* NONE - FIXME! */
/* cleanup */
ENDTEST ();
TEST ("gst_object_unparent: invalid object");
/* setup */
/* action */
gst_object_unparent ((GstObject *) gtkobject);
/* assertions */
/* NONE - FIXME! */
/* cleanup */
ENDTEST ();
TEST ("gst_object_unparent: no parent");
/* setup */
object = gst_object_new ();
/* category cleanup */
g_free (object);
g_free (parent);
g_free (newparent);
g_free (gtkobject);
fprintf (stderr, "\n\nTotal tests:\t%d\n", _total_tests);
fprintf (stderr, "Total passed:\t%d\n", _passed_tests);
fprintf (stderr, "Total FAILED:\t%d\n", _total_tests - _passed_tests);
}

95
docs/random/omega/testing/gstobject.txt
View file

@ -1,95 +0,0 @@
Subject: GstObject
Areas to test
=============
Creation
Refcounting
Destruction
Flags
Locking
Parentage
Path string
Tests
=====
Creation
--------
Create an object
Does it return !NULL
GST_IS_OBJECT() ?
Refcounting
-----------
Create new object
object->refcount == 1, GTK_OBJECT_FLOATING(object) == TRUE
Increment refcount
object->refcount == 2
Sink object
object->refcount == 1, GTK_OBJECT_FLOATING(object) == FALSE
Increment refcount
object->refcount == 2
Decrement refcount
object->refcount == 1
Destruction
-----------
???
Flags (start with new object)
-----
Create new object
Verify that all flags are unset
Set a flag
Verify it's set
Unset a flag
Verify it's not set
Locking (start with new object)
-------
Lock an object
Try to lock, get false
Parentage (start with new object, check refcount == 1)
---------
gst_object_set_parent: (start with new parent object)
Pass NULL...
Pass !NULL, but not Object...
Pass NULL parent...
Pass !NULL parent, but not Object...
Pass valid Object, and parent == object
object->refcount == 1
GTK_OBJECT_FLOATING(object) == TRUE
object->parent == NULL
Pass valid Object
object->refcount == 1
GTK_OBJECT_FLOATING(object) == FALSE
object->parent == parent
The "parent_set" signal should fire with the object and parent as args
Pass Object with parent already set
object->parent should not equal new parent
object->refcount == 1
gst_object_get_parent:
Pass NULL...
Pass !NULL, not Object...
Pass valid object with no parent
Get NULL
Pass valid object with parent
Get parent pointer
gst_object_unparent:
Pass NULL, with no parent
no effect
Pass !NULL, not Object, NULL parent
pointer not mangled
Pass valid object, with no parent
object->parent == NULL
object->refcount = 1
Pass NULL, with valid parent
no effect
Pass !NULL, not Object, with valid object as parent
pointer not mangled
Pass valid object, with valid parent
object->parent == NULL
object->refcount == 0

77
docs/random/omega/type-properties
View file

@ -1,77 +0,0 @@
OUTDATED
--------
A type properties system might look like following:
1) Type definition includes the properties and their ranges:
audio/mp3
layer: 1 - 3
bitrate: 8 - 320
audio/raw
format: bitfield (using asound.h definitions)
depth: 8 - 32
rate: 4000 - 96000
channels: 1 - n
interleave: boolean
video/raw
format: 32-bit FOURCC
bpp: 1 - 32
width: 1 - n
height: 1 - n
framerate: 32-bit float
etc.
2) An element can specify what subtypes it can deal with by creating a list of property tables:
mpg123: audio/mp3
layer: 1 - 3
bitrate: 8 - 320
osssink:
format: S8, S16, etc.
depth: 8 - 16
rate: 8000 - 48000
channels: 1 - 2
interleave: true
And you could list several of these, so for instance if the card only supports 8-bit at up to 22KHz in
mono, you can remove S8 from the above list and add a second entry:
osssink:
format: S8
depth: 8
rate: 8000 - 22050
channels: 1
interleave: false (irrelevant)
The obvious problem with these examples is that the rate isn't really 8000 - 48000, it's 8000, 11025,
16000, 22050, 44100, and 48000. However, we may be able to leave these to pad connect time.
struct _type_definition {
char *mime_type;
....
GData *properties;
}
gst_type_add_property_int(_type *type,gchar *propname,int min,int max) {
struct _type_prop_int prop_int;
GQuark quark = g_quark_from_string(propname);
prop_int->id = quark;
prop_int->min = min;
prop_int->max = max;
g_datalist_id_set_data(type->properties,quark,&prop_int);
}

269
docs/random/phonon-gst
View file

@ -1,269 +0,0 @@
Phonon backend
--------------
The phonon design is based around forming graphs using 3 basic components:
- a source component that generates raw audio/video/subtitle data, aka
MediaObject.
- an effect component that applies effects to raw audio/video known as
AudioPath/VideoPath respectively. Subtitles are routed to a VideoPath
- output components that render audio or video called AudioOutput and
VideoOutput.
there is also a special input object that allows for feeding raw data in the
pipeline and specialized sinks to retrieve audio samples and video frames from
the pipeline.
A typical graph or a source that produces an audio and a video stream that
need to be played. The VideoPath and AudioPath typically contain no filters
in this case:
+----+ +---
| FX | | ...
+----+ +---
V V
+-----------+ +-------------+
----->| VideoPath |----->| VideoOutput |
| +-----------+ +-------------+
+-------------+
| MediaObject |
+-------------+
| +-----------+ +-------------+
+---->| AudioPath |----->| AudioOutput |
+-----------+ +-------------+
^ ^
+----+ +---
| FX | | ...
+-+--+ +---
- This is very similar to a regular gstreamer playback pipeline.
A typical graph of playing and crosfading two sources:
+--------+
| volume |
+--------+
V
+-------------+ +-----------+
| MediaObject |--->| AudioPath |\
+-------------+ +-----------+ \ +-------------+
---->| AudioOutput |
+-------------+ +-----------+ / +-------------+
| MediaObject |--->| AudioPath |/
+-------------+ +-----------+
^
+--------+
| volume |
+--------+
- As soon as two audio paths are connected to one sink, the input signals are
mixed before sending them to the sink. The mixing is typically done in the
audio sink by an element such as adder.
Other types of graphs are possible too:
+-----------+
/| AudioPath |\
+-------------+ / +-----------+ \ +-------------+
| MediaObject |-- ---->| AudioOutput |
+-------------+ \ +-----------+ / +-------------+
\| AudioPath |/
+-----------+
- This graph sends the same out data to 2 effect filter graphs and then mixes
it to an audio output. The splitting of the graph typically happens with a
tee element after the media object.
Questions
---------
1) do the following chains run
- synchronized with a shared clock?
+-------------+ +-----------+ +-------------+
| MediaObject |--->| AudioPath |--->| AudioOutput |
+-------------+ +-----------+ +-------------+
+-------------+ +-----------+ +-------------+
| MediaObject |--->| VideoPath |--->| VideoOutput |
+-------------+ +-----------+ +-------------+
- no API to set both MediaObjects atomically to play so it is assumed that
the playback starts and follows the rate of the global clock as soon as
the MediaObject is set to play. This makes unconnected chains run as if
they were in different GstPipelines.
2) Threading:
- Can signals be emitted from any thread?
- what operations are permitted from a signal handler?
3) Error reporting
- How does error reporting work?
* an audio/video device/port is busy.
* a fatal decoding error occurred.
* a media type is not supported
General
-------
- Setting up KDE and Phonon build environment
- Testing, identifying test applications, building test cases
- Asking questions to Phonon maintainers/designers
Essential classes
-----------------
These classes are essential to implement a backend and should be implemented
first.
Phonon::BackendCapabilities
Mostly exposes features in the registry like available decoders and effects.
Phonon::Factory
Entry point for the GStreamer backend. Provides methods to create instances of
object from our backed.
Simple playback
---------------
The following classes need to be implemented in order to have simple playback
capabilities from the backend.
Phonon::AudioOutput
- Wrapper around audiosinks. Also needs provision for rate and format
conversions.
- Mixing capabilities in the case when 2 audio paths are routed to it.
Notes:
* is the volume related to the device or to the connection to the device.
Phonon::VideoWidget
- Wrapper around videosinks. Also needs provision for colorspace and size
conversions. Extends QWidget and probably needs to hook into the XOverlay
stuff to draw in the QT widget. Supports fullscreen mode with a switch.
- Needs mixing capabilities in the case when 2 video paths are routed to it.
Phonon::AbstractMediaProducer
- contains stream selection
- play/pause/stop
- seeking
- periodically performs tick callbacks.
Phonon::MediaObject:
- The object that decodes the media into raw audio/video/subtitle.
This object will use the GStreamer decodebin element to perform the
typefinding and decoding.
Phonon::AudioPath/Phonon::VideoPath
- Simple container for audio/video effect plugins.
- Handles adding/removing of effects, making sure that the streaming is not
interrupted and the formats are all compatible.
Effect support
--------------
Phonon::Visualization
Connects an AudioPath to a VideoWidget and allows for selection of a
visualisation plugin.
Phonon::AudioEffect/Phonon::VideoEffect
Base classes
Phonon::VolumeFaderEffect
Allows fade-in and fade-out with a configurable curve and time. Needs
GstController.
Phonon::BrightnessControl
Controls the brightness of video.
Playlist support
----------------
Phonon::MediaQueue:
?? don't know yet where this fits in.
Capture
-------
Phonon::AvCapture
Synchronized audio and video capture.
Phonon::AudioWriter
Compress audio.
Advanced features
-----------------
Phonon::ByteStream
Feed raw data into the pipeline. Used for streaming network access.
Implementation:
Possibly a specialized source element connected to a decodebin.
Notes:
* Phonon::ByteStream::writeData
- can it block?
* Phonon::ByteStream::setStreamSeekable
- If called before starting the ByteStream, decodebin might operate in pull
based mode when supported. Else the source is activated in push mode.
- If called after starting ByteStream, the Phonon::ByteStream::seekStream
signal can be called for push-based seekable streams.
* Can the signals be emitted from a streaming thread?
Phonon::AudioDataOutput/Phonon::VideoDataOutput/
Receive raw audio/video data from the pipeline. Used to allow applications to
deal with the raw data themselves.
Implementation:
Possibly a specialized sink element.
Notes :
* Phonon::AudioDataOutput::dataReady
- can this be emitted from the streaming threads?
* Phonon::AudioDataOutput::endOfMedia
- can this be emitted from the streaming threads?
- We need to grab this EOS message synchronously from the bus.
- should be emitted _before_ sending the last dataReady. This means we need
to cache at least one dataReady.
* Phonon::AudioDataOutput::setDataSize
- can this be a _suggested_ data size or does every callback need to be of
this size?

93
docs/random/plugins
View file

@ -1,93 +0,0 @@
SOMEWHAT OUTDATED
-----------------
We describe how the plugin system works.
Plugins
-------
plugins are basically shared libraries with a plugin_init
function.
they provide the GStreamer core library with the following
information:
- element factories
- type factories
- metadata factories?
ElementFactory
--------------
element factories provide the core library with elements (duh)
an element factory has the following information:
- a unique name for the element factory
- strings describing the element (name, description, copyright,...)
- a description of the media types it accepts (as capabilities)
- a description of the media types it outputs (as capabilities)
TypeFactory
-----------
has the following properties:
- a mime type
- file extensions that may provide a hint for this type
- a function to detect this type
- a string to detect this type (file(1) like)
XML registry
------------
The complete plugin tree will be exported into an XML description so
that the definitions of the factories can be obtained without having
to load and plugin_init the plugins.
Loading of plugins
------------------
at some point, the plugin will need to be read into memory before
any elements it provides can be used. the XML description of the
loaded plugin will need to be updated.
We will have the following methods for (implicitly) loading plugins:
gst_plugin_load_all()
remove the complete XML tree and read all the plugins in the
paths. The plugin_init method will be called and the XML tree
will be rebuild in memory.
gst_plugin_load (name)
The plugin will be located in the tree and if it already loaded, the
function returns. If it is not loaded, the XML entry is removed and
the plugin is loaded. The plugin_init is called so that the XML tree
is reconstructed.
gst_elementfactory_create (factory, name);
The plugin providing the element will be located, if the plugin is
already loaded, an element with the given name is created.
if the plugin is not loaded, gst_plugin_load is called.
the loaded factory is located again and the element is created.
The typefind function is called
When the plugin description is read from the XML file, the typefind
function is hooked up to a dummy typefind function. The dummy typefind
function will locate the plugin it belongs to and call gst_plugin_load.
after the loading of the plugin, the real typefind function is called.

941
docs/random/plugins.dia
View file

@ -1,941 +0,0 @@
<?xml version="1.0"?>
<diagram xmlns:dia="http://www.lysator.liu.se/~alla/dia/">
<diagramdata>
<attribute name="background">
<color val="#ffffff"/>
</attribute>
<attribute name="paper">
<composite type="paper">
<attribute name="name">
<string>#A4#</string>
</attribute>
<attribute name="tmargin">
<real val="2.82"/>
</attribute>
<attribute name="bmargin">
<real val="2.82"/>
</attribute>
<attribute name="lmargin">
<real val="2.82"/>
</attribute>
<attribute name="rmargin">
<real val="2.82"/>
</attribute>
<attribute name="is_portrait">
<boolean val="true"/>
</attribute>
<attribute name="scaling">
<real val="1"/>
</attribute>
<attribute name="fitto">
<boolean val="false"/>
</attribute>
</composite>
</attribute>
<attribute name="grid">
<composite type="grid">
<attribute name="width_x">
<real val="1"/>
</attribute>
<attribute name="width_y">
<real val="1"/>
</attribute>
<attribute name="visible_x">
<int val="1"/>
</attribute>
<attribute name="visible_y">
<int val="1"/>
</attribute>
</composite>
</attribute>
<attribute name="guides">
<composite type="guides">
<attribute name="hguides"/>
<attribute name="vguides"/>
</composite>
</attribute>
</diagramdata>
<layer name="Background" visible="true">
<object type="UML - Class" version="0" id="O0">
<attribute name="obj_pos">
<point val="0.85,4.15"/>
</attribute>
<attribute name="obj_bb">
<rectangle val="0.8,4.1;12.3656,11.4"/>
</attribute>
<attribute name="elem_corner">
<point val="0.85,4.15"/>
</attribute>
<attribute name="elem_width">
<real val="11.4656"/>
</attribute>
<attribute name="elem_height">
<real val="7.2"/>
</attribute>
<attribute name="name">
<string>#GstPlugin#</string>
</attribute>
<attribute name="stereotype">
<string/>
</attribute>
<attribute name="abstract">
<boolean val="false"/>
</attribute>
<attribute name="suppress_attributes">
<boolean val="false"/>
</attribute>
<attribute name="suppress_operations">
<boolean val="false"/>
</attribute>
<attribute name="visible_attributes">
<boolean val="true"/>
</attribute>
<attribute name="visible_operations">
<boolean val="false"/>
</attribute>
<attribute name="attributes">
<composite type="umlattribute">
<attribute name="name">
<string>#name#</string>
</attribute>
<attribute name="type">
<string>#gchar *#</string>
</attribute>
<attribute name="value">
<string/>
</attribute>
<attribute name="visibility">
<enum val="0"/>
</attribute>
<attribute name="abstract">
<boolean val="false"/>
</attribute>
<attribute name="class_scope">
<boolean val="false"/>
</attribute>
</composite>
<composite type="umlattribute">
<attribute name="name">
<string>#longname#</string>
</attribute>
<attribute name="type">
<string>#char *#</string>
</attribute>
<attribute name="value">
<string/>
</attribute>
<attribute name="visibility">
<enum val="0"/>
</attribute>
<attribute name="abstract">
<boolean val="false"/>
</attribute>
<attribute name="class_scope">
<boolean val="false"/>
</attribute>
</composite>
<composite type="umlattribute">
<attribute name="name">
<string>#filename#</string>
</attribute>
<attribute name="type">
<string>#gchar *#</string>
</attribute>
<attribute name="value">
<string/>
</attribute>
<attribute name="visibility">
<enum val="0"/>
</attribute>
<attribute name="abstract">
<boolean val="false"/>
</attribute>
<attribute name="class_scope">
<boolean val="false"/>
</attribute>
</composite>
<composite type="umlattribute">
<attribute name="name">
<string>#loaded#</string>
</attribute>
<attribute name="type">
<string>#gboolean#</string>
</attribute>
<attribute name="value">
<string/>
</attribute>
<attribute name="visibility">
<enum val="0"/>
</attribute>
<attribute name="abstract">
<boolean val="false"/>
</attribute>
<attribute name="class_scope">
<boolean val="false"/>
</attribute>
</composite>
<composite type="umlattribute">
<attribute name="name">
<string>#elements#</string>
</attribute>
<attribute name="type">
<string>#GList *#</string>
</attribute>
<attribute name="value">
<string/>
</attribute>
<attribute name="visibility">
<enum val="0"/>
</attribute>
<attribute name="abstract">
<boolean val="false"/>
</attribute>
<attribute name="class_scope">
<boolean val="false"/>
</attribute>
</composite>
<composite type="umlattribute">
<attribute name="name">
<string>#types#</string>
</attribute>
<attribute name="type">
<string>#GList *#</string>
</attribute>
<attribute name="value">
<string/>
</attribute>
<attribute name="visibility">
<enum val="0"/>
</attribute>
<attribute name="abstract">
<boolean val="false"/>
</attribute>
<attribute name="class_scope">
<boolean val="false"/>
</attribute>
</composite>
<composite type="umlattribute">
<attribute name="name">
<string>#autopluggers#</string>
</attribute>
<attribute name="type">
<string>#GList *#</string>
</attribute>
<attribute name="value">
<string/>
</attribute>
<attribute name="visibility">
<enum val="0"/>
</attribute>
<attribute name="abstract">
<boolean val="false"/>
</attribute>
<attribute name="class_scope">
<boolean val="false"/>
</attribute>
</composite>
</attribute>
<attribute name="operations"/>
<attribute name="template">
<boolean val="false"/>
</attribute>
<attribute name="templates"/>
</object>
<object type="UML - Class" version="0" id="O1">
<attribute name="obj_pos">
<point val="17,9"/>
</attribute>
<attribute name="obj_bb">
<rectangle val="16.95,8.95;31.9092,13.85"/>
</attribute>
<attribute name="elem_corner">
<point val="17,9"/>
</attribute>
<attribute name="elem_width">
<real val="14.8592"/>
</attribute>
<attribute name="elem_height">
<real val="4.8"/>
</attribute>
<attribute name="name">
<string>#GstElementFactory#</string>
</attribute>
<attribute name="stereotype">
<string/>
</attribute>
<attribute name="abstract">
<boolean val="false"/>
</attribute>
<attribute name="suppress_attributes">
<boolean val="false"/>
</attribute>
<attribute name="suppress_operations">
<boolean val="false"/>
</attribute>
<attribute name="visible_attributes">
<boolean val="true"/>
</attribute>
<attribute name="visible_operations">
<boolean val="false"/>
</attribute>
<attribute name="attributes">
<composite type="umlattribute">
<attribute name="name">
<string>#name#</string>
</attribute>
<attribute name="type">
<string>#gchar *#</string>
</attribute>
<attribute name="value">
<string/>
</attribute>
<attribute name="visibility">
<enum val="0"/>
</attribute>
<attribute name="abstract">
<boolean val="false"/>
</attribute>
<attribute name="class_scope">
<boolean val="false"/>
</attribute>
</composite>
<composite type="umlattribute">
<attribute name="name">
<string>#type#</string>
</attribute>
<attribute name="type">
<string>#GtkType#</string>
</attribute>
<attribute name="value">
<string/>
</attribute>
<attribute name="visibility">
<enum val="0"/>
</attribute>
<attribute name="abstract">
<boolean val="false"/>
</attribute>
<attribute name="class_scope">
<boolean val="false"/>
</attribute>
</composite>
<composite type="umlattribute">
<attribute name="name">
<string>#details#</string>
</attribute>
<attribute name="type">
<string>#GstElementDetails *#</string>
</attribute>
<attribute name="value">
<string/>
</attribute>
<attribute name="visibility">
<enum val="0"/>
</attribute>
<attribute name="abstract">
<boolean val="false"/>
</attribute>
<attribute name="class_scope">
<boolean val="false"/>
</attribute>
</composite>
<composite type="umlattribute">
<attribute name="name">
<string>#padtemplates#</string>
</attribute>
<attribute name="type">
<string>#GList *#</string>
</attribute>
<attribute name="value">
<string/>
</attribute>
<attribute name="visibility">
<enum val="0"/>
</attribute>
<attribute name="abstract">
<boolean val="false"/>
</attribute>
<attribute name="class_scope">
<boolean val="false"/>
</attribute>
</composite>
</attribute>
<attribute name="operations"/>
<attribute name="template">
<boolean val="false"/>
</attribute>
<attribute name="templates"/>
</object>
<object type="UML - Association" version="0" id="O2">
<attribute name="obj_pos">
<point val="12.3156,9.25"/>
</attribute>
<attribute name="obj_bb">
<rectangle val="10.8656,7.8;18.45,11.15"/>
</attribute>
<attribute name="orth_points">
<point val="12.3156,9.25"/>
<point val="15,9.25"/>
<point val="15,9.7"/>
<point val="17,9.7"/>
</attribute>
<attribute name="orth_orient">
<enum val="0"/>
<enum val="1"/>
<enum val="0"/>
</attribute>
<attribute name="name">
<string/>
</attribute>
<attribute name="direction">
<enum val="0"/>
</attribute>
<attribute name="ends">
<composite>
<attribute name="role">
<string/>
</attribute>
<attribute name="multiplicity">
<string>#1#</string>
</attribute>
<attribute name="arrow">
<boolean val="false"/>
</attribute>
<attribute name="aggregate">
<enum val="2"/>
</attribute>
</composite>
<composite>
<attribute name="role">
<string/>
</attribute>
<attribute name="multiplicity">
<string>#*#</string>
</attribute>
<attribute name="arrow">
<boolean val="false"/>
</attribute>
<attribute name="aggregate">
<enum val="0"/>
</attribute>
</composite>
</attribute>
<connections>
<connection handle="0" to="O0" connection="17"/>
<connection handle="1" to="O1" connection="3"/>
</connections>
</object>
<object type="UML - Class" version="0" id="O3">
<attribute name="obj_pos">
<point val="21,2"/>
</attribute>
<attribute name="obj_bb">
<rectangle val="20.95,1.95;34.9396,6.85"/>
</attribute>
<attribute name="elem_corner">
<point val="21,2"/>
</attribute>
<attribute name="elem_width">
<real val="13.8896"/>
</attribute>
<attribute name="elem_height">
<real val="4.8"/>
</attribute>
<attribute name="name">
<string>#GstPadTemplate#</string>
</attribute>
<attribute name="stereotype">
<string/>
</attribute>
<attribute name="abstract">
<boolean val="false"/>
</attribute>
<attribute name="suppress_attributes">
<boolean val="false"/>
</attribute>
<attribute name="suppress_operations">
<boolean val="false"/>
</attribute>
<attribute name="visible_attributes">
<boolean val="true"/>
</attribute>
<attribute name="visible_operations">
<boolean val="false"/>
</attribute>
<attribute name="attributes">
<composite type="umlattribute">
<attribute name="name">
<string>#name_template#</string>
</attribute>
<attribute name="type">
<string>#gchar *#</string>
</attribute>
<attribute name="value">
<string/>
</attribute>
<attribute name="visibility">
<enum val="0"/>
</attribute>
<attribute name="abstract">
<boolean val="false"/>
</attribute>
<attribute name="class_scope">
<boolean val="false"/>
</attribute>
</composite>
<composite type="umlattribute">
<attribute name="name">
<string>#direction#</string>
</attribute>
<attribute name="type">
<string>#GstPadDirection#</string>
</attribute>
<attribute name="value">
<string/>
</attribute>
<attribute name="visibility">
<enum val="0"/>
</attribute>
<attribute name="abstract">
<boolean val="false"/>
</attribute>
<attribute name="class_scope">
<boolean val="false"/>
</attribute>
</composite>
<composite type="umlattribute">
<attribute name="name">
<string>#presence#</string>
</attribute>
<attribute name="type">
<string>#GstPadPresence#</string>
</attribute>
<attribute name="value">
<string/>
</attribute>
<attribute name="visibility">
<enum val="0"/>
</attribute>
<attribute name="abstract">
<boolean val="false"/>
</attribute>
<attribute name="class_scope">
<boolean val="false"/>
</attribute>
</composite>
<composite type="umlattribute">
<attribute name="name">
<string>#caps#</string>
</attribute>
<attribute name="type">
<string>#GstCaps *#</string>
</attribute>
<attribute name="value">
<string/>
</attribute>
<attribute name="visibility">
<enum val="0"/>
</attribute>
<attribute name="abstract">
<boolean val="false"/>
</attribute>
<attribute name="class_scope">
<boolean val="false"/>
</attribute>
</composite>
</attribute>
<attribute name="operations"/>
<attribute name="template">
<boolean val="false"/>
</attribute>
<attribute name="templates"/>
</object>
<object type="UML - Association" version="0" id="O4">
<attribute name="obj_pos">
<point val="31.8592,13.3"/>
</attribute>
<attribute name="obj_bb">
<rectangle val="30.4092,1.25;38.45,14.75"/>
</attribute>
<attribute name="orth_points">
<point val="31.8592,13.3"/>
<point val="37,13.3"/>
<point val="37,2.7"/>
<point val="34.8896,2.7"/>
</attribute>
<attribute name="orth_orient">
<enum val="0"/>
<enum val="1"/>
<enum val="0"/>
</attribute>
<attribute name="name">
<string/>
</attribute>
<attribute name="direction">
<enum val="0"/>
</attribute>
<attribute name="ends">
<composite>
<attribute name="role">
<string/>
</attribute>
<attribute name="multiplicity">
<string>#1#</string>
</attribute>
<attribute name="arrow">
<boolean val="false"/>
</attribute>
<attribute name="aggregate">
<enum val="2"/>
</attribute>
</composite>
<composite>
<attribute name="role">
<string/>
</attribute>
<attribute name="multiplicity">
<string>#*#</string>
</attribute>
<attribute name="arrow">
<boolean val="false"/>
</attribute>
<attribute name="aggregate">
<enum val="0"/>
</attribute>
</composite>
</attribute>
<connections>
<connection handle="0" to="O1" connection="15"/>
<connection handle="1" to="O3" connection="4"/>
</connections>
</object>
<object type="UML - Class" version="0" id="O5">
<attribute name="obj_pos">
<point val="17.1,14.85"/>
</attribute>
<attribute name="obj_bb">
<rectangle val="17.05,14.8;32.494,18.9"/>
</attribute>
<attribute name="elem_corner">
<point val="17.1,14.85"/>
</attribute>
<attribute name="elem_width">
<real val="15.344"/>
</attribute>
<attribute name="elem_height">
<real val="4"/>
</attribute>
<attribute name="name">
<string>#GstTypeFactory#</string>
</attribute>
<attribute name="stereotype">
<string/>
</attribute>
<attribute name="abstract">
<boolean val="false"/>
</attribute>
<attribute name="suppress_attributes">
<boolean val="false"/>
</attribute>
<attribute name="suppress_operations">
<boolean val="false"/>
</attribute>
<attribute name="visible_attributes">
<boolean val="true"/>
</attribute>
<attribute name="visible_operations">
<boolean val="false"/>
</attribute>
<attribute name="attributes">
<composite type="umlattribute">
<attribute name="name">
<string>#mime#</string>
</attribute>
<attribute name="type">
<string>#gchar *#</string>
</attribute>
<attribute name="value">
<string/>
</attribute>
<attribute name="visibility">
<enum val="0"/>
</attribute>
<attribute name="abstract">
<boolean val="false"/>
</attribute>
<attribute name="class_scope">
<boolean val="false"/>
</attribute>
</composite>
<composite type="umlattribute">
<attribute name="name">
<string>#exts#</string>
</attribute>
<attribute name="type">
<string>#gchar *#</string>
</attribute>
<attribute name="value">
<string/>
</attribute>
<attribute name="visibility">
<enum val="0"/>
</attribute>
<attribute name="abstract">
<boolean val="false"/>
</attribute>
<attribute name="class_scope">
<boolean val="false"/>
</attribute>
</composite>
<composite type="umlattribute">
<attribute name="name">
<string>#typefindfunc#</string>
</attribute>
<attribute name="type">
<string>#GstTypeFindFunc#</string>
</attribute>
<attribute name="value">
<string/>
</attribute>
<attribute name="visibility">
<enum val="0"/>
</attribute>
<attribute name="abstract">
<boolean val="false"/>
</attribute>
<attribute name="class_scope">
<boolean val="false"/>
</attribute>
</composite>
</attribute>
<attribute name="operations"/>
<attribute name="template">
<boolean val="false"/>
</attribute>
<attribute name="templates"/>
</object>
<object type="UML - Association" version="0" id="O6">
<attribute name="obj_pos">
<point val="12.3156,10.05"/>
</attribute>
<attribute name="obj_bb">
<rectangle val="10.8656,8.6;18.55,17"/>
</attribute>
<attribute name="orth_points">
<point val="12.3156,10.05"/>
<point val="15.7,10.05"/>
<point val="15.7,15.55"/>
<point val="17.1,15.55"/>
</attribute>
<attribute name="orth_orient">
<enum val="0"/>
<enum val="1"/>
<enum val="0"/>
</attribute>
<attribute name="name">
<string/>
</attribute>
<attribute name="direction">
<enum val="0"/>
</attribute>
<attribute name="ends">
<composite>
<attribute name="role">
<string/>
</attribute>
<attribute name="multiplicity">
<string>#1#</string>
</attribute>
<attribute name="arrow">
<boolean val="false"/>
</attribute>
<attribute name="aggregate">
<enum val="2"/>
</attribute>
</composite>
<composite>
<attribute name="role">
<string/>
</attribute>
<attribute name="multiplicity">
<string>#*#</string>
</attribute>
<attribute name="arrow">
<boolean val="false"/>
</attribute>
<attribute name="aggregate">
<enum val="0"/>
</attribute>
</composite>
</attribute>
<connections>
<connection handle="0" to="O0" connection="19"/>
<connection handle="1" to="O5" connection="3"/>
</connections>
</object>
<object type="UML - Class" version="0" id="O7">
<attribute name="obj_pos">
<point val="17.065,19.975"/>
</attribute>
<attribute name="obj_bb">
<rectangle val="17.015,19.925;27.5201,24.025"/>
</attribute>
<attribute name="elem_corner">
<point val="17.065,19.975"/>
</attribute>
<attribute name="elem_width">
<real val="10.4051"/>
</attribute>
<attribute name="elem_height">
<real val="4"/>
</attribute>
<attribute name="name">
<string>#GstAutoplugFactory#</string>
</attribute>
<attribute name="stereotype">
<string/>
</attribute>
<attribute name="abstract">
<boolean val="false"/>
</attribute>
<attribute name="suppress_attributes">
<boolean val="false"/>
</attribute>
<attribute name="suppress_operations">
<boolean val="false"/>
</attribute>
<attribute name="visible_attributes">
<boolean val="true"/>
</attribute>
<attribute name="visible_operations">
<boolean val="false"/>
</attribute>
<attribute name="attributes">
<composite type="umlattribute">
<attribute name="name">
<string>#name#</string>
</attribute>
<attribute name="type">
<string>#gchar *#</string>
</attribute>
<attribute name="value">
<string/>
</attribute>
<attribute name="visibility">
<enum val="0"/>
</attribute>
<attribute name="abstract">
<boolean val="false"/>
</attribute>
<attribute name="class_scope">
<boolean val="false"/>
</attribute>
</composite>
<composite type="umlattribute">
<attribute name="name">
<string>#longdesc#</string>
</attribute>
<attribute name="type">
<string>#gchar *#</string>
</attribute>
<attribute name="value">
<string/>
</attribute>
<attribute name="visibility">
<enum val="0"/>
</attribute>
<attribute name="abstract">
<boolean val="false"/>
</attribute>
<attribute name="class_scope">
<boolean val="false"/>
</attribute>
</composite>
<composite type="umlattribute">
<attribute name="name">
<string>#type#</string>
</attribute>
<attribute name="type">
<string>#GtkType#</string>
</attribute>
<attribute name="value">
<string/>
</attribute>
<attribute name="visibility">
<enum val="0"/>
</attribute>
<attribute name="abstract">
<boolean val="false"/>
</attribute>
<attribute name="class_scope">
<boolean val="false"/>
</attribute>
</composite>
</attribute>
<attribute name="operations"/>
<attribute name="template">
<boolean val="false"/>
</attribute>
<attribute name="templates"/>
</object>
<object type="UML - Association" version="0" id="O8">
<attribute name="obj_pos">
<point val="12.3156,10.85"/>
</attribute>
<attribute name="obj_bb">
<rectangle val="10.8656,9.4;18.515,22.125"/>
</attribute>
<attribute name="orth_points">
<point val="12.3156,10.85"/>
<point val="14.8,10.85"/>
<point val="14.8,20.675"/>
<point val="17.065,20.675"/>
</attribute>
<attribute name="orth_orient">
<enum val="0"/>
<enum val="1"/>
<enum val="0"/>
</attribute>
<attribute name="name">
<string/>
</attribute>
<attribute name="direction">
<enum val="0"/>
</attribute>
<attribute name="ends">
<composite>
<attribute name="role">
<string/>
</attribute>
<attribute name="multiplicity">
<string>#1#</string>
</attribute>
<attribute name="arrow">
<boolean val="false"/>
</attribute>
<attribute name="aggregate">
<enum val="2"/>
</attribute>
</composite>
<composite>
<attribute name="role">
<string/>
</attribute>
<attribute name="multiplicity">
<string>#*#</string>
</attribute>
<attribute name="arrow">
<boolean val="false"/>
</attribute>
<attribute name="aggregate">
<enum val="0"/>
</attribute>
</composite>
</attribute>
<connections>
<connection handle="0" to="O0" connection="21"/>
<connection handle="1" to="O7" connection="3"/>
</connections>
</object>
</layer>
</diagram>

5
docs/random/porting-to-0.11.txt
View file

@ -1,5 +0,0 @@
This guide has been renamed to porting-to-1.0.txt.
The latest version can be found at:
http://cgit.freedesktop.org/gstreamer/gstreamer/tree/docs/random/porting-to-1.0.txt

38
docs/random/queue
View file

@ -1,38 +0,0 @@
thread 1 thread2
// the queue is empty
while (!queue->level_buffers) {
STATUS("queue: %s U released lock\n");
GST_OBJECT_UNLOCK (queue);
// thread1 is scheduled and puts a lot of buffers
// in the queue
// thread1 has put the last buffer on the queue
// here. A signal is going to be emitted
tosignal = (queue->level_buffers >= 0);
queue->level_buffers++;
/* we can unlock now */
GST_OBJECT_UNLOCK (queue);
if (tosignal) {
g_mutex_lock (queue->emptylock);
g_cond_signal (queue->emptycond);
g_mutex_unlock (queue->emptylock);
}
g_mutex_lock (queue->emptylock);
// wait forever
g_cond_wait (queue->emptycond, queue->emptylock);
g_mutex_unlock (queue->emptylock);
GST_OBJECT_LOCK (queue);
}
// thread 1 will also wait forever because the
// queue is filled....

3
docs/random/richardb/syncmail
View file

@ -1,3 +0,0 @@
The syncmail script in CVSROOT is a modified version of the script
taken from the sourceforge snippets page, at:
http://sourceforge.net/snippet/detail.php?type=snippet&id=100414

44
docs/random/rtp
View file

@ -1,44 +0,0 @@
RTP client subsystem proposal.
Ramon Garcia Fernandez
A brief description of the RTP protocol.
The RTP protocol uses two connections: one for passing data and another for control. The control connection is used for starting, finishing, and passing statistics of lost packets. On the data connection, packets are transmitted containing the media. These packets have a 7 bit field that says what codec is the data transmitted with. This codec is variable. However data cannot change from audio to video in packets of the same connection. All the packets belong to the same logical stream, that is, for instanace, to the same speech, or to the same song. Different codecs in the same stream is used, for instance, to insert comfort noise.
Each codec is packeted in a specific way in RTP packets. This is necessary to minimize the damage produced by lost packets. Therefore, packets should be arranged so that each packet can be decoded independently. If a packet is lost, that shouldn't preclude the decoding of following packets.
Suggested implementation:
The implementation that I suggest contains an rtpdec element. This element has one sink pad for the data, one src pad for the decoded data, and a pair of src and sink pads for control messages. The decoded data that comes from rtpdec has no RTP dependency. It is in the format expected by the codec that it decodes it. Therefore, rtpdec must do codec specific processing of data to take into account that different codecs are stored differently in RTP.
A possible pipeline is:
udpcon name="data" udpcon name="control" data.src!rtpdec.datasrc control.src!rtpdec.controlin control.sink!rtpdec.controlout rtpdec name="rtpdec" ! mp3decode ! osssink
udpcon is an element (not yet written) that provides a src and a sink pad to read and write to a socket. There are two udpcon, one for reading the data socket and another for reading and writing to the control socket.
To handle variable types, the following would work (the UDP part left out for clarity is identical to the one above):
[udp] rtpdec ! spider ! osssink
rtpdec has a property that tells it the mapping of the payload bits to mime type. This is necessary, because that mapping is not fixed. It has to be obtained at runtime through other mechanisms, such as a SDP file obtained by http or rtsp.
A possible implementation of rtpdec is a bin that contains two elements: a media independent _rtpparse and a media dependent rtp-dec-mp3. Thus the pipeline would be
udpsrc ! _rtpparse ! rtp-dec-mp3 ! mp3decode ! osssink
\________________________/
rtpdec bin
Another possible implementation is, that rtpdec opens a plugin that contains the code to convert the codec specific packets into the input that the normal decoder for that codec expects.
Ronald said that this mechanism would be complicated because it would require to duplicate the functionality of the plugin loader. It shouldn't be like that because it should use the normal plugin loader. However, this is an issue that I do not fully understand.
Ronald suggested to use inheritance. Thus the user should insert a codec specific rtp element in the pipeline. Such as:
udpsrc ! rtp-mp3 ! mp3decode ! osssink
Reuse of RTP logic would be achieved through inheritance.
This looks more logical, because inheritance reflects the fact that rtp-mp3 "is an" specialization of rtp. However, there are several issues. As stated above, it is possible in RTP to switch the encoding of the media at any time. If this happens, some state must be kept, such as statistics of packets received and sent.

12
docs/random/signal
View file

@ -1,12 +0,0 @@
Random notes on signals:
* Use a BOXED (and not a POINTER) marshaller when your signal will have a
GstBuffer argument:
g_signal_new ("handoff", G_TYPE_FROM_CLASS (klass), G_SIGNAL_RUN_LAST,
G_STRUCT_OFFSET (GstIdentityClass, handoff), NULL, NULL,
gst_marshal_VOID__BOXED, G_TYPE_NONE, 1,
GST_TYPE_BUFFER | G_SIGNAL_TYPE_STATIC_SCOPE);
* For GstBuffers arguments, consider using G_SIGNAL_TYPE_STATIC_SCOPE as it
can prevent an extra copy of the buffer

324
docs/random/sources
View file

@ -1,324 +0,0 @@
There are a number of different ways of coding a GstSrc. I'll try to
outline them and how the function here:
1a) Simple push-function based with single output
*------* *------
! ! !
! src !--->--! plugin
! ! !
*------* *------
This is the style that all the existing sources use. There is a single
output pad, and a _push function that's global to the whole element. The
_push function simply constructs buffers and pushes them out the pad.
Chained (current implementation):
bin src pad1 pad2 plugin
! (= pad1->peer)
gst_bin_iterate
!
! (find entry)
!
! gst_src_push
!---------------->!
! (create buffer)
!
! gst_pad_push
!---------------->!
!
! pad1->chainfunc (pad1->peer)
!------------------------------->!
! !
: :
(more chaining)
: :
!<-------------------------------!
!<----------------!
!<-----------------!
!
iteration ends
!
---
-
Typically this will be the/an entry into the Bin. The Bin's iterate
function simply calls the Src's _push function. When the _push function
pushes a buffer out it's pad, the chain function of the peer pad is
called, presumably causing a push out the other side of that element, and
eventually data gets to the other end. The stack unrolls, and the
iteration ends for that Src.
Cothreaded:
Again, the source would generally be an entry into the Bin. A loopfunc
will be constructed around it, which will simply loop calling the Src's
_push function as in the non-cothreaded case. When the _push function
pushes a buffer, it finds a pushfunc attached to the pad, drops the buffer
in the pen, and calls the pushfunc provided by the Bin. This causes a
switch to the next element, then the next, to the end, at which point a
buffer pull will travel back down the chain. The _push function gets
context and finishes, at which point the loopfunc wrapper simply calls it
again in the next iteration.
(current implementation):
bin cothread1 src pad1 pad2 cothread2 plugin
! (src) (= pad2->peer) (plugin)
gst_bin_iterate
!
! (first entry)
!
! cothread_switch
!---------------->!
! gst_src_push
!---------------->!
! (create buffer)
!
! gst_pad_push (pad1)
!
!--------------------!
! (fill bufpen)
!
! cothread switch
----------------------->!
! gst_pad_pull (pad2)
!
!<----------!
!
! (get buffer from bufpen)
!
!---------->!
! pad2->chainfunc
!------------->!
!
:
:
!<-------------!
! gst_pad_pull (pad2)
!<----------!
!
! (bufpen empty)
!<----------!
!<-------------------------------------! cothread switch
!<----------------!
!
iteration ends
!
:
:
!
next iteration
!
! cothread_switch
!---------------->!
! gst_src_push
!---------------->!
! (create buffer)
!
! gst_pad_push (pad1)
!
!--------------------!
! (fill bufpen)
!
! cothread switch
!---------->!
! (get buffer from bufpen)
!
!---------->!
! pad2->chainfunc
!------------->!
!
:
:
!<-------------!
! gst_pad_pull (pad2)
!<----------!
!
! (bufpen empty)
!<----------!
!<-------------------------------------! cothread switch
!<----------------!
!
iteration ends
!
:
-----------------------------------------------------------------------------------------------
1b) Simple push-function based with multiple output
Chained:
Similar to the single output variant, except several chains are spawned
off, one per push, hanging off whichever pad the buffer is pushed off of.
The stack will grow and unwind as many times as buffers are pushed out.
(current implementation)
bin src pad1 pad2 plugin
! (= pad1->peer)
gst_bin_iterate
!
! (find entry)
!
! gst_src_push
!---------------->!
! (create buffer)
!
! gst_pad_push
!---------------->!
!
! pad1->chainfunc (pad1->peer)
!------------------------------->!
! !
: :
(more chaining)
: :
!<-------------------------------!
!<----------------!
! (create buffer)
!
! gst_pad_push
!---------------->!
!
! pad1->chainfunc (pad1->peer)
!------------------------------->!
! !
: :
(more chaining)
: :
!<-------------------------------!
!<----------------!
:
(more pushes)
:
!<-----------------!
!
iteration ends
!
---
-
Cothreaded:
Also similar to the single output variant. When the pull winds its way
back from the first push, execution returns to the Src's _push function,
which simply goes off and pushes out another buffer, causing another
series of context switches. Eventually the loopfunc wrapper starts over,
round and round we go.
-----------------------------------------------------------------------------------------------
2) Pull-function based with single output
Similar to a regular filter with a chain function associated with each
pad, this kind of source doesn't provide a src-wide push function, but
does provide pullfuncs for its pad. A pullfunc puts a buffer in the pen
and exits.
Chained:
bin src pad1 pad2 plugin
! (= pad1->peer)
gst_bin_iterate
!
! (find entry)
!
! (find src pad
! of entry element)
!
! gst_pad_pull
!------------------------------------------------>!
? !
!<------------------------------!
!
! (create buffer)
!
! gst_pad_push
!------------------------------>!
! (bufpen filled)
(return buffer) !
!<------------------------------------------------!
!
! gst_pad_chain
!------------------------------------------------------------------->!
!
:
(more chaining)
:
!<-------------------------------------------------------------------!
!
iteration ends
!
---
-
As usual, is likely to be an entry into a Bin. The Bin iterate code must
explicitly pull a buffer and pass it on to the peer.
Cothreaded:
bin cothread1 src pad1 pad2 cothread2 plugin
! (src) (= pad2->peer) (plugin)
gst_bin_iterate
!
! (first entry)
!
! cothread_switch
!---------------->! gst_pad_pull
!------------------------------------------------>!
? !
!<------------------------------!
!
! (create buffer)
!
! gst_pad_push
!------------------------------>!
! (bufpen filled)
(return buffer) !
!<------------------------------------------------!
!
! pad_chain
!--------------------------------------! cothread switch
|---------------------->!
! gst_pad_pull (pad2)
!
!<----------!
!
! (get buffer from bufpen)
!
!---------->!
! pad2->chainfunc
!------------->!
!
:
:
!<-------------!
! gst_pad_pull (pad2)
!<----------!
!
! (bufpen empty)
!<-----------!
! cothread switch
!-------------------!
!<----------------!
!<----------------!
!
iteration ends
!
:

192
docs/random/status-0.11-14-jun-2011.txt
View file

@ -1,192 +0,0 @@
0.11 status 1 jun 2011
----------------------
Hello again GStreamer hackers,
Here's another status update from the 0.11 branch. This one is packed
with new stuff because it really took too long between the last
update, but well..
Misc Core Changes
-----------------
The first set of big core change was done by über hacker Sebastian Dröge.
He removed our special GST_BOILERPLATE macro. Because of some API strangeness
we needed to do actions in the base_init method, which the GST_BOILERPLATE
macro set up for us. The base_init concept is unfortunately not supported
by almost all language bindings so we wanted to clean this up and use the
regular G_DEFINE_TYPE macros. By inheriting metadata and pad templates
from the parent, the GST_BOILERPLATE macro could be removed.
Removal of gst_pad_alloc_buffer
-------------------------------
The next big change was the removal of gst_pad_alloc_buffer along with
all the pad_alloc functions. The pad_alloc function were used to both
implement allocation of buffers and notify upstream elements of new media
formats. The functionality is replace by 3 new things: GstBufferPool (as
noted in the previous status update), a new ALLOCATION query and a new
RECONFIGURE event.
The ALLOCATION query is used to query the downstream elements about the
buffer properties they would like. These properties include the size,
prefix and alignment of the memory. The downstream element can also propose
a GstBufferPool. It is then the upstream element that decides how it
will allocate buffers. This allows us, for example, to make the video
decoders specify their required memory alignment to the videosink.
Since new format suggestions are no longer piggybacked upstream with the
allocation, something new was needed to notify upstream elements of a
format change. It started with bringing in the new RENEGOTIATE event
that Thiago Santos had been experimenting with in the 0.10 branch. The
new event basically lets upstream elements know that new formats are
possible somewhere downstream and it would make them renegotiate a new
format before pushing a new buffer.
New format changes can also happen when new elements are linked, so
gst_pad_link() will also send this RENEGOTIATE event upstream. As it turns
out, adding new elements could also signal the availability of a new
bufferpool or different ALLOCATION properties. The RENEGOTIATE event was
thus renamed to the more generic RECONFIGURE event.
The most impressive result of these changes is that most of the
complicated code in GstBaseTransform could simply be removed.
Sticky Events
-------------
The idea for the sticky events comes from the observation that a lot of the
serialized downstream events (SEGMENT, EOS, TAGS, ..) are really context
for the buffers that follow. The design called for making these events
sticky on the pads, meaning that they become a property of the pads
they travel over.
The result is that the current timing information (SEGMENT event) or the
current stream medatadata (TAG event) that is handled by the pad are
directly accessible by looking at the last event that traveled on the pad.
By making the events stick on the pads, we can propagate them downstream
automatically when new pads are linked. This should solve one of the
biggest 0.10 problems when dealing with dynamic pipelines, the loss of
events (mostly NEWSEGMENT events).
It was then only natural to also make the CAPS event a sticky downstream
serialized event. CAPS are indeed also context for the buffers that follow.
The caps property was also removed from GstBuffer, making caps negotiation
separate from the buffer dataflow again.
GstSegment changes
------------------
For the sticky events to work with SEGMENT events, we needed to change
the SEGMENT event so that it became selfcontained. The complicated segment
accumulation logic of 0.10 was simply removed and replaced with pad offsets.
It is now possible to tweak the timing of the data coming from a pad by
using the pad offset property.
GstCaps optimizations
---------------------
It doesn't look like we'll be able to implement GstCaps iterators to 0.11 so
Sebastian was looking for other ways to improve caps performance. One of the
low hanging fruits was to pass the complete GstCaps structure to the
GstBaseTransform transform_caps method. This allows for better and smarter
implementations of the function at the cost of marginally more complex code.
Sebastian also added a filter caps parameter to the gst_pad_get_caps()
function. This allows the caller to pass preferences and possible caps to
the function. The result is that the pad can make much better negotiation
decisions.
Sebastian also optimized gst_caps_is_subset()in the 0.10 branch and ported
the results to 0.11 as well.
with the still planned caps simplifications later, these changes should
substantially improve caps negotiation speed.
Pad probes
----------
Quite a few iterations were needed to reimplement the pad blocking and pad
probes. The new infrastructure allows you to add callbacks for the various
stages in the data transfer on a pad.
It's now possible to be notified when no data is going over a pad with the
IDLE probe. This should also fix one of the big problems with implementing
dynamic pipelines and the old pad blocking.
Not all features that we want to add to the new probes are implemented yet
but they can be added later without much trouble.
Other gems
----------
A new SCHEDULING query was added to get more information about the upstream
elements scheduling properties.
Tim-Philipp Müller finally removed our dependency on libxml2. Back in the
day, we used XML for serialization of objects and the registry. Object
serialization had long been deprecated and the XML registry was replaced
with a much faster binary registry.
Tim also removed the unversioned gst-launch, gst-inspect and gst-typefind
scripts. They were always confusing and annoying for packagers and users,
it is just easier
Edward Hervey spent some time keeping the unit tests working and making
sure the API docs are up to data.
GstIterator was made more binding friendly By Johan Dahlin and Sebastian.
And what about the plugins..
----------------------------
As usual, all -base plugins are kept in sync with the core API updates and
the new features. They are always a good place to see how the new API can
be used and how to port things.
What's next
-----------
Almost all of the features laid out in the GStreamer conference last year
are now implemented. We also have quite a few new cool features that
slipped in. Things are shaping up indeed.
The last big missing piece is the redesign of the caps fields for raw audio
and video. We plan to finish that in the next weeks.
There are also a few smaller things we would like to do: use GstQuery
for the various caps functions, do GstFlowReturn for the query/event
functions, ...
Meanwhile we start the stabilization phase and we will do a first prerelease
of 0.11 this week to bring things to a wider audience. Now is the time to
catch up and start porting your plugins to 0.11.
There is still a lot of work to be done to port plugins to 0.11 before we
can release 1.0. I ask everyone again (and especially maintainers) to help
us porting plugins, it's really a lot of work!
Have a nice hacking week,
Wim

18
docs/random/styleguide
View file

@ -1,18 +0,0 @@
Some notes about various stylistic issues
-----------------------------------------
(This file should end up as a nice web page eventually)
CODING STYLE
------------
- No //-style comments allowed. Never. Reason: Forte doesn't support it.
- Our indentation style is codified in gstreamer/tools/gst-indent
- Don't mix tabs with spaces when writing new code.
- use GST_*_OBJECT as much as possible so it's clear which object you're
logging for
SUBMITTING CODE
---------------
- new code submitted to core, plugins-base, or plugins-good needs to pass a
review from another committer.
- submitting new base classes also means you look at targets for assimilation.

37
docs/random/testing/syntax
View file

@ -1,37 +0,0 @@
The test program
----------------
We need a test program that:
- can read a file with the test specs
- construct a pipeline using the pipeline definition in the spec file
(using gst_parse_launch ())
- add signals to elements in the pipeline
- run the pipeline for a fixed (in time/number of iterations) period
- record the fireing of the signals
- compare the signals to the expected results.
spec file contents:
-------------------
tcN: name-of-the-tescase
tcP: pipeline of the testcase in gst_parse_launch () syntax.
tcS: id1, element, signalname (attach to signal in an element)
tcS: id2, element, signalname
...
tcI: the number of iterations on the top bin
tcT: a timeout value in mSecs
tcR: id1,1,id2,1,.. (the pattern of signals triggered)
or
tcR: id1==id2,... (denote an equal number of signals)
/n
signal trigger patterns
-----------------------

20
docs/random/thaytan/opengl
View file

@ -1,20 +0,0 @@
# OpenGL usage
How to set up openGL using plugins in such a way that they can
render to an on-screen window, or offscreen equally?
eg:
superfoo3d ! glwindow
or
superfoo3d ! gloffscreen ! xvideowindow
This would imply that there is some mime type which connects a glwindow/gloffscreen and a GL using component sensibly. I don't see that there is any actual
data to send, however - the only purpose of the glwindow/gloffscreen is to
set up an openGL context and then 'output' it somehow one the superfoo3d has
finished drawing each frame.
In the case of glwindow, 'output' it means translate into an on-screen window,
but for gloffscreen it means produce a video packet and go.
These components really need some other 2 way communication, rather than the
pads metaphor, I think?

28
docs/random/thaytan/video-overlays
View file

@ -1,28 +0,0 @@
# Compositing and video overlays.
We want a way to composite a video frame and an overlay frame. The video frame is as we expect, the overlay is a normal video frame + an alpha mask. In the following monologue, please consider that I mean RGB/I420/YV12/YUY2 wherever I say RGB.
So we have a plugin with 2 sinks and one src:
+----------------+
|video |
| video|
|overlay |
+----------------+
What should the format for the overlay data look like?
Ideas:
* RGBA - 4 byte per pixel video frame
* RGB+A - 3 Bytes per pixel video frame, then 1 byte per pixel overlay frame.
A new mime type, or a variation on video/raw? I'm not sure
* Overlay is actually 2 sinks, one takes a RGB/YUV data the other the alpha channel.
I'm not sure which approach is better, but I think it is probably neatest to
use RGB+A, and then have a separate plugin which has 2 sinks and converts an
RGB/YUV stream plus an alpha stream into an RGB+A stream. The benefit of RGB+A over RGBA in this scenario, is that it is easier (correct me if I'm wrong) to optimise 2 block copies which appends an alpha frame to a RGB frame than it is to
do the many copies required to interleave them into an RGBA stream.
So, I see this producing a few new plugins:
videooverlay - takes an RGB and an RGB+A frame from 2 sinks, does the overlay (according to some properties) and outputs a result frame in RGB or RGB+A (as negotiated) format on 1 src.
rgb2rgba - takes 1 RGB frame and one A frame from 2 sinks and outputs RGB+A on 1 src. If the A sink is not connected, we just add a fixed alpha channel based on an object-property.
rgb2rgba - takes an RGB+A frame and discards the RGB component.
textoverlay - This plugin, instead of taking a video frame and overlaying text, can just output an RGB+A stream with appropriate timestamping. This prevents duplicating code to composite with an alpha mask and allows us to optimise it in one spot only.

121
docs/random/thomasvs/0.10
View file

@ -1,121 +0,0 @@
general
-------
- write script to check for:
invalid // comments
invalid %lld
invalid LL for 64bit ints
wrong whitespacing
trailing space
gstreamer
---------
- reorganize tests and examples into
- testsuite
- check: unit tests
- examples: example code
- interactive: interactive tests
- move gst/base to libs/gst/base ?
(but elements link against them)
- move elements out of gst/ dir ?
- check/gst/pipelines: currently disabled, random failures
gst-plugins-base
----------------
- gst-libs/gst/audio:
- DONE: audiofilter folded in
- gst:
- adder: needs docs, an example, and a test
- audioconvert: ok
- audiorate: needs docs and tests
- audioresample: David needs to fix this
- audioscale: needs to go
- audiotestsrc: ok
- ffmpegcolorspace: needs a test
- playback: example
- sine: removed, DONE
- subparse:
- works, but how do we link it so we can display it ?
- example ?
- test with different files/sources ? can be inlined
- tags:
- contained a very small code file that wasn't built, and a copy of a header
that was in the tag lib; removed; DONE
- tcp:
- works
- need tests
- need docs
- need possible porting to GNet (later)
- typefind:
- works
- need tests - this definitely could use it
- is there any way they can be documented ?
- should the plugin docs show a list of them ?
- videorate:
- needs tests, docs
- videoscale:
- needs tests
- negotiation
- par conversion
- different scale algorithms
- needs docs
- negotiation with five out of six free variables (src/sink w/h/par)
- videotestsrc:
- could use tests for all possible caps
- volume: OK
- ext:
- alsa:
- needs docs; esp. params and common practices
- needs interactive tests; depends on having such a setup available
- cdparanoia:
- needs docs, and interactive test
- remains in -base until cdio is proven to be better on all counts
- gnomevfs:
- needs docs (how to use proxy, link to gnomevfs docs, explanation
about need for homedir to create .gnome2 dir in, ...)
- needs test; test could use local files and http urls
- libvisual
- needs docs (easy)
- needs test
- ogg, vorbis, theora
- needs docs
- needs test
- sys
- v4l
- needs interactive test
- needs lots of docs
- ximage
- interactive test should go somewhere
- docs ok
- xvimage
- interactive test should go somewhere
- docs ok
gst-plugins-good
----------------
- gst:
- alpha, alphacolor: document with example
- auparse: crashes on e.g.
gst-launch -v filesrc location=/usr/share/emacs/site-lisp/emacspeak/sounds/default-8k/ask-short-question.au ! auparse ! osssink
-> will move to bad
- autodetect: OK
- videofilter:
- is the lib still needed, given basetransform ?
yes
- currently installs a lib; should not install, or move to a dir, with pc
file, possibly in -base
DONE: moved to -base
- ext:
- aasink: properties need looking at
- width, height: why are they there ? caps don't match
- frames-displayed: in base class ?
- frame-time: what's this ?
- cairo:
- cairotimeoverlay works
- cairotextoverlay ? pipeline ?
- flac:
- flacenc:
gst-launch -v audiotestsrc wave=white-noise ! flacenc ! filesink location=white-noise.flac does not produce a correct file
- flacdec works, but
gst-launch gnomevfssrc location=http://gstreamer.freedesktop.org/media/small/dark.441-16-s.flac ! flacdec ! autoaudiosink
does not

108
docs/random/thomasvs/0.4.0
View file

@ -1,108 +0,0 @@
Set of 0.4.0 proposals
* module versioning
- we need an agreement on how we are going to version the separate modules.
This needs to meet some requirements :
a) we shouldn't be forced to release all of the modules together
every time (if the plugins have been updated, and still work against
a previous core, then only release plugins, for example)
We should start treating the different modules as separate projects
(albeit still closely tied of course)
b) it should be clear for other people what packages they need to download
c) major API breakage needs to be clear; we don't expect really old players
to keep working with really new cores.
- my idea (which others seemed to agree with) would be to let the micro
numbers run independently. Only when the core API has changed to the
point that other modules are not compatible with them anymore
should we upgrade the minor version.
We can assume/assert that modules with the same minor number will be able
to work with each other.
- Example schedule :
* we release 0.4.0 versions of all of the modules
* plugins get updated a few times : 0.4.1 and 0.4.2
* core gets a new scheduler, doesn't affect other modules : 0.4.1
* gst-player gets rapid releases due to arik's recovery: 0.4.1-0.4.5
* core gets a major new update re: events : 0.5.0
* some days later, other modules have been updated to the new core
and the new core starts being useful to other people as well
* release practice
- we should have a simple way to cut releases; something that makes
the necessary adaptions to the source tree
This could also be a simple check list of things that need to pass
- cvs tarballs/packages should be easily distinguishable from pre-release
tarballs/packages and actual released ones.
my idea here would be :
a) releases are named as normal
e.g. gstreamer-0.4.0.tar.gz
gstreamer-0.4.0-1.i386.rpm
b) as soon as the release is made, we are back in "cvs" mode
i'd use a ".1" for a fourth version number for all cvs versions
so as soon as we release 0.4.0, I'd add a fourth ".1" version number.
this one would be used during the whole of the cvs period, no
reason to up this manually in between
The packages (rpms in any case, don't know about debs) would still
keep the date as the revision number like they have now, in order
to make it easy to work with snapshots.
c) when we are ready to release this module, I would go back to maintainer
mode, but keep the fourth version number and increase that for each cut.
So we'd stop developing the module, switch to maintainer mode, up the
version number to
gstreamer-0.4.0.2.tar.gz
and test that.
d) when we are happy with the precuts, we drop the fourth number and make
a release
Summary :
* all "official" released versions have sane versioning with three numbers
* all "cvs" versions are clearly recognizable by the fourth .1
* all "precuts" are also recognizable
* no tarballs will accidentally spill pretending to be real releases ;)
* upgrading rpm's all through this process is easy
* build code stuff duplicated between various modules
- how do we integrate these ? this pertains to stuff in autogen.sh,
duplicate stuff in configure.ac (which should probably be moved out to
custom .m4 files and yanked in), and maybe testing stuff
* what possible ways are there to build gstreamer ?
I would like to take stock of the combinations of deps possible to build
gstreamer. While most people want only glib2, I think there is merit
in still making glib1 work and willing to put in the effort. I just need
the possibilities mapped out once and for all ;) IMO the effort going into
making gstreamer build without libxml is the same as making it work with
older glib too. I mean, as soon as you allow variation, it isn't that hard
to allow for more than one variation. The bigger step is from zero to one.
so, what are they ?
- glib1 & gtk1
- glib1 & gtk1 & libxml
- glib2 & libxml2
* media suite
- media files on the site should be renamed to simple uniform names
- and split up based on size
- described by features/content
* when to branch in CVS ?
* automatic build testing
- tinderbox ? useful ?
- small build scripts
* automatic functionality testing
- an md5sink would be useful to do this
- something automated is needed; it should check if you have the plugins
that are needed to test other plugins
* media player
* packaging
- dependency libs should be easily available

3
docs/random/thomasvs/TODO
View file

@ -1,3 +0,0 @@
GSTREAMER
---------
- gst-i18n-lib.h is included funnily from base classes and elements

37
docs/random/thomasvs/docreview
View file

@ -1,37 +0,0 @@
Documentation review
* gstbuffer
- What are the flags in GstBuffer ? used anywhere ? defined how ?
* General
- how can we define common terms and make them cros-ref'd ?
e.g. timestamps in buffer, do we say everywhere that they're in nanosec ?
* Style
- when in doubt, try to conform to GTK+ reference docs
(in the gtk-doc tarball, doc/style-guide.txt)
- GtkMisc and GtkFontSelectionDialog are example templates.
- in the arg clarification, use as much cross-reffing as possible. Do it
only where it is useful in the explanation text.
- examples
- use active form instead of imperative describing functions; we describe
what the function does.
good : creates a new buffer.
bad : create new buffer
- use singular for enum names; this makes it more natural to reference to
it in the API docs
good : GstBufferFlag
bad : GstBufferFlags
- in arg clarification, use a period and start with a small letter.
Call the object you work on "a" instead of "the". Call the other objects
"the".
If the object in question is the return value, this means you call the
return value "a". If the object in question is the first argument
of the call, you call this argument "a" and the rest "the".
good : @buf: a pointer to query.
bad : @buf: The pointer to query

15
docs/random/thomasvs/features
View file

@ -1,15 +0,0 @@
Here's a list of features in GStreamer which plug-ins can make use of.
For each feature we will try to find a few plug-ins that show a good way
of implementing them.
seeking
caps negotiation
timestamps
clock interaction
signals
object argument handling
chain-based
loop-based
request pads
sometimes pads

16
docs/random/thomasvs/guadec-4
View file

@ -1,16 +0,0 @@
Presentation ideas for GUADEC 4 (thomasvs, April 8 2002)
* use gst-editor to create pipelines that make a karaoke machine
in different steps and using different features
1) pipeline 1: play the free software song by Richard Stallman
2) pipeline 2: do the same but add a visualization plugin
3) create a small video using actual RMS footage
4) pipeline 3: play this video and the song together
5) Stallman is a bit hard to understand. We want text.
pipeline 4: use the subtitle reader to overlay text
maybe also do a bouncing ball overlay !
6) Stallman can't sing. Let's pitch-shift him.
this will need MIDI or dynparams to control a pitch shifter
7) It sounds better, but still not quite there. Replace him with a festival
voice doing the pitch shifting.

75
docs/random/thomasvs/pthread
View file

@ -1,75 +0,0 @@
Some notes on use of pthreads in GStreamer
------------------------------------------
First off, pthreads are HARD. Remember that.
1) How I learned to debug glibc and pthreads and add debug code to it.
You have to trick your GStreamer test app in running against a modified
glibc.
I used Red Hat 7.3, downloaded the .src.rpm, installed it, applied the
patches included, and ran
./configure --prefix=/home/thomas/cvs --with-add-ons
make
make install
After quite some time this left me with recompiled libc and libpthread
libraries in /home/thomas/cvs/lib, as well as a new ld-linux.so.2
Now you need to use this new ld-linux.so ld loader to run your app,
preferably from inside of gdb so you can tell what's going on when it
crashes.
You can use ld-linux.so.2 to call your binaries:
ld-linux.so.2 .libs/thread1
to run the thread1 program with the new glibc.
If this is a GStreamer app, chances are it might not find some libraries
it needs that you could safely use from /usr/lib (like, zlib and popt).
Also, you want it to run in gdb, so this is my full line:
LD_LIBRARY_PATH=/usr/lib /home/thomas/cvs/lib/ld-linux.so.2 \
/usr/bin/gdb .libs/thread1
At this point you can start adding debug code to the pthreads implementation
in your glibc source tree. Just change, re-run make install, and restart
the test app in gdb.
Helpful --gst-mask is 0x00200100 to get thread info and scheduling info
(with mem alloc from cothreads)
2) What GStreamer does with pthreads.
Apps create a thread with gst_thread_new. This just allocates the GstThread
structure without actually doing much with it.
When a thread goes from NULL to READY, the gst_thread_change_state function
creates the actual pthread.
- we lock the thread->lock mutex
- we create attributes for the pthread
- by default the pthread is JOINABLE
- we ask the thread's scheduler for a preferred stack size and location
(FIXME: if the scheduler doesn't return one, what do we do ?)
- we create the pthread with the given attributes
- the pthread id is stored in thread->thread_id
- the created pthread starts executing gst_thread_main_loop (thread)
- the change_state function does a g_cond_wait
- this means it unlocks the mutex, waits until thread->cond is set
(which happens in gst_thread_main_loop),
then relocks the mutex and resumes execution
From the point of view of the created pthread, here's what happens.
gst_thread_main_loop (thread) gets called
- the thread's mutex gets locked
- the thread's scheduler's policy gets examined
- the scheduler gets set up (invokes the scheduler object's setup method)
FIXME: what are the prereqs of this _setup method ?
- basic and fast scheduler both call do_cothread_context_init
- basic: this calls cothread_context_init
- cothread_context_init
- fast: this calls cothread_create (NULL, 0, NULL, NULL))
(FINISHME)
(FOLDMEBACKTOREALDOCS)

39
docs/random/thomasvs/pwg
View file

@ -1,39 +0,0 @@
Stuff for the PWG
-----------------
* arguments
- how to add arguments
- create an identifier in the enum, starting with ARG_
example: ARG_RATE
- add the property by adding a
g_object_class_install_property line
FIXME: what is name/nick/blurb in the paramspec ?
- if the argument is readable, a block of code for it needs to be added
to the _get_property function.
- default value
- default value should be set in _init function
- default value can be specified in paramspec (but I don't think this
is used anywhere)
- things to check/possible problems
- do you have a _get_property function ?
- do you have a _set_property function ?
- do both have a default handler that handles invalid property ID's ?
- are the _get/_set_property handlers assigned to the class's struct ?
- do you have a g_object_class_install_property line for the argument ?
- are there restrictions on when your parameters can change ?
e.g. sample rate is not something that should be changed while PLAYING,
so it can only be changed in the NULL state
- did you use ARG_ ... consistently everywhere ?
- my own problems:
- how to set defaults and make the paramspec be right for them ?
* audio
- explanation of difference of width and depth
- guidelines on how to implement this properly

36
docs/random/thomasvs/registry
View file

@ -1,36 +0,0 @@
Reviewing the registry (thomasvs, April 8 2002)
* added a --gst-registry flag to the core which allows any gst app
to specify a different registry for loading/saving
some stuff to do this went into gstreamer/gst/gstregistry.h
* What location is used for writing ? (gst-register)
- if specified (using --gst-registry) then use the specified location
- if not specified :
- it can be written in the global location, do it there
(which should be sysconfdir/gstreamer) and reg.xml
- if not writable, then try ~/.gstreamer/reg.xml
* What location is used for reading ? (gst-whatever)
- if specified (using --gst-registry) then use the specified location
- if not specified :
- right now :
if local exists, only read local
if not, read global
- TODO: try reading GST_CONFIG_DIR/reg.xml first
then try reading ~/.gstreamer/reg.xml
AND replace every namespace collision with the new one
* actual variables stuff (gstregistry.c)
- use gst_registry_write_get to get a GstRegistryWrite struct back
listing the right location of dir, file and tmp file
- use gst_registry_read_get to get a GstRegistryRead struct back
listing the path of global and local file to read
* gst-register signals it's going to write to the registry (causing it to
be unlinked before the read in post_init ()) by setting a global variable,
_gst_init_write_registry

33
docs/random/types
View file

@ -1,33 +0,0 @@
GstTypes exist to try to make sure data eveyrone is talking about the
right kind of data. They aid quite a bit in autoplugging, in fact make it
possible. Each well-formed type includes a function (typefind) that will
take one or more buffers and determine whether or not it is indeed a
stream of that type, and possible a metadata to go with it. It may
provide related metadata structure IDs (and must if it provides metadata
from the typefind function).
Because multiple elements and plugins are very likely to be using the same
types, the process of creating/finding types is designed to be done with a
single function call. All operations on GstTypes occur via their guint16
ID numbers, with the GstType structure "private" to the GST library. A
plugin wishing to use a give type would contain a static struct of type
GstTypeFactory, which lists the MIME type, possible extensions (which may
overlap the mime-types file), a typefind function, and any other cruft I
decide to add.
A plugin init function would take this typefactory and hand it to the
gst_type_new() (FIXME: badly named) function, which would first search for
that same MIME type in the current list. If it found one, it would
compare the two to see if the new one is "better". Better is defined as
having more extensions (to be merged) or a typefind function verses none.
The point of returning an existing MIME type is a result of the goal of
unifying types enough to guarantee that, for instance, all MP3 decoders
will work interchangeably. If MP3 decoder A says "MIME type 'audio/mpeg'
with extensions 'mpeg3'" and decoder B says "MIME type 'audio/mpeg' with
extensions 'mp3'", we don't want to have two types defined, possibly with
two typefind functions. If we did, it's not obvious which of the two would
be tried first (luck) and if both would really identify streams as mp3
correctly in all cases. And whichever wins, we're stuck using the
associated decoder to play that stream. We lose the choice between any
valid mp3 decoder, and thus the whole point of the type system.

282
docs/random/types2
View file

@ -1,282 +0,0 @@
OUTDATED
--------
1. Introduction
---------------
The type system is used to attach meaning to the bytes in a GstBuffer.
A plugin can decide to add metadata to the GstBuffer, this metadata
will carry an associated typeid.
Types are also used by the plugins to expose the type of their pads to
the type system.
Types are essential for autoplugging.
We will explain the inner workings of the type system in this document, as
well as the API used in the plugins.
2. Type properties
------------------
a) identification of the type
Types are identified by one or more MIME types.
b) detection of types
The type of any given GstBuffer can be detected using
- a typefind function: a function that will inspect the bytes
in the buffer and return a gboolean indicating the
buffer is of the given type.
- a template for the bytes/bits in the data that must be
satisfied in order for the GstBuffer to be of the given
type.
- other properties that act more like a hint like:
the extension of the source filename.
the URI of the source.
3. Type registration
--------------------
a) the core libraries will create to types:
video/raw image/raw
audio/raw
b) all other types will be provided by the plugins
before a type can become available to other plugins, the plugin
has to register the type.
The system will keep a directed graph of the types and the plugins
that operate on them.
example:
video/mpeg
!
mpeg1parse
/ \
video/mpeg1 audio/mp3 -----\
! ! \ !
mpeg_play mpeg2dec mpg123 xing ...
\ / \ /
video/raw audio/raw-----\
! ! ! !
videosink SDLsink audiosink alsasink ...
The system can find the needed plugins to convert video/mpeg to
audio/raw using this graph.
4. type equivalence
-------------------
some types can have the same meaning for example:
audio/mp3 and audio/mpeg
or
video/raw and image/raw
a plugin that exposes its output type as audio/mpeg and another plugins
with input type audio/mp3 can be connected. The system has to know about
the equivalence of both types, even it they have a different mime type.
5. type hierarchy
-----------------
some plugins can output a specific subset of an already existing type.
example:
mp3parse inputs audio/mp3 and packs the stream into mp3 audio frames
with mime type: audio/mp3-frame
mpg123 only accepts audio/mp3-frame but not audio/mp3.
another mp3 decoder (libmpg123) can accept audio/mp3 (and thus also
audio/mp3-frame)
it must be possible to connect both libmpg123 and mpg123 to the mp3parse
element.
it must not be possible to connect mpg123 to an element that outputs only
audio/mp3 but not audio/mp3-frame.
We say that audio/mp3-frame is a more specific subset of type audio/mp3.
we can create a type hierarchy:
audio/mp3
/ \
audio/mp3-frame audio/mp3-layer12
/ \
audio/mp3-layer1 audio/mp3-layer2
6. multi-type pads
------------------
certain plugins might accept multiple non equivalent types in one of their
input pads. Consequently a plugin might output non equivalent types in
its output pad.
It must therefore be possible to specify multiple types for a pad.
example:
mpegdemux may be able to demux both MPEG1 and MPEG2 system streams.
we show the type hierarchy of the video/mpeg as follows:
video/mpeg
/ \
video/mpeg1 video/mpeg2 ---------------\
/ \ / \ !
mpeg1-system* mpeg1-video mpeg2-ts mpeg2-ps* mpeg2-video
the mpegdemux element might specify the type of the input pad as
one of video/mpeg1-system and video/mpeg2-ts
7. definition of types
----------------------
A plugin will provide the following information to the type system:
- a mime type string describing the hierarchy and where the types
they provide are located in that hierarchy.
- typefind functions for some of the types.
- extensions for some of the types
We will propose a syntax to define the type hierarchy
a) equivalent types :
separated with a | sign
( audio/mp3 | audio/mpeg )
b) type hierarchy :
in braces:
( audio/mp3 ( audio/mp3-frame))
c) multi-type pads
( mpegdemux ( video/mpeg1-system + video/mpeg2-ps) )
the pad will have type mpegdemux which is the parent for
type video/mpeg1-system or video/mpeg2-ps
mpegdemux
/ \
video/mpeg1-system video/mpeg2-ps
once the type hierarchy has been registered, the typeid of each
element can be obtained with:
guint16 gst_type_find_by_mime (gchar *mime)
extra typefind functions and/or extensions can be added to the
typeid afterwards.
8. type matching
----------------
The more specific typefind functions, the functions associated with
types in the leaf nodes, are tried first.
when a specific type has been found ex. video/mpeg1-system elements
that can handle this type or one of its parents are selected:
mpegdemux: mpeg1parse: supermpegdecoder:
video/mpeg video/mpeg video/mpeg
! !
mpegdemux video/mpeg1-system
!
video/mpeg1-system
In the above case, also the super mpeg element is selected because it stated
that it can handle all sorts of video/mpeg data.
example 2:
suppose the typefind functions finds an mp3 stream, following elements
are selected:
libmpg123 mp3parse:
audio/mp3 audio/mp3
mpg123 is not selected because its pad type is too specific (audio/mp3-frame):
mpg123
audio/mp3
!
audio/mp3-frame
if the typefind would find a mp3-frame type, all three objects would be selected.
8. consideration
----------------
It is clear that clear indications have to be given to the type hierarchy,
especially for the top nodes.
The more specific an element is in its mime type specification, the more flexible
and extendible the plugin system can be.

177
docs/random/types3
View file

@ -1,177 +0,0 @@
SOMEWHAT OUTDATED, design still holds though
--------------------------------------------
1. Introduction
---------------
The type system is used to attach meaning to the bytes in a GstBuffer.
A plugin can decide to add metadata to the GstBuffer, this metadata
will carry an associated typeid.
Types are also used by the plugins to expose the type of their pads to
the type system.
Types are essential for autoplugging.
We will explain the inner workings of the type system in this document, as
well as the API used in the plugins.
2. major types
--------------
major types are identified with mime types and are used to denote a
family of types.
More specific information about the type is given using properties. This
will allow us to be very specific without creating a lot of mime types.
3. API
------
Both a simple array based specification and a real API will be
provided to build the capabilities.
In the array based approach, we basically build an array of pointers.
Some macros will be available to specify ranges, boolean values, lists
and id's. (not sure if this can be done)
#define GST_TYPE_INT_RANGE(a, b) GST_TYPE_RANGE,(a),(b)
#define GST_TYPE_BOOLEAN(a) GST_TYPE_BOOLEAN,(a)
#define GST_TYPE_LIST(a...) GST_TYPE_LIST,(##a),NULL
for example:
static GstTypeCapsFactory mpg123_sink_caps[] = {
"audio/mp3",
"layer", GST_TYPE_INT_RANGE (1, 3),
"bitrate", GST_TYPE_INT_RANGE (8, 320),
"framed", GST_TYPE_BOOLEAN (true),
NULL
};
will expand to the array:
static GstTypeCapsFactory mpg123_sink_caps[] = {
"audio/mp3",
"layer", GST_TYPE_RANGE,1,3,
"bitrate", GST_TYPE_RANGE,8,320,
"famed", GST_TYPE_BOOLEAN,true,
NULL,
};
when we register the caps factory, the strings will be converted
into GQuarks and be stored into a GData Keyed Data List.
This will result in a GstTypeCaps structure:
struct _GstTypeCaps {
guint16 id; // if of the major type
GData *properties;
}
4. example using arrays
-----------------------
mpg123: an mpeg audio decoder.
// a factory for the major type we use
static GstTypeFactory mp3factory = {
"audio/mp3", // major type
".mp3 .mp2 .mp1 .mpga", // extensions
NULL, // typefind function
};
// capabilities of the sink pad
static GstTypeCapsFactory mpg123_sink_caps[] = {
"audio/mp3",
"layer", GST_TYPE_INT_RANGE (1, 3),
"bitrate", GST_TYPE_INT_RANGE (8, 320),
"framed", GST_TYPE_BOOLEAN (true),
NULL
};
// capabilities of the source pad
static GstTypeCapsFactory mpg123_src_caps[] = {
"audio/raw",
"rate", GST_TYPE_INT_RANGE (8000, 48000),
"channels", GST_TYPE_INT_RANGE (1, 2),
NULL
};
static GstTypeCaps *sinkcaps = NULL, *rawcaps = NULL;
static gst_mpg123_init (GstMpg123 *mpg123)
{
mpg123->sinpad = gst_pad_new ("sink", GST_PAD_SINK);
gst_element_add_pad (GST_ELEMENT (mpg123), mpg123->sinkpad);
gst_pad_set_caps (mpg123->sinkpad, sinkcaps);
...
}
GstPlugin *plugin_init (GModule *module)
{
...
plugin = gst_plugin_new ("mpg123");
gst_plugin_add_type_factory (plugin, mp3factory);
...
sinkcaps = gst_type_register_caps (mpg123_sink_caps, NULL);
rawcaps = gst_type_register_caps (mpg123_src_caps, NULL);
...
}
mpeg2dec: an mpeg video decoder that can do mpeg1 and mpeg2.
static GstTypeFactory mpegfactory = {
"video/mpeg", // major type
".mpg .mpeg", // extensions
NULL, // typefind function
};
static GstTypeCapsFactory mpeg2dec_sink_caps[] = {
"video/mpeg",
"mpegtype", GST_TYPE_LIST (
GST_TYPE_INT(1),
GST_TYPE_INT(2),
),
NULL
};
static GstTypeCapsFactory mpeg2dec_src_caps[] = {
"video/raw",
"fourcc", GST_TYPE_LIST (
GST_TYPE_INT32 (0x32315659),
GST_TYPE_INT32 (0x32...),
),
"width", GST_TYPE_INT_RANGE (16, 4096),
"height", GST_TYPE_INT_RANGE (16, 4096),
NULL
};
static GstTypeCaps *sinkcaps = NULL, *rawcaps = NULL;
GstPlugin *plugin_init (GModule *module)
{
...
plugin = gst_plugin_new ("mpeg2dec");
...
sinkcaps = gst_type_register_caps (mpeg2dec_sink_caps, NULL);
rawcaps = gst_type_register_caps (mpeg2dec_src_caps, NULL);
...
}
5. capability compatibility
--------------------------
Two pads are compatible if:
- The major types are equal
- range of the sink pad contains the range of the src pad

21
docs/random/use-cases-0.11.txt
View file

@ -1,21 +0,0 @@
Use cases for 0.11
------------------
- improve performance
* incremental caps
* cleanup caps fields
* Make miniobject a simple boxed type
* reverse negotiation with an event
- buffer metadata
* Handle strides for video
* express crop, scaling, region of interest, ...
* Better support for foreign objects such as Cairo, OpenGL, OMX, ...
- make dynamic pipelines easier
* sticky event propagation
* newsegment simplifications
* per-pad time offsets

44
docs/random/usecases
View file

@ -1,44 +0,0 @@
Use cases for audio applications
--------------------------------
ANALYSIS
========
* find start and end -x dB boundary
* find overall volume for complete track (peak/RMS/average)
* do level-triggered playback
CUE FILES
=========
Given a long audio file and a .cue file:
- an element should be able to read an audio input stream and a cue input
stream and send new-media events based on the .cue file information
- a audiocdsink should be able to record this stream straight to disk
Use cases for core functionality
--------------------------------
THREADS
=======
* { sinesrc } ! { queue ! osssink }
- play thread
- pause input thread
- output thread should keep playing as long as queue not empty
- play input thread again
- no gaps should be heard
Use cases for video applications
--------------------------------
THUMBNAILING
============
- load file
- figure out if it's playable
- ask length of file
- seek to middle
- snapshot one frame
- save to disk

78
docs/random/vis-transform
View file

@ -1,78 +0,0 @@
Duh, an 'easy' way to replicate Giess's behavior:
For each frame, you have to mutate it by a transform matrix. This is
easy, thought not cheap. First you precalculate the transform matrix how
you want it, based on whatever rotations or whatever you want.
The data stored in each spot on the matrix tells you how to transform a
single pixel. The simple case is dx,dy, where both are relatively small.
The probably ought to be a byte in any case, so you can scale the
transform matrix on slow machines. A more complex case is some trick
whereby a single pixel ends up splattered in several places. Idea below.
The matrix consists of some number of 8bit arrays of the same size as the
image. They'd probably be line-interleaved or better to help with cache
effects (which are VERY serious here). Each channel represents some
aspect of the transform. The first two would likely be dx and dy, the
third might be a multiplier if that wasn't done statically.
The idea: any number of transform sets could be applied, given available
processing power. Just set the static scalar or the multiplier matrices
so you don't completely swamp the output pixels.
Note that this is fastest in 8-bit, but theoretically could be applied to
32 bit. 15 and 16 are hard, since you can't easily apply the multipliers
unless they're 1/2^n, and even then it's significantly heavier (you'd have
to mask the top n bits of each color out).
This SCREAMS for MMX, in case you haven't figured it out yet.
Unfortunately, MMX is only directly useful for the scalar matrix, unless
you do a trick where all the pixels in that fit in 64 bits (8 8bit, 4
16bit, or 2 32bit) are always moved in a group. This is very possible,
and might be a significant perf increase by being able to use MMX all the
way through. Otherwise you have to place each pixel by extracting the MMX
stuff back into normal registers, and that just plain sucks.
A pseudo-C implementation:
----- BEGIN -----
gint x,y; /* image x and y size */
guchar old_image[x][y]; /* original image */
guchar new_image[x][y]; /* new image */
gchar x_xform[x][y]; /* dx matrix */
gchar y_xform[x][y]; /* dy matrix */
guchar s_xform[x][y]; /* intensity scalar matrix */
guchar scalar; /* global scalar */
gint i,j; /* indixes */
gulong p; /* pixel value in question */
guchar u,v,w; /* modifier variables */
/* clear the new image, we don't want anything getting in the way */
/* NOT NECESSARILY A GOOD THING, THOUGH */
memset(new_image,0,x*y);
/* loop through all the lines in the image */
for (j=0;j<y;j++) {
/* loop through all the pixels in the line */
for (i=0;i<x;i++) {
p = old_image[i][j];
u = x_xform[i][j];
v = y_xform[i][j];
w = s_xform[i][j];
new_image[i+u][j+v] = (guchar)((p<<14) / (w * scalar));
}
}
----- END -----
Note that the above really, *REALLY* sucks performance-wise. Throw it a
80x60 image and it'll swamp my poor laptop. Also note that I simply set
the pixel value, not merge it. That means you'd better be sure your
transform matrix doesn't have overlapping destinations.
Other notes about the above code: x_xform and y_xform are signed chars,
which means pixels can move in all directions. The intensity matrix is
unsigned, with a range from 0 to 255, so is the global scalar. Note the
shift of 14bits (2 * 7bits), then divide by each. That means identity for
both scalars is at 128. The FP range of each is thus 0.0 to 2.0. Very
handy.

32
docs/random/wingo/porting-plugins-to-0.9
View file

@ -1,32 +0,0 @@
-*- outline -*-
Some collected notes porting plugins to GStreamer 0.9.
* General notes
Consider using the base classes. There are base classes for sources,
sinks, and "transformers", which are plugins that have the same caps on
their source and sink sides.
These base classes offer interfaces that are much more appropriate to
the function of specific plugins than the general GstElement interface.
* Functions that changed
** gst_pad_use_explicit_caps
** gst_pad_set_explicit_caps
Not needed; just set the caps on the first buffer you push out.
** gst_element_set_loop_function
Use gst_pad_set_loop_function. Note that loop-based elements participate
more in the scheduling process in 0.9, so you'll need to add hooks to
the activate function to start up a task.
refcounting:
_get_ accessors
GValue, GstObject, and refcounts (arrrrgh)

76
docs/random/wingo/threadsafe-properties
View file

@ -1,76 +0,0 @@
-*- Mode: text -*-
Sometimes it's difficult to use GStreamer in real applications where the GUI and
the GStreamer pipeline are in different threads. You have to somehow make sure
that the object is not modifying its properties while you get and set them. This
document is a brief outline of a potential fix to this issue.
* When is it safe to get or set properties?
Well, it is only safe to do so when other threads are not accessing an object.
So, from the main thread, it is only safe to query or modify properties on an
object when it is not in the RUNNING state, because when it is RUNNING its
properties are potentially changing in the thread of execution.
The only place that it is safe to get or set properties while in the RUNNING
state is from within the thread of execution. Thus, something within the
iteration loop of the thread must check to see if there are pending property
changes or queries. There are two places this could be done, from within
gstthread itself and from within the scheduler. Doing it from gstthread sounds
like a good idea because it keeps more code out of the scheduler, but is also
bad in a way because it requires a gstthread-specific api to proxy prop_set and
prop_get. Setting it in the scheduler sounds good because of its finer
granularity, but might be bad because it would clutter the scheduler a bit more.
I propose to go with the scheduler-based solution based on system response time
and out of potentials for integration with gst_clock_wait.
* Implementation
We do want to preserve some measure of generality, however. Considering that
more threadsafety issues could pop up in the future, it would be nice to have
one function to call from the scheduler, in the interests of code
simplification. This function will be present in some elements and not in
others, and will be modified at run time, so we will make it a function pointer
within the GstElement struct.
struct _GstElement {
...
void (*pre_run_func) (GstElement *);
void (*post_run_func) (GstElement *);
}
Only the managing bin of an element is allowed to set that function, because
presumably that bin would know something about how to schedule the element.
Then, in the scheduler, before we call chain functions and before we switch into
loop functions:
if (element->pre_run_func)
element->pre_run_func (element);
Then, to get or set properties, we use the new functions gst_element_get or
gst_element_set. _set would add the property name and a gvalue onto a queue
(probably a GAsyncQueue). Then the pre_run_func would go ahead and set the
properties. _get is a little more tricky; _set doesn't hardly block at all,
although it's not instantaneous. With _get though, you really don't know what
the properties are until you query them. The best thing would be to connect to
the ::notify signal, which executes within the thread of interest. However, say
you really want to use _get. Hmm. I think that it would have to block. On what?
Well, probably on an element's mutex. So it seems we might need a
post_run_func too, to unlock the mutex. We can use the GstObject lock for this.
But we need a little more. How do we know whether or not just calling
g_object_get/set is ok? I'm thinking this whole prop set/get proxy thing should
not be abstracted away, that it should be contained in gstelement.c. There are
more kinds of bins that need threadsafety than just gstthread (I'm thinking
about jack here). So, we can add on a GAsyncQueue *prop_set_queue; to the
GstElement struct, and only initialize it in certain managing bins. Then, we can
set a flag on gstelement, GST_ELEMENT_USE_THREADSAFE_PROPERTIES. If this flag is
set (by the managing bin), do all this complicated mess; otherwise use the
gobject native functionality.
So, this is the plan. We'll see how the implementation goes. This should make MT
gst programming much easier.
wingo.
25 May 2002.

62
docs/random/wingo/without-factories
View file

@ -1,62 +0,0 @@
-*- outline -*-
* Creating Elements Without Factories
** The purpose of factories
On a typical GStreamer system, there are approximately 6.022*10^23
plugins. GStreamer knows about all of them because of the registry. The
goal is to avoid initializing each one of them, when maybe for your
application you only need one or two.
The problem becomes, how do you create an instance of the plugin? The
normal way to instantiate a class is via g_object_new (TYPE, ARGS...).
In the case that the plugin isn't loaded, you don't know its type, and
can't even get it from the type name.
Element factories exist to solve this problem by associating names (like
"sinesrc" or "identity") with certain types that are provided by the
plugin. Then when the user asks for "sinesrc", the appropriate plugin is
loaded, its types are initialized, and then gst_element_factory_create
creates the object for you.
** Why not factories?
To review, factories (1) allow plugins to remain unloaded if not
necessary, and (2) make it easy to create elements.
If you are writing an application that has custom elements (as is the
case with most serious applications), you will probably have the plugin
loaded up already, and you will have access to the type of the element.
To muck about creating a plugin for the app, registering the element
with the plugin, and then creating it with the element factory API
actually takes more work than the normal way.
** g_object_new
So you want to avoid factories. To create objects with a simple
g_object_new call is our strategy. However, to preserve the same
semantics as gst_element_factory_create, we need to know what else is
needed to initialize a GStreamer element.
The other things that gst_element_factory_create does are as follows:
*** Sets the ->elementfactory member on the element class
Note that anything trying to get the factory won't work (e.g.
gst_element_get_factory). Thankfully this is less of a problem after the
0.7 plugin system changes.
*** Initializes the name of the element
To do this ourselves, we either call gst_object_set_name, or when we
set the "name" property when creating the object.
** Summary
To create a GStreamer element when you know the type, you can just use
g_object_new (get_type_of_my_element (),
"name", the_name_you_want_possibly_null,
... any other properties ...
NULL);

26
docs/random/zaheerm/dvb-interface.txt
View file

@ -1,26 +0,0 @@
Interface:
- type (string like caps) channels/x-mpegts-pat
- GstStructure to describe the what the list is, name, where from etc. (how many entries you can select, if multiple tried, first one picked)
- method to list entries (channels)
- method to get current entries
- method to set list of entries
- each entry contains an id (32 bit int say), GstStructure (with additional stuff)
signals to say when entry list changes
to say when current entry changes
So for a transport stream, this interface would be exported and it could contain a list of interfaces...eg it would export PAT, and PAT list would contain interfaces in the GstStructure for the PMT.
dvbsrc -> list of frequencies
flutsdemux -> list of programs (PAT) and list of PMT streams per program
get list of frequencies, select frequency
get list of programs, select programs
get list of streams, select streams
only give program details in PAT list, when we have seen the PMT for that program.
possibly another interface to select multiple list interfaces from an element. (interface selector interface)
--
other option is to tag