mirror of
https://gitlab.freedesktop.org/gstreamer/gstreamer.git
synced 2025-01-13 10:55:34 +00:00
added all of my un-committed random doc files, as backup <g>
Original commit message from CVS: added all of my un-committed random doc files, as backup <g>
This commit is contained in:
parent
a6a48cf180
commit
5608ed3da1
20 changed files with 1140 additions and 0 deletions
19
docs/random/omega/EOS/chain-walkthrough
Normal file
19
docs/random/omega/EOS/chain-walkthrough
Normal file
|
@ -0,0 +1,19 @@
|
|||
disksrc -> mp3parse -> mpg123 -> audiosink
|
||||
|
||||
disksrc reads last 4K chunk from disk. Sets EOS on the buffer, pushes buffer out to peer.
|
||||
disksrc:src sets itself to EOS and chains to mp3parse.
|
||||
|
||||
mp3parse takes the buffer and slices it up into several.
|
||||
Each buffer gets chained off to mpg123.
|
||||
|
||||
mpg123 decoders each frame and sends it to audiosink.
|
||||
|
||||
audiosink dumps the audio data to the sound card.
|
||||
|
||||
mpg123 returns from the chain, and returns.
|
||||
|
||||
mp3parse returns from each of the several chains,a nd returns.
|
||||
|
||||
disksrc:src returns from the chain, signals EOS, and sets disksrc's EOS flag
|
||||
|
||||
At this point the pipeline state code takes over and starts to set everyone to READY
|
21
docs/random/omega/IDEAS
Normal file
21
docs/random/omega/IDEAS
Normal file
|
@ -0,0 +1,21 @@
|
|||
Plugin Registry:
|
||||
Have a web-accessible database of plugins from everywhere, including their merit values. gstplugin.c
|
||||
and the autoplug code (should we abstract out autoplug into a special file? I think so) can make use of
|
||||
this database via another module (gstwebregistry.c?) to look up stuff. A copy of the registry (gzip'd
|
||||
XML) could even be cached. System and user options would determine whether this registry is checked
|
||||
and/or updated automatically. The registry could simply be merged with the local machine and user
|
||||
registries, with the state bit set to "don't even have it". Autodownload/install code should be
|
||||
provided, though designed such that it's not toolkit/OS specific.
|
||||
|
||||
Merit:
|
||||
Plugins and even type definition should carry merit values, allowing the system to determine which
|
||||
plugin or type definition is better. This can be tied into the web registry setup, where merit values
|
||||
can be updated without requiring an update of the plugins themselves. They can also be guaranteed
|
||||
unique if there's a range reserved for registered objects. Unfortunately, that might get us into some
|
||||
political wars. We could leave that up to the users via some voting system.
|
||||
|
||||
Scheduling state:
|
||||
All the scheduling info for a Bin should be contained in a single object of some form. This can be
|
||||
saved off and restored seamlessly at some point in the future. This would be ideal for some autoplug
|
||||
cases, where the rest of the pipeline simply ceases to exist temporarily, and scheduling entry points
|
||||
may need to be modified until the autoplug stage is finished.
|
54
docs/random/omega/TODO-0.1.0
Normal file
54
docs/random/omega/TODO-0.1.0
Normal file
|
@ -0,0 +1,54 @@
|
|||
mandatory:
|
||||
-----------
|
||||
|
||||
(done) Fix compile warnings in gst/*
|
||||
|
||||
Make sure all common media types work with autoplug and gstmediaplay
|
||||
|
||||
Add interface to control the level of output, both compile- and run-time
|
||||
|
||||
(done) Make sure the build is capable of being run without any debugging noise
|
||||
|
||||
Make sure that the absense of any of the optional libraries will not be fatal
|
||||
|
||||
Make sure all the old plugins don't build normally, via some configure option?
|
||||
|
||||
Try to get more of the INFO calls worked out, removing gst_info
|
||||
|
||||
Decide whether DEBUG should use the category system, and if so, implement it
|
||||
|
||||
Icon for gstplay, .desktop file, etc.
|
||||
|
||||
Build tarballs
|
||||
|
||||
Build RPMs
|
||||
|
||||
Figure out which docs we're going to make 'offical'
|
||||
|
||||
UPDATE THE WEB SITE
|
||||
|
||||
optional:
|
||||
---------
|
||||
|
||||
Fix gstreamer-launch to do named pads
|
||||
|
||||
Need to fix EOS subsystem so -launch can play out
|
||||
|
||||
gstreamer-launch should play out till EOS
|
||||
|
||||
Finish LADSPA plugin to data-moving stage
|
||||
|
||||
|
||||
|
||||
things to remember for the anouncement:
|
||||
---------------------------------------
|
||||
|
||||
Build requirements list:
|
||||
libtool 1.3.5 or patched to "pass_all"
|
||||
optional:
|
||||
libcdparanoia
|
||||
libmp3lame
|
||||
libxaudio
|
||||
libXv
|
||||
libgdk-pixbuf
|
||||
libglade
|
3
docs/random/omega/TYPE_FOURCC
Normal file
3
docs/random/omega/TYPE_FOURCC
Normal file
|
@ -0,0 +1,3 @@
|
|||
#define GST_TYPE_FOURCC(f) \
|
||||
(((f)[0]) & ((f)[1] << 8) & ((f)[2] << 16) & ((f)[3] << 24))
|
||||
|
320
docs/random/omega/caps2
Normal file
320
docs/random/omega/caps2
Normal file
|
@ -0,0 +1,320 @@
|
|||
The elementfactory for a given element will contain some information about the capabilities of element's
|
||||
pads or potential pads. An indication will be provided as to whether the pad always exists, always
|
||||
exists once data is present, or *might* exist once data is present (the latter case is for things like
|
||||
the MPEG system parsers, where an audio stream might or might not exist).
|
||||
|
||||
|
||||
First, an entirely normal example:
|
||||
----------------------------------
|
||||
|
||||
(-----------) (----------) (-------------)
|
||||
! disksrc ! ! mpg123 ! ! audiosink !
|
||||
! src sink src sink !
|
||||
! ! ! ! ! !
|
||||
(-----------) (----------) (-------------)
|
||||
|
||||
We start with only the disksrc. The typefind filter is attached to the disksrc, and via typefind magic
|
||||
the properties of the disksrc are found to be:
|
||||
|
||||
disksrc->src->caps = {
|
||||
"audio/mp3",
|
||||
"layer", GST_CAPS_INT (3),
|
||||
"bitrate", GST_CAPS_INT (128),
|
||||
NULL
|
||||
};
|
||||
|
||||
A look through the plugin registry shows that we have an element called mpg123 that has the following
|
||||
caps:
|
||||
|
||||
static GstCapsFactory mpg123_sink_caps = {
|
||||
"audio/mp3",
|
||||
"layer", GST_CAPS_INT_RANGE (1, 3),
|
||||
"bitrate", GST_CAPS_INT_RANGE (8, 320),
|
||||
NULL
|
||||
};
|
||||
|
||||
The caps of the disksrc fit within those parameters, so we instantiate an mpg123 and attach it to the
|
||||
disksrc. The connection succeeds negotiation and as a result the mpg123 specifies its output caps as:
|
||||
|
||||
mpg123->src->caps = {
|
||||
"audio/raw",
|
||||
"format", GST_CAPS_BITFIELD (S16),
|
||||
"depth", GST_CAPS_INT (16),
|
||||
"rate", GST_CAPS_INT (44100),
|
||||
"channels", GST_CAPS_INT (2),
|
||||
NULL
|
||||
};
|
||||
|
||||
Again from the plugin registry we find an element audiosink that has appropriate caps:
|
||||
|
||||
static GstCapsFactory audiosink_src_caps = {
|
||||
"audio/raw",
|
||||
"format", GST_CAPS_BITFIELD (S16,....),
|
||||
"depth", GST_CAPS_INT (16),
|
||||
"rate", GST_CAPS_INT_RANGE (4000, 96000),
|
||||
"channels", GST_CAPS_INT_RANGE (1, 2),
|
||||
NULL
|
||||
};
|
||||
|
||||
A copy of the audiosink is instantiated and attached, negotiation goes smoothly, and we're done. No
|
||||
dataflow has occured, no failure found, etc. An ideal autoplug.
|
||||
|
||||
|
||||
Now, a slightly more convoluted example:
|
||||
----------------------------------------
|
||||
|
||||
Start with the same graph:
|
||||
|
||||
(-----------) (----------) (-------------)
|
||||
! disksrc ! ! mpg123 ! ! audiosink !
|
||||
! src sink src sink !
|
||||
! ! ! ! ! !
|
||||
(-----------) (----------) (-------------)
|
||||
|
||||
Run typefind on the disksrc's output, get the same output caps:
|
||||
|
||||
disksrc->src->caps = {
|
||||
"audio/mp3",
|
||||
"layer", GST_CAPS_INT (3),
|
||||
"bitrate", GST_CAPS_INT (128),
|
||||
NULL
|
||||
};
|
||||
|
||||
Find and attach mpg123, get the following output caps this time:
|
||||
|
||||
mpg123->src->caps = {
|
||||
"audio/raw",
|
||||
"format", GST_CAPS_BITFIELD (S16),
|
||||
"depth", GST_CAPS_INT (16),
|
||||
"rate", GST_CAPS_INT (44100),
|
||||
"channels", GST_CAPS_INT (1),
|
||||
NULL
|
||||
};
|
||||
|
||||
Note that this time we have a mono output. A look into the audiosink caps shows that we have a match.
|
||||
So we instantiate a copy. Oops. We now find that the caps for the input pad on our audiosink have
|
||||
changed:
|
||||
|
||||
mpg123->src->caps = {
|
||||
"audio/raw",
|
||||
"format", GST_CAPS_BITFIELD (S16,...),
|
||||
"depth", GST_CAPS_INT (16),
|
||||
"rate", GST_CAPS_INT (11025, 48000),
|
||||
"channels", GST_CAPS_INT (2),
|
||||
NULL
|
||||
};
|
||||
|
||||
Whoops. It seems that the sound card we've got in this machine (FIXME how on earth to deal with
|
||||
multiple sound cards???) doesn't support mono output *at all*. This is a problem. We now find that we
|
||||
hae no options as far as directly matching the mpg123 to the audiosink.
|
||||
|
||||
A look through our (ficticious) plugin registry shows at least one element that at least has audio/raw
|
||||
on both input and ouput (since both mpg123 and audiosink have open pads with this mime type). A closerlook shows that its caps are:
|
||||
|
||||
static GstCapsFactory mono2stereo_sink_caps = {
|
||||
"audio/raw",
|
||||
"channels", GST_CAPS_INT (1),
|
||||
NULL
|
||||
};
|
||||
static GstCapsFactory mono2stereo_src_caps = {
|
||||
"audio/raw",
|
||||
"channels", GST_CAPS_INT (2),
|
||||
NULL
|
||||
};
|
||||
|
||||
Wow, that's a perfect match. Instantiate, attach to mpg123, no problems. Attach to audiosink, no
|
||||
problems. Done. When we start up the pipeline, we should get absolutely no callbacks from pads saying
|
||||
"help me, I've fallen and..., er, I don't like this buffer!".
|
||||
|
||||
|
||||
A really messy case:
|
||||
--------------------
|
||||
|
||||
Start with a disksrc, typefind it, get the following:
|
||||
|
||||
disksrc->src->caps = {
|
||||
"audio/mp3",
|
||||
"layer", GST_CAPS_INT (3),
|
||||
"bitrate", GST_CAPS_INT (128),
|
||||
NULL
|
||||
};
|
||||
|
||||
Look through the plugin registry, find mpg123. Instantiate it, attach it. It spits out audio
|
||||
parameters as usual:
|
||||
|
||||
mpg123->src->caps = {
|
||||
"audio/raw",
|
||||
"format", GST_CAPS_BITFIELD (S16),
|
||||
"depth", GST_CAPS_INT (16),
|
||||
"rate", GST_CAPS_INT (44100),
|
||||
"channels", GST_CAPS_INT (2),
|
||||
NULL
|
||||
};
|
||||
|
||||
Now we instantiate an audiosink plugin. This time, we're sunk:
|
||||
|
||||
mpg123->src->caps = {
|
||||
"audio/raw",
|
||||
"format", GST_CAPS_BITFIELD (S8,U8),
|
||||
"depth", GST_CAPS_INT (8),
|
||||
"rate", GST_CAPS_INT_RANGE (11025, 22050),
|
||||
"channels", GST_CAPS_INT (1),
|
||||
NULL
|
||||
};
|
||||
|
||||
ACK! It's one of those Disney Sound Source things. We've got a problem here that isn't obviously
|
||||
solvable. However, there happens to be another mp3 decoder sitting around. It's got the same
|
||||
properties as mpg123, but a lower merit value. Let's instantiate one and attach it. We get the
|
||||
following output pad caps:
|
||||
|
||||
mp3decoder->src->caps = {
|
||||
"audio/raw",
|
||||
"format", GST_CAPS_BITFIELD (S8,S16),
|
||||
"depth", GST_CAPS_INT_RANGE (8,16),
|
||||
"rate", GST_CAPS_INT_RANGE (8000, 44100),
|
||||
"channels", GST_CAPS_INT (1,2),
|
||||
NULL
|
||||
};
|
||||
|
||||
Well, that matches the audiosink. We try attaching it, and during negotiation the mp3decoder finds
|
||||
sufficient common ground with the castrated audiosink and sets its output pad to match the best of the
|
||||
options: S8 at 22050 KHz.
|
||||
|
||||
|
||||
|
||||
Next to impossible scenario: DVD
|
||||
--------------------------------
|
||||
|
||||
Start with a dvdsrc. It's output pad caps are:
|
||||
|
||||
static GstCapsFactory dvdsrc_src_caps = {
|
||||
"video/mpeg",
|
||||
"mpegversion", GST_CAPS_INT (2),
|
||||
"systemstream", GST_CAPS_BOOLEAN (TRUE),
|
||||
NULL
|
||||
};
|
||||
|
||||
The type would be classified as incomplete via some mechanism. This might cause the autoplug code to go
|
||||
and run the typefind function. It would flesh the type out to the following:
|
||||
|
||||
dvdsrc->src->caps = {
|
||||
"video/mpeg",
|
||||
"mpegversion", GST_CAPS_INT (2),
|
||||
"systemstream", GST_CAPS_BOOLEAN (TRUE),
|
||||
"videostreams", GST_CAPS_INT (1),
|
||||
"audiostreams", GST_CAPS_INT (3),
|
||||
"bitrate", GST_CAPS_INT (40960),
|
||||
NULL,
|
||||
};
|
||||
|
||||
Wow, that helped a lot. A check through the plugin registry shows that the mpeg2parse will match those
|
||||
properties:
|
||||
|
||||
static GstCapsFactory mpeg2parse_sink_caps = {
|
||||
"video/mpeg",
|
||||
"mpegversion", GST_CAPS_INT (2),
|
||||
"systemstream", GST_CAPS_BOOLEAN (TRUE),
|
||||
NULL
|
||||
};
|
||||
|
||||
(In retrospect, it may not be necessary to run typefind if there's match this good right away. Only run
|
||||
typefind when there's no exact match.)
|
||||
Since there are no output pads yet, we have to actually push data through the pipeline. The moment a
|
||||
buffer or two get to the mpeg2parse element, it promptly goes and creates an output pad, probably of the
|
||||
following caps:
|
||||
|
||||
mpeg2parse_video_src_caps = {
|
||||
"video/mpeg",
|
||||
"mpegversion", GST_CAPS_RANGE (1,2),
|
||||
"systemstream", GST_CAPS_BOOLEAN (FALSE),
|
||||
NULL
|
||||
};
|
||||
|
||||
This seems to be a task for typefind again. But since data is flowing, we have to be careful with the
|
||||
buffers. (This is the case in any typefind maneuver, but moreso when one really can't rewind the
|
||||
source without consequences) The autoplug system attaches a special pseudo-element to mpeg2parse's new
|
||||
output pad, and attaches the typefind element to the end of that. The pseudo-element takes the buffer,
|
||||
stores it, and passes a copy off to the attached element, in this case typefind. This repeats until
|
||||
typefind has determined the type, at which point the typefind is removed, and the newly found element is
|
||||
attached instead.
|
||||
|
||||
The pseudo-element is 'rewound' and the stored buffers flow out and into the newly attached element.
|
||||
When the cache of buffers is gone, a signal fires and the autoplug system removes the pseudo-element and
|
||||
reconnects the pipeline.
|
||||
|
||||
In this case, the typefind function will find the following:
|
||||
|
||||
mpeg2parse_video_src_caps = {
|
||||
"video/mpeg",
|
||||
"mpegversion", GST_CAPS_INT (2),
|
||||
"systemstream", GST_CAPS_BOOLEAN (FALSE),
|
||||
"bitrate", GST_CAPS_INT (36864),
|
||||
"width", GST_CAPS_INT (720),
|
||||
"height", GST_CAPS_INT (480),
|
||||
"framerate", GST_CAPS_FLOAT (29.97003),
|
||||
"chromaformat", GST_CAPS_INT (1), [GST_CAPS_STRING ("4:2:0") ?]
|
||||
NULL
|
||||
};
|
||||
|
||||
Back to the plugin registry, we find our only choice is mpeg2dec, which has input caps of:
|
||||
|
||||
static GstCapsFactory mpeg2dec_sink_caps = {
|
||||
"video/mpeg",
|
||||
"mpegversion", GST_CAPS_RANGE (1,2),
|
||||
"systemstream", GST_CAPS_BOOLEAN (FALSE),
|
||||
NULL
|
||||
};
|
||||
|
||||
Once again it just so happens that we really didn't need to do the typefind at all. But it can't hurt
|
||||
unless the typefind is slow and painful, which we can guess won't be the case since the choices are
|
||||
rather limited by the fact that there's already a MIME type attached, meaning we can drastically reduce
|
||||
the number of typefind functions we try (down to one, actually).
|
||||
|
||||
However, since we *have* run the typefind, upon attachment of the input pad of mpeg2dec, the output pad
|
||||
looks like the following:
|
||||
|
||||
mpeg2dec_src_caps = {
|
||||
"video/raw",
|
||||
"fourcc", GST_CAPS_LIST (
|
||||
GST_CAPS_FOURCC ("YV12"), [identical...]
|
||||
GST_CAPS_FOURCC ("IYUV"),
|
||||
GST_CAPS_FOURCC ("I420"),
|
||||
),
|
||||
"width", GST_CAPS_INT (720),
|
||||
"height", GST_CAPS_INT (480),
|
||||
"framerate", GST_CAPS_FLOAT (29.97003),
|
||||
NULL
|
||||
};
|
||||
|
||||
Currently only videosink supports the output of video/raw. It claims a list of FOURCCs but nothing
|
||||
more:
|
||||
|
||||
static GstCapsFactory videosink_sink_caps = {
|
||||
"video/raw",
|
||||
"fourcc", GST_CAP_LIST ( GST_CAPS_FOURCC ("YV12"),
|
||||
GST_CAPS_FOURCC ("IYUV"), GST_CAPS_FOURCC ("I420"),
|
||||
GST_CAPS_FOURCC ("YUY2"), GST_CAPS_FOURCC ("UYVY"),
|
||||
[ etc... ],
|
||||
),
|
||||
NULL
|
||||
};
|
||||
|
||||
When instantiated, we potentially have the same problem as with the audiosink: we don't necessarily know
|
||||
which hardware output to use. Somehow we have to solve the problem of setting some element arguments
|
||||
before we can get useful information out of them as to the properties. In this case anyway, if the
|
||||
videosink were to find only one output possibility, it would trim the list of FOURCCs it can deal with
|
||||
to what the hardware can handle, as well as add further properties:
|
||||
|
||||
videosink_sink_caps = {
|
||||
"video/raw",
|
||||
"fourcc", GST_CAPS_LIST (GST_CAPS_FOURCC ("YV12"),
|
||||
GST_CAPS_FOURCC ("YUY2"),
|
||||
),
|
||||
"width", GST_CAPS_INT_RANGE (4,1020),
|
||||
"height", GST_CAPS_INT_RANGE (4,1020),
|
||||
NULL
|
||||
};
|
||||
|
||||
We can now connect the mpeg2dec output to the videosink, and we now have displaying video.
|
||||
|
||||
. . . .
|
20
docs/random/omega/caps3
Normal file
20
docs/random/omega/caps3
Normal file
|
@ -0,0 +1,20 @@
|
|||
/* fake mp3 where bitrates 8 through 32 can't be stereo */
|
||||
static GstCapsListFactory mpg123_sink_caps = {
|
||||
{
|
||||
"audio/mp3",
|
||||
"layer", GST_CAPS_INT_RANGE (1, 3),
|
||||
"bitrate", GST_CAPS_INT_RANGE (32, 320),
|
||||
"channels", GST_CAPS_INT_RANGE (1, 2),
|
||||
"framed", GST_CAPS_BOOLEAN (TRUE),
|
||||
NULL
|
||||
},
|
||||
{
|
||||
"audio/mp3",
|
||||
"layer", GST_CAPS_INT_RANGE (1, 3),
|
||||
"bitrate", GST_CAPS_INT_RANGE (8, 32),
|
||||
"channels", GST_CAPS_INT_RANGE (1),
|
||||
"framed", GST_CAPS_BOOLEAN (TRUE),
|
||||
NULL
|
||||
},
|
||||
NULL
|
||||
};
|
30
docs/random/omega/debug-commit
Normal file
30
docs/random/omega/debug-commit
Normal file
|
@ -0,0 +1,30 @@
|
|||
Changes made to the DEBUG system. New header file gstdebug.h holds the stuff to keep it out of gst.h's
|
||||
hair. DEBUG prints out the process id, cothread id, source filename and line number. Two new macros
|
||||
DEBUG_ENTER and DEBUG_LEAVE are used to show the entry and exit of a given function. This eventually
|
||||
might be used to construct call trace graphs, even taking cothreads into account. This would be quite
|
||||
useful in visualizing the scheduling mechanism.
|
||||
|
||||
Minor changes to various debug messages.
|
||||
|
||||
Also sitting in gstdebug.h is a prototypical DEBUG_ENTER that's capable of performing DEBUG_LEAVE
|
||||
automatically. It does this by utilizing a little-known GCC extension that allows one to call a
|
||||
function with the same parameters as the current function. The macro uses this to basically call
|
||||
itself. A boolean is used to ensure that when it calls itself it actually runs the body of the
|
||||
function. In the meantime it prints stuff out before and after the real function, as well as
|
||||
constructing a debugging string. This can be used eventually to provide call-wide data on the DEBUG
|
||||
lines, instead of having to replicate data on each call to DEBUG. More research is needed into how this
|
||||
would most cleanly be fit into some other chunk of code, like GStreamer (I think of this DEBUG trick as
|
||||
a separate project, sorta).
|
||||
|
||||
Unfortunately, the aforementioned DEBUG trick interacts quite poorly with cothreads. Almost any time
|
||||
it's used in a function that has anything remotely to do with a cothread context (as in, it runs in
|
||||
one), a segfault results from the __builtin_apply call, which is the heart of the whole thing. If
|
||||
someone who really knows assembly could analyze the resulting code to see what's really going on, we
|
||||
might find a way to fix either the macro or the cothreads (I'm thinking that there's something we missed
|
||||
in constructing the cothreads themselves) so this works in all cases.
|
||||
|
||||
In the meantime, please insert both DEBUG_ENTER and DEBUG_LEAVE in your functions. Be sure to put
|
||||
DEBUG_ENTER after your variable declarations and before any functional code, not to put the function
|
||||
name in any DEBUG strings (it's already there, trust me), and put a DEBUG_LEAVE if you care enough.
|
||||
|
||||
Changes are going to happen in the way DEBUGs and other printouts occur, so stay tuned.
|
48
docs/random/omega/eos
Normal file
48
docs/random/omega/eos
Normal file
|
@ -0,0 +1,48 @@
|
|||
What I propose for EOS condition is th following: As is stands, EOS is a
|
||||
flag in a buffer. However, it's also a signal from an element, which is
|
||||
odd. What I propose is that EOS become a pad thing. Here's how it would
|
||||
work:
|
||||
|
||||
When a source has a final buffer, it marks it as EOS. It passes this on
|
||||
through the pad, and the pads set a 'pending' EOS state on the way
|
||||
through. When another push or pull happens, only then does the pad signal
|
||||
EOS. This means that EOS doesn't happen until the buffer has passed all
|
||||
the way to the end, and a request for another buffer is starting to work
|
||||
its way back. It gets stopped almost immediately.
|
||||
|
||||
This gets a bit messy in complex cases, haven't thought them all out.
|
||||
Assuming everything is reasonably symmetric, it should work out cleanly.
|
||||
We may have to add the ability to force buffers to pass even post EOS, but
|
||||
that sounds like a hack.
|
||||
|
||||
|
||||
|
||||
Example: DVD
|
||||
============
|
||||
|
||||
|
||||
------------ |-------|
|
||||
-| mpeg2dec |-----| |
|
||||
------------------------ / ------------ | |
|
||||
| video0|------/ ------------- | | -------------
|
||||
| | -| subtitle0 |-----| merge |-------| videosink |
|
||||
| subtitle0|------/ ------------- | | -------------
|
||||
---------- | | ------------- | |
|
||||
| dvdsrc |---| mpeg2parse subtitle1|--------| subtitle1 |-----| |
|
||||
---------- | | ------------- |-------|
|
||||
| audio0|--------\ ----------- |----------|
|
||||
| | -| ac3dec0 |-----| |
|
||||
| audio1|-------\ ----------- | | -------------
|
||||
------------------------ \ ----------- | switcher |----| audiosink |
|
||||
-| ac3dec1 |-----| | -------------
|
||||
----------- |----------|
|
||||
|
||||
When dvdsrc reads its last buffer, it sets EOS on the buffer and passes it. The pad (probably audio1,
|
||||
since video and audio are interleaved and subtitles don't usually end up at the end of a movie) would
|
||||
set EOS pending, and pass it to mpeg2dec which would pass the EOS to the merge. Merge would then ask
|
||||
for a buffer from subtitle1, since that's the one we're showing subtitles on right now. It would then
|
||||
pull from the mpeg2parse. This would cause a switch back the parser, who's next task is to go and set
|
||||
all the output pads to EOS. It may have to accomplish this by sending a zero-byte buffer.
|
||||
|
||||
Well, this example isn't actually very good. I'll go ahead and implement it, and we can see what
|
||||
happens. The current setup is so lacking that anything is better....
|
3
docs/random/omega/filterfactory
Normal file
3
docs/random/omega/filterfactory
Normal file
|
@ -0,0 +1,3 @@
|
|||
A very useful feature would be a FilterFactory, i.e. some system whereby filter writers can avoid
|
||||
getting their hands messy with the details of building and operating a G[tk]Object. A couple of
|
||||
structures and a couple of functions should do it.
|
61
docs/random/omega/output_policies
Normal file
61
docs/random/omega/output_policies
Normal file
|
@ -0,0 +1,61 @@
|
|||
GStreamer polices for various forms of library output, including error cases.
|
||||
|
||||
|
||||
g_return_...
|
||||
============
|
||||
The parameters of a function are checked for validity (non-NULL, correct type, etc.) with g_return[_val]_if_fail, i.e.:
|
||||
|
||||
gst_element_connect (GstElement *src, gchar *srcpadname,
|
||||
GstElement *dest, gchar *destpadname) {
|
||||
g_return_if_fail (src != NULL);
|
||||
g_return_if_fail (GST_IS_ELEMENT(src));
|
||||
g_return_if_fail (srcpadname != NULL);
|
||||
g_return_if_fail (dest != NULL);
|
||||
g_return_if_fail (GST_IS_ELEMENT(dest));
|
||||
g_return_if_fail (destpadname != NULL);
|
||||
|
||||
This will inform the user of the library of any basic problems with the arguments they pass, such as a NULL pointer.
|
||||
|
||||
|
||||
ERROR
|
||||
=====
|
||||
The ERROR macro will be used whenever there is some other type of flaw in the data passed, at a GStreamer-specific
|
||||
level:
|
||||
|
||||
srcpad = gst_element_get_pad (src, srcpadname);
|
||||
if (srcpad == NULL) {
|
||||
ERROR(src,"source element has no pad \"%s\"",srcpadname);
|
||||
return;
|
||||
}
|
||||
|
||||
An ERROR will generally cause ceasation of the application, and ideally launch gdb.
|
||||
|
||||
|
||||
INFO
|
||||
====
|
||||
The INFO macro will be used to output any interesting state from the library, such as plugin loading and various events
|
||||
of interest. They will be separated into categories that can be individually enabled and disabled.
|
||||
|
||||
Categories (first draft of list, unordered):
|
||||
|
||||
cothreads [cothreads.c]
|
||||
gst initialization [gst.c]
|
||||
autoplug results [autoplug.c]
|
||||
autoplug attempts [autoplug.c]
|
||||
bin parentage issues [gstbin.c]
|
||||
plan generation [gstbin.c]
|
||||
schedule generation [gstschedule.c]
|
||||
schedule implementation [gstpad.c, gstbin.c, etc.]
|
||||
buffer operations [gstbuffer.h]
|
||||
caps matching [gstcaps.c]
|
||||
clock stuff [gstclock.c]
|
||||
element pad operations [gstelement.c]
|
||||
elementfactory operations [gstelementfactory.c]
|
||||
pad creation/connection [gstpad.c]
|
||||
pipeline stuff [gstpipeline.c]
|
||||
plugin loading [gstplugin.c]
|
||||
plugin loading errors [gstplugin.c]
|
||||
properties operations [gstprops.c]
|
||||
thread creation/management [gstthread.c]
|
||||
type operations [gsttype.c]
|
||||
XML load/save [gstxml.c]
|
37
docs/random/omega/pad-negotiation
Normal file
37
docs/random/omega/pad-negotiation
Normal file
|
@ -0,0 +1,37 @@
|
|||
When two pads are connected, a negotiation phase is going to have to
|
||||
happen. Ideally, the caps of the two pads will both be fully-specified,
|
||||
and match. That's the ideal case, but may rarely happen in practice.
|
||||
|
||||
It'll work the following way:
|
||||
|
||||
1) gst_pad_connect(pad1,pad2) is called by something
|
||||
2) pad1's negotiate() method is called, with pad2 as argument
|
||||
3) negotiate() repeatedly calls pad2's set_caps() method
|
||||
|
||||
At some point, the two pads will agree on a set of caps, and proceed by
|
||||
returning TRUE from negotiate(), at which point gst_pad_connect()
|
||||
finishes. If it returns FALSE, gst_pad_connect() is forced to fail.
|
||||
|
||||
Now, obviously the algorithm used to find matching caps can get
|
||||
complicated. But in some cases it'll be simple. Ideally, if there is no
|
||||
negotiate() function for pad1, there'll be a function that will go through
|
||||
the options and try to make pad1 and pad2 meet in the middle, with no
|
||||
specific knowledge of what the caps actually mean.
|
||||
|
||||
Another detail is deciding which pads are pad1 and pad2. In the autoplug
|
||||
case, the code will sometimes know which of the pads have the more
|
||||
specific caps. In others, you may not. Either you can guess, and
|
||||
possibly lose to having the slower of the two pad's negotiate() methods do
|
||||
the work, or you might be able to actually guess at which is the most
|
||||
specific set of caps:
|
||||
|
||||
For any given typeid in the union of both pads, look at all properties
|
||||
For each property, find the smallest range, assign this a 1.0
|
||||
For all other instances of this property, assign relative to 1.0
|
||||
For each pad1,pad2
|
||||
Take the assigned value of every property, and multiply together
|
||||
|
||||
Whichever value is lower between pad1 and pad2 is most likely to be the
|
||||
most specific set of caps. The trick is implementing the above
|
||||
efficiently, but on the surface there appear to be more than enough
|
||||
potential optimizations.
|
43
docs/random/omega/padtemplates
Normal file
43
docs/random/omega/padtemplates
Normal file
|
@ -0,0 +1,43 @@
|
|||
typedef gpointer GstCapsFactoryEntry;
|
||||
typedef GstCapsFactoryEntry GstCapsFactory[];
|
||||
typedef GstCapsFactory *GstCapsListFactory[];
|
||||
|
||||
typedef gpointer GstPadFactoryEntry;
|
||||
typedef GstPadFactoryEntry GstPadFactory[];
|
||||
typedef GstPadFactory *GstPadListFactory[];
|
||||
|
||||
#define GST_PADFACTORY_SRC GINT_TO_POINTER (GST_PAD_SRC)
|
||||
#define GST_PADFACTORY_ALWAYS GINT_TO_POINTER (GST_PAD_ALWAYS)
|
||||
|
||||
typedef struct GstPadTemplate {
|
||||
gchar *name_template;
|
||||
gint direction;
|
||||
gint presence;
|
||||
GList *caps;
|
||||
};
|
||||
|
||||
GstPadFactory mpg123_src_padfactory = {
|
||||
"src"
|
||||
GST_PADFACTORY_SRC,
|
||||
GST_PADFACTORY_ALWAYS,
|
||||
|
||||
"audio/raw"
|
||||
"samplerate", GST_PROPS_INT (44100),
|
||||
. . .
|
||||
NULL
|
||||
};
|
||||
|
||||
static GstPadTemplate *srcpadtemplate;
|
||||
|
||||
mpg123_new(GstMpg123 *mpg123) {
|
||||
mpg123->srcpad = gst_pad_new_template("src", srcpadtemplate);
|
||||
. . .
|
||||
}
|
||||
|
||||
plugin_init() {
|
||||
GstElementFactory *factory;
|
||||
|
||||
factory = gst_elementfactory_new("mpg123",. . .);
|
||||
srcpadtemplate = gst_padfactory_new(mpg123_src_padfactory);
|
||||
get_elementfactory_add_padtemplate (srcpadtemplate);
|
||||
}
|
84
docs/random/omega/plan-generation
Normal file
84
docs/random/omega/plan-generation
Normal file
|
@ -0,0 +1,84 @@
|
|||
Plan generation happens at transition from NULL to READY (and PLAYING to READY right now, need to fix
|
||||
that). By way of some logic in gst_bin_change_state(), gst_bin_create_plan() is only called for the
|
||||
outer Bin, usually a Pipeline. This keeps things from getting nasty later on.
|
||||
|
||||
A major new concept in plan generation is that of the 'manager'. This is the element that is reponsible
|
||||
for running a given element. In general, Pipelines and Threads are the only managing-capable elements
|
||||
(have the MANAGER flag set), since they are the only ones with real scheduling authority (because they
|
||||
have a process context to play with, basically).
|
||||
|
||||
gst_bin_set_manager() is called to set the manager element of the bin and all it's children and their
|
||||
children. However, there's one important trick: it won't recurse into child Bins that have the MANAGER
|
||||
flag set. This avoids some highly redundant recursion.
|
||||
|
||||
When create_plan() is called on the outside Pipeline, the first thing it does is call
|
||||
set_manager(self,self). As noted above, this recursion will not proceed into child Bins that have the
|
||||
MANAGER flag set.
|
||||
|
||||
The next step is to recursively generate the plan (yes, head-recursive). This gives child Bins the
|
||||
opportunity to generate their plan first, causing a inside-to-outside sequence. This matches the way
|
||||
the scheduling is arranged now, where the plan for a Src/Connection outside a Bin is handled by that
|
||||
Bin, not it's parent. But we must be very careful not to stomp on that plan in the parent Bin.
|
||||
|
||||
Because create_plan() is called on all Bins, but we can only set up scheduling state in MANAGER bins,
|
||||
create_plan() must perform create_plan() recursion, but not do anything else *unless* the MANAGER bit is
|
||||
set. It shouldn't even call set_manager() unless it's a MANAGER itself, because calling it otherwise
|
||||
would waste time doing the work again. Basically, from the standpoing of setting the manager,
|
||||
create_plan() recursion starts it when the current Bin is a MANAGER, and set_manager() stops when it
|
||||
finds the next one. create_plan()'s further recursion eventually starts the process back up again
|
||||
furtuer down the hierarchy, until everything is covered.
|
||||
|
||||
For all MANAGER Bins, the last step is to actually create the scheduling plan. This is still one of the
|
||||
nastiest chunks of code in the whole project, and probably will do nothing but get worse from now on (it
|
||||
got better recently, but only because I took a chainsaw to the code and broke everthing...). It will
|
||||
remain similar to what it is now, but with some definite differences.
|
||||
|
||||
First task is now to find all the elements that we're responsible for. This is normally a recursive
|
||||
process, because the structure is an arbitrary tree. However, something like the following should work
|
||||
(bin is self):
|
||||
|
||||
GSList *elements = NULL;
|
||||
GList *children;
|
||||
GSList *waiting_bins = NULL;
|
||||
GstBin *waiting_bin;
|
||||
|
||||
waiting_bins = g_slist_prepend (waiting_bins,bin);
|
||||
|
||||
while (waiting_bins) {
|
||||
// retrieve the top of the stack and pop it
|
||||
waiting_bin = GST_BIN (waiting_bins->data);
|
||||
waiting_bins = g_slist_remove (waiting_bins,waiting_bin);
|
||||
|
||||
// walk the list of elements, and find bins
|
||||
children = waiting_bin->children;
|
||||
while (children) {
|
||||
// add it to the list of elements
|
||||
elements = g_slist_prepend (elements, children->data);
|
||||
|
||||
// if it's a bin and it's not a managing bin,
|
||||
// shove it on the list of bins to recurse into
|
||||
if (GST_IS_BIN (children->data) &&
|
||||
!GST_FLAG_IS_SET (GST_ELEMENT (children->data)))
|
||||
waiting_bins = g_slist_prepend (waiting_bins,children->data);
|
||||
|
||||
children = g_list_next (children);
|
||||
}
|
||||
}
|
||||
|
||||
The code makes the assumption that the manager of every element is the same until such time as a
|
||||
different managing parent appears in the hierarchy. This is the result of the aforementioned nested
|
||||
recursion of create_plan() and set_manager(), but may not remain the case forever. The above loop
|
||||
should probably be slightly re-written to work solely on whether or not the Bin in question is the
|
||||
element's manager. This means that the child Bins are *always* recursed into, in case there's a rogue
|
||||
element inside of one of them that's supposed to be managed.
|
||||
|
||||
At the same time all the elements managed by this bin are found (i.e. in the inner loop), we can
|
||||
determine some useful bits of information, such as testing for several cases that require the use of
|
||||
cothreads. The availability of manager information at this point may aid significantly in this
|
||||
decision.
|
||||
|
||||
Finally, the scheduling plan is generated, based on all the elements to be managed by the Bin (the list
|
||||
of which may span several 'generations' of Bins and elements). Elements which have peers in child
|
||||
self-managed Bins are left alone on for the pad in that makes that connection. This should keep the
|
||||
parent Bins from stepping all over state set up by the child Bins, by establishing clear implicit
|
||||
ownership on the pad level, based on the managing Bins' relationship to the pad.
|
20
docs/random/omega/sched-case
Normal file
20
docs/random/omega/sched-case
Normal file
|
@ -0,0 +1,20 @@
|
|||
Case 1:
|
||||
|
||||
|
||||
---------------------------------------------------
|
||||
| pipeline |
|
||||
| --------------- ---------------- |
|
||||
| | bin | | thread | |
|
||||
| | ----------- | ------------ | ------------ | |
|
||||
| | | fakesrc | | | queue | | | fakesink | | |
|
||||
| | | src>|-|--|<sink src>|--|-|<sink | | |
|
||||
| | ----------- | ------------ | ------------ | |
|
||||
| --------------- ---------------- |
|
||||
---------------------------------------------------
|
||||
|
||||
Pipeline manages: fakesrc, queue
|
||||
Thread manages: fakesink
|
||||
Both forced to use cothreads.
|
||||
|
||||
First thing the thread does is try to pull from the queue. Because it's a chain function, it runs in
|
||||
_chain_wrapper, which calls gst_pad_pull().
|
100
docs/random/omega/sched-commit1
Normal file
100
docs/random/omega/sched-commit1
Normal file
|
@ -0,0 +1,100 @@
|
|||
Changed the way things are scheduled, especially sources. A Src used to
|
||||
have a push() function, and optionally a pushregion() to deal with async
|
||||
reads, etc. That whole thing has gone away, in favor of providing a
|
||||
pull() function for the output (Src) pad instead, ala chain functions.
|
||||
This makes constructing cothreaded schedules out of non-loop elements
|
||||
somewhat easier. Basically there was always a question as to which pad
|
||||
was being dealt with. In the pullregion case, cothread-specific data was
|
||||
used to try to pass the region struct to the right place, which is a slow
|
||||
hack. And in general, the push function severely limited the kind of
|
||||
tricks that could be played when there's more than one output pad, such as
|
||||
a multi-out file reader with async capabilities on each pad independently.
|
||||
|
||||
This changes the way cothread scheduling occurs. Instead of the hack to
|
||||
deal with Src's by calling their push() function (or optionally the
|
||||
pushregion(), in certain cases), we now are working towards a general
|
||||
mechanism where pads are the only thing that are dealt with directly.
|
||||
|
||||
An optimization was made in the process of doing this: the loopfunction
|
||||
actually run as the outer [stack] frame of the cothread is now set more
|
||||
intelligently in create_plan() based on what kind of element it is. We
|
||||
now have:
|
||||
|
||||
loopfunc_wrapper: used for loop-based elements, it simply calls the
|
||||
loopfunc in a loop, paying attention to COTHREAD_STOPPING (see
|
||||
below). It currently does other, soon to be depracated, stuff.
|
||||
|
||||
pullsrc_wrapper: wraps a Src that's not loop-based (since your options
|
||||
are now loop- or pull-based)
|
||||
|
||||
There will be a couple more to deal with other cases, such as Connections
|
||||
and chain-based elements. The general idea is that it's a lot more
|
||||
efficient to make the decisions once in create_plan than to keep doing
|
||||
this huge if/else chain in the wrapper. Just choose the right wrapper up
|
||||
front. It'll be most apparent performance-wise in the case of whichever
|
||||
element context is switched to first for each iteration, since the whole
|
||||
wrapper setup is done for every iteration.
|
||||
|
||||
The tricky part is that there is now a bit of overloading of the function
|
||||
pointers in a pad. The current meanings (possibly to change a bit more
|
||||
soon) are:
|
||||
|
||||
chainfunc: as always, chainfunc pointer is mirrored between peer pads
|
||||
(this may change, and the chain func may end up in pushfunc)
|
||||
pushfunc: SrcPad: gst_pad_pushfunc_proxy, cothread_switch to peer
|
||||
SinkPad: none (may take over chainfunc, see below) pullfunc:
|
||||
SrcPad: Src or Connection's function to construct buffers
|
||||
SinkPad: gst_pad_pullfunc_proxy, cothread_switch to peer
|
||||
|
||||
There are a number of issues remaining with the scheduling, not the least
|
||||
of which is the fact that Connections are still dealt with the old way,
|
||||
with _push() functions and such. I'm trying to figure out a way to unify
|
||||
the system so it makes sense. Following the scheduling system is hard
|
||||
enough, trying to change it is murder.
|
||||
|
||||
|
||||
Another useful scheduling addition, mentioned above, is COTHREAD_STOPPING.
|
||||
It's an element flag that's used to signal whatever code is running in
|
||||
cothread context that it should be finishing up and exiting soon. An
|
||||
example of this is in plugins/cobin/spindentity.c. All the loops should
|
||||
now be composed of do/while loops, rather than while(1) loops:
|
||||
|
||||
do {
|
||||
buf = gst_pad_pull(spindentity->sinkpad);
|
||||
gst_pad_push(spindentity->srcpad,buf);
|
||||
} while (!GST_ELEMENT_IS_COTHREAD_STOPPING(element));
|
||||
|
||||
The reason for this is that COTHREAD_STOPPING may be set before the above
|
||||
loop ever gets started. It wouldn't do for the body of the loop to never
|
||||
once get called, that would simply stall the pipeline. Note that only the
|
||||
core library code is ever responsible for setting and unsetting this flag.
|
||||
All elements have to do is respond to it by cleanly exiting the loop and
|
||||
the function holding it.
|
||||
|
||||
This is needed primarily to allow iterations to occur properly.
|
||||
Basically, there's a single entry point in the cothread scheduling loop,
|
||||
gst_bin_iterate_func() simply switches to this cothread. If the element
|
||||
in this context is allowed to loop infinitely, nothing would even switch
|
||||
back to the context from which the iterate() was originally called. This
|
||||
is a bit of a problem. The solution is for there to be an implicit switch
|
||||
back to the originating context. Now, even I'm not sure exactly how this
|
||||
works, but if the cothread that's switched to actually returns, execution
|
||||
returns back to the calling context, i.e. iterate_func().
|
||||
|
||||
COTHREAD_STOPPING is therefore set just before switching into this
|
||||
(currently randomly chosen) context, on the assumption that it will return
|
||||
promptly after finishing its duties. The burden of clearing the flag
|
||||
falls to the various wrapper functions provided by the Bin code, thus
|
||||
element writers don't have to worry about doing that at all (and simply
|
||||
shouldn't).
|
||||
|
||||
|
||||
Related changes:
|
||||
All the sources in elements/ have been changed to reflect the new system.
|
||||
|
||||
|
||||
FIXMEs:
|
||||
1) gstpipeline.c calls gst_src_push at some point, dunno why, it's
|
||||
commented out now.
|
||||
2) any other sources, including vcdsrc, dvdsrc, and v4lsrc will break
|
||||
badly and need to be modified to work as pull-based sources.
|
89
docs/random/omega/sched/chains
Normal file
89
docs/random/omega/sched/chains
Normal file
|
@ -0,0 +1,89 @@
|
|||
A new concept in scheduling is that of chains of elements that need to be schedule separately, even in the same set of
|
||||
managed elements (which is the set of elements that the Bin in question [a pipeline or thread] is responsible for).
|
||||
An example would by anywhere you have a non-blocking queue in place for buffering. This kind of element might be
|
||||
useful in cases where the scheduling on a buffer level is tight enough that deadlocks might occur.
|
||||
|
||||
The scheduler will find chains by traversing the pipeline through the list of managed elements. A chain boundary is
|
||||
anywhere you have a 'DECOUPLED' element. A DECOUPLED element is one where there is no direct correlation between the
|
||||
activities of the various pads. A source fits this description, although the normal single-pad source is the
|
||||
degenerate case. A queue more properly fits the bill, since pushing a buffer at the sink pad doesn't trigger anything
|
||||
on the src pad, and vice versa. A multi-src async source is probably the best example, since you want to leave the
|
||||
scheduling up to the elements connected to it.
|
||||
|
||||
Anyway, first the simple case:
|
||||
|
||||
fakesrc -> fakesink
|
||||
|
||||
Both of them should probably have the DECOUPLED bit set, at least to be true to the nature of the actual fake elements.
|
||||
These two end up being a chain, and scheduling has to be set up for the chain. There are no cothreaded elements in the
|
||||
chain, which means it's relatively easy. The goal is to find a single entry into the chain, which can be called in a
|
||||
loop to get things done. Since the fakesrc is DECOUPLED, and we'd be messing with the source pad, it has lower
|
||||
priority than a DECOUPLED sink pad, so the fakesrc's sink pad is the ideal entry into the chain. This can be
|
||||
simplified into saying that the fakesink is the entry.
|
||||
|
||||
In the end, the code to do this boils down to:
|
||||
|
||||
buf = gst_pad_pull (fakesink->sinkpad);
|
||||
gst_pad_push (fakesink->sinkpad, buf);
|
||||
|
||||
Because of the way things are no implemented for scheduling, turning it around and making the source the entry has no
|
||||
effect as far as the efficiency. That's because _get no longer directly calls gst_pad_push(), so we have to do it
|
||||
outside. No big deal, it boils down to the same thing I think, modulo a cache-line of stack (i.e. one or two fewer
|
||||
this way).
|
||||
|
||||
If we put an identity in the middle:
|
||||
|
||||
fakesrc -> identity -> fakesink
|
||||
|
||||
then we have the same thing, except that there's now an element that isn't DECOUPLED, so it gets higher priority. That
|
||||
means the identity is now the entry, and when we push the buffer into its chain function, the fakesink gets called.
|
||||
|
||||
Now, we can make this much more complex, with the following elementary echo meta-filter:
|
||||
|
||||
|=====| -> delay1 -> |=====|
|
||||
| | | |
|
||||
-> queue -> | tee | -> delay2 -> | mix | -> queue ->
|
||||
| | | |
|
||||
|=====| -> delay3 -> |=====|
|
||||
|
||||
The tee takes a buffer in and spits three out, delay shifts the timestamps around and possibly reframes things to be
|
||||
friendly. mix takes the three buffers and simply sums them (they're all audio). The tee element takes one buffer in
|
||||
and promptly spits three out, one after another. Delay takes an element and immediately spits out a buffer (it
|
||||
zero-pads at the beginning [the duration of the delay] for the sake of argument). Mix in this case is chained, but
|
||||
assumes that buffer will arrive in order. On the last chain, it does a push of the newly mixed audio buffer.
|
||||
|
||||
The queues are both DECOUPLED, so they have lower weight. That leaves a bunch of other elements sitting there ripe for
|
||||
entry. But if we were to take delay1, what would happen? Well we can't, since there's no _get function on the tee's
|
||||
src pads.
|
||||
|
||||
This just re-enforces the idea that the left-most (closest to the source, for you right-to-left people) element should
|
||||
get to be the entry. But what if we have multiple left-most elements?:
|
||||
|
||||
-> queue -> eq1 -> |=====|
|
||||
| mix | -> queue
|
||||
-> queue -> eq2 -> |=====|
|
||||
|
||||
If eq1 is the only entry, we call pull on the queue, then chain over to mix. Mix then doesn't do anything with it,
|
||||
since it's waiting for another buffer before doing anything. That means we have to do the same with eq2, and have it
|
||||
chain to mix, at which point mix will do its magic and chain out to the right-hand side. Figure out to actually use
|
||||
both entries is hard, because the idea at this point is that there's only a single entry to a chain.
|
||||
|
||||
Does this mean that we should make mix a DECOUPLED element? That would fix it to some extent, giving us three chains
|
||||
in the above case. Each eq chain would be driven by the eq element, pulling from the queue and pushing into the mixer.
|
||||
The mixer -> queue chain is problematic, because there is no possibly entry. The mixer side has no _get function
|
||||
(since the push always happens upon the receipt of a buffer from the second sink pad), which means that that those two
|
||||
pads have no possible entrance.
|
||||
|
||||
Cothreads make this case much easier, since the mix element would drive things, forcing the eq elements to pull and
|
||||
process buffers in order as needed. It may be that the best option in the case where there are any multi-sinkpad
|
||||
elements is to turn them into cothreads.
|
||||
|
||||
|
||||
Now, on to cothreaded cases. The simplest possible is to turn all the elements into cothreads. I may punt on this and
|
||||
do just that for the moment, but there's still the question of what to do at the ends of the chain, where the DECOUPLED
|
||||
elements are. The easiest is to simply always make then chained, so there's never any worry about who owns the
|
||||
cothread context for the element, simply because there never will be one.
|
||||
|
||||
fakesrc -> queue -> @identity -> fakesink
|
||||
|
||||
We just set it up so both ends of the queue are chained, and all is well.
|
39
docs/random/omega/sched/walkthrough-72
Normal file
39
docs/random/omega/sched/walkthrough-72
Normal file
|
@ -0,0 +1,39 @@
|
|||
72)
|
||||
|
||||
[-pipeline---------------------------------------------------------------------------------------------]
|
||||
! [-bin-----------------------------] [-thread---------] !
|
||||
! ! [--------] [---------] ! [------] [---------] [------] ! [--------] ! !
|
||||
! ! !faksesrc! !identity1! ! !queue1! !identity2! !queue2! ! !fakesink! ! !
|
||||
! ! ! src --- sink * src --- sink n src -- sink src -- sink src -- sink ! ! !
|
||||
! ! [--------] [---------] ! [------] [---------] [------] ! [--------] ! !
|
||||
! [---------------------------------] [----------------] !
|
||||
[------------------------------------------------------------------------------------------------------]
|
||||
|
||||
|
||||
-----
|
||||
Ideally, you'd end up with the following sub-pipelines
|
||||
|
||||
pipeline:
|
||||
fakesrc -> identity1 -> queue1
|
||||
queue1 -> identity2 -> queue2
|
||||
|
||||
thread:
|
||||
queue2 -> fakesink
|
||||
|
||||
|
||||
They'd be scheduled as following:
|
||||
|
||||
fakesrc: passive chained, pulled by identity1
|
||||
identity1: loopfunc cothreaded, ENTRY
|
||||
queue1:sink: passive chained, pushed by identity1
|
||||
|
||||
queue1:src: _get-based iteration, ENTRY
|
||||
identity2: chained by queue1
|
||||
queue2:src: passively chained, pushed by identity2
|
||||
|
||||
queue2:sink: passively chained, pulled by fakesink
|
||||
fakesink: loopfunc cothreaded, ENTRY
|
||||
|
||||
|
||||
-----
|
||||
Most likely, we'd end up with the following
|
13
docs/random/omega/sched2
Normal file
13
docs/random/omega/sched2
Normal file
|
@ -0,0 +1,13 @@
|
|||
Currently, when an element wants to push or pull, they call gst_pad_*. These functions then decide what
|
||||
to do based on the selection of pointers in the pad. There are at least 2 options for each, plus some
|
||||
header stuff to do various checking of flags. Unfortunately, because of the selection of pointers in
|
||||
the pad, there is at least one case where scheduling has to be tricked, by providing a pointer to a
|
||||
function with no body.
|
||||
|
||||
What I propse is that these functionos be replaced with macros that call a function pointer directly.
|
||||
The default functions (provided by GstPad) would be capable of chaining, that's about it. When a
|
||||
schedule is formed, these get replaced with more specific functions, provided by GstBin or a subclass.
|
||||
|
||||
In the chain case, the pad_push_func might even be replaced brute force with the chain function, since
|
||||
they have the same prototype. In the cothreaded case, various functions would provide the ability to
|
||||
switch, deal with bufpens, etc.
|
63
docs/random/omega/scheduling
Normal file
63
docs/random/omega/scheduling
Normal file
|
@ -0,0 +1,63 @@
|
|||
0) definitions:
|
||||
|
||||
All pads without further specifiers are assumed to belong to the element
|
||||
in question. The pad's peer is always denoted with ->peer. If there's
|
||||
question, pads will be prefixed with self-> and other-> as necessary.
|
||||
|
||||
All elements in this document have at most one source and one sink pad,
|
||||
called srcpad and sinkpad. Multi-pad cases are supposed to be simple
|
||||
extrapolations except in a couple strange cases, to be covered elsewhere.
|
||||
|
||||
|
||||
1) loop functions:
|
||||
|
||||
A loop function will call gst_pad_pull(sinkpad), do something, and call
|
||||
gst_pad_push(srcpad).
|
||||
|
||||
gst_pad_pull first checks to see if there's a buffer in the pen. If not,
|
||||
it calls that pad's pullfunc handler, passing it the peer pad. When that
|
||||
finishes, we check again and return the buffer. If no buffer, we squawk.
|
||||
The pullfunc handler simply causes a cothread switch to the peer pad's
|
||||
context;
|
||||
|
||||
gst_pad_push places the buffer in the peer pad's pen, and calls the local
|
||||
pad's pushfunc. The pushfunc simply causes a switch to the peer pad's
|
||||
context.
|
||||
|
||||
|
||||
2) chain functions
|
||||
|
||||
The loopfunc constructed around a chain function starts by finding all
|
||||
sink pads. For each sink pad, it calls gst_pad_pull. This causes a
|
||||
switch to the peer pad's context and a buffer to appear, which is
|
||||
returned. The pad is the handed off to the chain function, which
|
||||
presumably does some processing and calls gst_pad_push(), which causes a
|
||||
context switch to that pad's peer.
|
||||
|
||||
|
||||
|
||||
3) source
|
||||
|
||||
The loopfunc must repeatedly call the srcpad's pull
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
--------------------------------- ---------------------------------
|
||||
srcpad | | sinkpad
|
||||
| |
|
||||
GstPad *peer; | <-p-> | GstPad *peer;
|
||||
pointer to peer pad | | pointer to peer pad
|
||||
| |
|
||||
funcptr *pushfunc; | --s-> | funcptr *pushfunc;
|
||||
causes switch to peer ctx | | element uses buffer in pen,
|
||||
| |
|
||||
funcptr *pullfunc; | <-s-- | funcptr *pullfunc;
|
||||
element puts buffer in pen, | | causes switch to peer ctx
|
||||
calls gst_pad_push() | |
|
||||
| |
|
||||
funcptr *pullregion func | <-s-- | funcptr *pullregionfunc;
|
||||
element puts region in pen | | causes switch to peer ctx
|
||||
calls gst_pad_push() | |
|
||||
--------------------------------- ---------------------------------
|
73
docs/random/omega/type-properties
Normal file
73
docs/random/omega/type-properties
Normal file
|
@ -0,0 +1,73 @@
|
|||
A type properties system might look like following:
|
||||
|
||||
|
||||
1) Type definition includes the properties and their ranges:
|
||||
|
||||
audio/mp3
|
||||
layer: 1 - 3
|
||||
bitrate: 8 - 320
|
||||
|
||||
audio/raw
|
||||
format: bitfield (using asound.h definitions)
|
||||
depth: 8 - 32
|
||||
rate: 4000 - 96000
|
||||
channels: 1 - n
|
||||
interleave: boolean
|
||||
|
||||
video/raw
|
||||
format: 32-bit FOURCC
|
||||
bpp: 1 - 32
|
||||
width: 1 - n
|
||||
height: 1 - n
|
||||
framerate: 32-bit float
|
||||
|
||||
etc.
|
||||
|
||||
|
||||
2) An element can specify what subtypes it can deal with by creating a list of property tables:
|
||||
|
||||
mpg123: audio/mp3
|
||||
layer: 1 - 3
|
||||
bitrate: 8 - 320
|
||||
|
||||
osssink:
|
||||
format: S8, S16, etc.
|
||||
depth: 8 - 16
|
||||
rate: 8000 - 48000
|
||||
channels: 1 - 2
|
||||
interleave: true
|
||||
|
||||
And you could list several of these, so for instance if the card only supports 8-bit at up to 22KHz in
|
||||
mono, you can remove S8 from the above list and add a second entry:
|
||||
|
||||
osssink:
|
||||
format: S8
|
||||
depth: 8
|
||||
rate: 8000 - 22050
|
||||
channels: 1
|
||||
interleave: false (irrelevant)
|
||||
|
||||
The obvious problem with these examples is that the rate isn't really 8000 - 48000, it's 8000, 11025,
|
||||
16000, 22050, 44100, and 48000. However, we may be able to leave these to pad connect time.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
struct _type_definition {
|
||||
char *mime_type;
|
||||
|
||||
....
|
||||
|
||||
GData *properties;
|
||||
}
|
||||
|
||||
gst_type_add_property_int(_type *type,gchar *propname,int min,int max) {
|
||||
struct _type_prop_int prop_int;
|
||||
GQuark quark = g_quark_from_string(propname);
|
||||
|
||||
prop_int->id = quark;
|
||||
prop_int->min = min;
|
||||
prop_int->max = max;
|
||||
g_datalist_id_set_data(type->properties,quark,&prop_int);
|
||||
}
|
Loading…
Reference in a new issue