mirror of
https://gitlab.freedesktop.org/gstreamer/gstreamer.git
synced 2024-12-16 05:16:36 +00:00
145 lines
6.5 KiB
Text
145 lines
6.5 KiB
Text
|
Introduction (1)
|
||
|
(sorry, no cool logo/graphic yet, ideas?)
|
||
|
GStreamer is a library and set of tools to build arbitrary,
|
||
|
reconfigurable filter graphs. It derives from the OGI Pipeline
|
||
|
and DirectShow (docs, no experience), and is in its second
|
||
|
generation (first was completed/abandonded *in* Atlanta on the
|
||
|
way to the Linux Expo).
|
||
|
0.1.0 release is scheduled for Oct 31, 1999.
|
||
|
Will cover Background, Goals, Design, and Futures
|
||
|
Why do we need this?
|
||
|
launch reads the command line to create the graph, from .so's
|
||
|
Connections (queues) are made by launcher, lots of switchout code
|
||
|
Argument handling is messy, at start-time only
|
||
|
...thus there is basically only one state: running
|
||
|
There is no real master process capable of seeing the whole
|
||
|
pipeline as a graph, so wiring A->B->C with some backchannel
|
||
|
(parameter, not data stream) from C to A is hard
|
||
|
Channels try to do IPC, file, and network I/O, excess abstraction
|
||
|
Goals (1)
|
||
|
Provide a clean way to both build graphs and write new elements
|
||
|
Make things easier by providing auto-connect, stock sub-graphs
|
||
|
Include tools sorely lacking in OGI pipeline, like editor, saves
|
||
|
Enable Linux to catch up with M$'s world, allowing commercial
|
||
|
plugins to a stable API (as of 1.0) so we don't end up with N
|
||
|
wheels from N-M different people (multiple projects)
|
||
|
Overview (1)
|
||
|
Object hierarchy capable of run-time discovery, based on GtkObject
|
||
|
Deeply nested parent-child relationships enable threads, blackboxes
|
||
|
Buffers can point to anything, are typed, and can carry metadata
|
||
|
Plugins can be loaded at any point, and registry reduces loads
|
||
|
Symbiotic editor lets you design/run/save graphs visually
|
||
|
What are filter graphs? (1)
|
||
|
Filters take data in and spit data out, doing something to it
|
||
|
Filters have N>=0 inputs and M>=0 outputs
|
||
|
Filter graphs are many filters connected together, like a circuit
|
||
|
The goal is typically to move data from 'left' to 'right', towards
|
||
|
some kind of user-visible conclusion, i.e. audio or video
|
||
|
Architecture (3?)
|
||
|
- Graphs of Elements
|
||
|
(here lies screen-grab from editor)
|
||
|
Element is core Object, Bins hold (and are) Elements
|
||
|
Pads are fundamental to an Element, are cross-wired with pointers
|
||
|
Since Bins hold Elements, and Bins are Elements, Bins hold Bins
|
||
|
'Ghostpads' provide interfaces for Bins without native interfaces
|
||
|
# Threads are type of Bin that actually run in separate threads
|
||
|
|
||
|
- States
|
||
|
(table of states, invariants, and descriptions)
|
||
|
COMPLETE Element has all needed information
|
||
|
RUNNING Element has acquired resources, ready to go
|
||
|
DISCOVERY ... (unimplemented)
|
||
|
PREROLL ... (unimplemented)
|
||
|
PLAYING Element is actively trading data
|
||
|
PAUSED Special state where things just don't run (?..)
|
||
|
States are used to keep elements in check
|
||
|
|
||
|
- Buffers
|
||
|
Buffers designed to be versatile, with arbitrary typing/metadata
|
||
|
Has pointer to data, length, so can point to someone else's data
|
||
|
Type system (crude so far) ensures buffers don't go stray
|
||
|
Metadata can be attached, such as the audio parameters
|
||
|
Ref-counting and copy-on-write avoids most copies, not complete
|
||
|
Sub-buffers can be created from bigger buffer, limitting copies
|
||
|
Gtk+ Object System (2)
|
||
|
- Pros
|
||
|
C-language object system, well tested (Gtk+, Gnome...)
|
||
|
Arguments of any fundamental type, read/write, built-in hooks
|
||
|
Signals used for hooks into object events, overridable
|
||
|
Run-time discovery of args, signals (quarks)
|
||
|
- Cons
|
||
|
No multiple-inheritance, though I haven't *needed* it
|
||
|
There are some holes (can't attach quarks to *eveything*)
|
||
|
|
||
|
- Design
|
||
|
Classes, instances are structs; 1st item is parent class
|
||
|
Type system allows clean casting, ^^^^^^
|
||
|
Arguments are set/get by string, use functions to do the work,
|
||
|
thus setting an arg might trigger a redraw of the GUI
|
||
|
Signals are strings, use marshallers, various firing methods
|
||
|
Basic GStreamer objects (1)
|
||
|
- Elements
|
||
|
(show class,instance structs)
|
||
|
Very simple, just provides a means to handle pads, state
|
||
|
- Bins
|
||
|
(show class,instance structs)
|
||
|
Supports children, handles group state transitions
|
||
|
Pads (1)
|
||
|
Pad list type, direction, and chaining function ptr
|
||
|
When creating a sink pad (!src) you set the chaining function
|
||
|
gst_pad_connect() sets the peers, and copies chain function to src
|
||
|
Passing buffer to a src pad transparently calls the chain function
|
||
|
(graph goes here...)
|
||
|
Sources (1)
|
||
|
Source provides functions to push data
|
||
|
Regular push() function just takes next N bytes and sends them
|
||
|
Async push_region() grabs N bytes at offset O and sends them
|
||
|
EOF signal [will] reset the state from PLAYING down to !RUNNING
|
||
|
"location" argument is global by convention, for filenames...URIs
|
||
|
Connections (1)
|
||
|
Special type of Filter that
|
||
|
Threads (1)
|
||
|
Special case of Bin that actually creates a thread transparently
|
||
|
When RUNNING, thread exists, mutex/cond used to go [!]PLAYING
|
||
|
Automatically determines how to start sub-graph
|
||
|
Looks for both Sources and Elements wired to Connection
|
||
|
Will cooperate with Pipelines when threading is not available
|
||
|
Typing and Metadata (1)
|
||
|
- Types
|
||
|
Based on MIME types, set up as quarks, and dealt with as int
|
||
|
Usable entirely at run-time, since they're registerable by plugins
|
||
|
- Metadata
|
||
|
Also registered as an int, but must be compile time due to structs
|
||
|
Have refcounts and CoW semantics, since they travel with buffers
|
||
|
Plugins (1)
|
||
|
Plugin architecture designed around class system
|
||
|
Arguments and signals provide interface over standard base class
|
||
|
Each Element defined by ElementFactory, which is queried by name
|
||
|
At plugin load, any number of ElementFactories and Types registered
|
||
|
Element registers against Type as source or sink
|
||
|
Editor (2+?)
|
||
|
(show filter graph snapshot, a different one, more complex)
|
||
|
Built as a parallel object hierarchy on top of GNOME Canvas
|
||
|
Every object in filter graph has equivalent in editor, plus some
|
||
|
Canvas is designed with groups and signal-propagation, so...
|
||
|
Why not build the whole thing as subclasses of CanvasGroup?
|
||
|
|
||
|
...because updates get messy/recursive (the way I *was* doing it)
|
||
|
Solution is to modify objects so they own Group rather than being
|
||
|
Relatively trivial modification, but requires lots of repointering
|
||
|
Still a genealogical mess of parents and children...
|
||
|
XML
|
||
|
The goal is to use XML heavily, with an eye towards DOM
|
||
|
Used for both saving and providing pre-build components
|
||
|
Both graph and editor will have namespace, they'll interleave
|
||
|
A generic save function will exist for Elements, with hooks
|
||
|
Saving an EditorElement will also save Element
|
||
|
Also used for a plugin registry, to avoid loading all plugins
|
||
|
|
||
|
|
||
|
|
||
|
|
||
|
leaky bucket is trivial
|
||
|
applications - generic conferencing tool (repluggable codecs), mixing
|
||
|
environment (data flow graphs)
|