mirror of
https://gitlab.freedesktop.org/gstreamer/gstreamer.git
synced 2024-11-25 11:11:08 +00:00
Added the type system design.
Original commit message from CVS: Added the type system design.
This commit is contained in:
Wim Taymans
parent
933bf32b3f
commit
1eb3884bc6
1 changed files with 278 additions and 0 deletions
278
docs/random/types2
Normal file
278
docs/random/types2
Normal file
|
|
@ -0,0 +1,278 @@
|
|||
1. Introduction
|
||||
---------------
|
||||
|
||||
The type system is used to attach meaning to the bytes in a GstBuffer.
|
||||
A plugin can decide to add metadata to the GstBuffer, this metadata
|
||||
will carry an associated typeid.
|
||||
|
||||
Types are also used by the plugins to expose the type of their pads to
|
||||
the type system.
|
||||
|
||||
Types are essential for autoplugging.
|
||||
|
||||
We will explain the inner workings of the type system in this document, as
|
||||
well as the API used in the plugins.
|
||||
|
||||
|
||||
2. Type properties
|
||||
------------------
|
||||
|
||||
a) identification of the type
|
||||
|
||||
Types are identified by one or more MIME types.
|
||||
|
||||
b) detection of types
|
||||
|
||||
The type of any given GstBuffer can be detected using
|
||||
|
||||
- a typefind function: a function that will inspect the bytes
|
||||
in the buffer and return a gboolean indicating the
|
||||
buffer is of the given type.
|
||||
|
||||
- a template for the bytes/bits in the data that must be
|
||||
satisfied in order for the GstBuffer to be of the given
|
||||
type.
|
||||
|
||||
- other properties that act more like a hint like:
|
||||
the extension of the source filename.
|
||||
the URI of the source.
|
||||
|
||||
|
||||
3. Type registration
|
||||
--------------------
|
||||
|
||||
a) the core libraries will create to types:
|
||||
|
||||
video/raw image/raw
|
||||
audio/raw
|
||||
|
||||
b) all other types will be provided by the plugins
|
||||
|
||||
before a type can become available to other plugins, the plugin
|
||||
has to register the type.
|
||||
|
||||
The system will keep a directed graph of the types and the plugins
|
||||
that operate on them.
|
||||
|
||||
example:
|
||||
|
||||
video/mpeg
|
||||
!
|
||||
mpeg1parse
|
||||
/ \
|
||||
video/mpeg1 audio/mp3 -----\
|
||||
! ! \ !
|
||||
mpeg_play mpeg2dec mpg123 xing ...
|
||||
\ / \ /
|
||||
video/raw audio/raw-----\
|
||||
! ! ! !
|
||||
videosink SDLsink audiosink alsasink ...
|
||||
|
||||
|
||||
The system can find the needed plugins to convert video/mpeg to
|
||||
audio/raw using this graph.
|
||||
|
||||
|
||||
4. type equivalence
|
||||
-------------------
|
||||
|
||||
some types can have the same meaning for example:
|
||||
|
||||
audio/mp3 and audio/mpeg
|
||||
|
||||
or
|
||||
|
||||
video/raw and image/raw
|
||||
|
||||
|
||||
a plugin that exposes its output type as audio/mpeg and another plugins
|
||||
with input type audio/mp3 can be connected. The system has to know about
|
||||
the equivalence of both types, even it they have a different mime type.
|
||||
|
||||
|
||||
5. type hierarchy
|
||||
-----------------
|
||||
|
||||
some plugins can ouput a specific subset of an allready existing type.
|
||||
|
||||
example:
|
||||
|
||||
mp3parse inputs audio/mp3 and packs the stream into mp3 audio frames
|
||||
with mime type: audio/mp3-frame
|
||||
|
||||
mpg123 only accepts audio/mp3-frame but not audio/mp3.
|
||||
|
||||
another mp3 decoder (libmpg123) can accept audio/mp3 (and thus also
|
||||
audio/mp3-frame)
|
||||
|
||||
it must be possible to connect both libmpg123 and mpg123 to the mp3parse
|
||||
element.
|
||||
it must not be possible to connect mpg123 to an element that outputs only
|
||||
audio/mp3 but not audio/mp3-frame.
|
||||
|
||||
|
||||
We say that audio/mp3-frame is a more specific subset of type audio/mp3.
|
||||
|
||||
we can create a type hierarchy:
|
||||
|
||||
audio/mp3
|
||||
/ \
|
||||
audio/mp3-frame audio/mp3-layer12
|
||||
/ \
|
||||
audio/mp3-layer1 audio/mp3-layer2
|
||||
|
||||
|
||||
6. multi-type pads
|
||||
------------------
|
||||
|
||||
certain plugins might accept multiple non equivalent types in one of their
|
||||
input pads. Consequently a plugin might output non equivalent types in
|
||||
its output pad.
|
||||
|
||||
It must therefore be possible to specify multiple types for a pad.
|
||||
|
||||
example:
|
||||
|
||||
mpegdemux may be able to demux both MPEG1 and MPEG2 system streams.
|
||||
we show the type hierarchy of the video/mpeg as follows:
|
||||
|
||||
video/mpeg
|
||||
/ \
|
||||
video/mpeg1 video/mpeg2 ---------------\
|
||||
/ \ / \ !
|
||||
mpeg1-system* mpeg1-video mpeg2-ts mpeg2-ps* mpeg2-video
|
||||
|
||||
|
||||
the mpegdemux element might specify the type of the input pad as
|
||||
one of video/mpeg1-system and video/mpeg2-ts
|
||||
|
||||
|
||||
|
||||
7. definition of types
|
||||
----------------------
|
||||
|
||||
A plugin will provide the following information to the type system:
|
||||
|
||||
- a mime type string describing the hierarchy and where the types
|
||||
they provide are located in that hierarchy.
|
||||
|
||||
- typefind functions for some of the types.
|
||||
|
||||
- extensions for some of the types
|
||||
|
||||
We will propose a syntax to define the type hierarchy
|
||||
|
||||
a) equivalent types :
|
||||
|
||||
separated with a | sign
|
||||
|
||||
( audio/mp3 | audio/mpeg )
|
||||
|
||||
b) type hierarchy :
|
||||
|
||||
in braces:
|
||||
|
||||
( audio/mp3 ( audio/mp3-frame))
|
||||
|
||||
c) multi-type pads
|
||||
|
||||
( mpegdemux ( video/mpeg1-system + video/mpeg2-ps) )
|
||||
|
||||
the pad will have type mpegdemux which is the parent for
|
||||
type video/mpeg1-system or video/mpeg2-ps
|
||||
|
||||
mpegdemux
|
||||
/ \
|
||||
video/mpeg1-system video/mpeg2-ps
|
||||
|
||||
|
||||
|
||||
once the type hierarchy has been registered, the typeid of each
|
||||
element can be obtained with:
|
||||
|
||||
guint16 gst_type_find_by_mime (gchar *mime)
|
||||
|
||||
extra typefind functions and/or extensions can be added to the
|
||||
typeid afterwards.
|
||||
|
||||
|
||||
8. type matching
|
||||
----------------
|
||||
|
||||
The more specific typefind functions, the functions associated with
|
||||
types in the leaf nodes, are tried first.
|
||||
|
||||
when a specific type has been found ex. video/mpeg1-system elements
|
||||
that can handle this type or one of its parents are selected:
|
||||
|
||||
mpegdemux: mpeg1parse: supermpegdecoder:
|
||||
|
||||
video/mpeg video/mpeg video/mpeg
|
||||
! !
|
||||
mpegdemux video/mpeg1-system
|
||||
!
|
||||
video/mpeg1-system
|
||||
|
||||
In the above case, also the super mpeg element is selected because it stated
|
||||
that it can handle all sorts of video/mpeg data.
|
||||
|
||||
|
||||
example 2:
|
||||
|
||||
suppose the typefind functions finds an mp3 stream, fillowing elments
|
||||
are selected:
|
||||
|
||||
libmpg123 mp3parse:
|
||||
|
||||
audio/mp3 audio/mp3
|
||||
|
||||
mpg123 is not selected because its pad type is too specific (audio/mp3-frame):
|
||||
|
||||
mpg123
|
||||
|
||||
audio/mp3
|
||||
!
|
||||
audio/mp3-frame
|
||||
|
||||
if the typefind would find a mp3-frame type, all three objects would be selected.
|
||||
|
||||
|
||||
8. consideration
|
||||
----------------
|
||||
|
||||
It is clear that clear indications have to be given to the type hierarchy,
|
||||
especially for the top nodes.
|
||||
|
||||
The more specific an element is in its mime type specification, the more flexible
|
||||
and extendible the plugin system can be.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Loading…
Reference in a new issue