mirror of
https://gitlab.freedesktop.org/gstreamer/gstreamer.git
synced 2024-09-08 21:28:45 +00:00
282 lines
6.3 KiB
Text
282 lines
6.3 KiB
Text
OUTDATED
|
|
--------
|
|
|
|
|
|
1. Introduction
|
|
---------------
|
|
|
|
The type system is used to attach meaning to the bytes in a GstBuffer.
|
|
A plugin can decide to add metadata to the GstBuffer, this metadata
|
|
will carry an associated typeid.
|
|
|
|
Types are also used by the plugins to expose the type of their pads to
|
|
the type system.
|
|
|
|
Types are essential for autoplugging.
|
|
|
|
We will explain the inner workings of the type system in this document, as
|
|
well as the API used in the plugins.
|
|
|
|
|
|
2. Type properties
|
|
------------------
|
|
|
|
a) identification of the type
|
|
|
|
Types are identified by one or more MIME types.
|
|
|
|
b) detection of types
|
|
|
|
The type of any given GstBuffer can be detected using
|
|
|
|
- a typefind function: a function that will inspect the bytes
|
|
in the buffer and return a gboolean indicating the
|
|
buffer is of the given type.
|
|
|
|
- a template for the bytes/bits in the data that must be
|
|
satisfied in order for the GstBuffer to be of the given
|
|
type.
|
|
|
|
- other properties that act more like a hint like:
|
|
the extension of the source filename.
|
|
the URI of the source.
|
|
|
|
|
|
3. Type registration
|
|
--------------------
|
|
|
|
a) the core libraries will create to types:
|
|
|
|
video/raw image/raw
|
|
audio/raw
|
|
|
|
b) all other types will be provided by the plugins
|
|
|
|
before a type can become available to other plugins, the plugin
|
|
has to register the type.
|
|
|
|
The system will keep a directed graph of the types and the plugins
|
|
that operate on them.
|
|
|
|
example:
|
|
|
|
video/mpeg
|
|
!
|
|
mpeg1parse
|
|
/ \
|
|
video/mpeg1 audio/mp3 -----\
|
|
! ! \ !
|
|
mpeg_play mpeg2dec mpg123 xing ...
|
|
\ / \ /
|
|
video/raw audio/raw-----\
|
|
! ! ! !
|
|
videosink SDLsink audiosink alsasink ...
|
|
|
|
|
|
The system can find the needed plugins to convert video/mpeg to
|
|
audio/raw using this graph.
|
|
|
|
|
|
4. type equivalence
|
|
-------------------
|
|
|
|
some types can have the same meaning for example:
|
|
|
|
audio/mp3 and audio/mpeg
|
|
|
|
or
|
|
|
|
video/raw and image/raw
|
|
|
|
|
|
a plugin that exposes its output type as audio/mpeg and another plugins
|
|
with input type audio/mp3 can be connected. The system has to know about
|
|
the equivalence of both types, even it they have a different mime type.
|
|
|
|
|
|
5. type hierarchy
|
|
-----------------
|
|
|
|
some plugins can ouput a specific subset of an already existing type.
|
|
|
|
example:
|
|
|
|
mp3parse inputs audio/mp3 and packs the stream into mp3 audio frames
|
|
with mime type: audio/mp3-frame
|
|
|
|
mpg123 only accepts audio/mp3-frame but not audio/mp3.
|
|
|
|
another mp3 decoder (libmpg123) can accept audio/mp3 (and thus also
|
|
audio/mp3-frame)
|
|
|
|
it must be possible to connect both libmpg123 and mpg123 to the mp3parse
|
|
element.
|
|
it must not be possible to connect mpg123 to an element that outputs only
|
|
audio/mp3 but not audio/mp3-frame.
|
|
|
|
|
|
We say that audio/mp3-frame is a more specific subset of type audio/mp3.
|
|
|
|
we can create a type hierarchy:
|
|
|
|
audio/mp3
|
|
/ \
|
|
audio/mp3-frame audio/mp3-layer12
|
|
/ \
|
|
audio/mp3-layer1 audio/mp3-layer2
|
|
|
|
|
|
6. multi-type pads
|
|
------------------
|
|
|
|
certain plugins might accept multiple non equivalent types in one of their
|
|
input pads. Consequently a plugin might output non equivalent types in
|
|
its output pad.
|
|
|
|
It must therefore be possible to specify multiple types for a pad.
|
|
|
|
example:
|
|
|
|
mpegdemux may be able to demux both MPEG1 and MPEG2 system streams.
|
|
we show the type hierarchy of the video/mpeg as follows:
|
|
|
|
video/mpeg
|
|
/ \
|
|
video/mpeg1 video/mpeg2 ---------------\
|
|
/ \ / \ !
|
|
mpeg1-system* mpeg1-video mpeg2-ts mpeg2-ps* mpeg2-video
|
|
|
|
|
|
the mpegdemux element might specify the type of the input pad as
|
|
one of video/mpeg1-system and video/mpeg2-ts
|
|
|
|
|
|
|
|
7. definition of types
|
|
----------------------
|
|
|
|
A plugin will provide the following information to the type system:
|
|
|
|
- a mime type string describing the hierarchy and where the types
|
|
they provide are located in that hierarchy.
|
|
|
|
- typefind functions for some of the types.
|
|
|
|
- extensions for some of the types
|
|
|
|
We will propose a syntax to define the type hierarchy
|
|
|
|
a) equivalent types :
|
|
|
|
separated with a | sign
|
|
|
|
( audio/mp3 | audio/mpeg )
|
|
|
|
b) type hierarchy :
|
|
|
|
in braces:
|
|
|
|
( audio/mp3 ( audio/mp3-frame))
|
|
|
|
c) multi-type pads
|
|
|
|
( mpegdemux ( video/mpeg1-system + video/mpeg2-ps) )
|
|
|
|
the pad will have type mpegdemux which is the parent for
|
|
type video/mpeg1-system or video/mpeg2-ps
|
|
|
|
mpegdemux
|
|
/ \
|
|
video/mpeg1-system video/mpeg2-ps
|
|
|
|
|
|
|
|
once the type hierarchy has been registered, the typeid of each
|
|
element can be obtained with:
|
|
|
|
guint16 gst_type_find_by_mime (gchar *mime)
|
|
|
|
extra typefind functions and/or extensions can be added to the
|
|
typeid afterwards.
|
|
|
|
|
|
8. type matching
|
|
----------------
|
|
|
|
The more specific typefind functions, the functions associated with
|
|
types in the leaf nodes, are tried first.
|
|
|
|
when a specific type has been found ex. video/mpeg1-system elements
|
|
that can handle this type or one of its parents are selected:
|
|
|
|
mpegdemux: mpeg1parse: supermpegdecoder:
|
|
|
|
video/mpeg video/mpeg video/mpeg
|
|
! !
|
|
mpegdemux video/mpeg1-system
|
|
!
|
|
video/mpeg1-system
|
|
|
|
In the above case, also the super mpeg element is selected because it stated
|
|
that it can handle all sorts of video/mpeg data.
|
|
|
|
|
|
example 2:
|
|
|
|
suppose the typefind functions finds an mp3 stream, fillowing elments
|
|
are selected:
|
|
|
|
libmpg123 mp3parse:
|
|
|
|
audio/mp3 audio/mp3
|
|
|
|
mpg123 is not selected because its pad type is too specific (audio/mp3-frame):
|
|
|
|
mpg123
|
|
|
|
audio/mp3
|
|
!
|
|
audio/mp3-frame
|
|
|
|
if the typefind would find a mp3-frame type, all three objects would be selected.
|
|
|
|
|
|
8. consideration
|
|
----------------
|
|
|
|
It is clear that clear indications have to be given to the type hierarchy,
|
|
especially for the top nodes.
|
|
|
|
The more specific an element is in its mime type specification, the more flexible
|
|
and extendible the plugin system can be.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|