diff --git a/docs/random/types2 b/docs/random/types2 new file mode 100644 index 0000000000..189f037023 --- /dev/null +++ b/docs/random/types2 @@ -0,0 +1,278 @@ +1. Introduction +--------------- + +The type system is used to attach meaning to the bytes in a GstBuffer. +A plugin can decide to add metadata to the GstBuffer, this metadata +will carry an associated typeid. + +Types are also used by the plugins to expose the type of their pads to +the type system. + +Types are essential for autoplugging. + +We will explain the inner workings of the type system in this document, as +well as the API used in the plugins. + + +2. Type properties +------------------ + +a) identification of the type + + Types are identified by one or more MIME types. + +b) detection of types + + The type of any given GstBuffer can be detected using + + - a typefind function: a function that will inspect the bytes + in the buffer and return a gboolean indicating the + buffer is of the given type. + + - a template for the bytes/bits in the data that must be + satisfied in order for the GstBuffer to be of the given + type. + + - other properties that act more like a hint like: + the extension of the source filename. + the URI of the source. + + +3. Type registration +-------------------- + +a) the core libraries will create to types: + + video/raw image/raw + audio/raw + +b) all other types will be provided by the plugins + +before a type can become available to other plugins, the plugin +has to register the type. + +The system will keep a directed graph of the types and the plugins +that operate on them. + +example: + + video/mpeg + ! + mpeg1parse + / \ + video/mpeg1 audio/mp3 -----\ + ! ! \ ! + mpeg_play mpeg2dec mpg123 xing ... + \ / \ / + video/raw audio/raw-----\ + ! ! ! ! + videosink SDLsink audiosink alsasink ... + + +The system can find the needed plugins to convert video/mpeg to +audio/raw using this graph. + + +4. type equivalence +------------------- + +some types can have the same meaning for example: + + audio/mp3 and audio/mpeg + +or + + video/raw and image/raw + + +a plugin that exposes its output type as audio/mpeg and another plugins +with input type audio/mp3 can be connected. The system has to know about +the equivalence of both types, even it they have a different mime type. + + +5. type hierarchy +----------------- + +some plugins can ouput a specific subset of an allready existing type. + +example: + + mp3parse inputs audio/mp3 and packs the stream into mp3 audio frames + with mime type: audio/mp3-frame + + mpg123 only accepts audio/mp3-frame but not audio/mp3. + + another mp3 decoder (libmpg123) can accept audio/mp3 (and thus also + audio/mp3-frame) + + it must be possible to connect both libmpg123 and mpg123 to the mp3parse + element. + it must not be possible to connect mpg123 to an element that outputs only + audio/mp3 but not audio/mp3-frame. + + +We say that audio/mp3-frame is a more specific subset of type audio/mp3. + +we can create a type hierarchy: + + audio/mp3 + / \ + audio/mp3-frame audio/mp3-layer12 + / \ + audio/mp3-layer1 audio/mp3-layer2 + + +6. multi-type pads +------------------ + +certain plugins might accept multiple non equivalent types in one of their +input pads. Consequently a plugin might output non equivalent types in +its output pad. + +It must therefore be possible to specify multiple types for a pad. + +example: + + mpegdemux may be able to demux both MPEG1 and MPEG2 system streams. + we show the type hierarchy of the video/mpeg as follows: + + video/mpeg + / \ + video/mpeg1 video/mpeg2 ---------------\ + / \ / \ ! + mpeg1-system* mpeg1-video mpeg2-ts mpeg2-ps* mpeg2-video + + + the mpegdemux element might specify the type of the input pad as + one of video/mpeg1-system and video/mpeg2-ts + + + +7. definition of types +---------------------- + +A plugin will provide the following information to the type system: + + - a mime type string describing the hierarchy and where the types + they provide are located in that hierarchy. + + - typefind functions for some of the types. + + - extensions for some of the types + +We will propose a syntax to define the type hierarchy + +a) equivalent types : + + separated with a | sign + + ( audio/mp3 | audio/mpeg ) + +b) type hierarchy : + + in braces: + + ( audio/mp3 ( audio/mp3-frame)) + +c) multi-type pads + + ( mpegdemux ( video/mpeg1-system + video/mpeg2-ps) ) + + the pad will have type mpegdemux which is the parent for + type video/mpeg1-system or video/mpeg2-ps + + mpegdemux + / \ + video/mpeg1-system video/mpeg2-ps + + + +once the type hierarchy has been registered, the typeid of each +element can be obtained with: + + guint16 gst_type_find_by_mime (gchar *mime) + +extra typefind functions and/or extensions can be added to the +typeid afterwards. + + +8. type matching +---------------- + +The more specific typefind functions, the functions associated with +types in the leaf nodes, are tried first. + +when a specific type has been found ex. video/mpeg1-system elements +that can handle this type or one of its parents are selected: + + mpegdemux: mpeg1parse: supermpegdecoder: + + video/mpeg video/mpeg video/mpeg + ! ! + mpegdemux video/mpeg1-system + ! + video/mpeg1-system + +In the above case, also the super mpeg element is selected because it stated +that it can handle all sorts of video/mpeg data. + + +example 2: + + suppose the typefind functions finds an mp3 stream, fillowing elments + are selected: + + libmpg123 mp3parse: + + audio/mp3 audio/mp3 + + mpg123 is not selected because its pad type is too specific (audio/mp3-frame): + + mpg123 + + audio/mp3 + ! + audio/mp3-frame + + if the typefind would find a mp3-frame type, all three objects would be selected. + + +8. consideration +---------------- + +It is clear that clear indications have to be given to the type hierarchy, +especially for the top nodes. + +The more specific an element is in its mime type specification, the more flexible +and extendible the plugin system can be. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +