mirror of
https://gitlab.freedesktop.org/gstreamer/gstreamer.git
synced 2024-12-26 18:20:44 +00:00
222 lines
6.8 KiB
Markdown
222 lines
6.8 KiB
Markdown
# Element Klass definition
|
|
|
|
## Purpose
|
|
|
|
Applications should be able to retrieve elements from the registry of
|
|
existing elements based on specific capabilities or features of the
|
|
element.
|
|
|
|
A playback application might want to retrieve all the elements that can
|
|
be used for visualisation, for example, or a video editor might want to
|
|
select all video effect filters.
|
|
|
|
The topic of defining the klass of elements should be based on use
|
|
cases.
|
|
|
|
A list of classes that are used in a installation can be generated
|
|
using:
|
|
|
|
```
|
|
gst-inspect-1.0 -a | grep -ho Klass.\* | cut -c8- | sort | uniq
|
|
```
|
|
|
|
## Proposal
|
|
|
|
The `GstElementDetails` contains a field named klass that is a pointer to
|
|
a string describing the element type.
|
|
|
|
In this document we describe the format and contents of the string.
|
|
Elements should adhere to this specification although that is not
|
|
enforced to allow for wild (application specific) customisation.
|
|
|
|
### String format
|
|
|
|
```
|
|
<keyword>['/'<keyword]*
|
|
```
|
|
|
|
The string consists of an ***unordered*** list of keywords separated with a '/'
|
|
character. While the / suggests a hierarchy, this is ***not*** the case.
|
|
|
|
### Keyword categories
|
|
|
|
#### Functional
|
|
|
|
Categories are base on ***intended usage*** of the element. Some elements
|
|
might have other side-effects (especially for filers/effects). The purpose
|
|
is to list enough keywords so that applications can do meaningful filtering,
|
|
not to completely describe the functionality that is expressed in caps.
|
|
|
|
- Source: produces data
|
|
|
|
- Sink: consumes data
|
|
|
|
- Filter: filters/transforms data, no modification on the data is
|
|
intended (although it might be unavoidable). The filter can
|
|
decide on input and output caps independently of the stream
|
|
contents (GstBaseTransform).
|
|
|
|
- Effect: applies an effect to some data, changes to data are
|
|
intended. Examples are colorbalance, volume. These elements can
|
|
also be implemented with GstBaseTransform.
|
|
|
|
- Demuxer: splits audio, video, … from a stream
|
|
|
|
- Muxer: interleave audio, video, … into one stream, this is like
|
|
mixing but without losing or degrading each separate input
|
|
stream. The reverse operation is possible with a Demuxer that
|
|
reproduces the exact same input streams.
|
|
|
|
- Decoder: decodes encoded data into a raw format, there is
|
|
typically no relation between input caps and output caps. The
|
|
output caps are defined in the stream data. This separates the
|
|
Decoder from the Filter and Effect.
|
|
|
|
- Encoder: encodes raw data into an encoded format.
|
|
|
|
- Mixer: combine audio, video, .. this is like muxing but with
|
|
applying some algorithm so that the individual streams are not
|
|
extractable anymore, there is therefore no reverse operation to
|
|
mixing. (audio mixer, video mixer, …)
|
|
|
|
- Converter: convert audio into video, text to audio, … The
|
|
converter typically works on raw types only. The source media
|
|
type is listed first.
|
|
|
|
- Analyzer: reports about the stream contents.
|
|
|
|
- Control: controls some aspect of a hardware device
|
|
|
|
- Extracter: extracts tags/headers from a stream
|
|
|
|
- Formatter: adds tags/headers to a stream
|
|
|
|
- Connector: allows for new connections in the pipeline. (tee, …)
|
|
|
|
- …
|
|
|
|
#### Based on media type
|
|
|
|
Purpose is to make a selection for elements operating on the different
|
|
types of media. An audio application must be able to filter out the
|
|
elements operating on audio, for example.
|
|
|
|
- Audio: operates on audio data
|
|
|
|
- Video: operates on video data
|
|
|
|
- Image: operates on image data. Usually this media type can also
|
|
be used to make a video stream in which case it is added
|
|
together with the Video media type.
|
|
|
|
- Text: operates on text data
|
|
|
|
- Metadata: operates on metadata
|
|
|
|
- …
|
|
|
|
#### Extra features
|
|
|
|
The purpose is to further specialize the element, mostly for
|
|
application specific needs.
|
|
|
|
- Network: element is used in networked situations
|
|
|
|
- Protocol: implements some protocol (RTSP, HTTP, …)
|
|
|
|
- Payloader: encapsulate as payload (RTP, RDT,.. )
|
|
|
|
- Depayloader: strip a payload (RTP, RDT,.. )
|
|
|
|
- RTP: intended to be used in RTP applications
|
|
|
|
- Device: operates on some hardware device (disk, network, audio
|
|
card, video card, usb, …)
|
|
|
|
- Visualisation: intended to be used for audio visualisation
|
|
|
|
- Debug: intended usage is more for debugging purposes.
|
|
|
|
#### Categories found, but not yet in one of the above lists
|
|
|
|
- Bin: playbin, decodebin, bin, pipeline
|
|
|
|
- Codec: lots of decoders, encoder, demuxers should be removed?
|
|
|
|
- Generic: should be removed?
|
|
|
|
- File: like network, should go to Extra?
|
|
|
|
- Editor: gnonlin, textoverlays
|
|
|
|
- DVD, GDP, LADSPA, Parser, Player, Subtitle, Testing, …
|
|
|
|
### Suggested order:
|
|
|
|
```
|
|
<functional>[/<media type>]*[/<extra...>]*
|
|
```
|
|
|
|
### Examples:
|
|
|
|
apedemux : Extracter/Metadata
|
|
audiotestsrc : Source/Audio
|
|
autoaudiosink : Sink/Audio/Device
|
|
cairotimeoverlay : Mixer/Video/Text
|
|
dvdec : Decoder/Video
|
|
dvdemux : Demuxer
|
|
goom : Converter/Audio/Video
|
|
id3demux : Extracter/Metadata
|
|
udpsrc : Source/Network/Protocol/Device
|
|
videomixer : Mixer/Video
|
|
videoconvert : Filter/Video (intended use to convert video with as little
|
|
visible change as possible)
|
|
vertigotv : Effect/Video (intended use is to change the video)
|
|
volume : Effect/Audio (intended use is to change the audio data)
|
|
vorbisdec : Decoder/Audio
|
|
vorbisenc : Encoder/Audio
|
|
oggmux : Muxer
|
|
adder : Mixer/Audio
|
|
videobox : Effect/Video
|
|
alsamixer : Control/Audio/Device
|
|
audioconvert : Filter/Audio
|
|
audioresample : Filter/Audio
|
|
xvimagesink : Sink/Video/Device
|
|
navseek : Filter/Debug
|
|
decodebin : Decoder/Demuxer
|
|
level : Filter/Analyzer/Audio
|
|
tee : Connector/Debug
|
|
|
|
### open issues:
|
|
|
|
- how to differentiate physical devices from logical ones?
|
|
autoaudiosink : Sink/Audio/Device alsasink : Sink/Audio/Device
|
|
|
|
## Use cases
|
|
|
|
- get a list of all elements implementing a video effect (pitivi):
|
|
|
|
klass.contains (Effect & Video)
|
|
|
|
- get list of muxers (pitivi):
|
|
|
|
klass.contains (Muxer)
|
|
|
|
- get list of video encoders (pitivi):
|
|
|
|
klass.contains (Encoder & video)
|
|
|
|
- Get a list of all audio/video visualisations (totem):
|
|
|
|
klass.contains (Visualisation)
|
|
|
|
- Get a list of all decoders/demuxer/metadata parsers/vis (playbin):
|
|
|
|
klass.contains (Visualisation | Demuxer | Decoder | (Extractor & Metadata))
|
|
|
|
- Get a list of elements that can capture from an audio device
|
|
(gst-properties):
|
|
|
|
klass.contains (Source & Audio & Device)
|
|
|
|
- filters out audiotestsrc, since it is not a device
|