mirror of
https://gitlab.freedesktop.org/gstreamer/gstreamer.git
synced 2024-10-06 10:42:22 +00:00
docs: add some text about parser/decoder autoplugging issues
This commit is contained in:
parent
5ed90ffc2c
commit
dbc04a27ec
1 changed files with 86 additions and 2 deletions
|
@ -13,9 +13,9 @@ Description:
|
||||||
|
|
||||||
_ a GstTypeFindElement connected to the single sink pad
|
_ a GstTypeFindElement connected to the single sink pad
|
||||||
|
|
||||||
_ optionnaly a demuxer/parser
|
_ optionally a demuxer/parser
|
||||||
|
|
||||||
_ optionnaly one or more DecodeGroup
|
_ optionally one or more DecodeGroup
|
||||||
|
|
||||||
* Autoplugging
|
* Autoplugging
|
||||||
|
|
||||||
|
@ -203,3 +203,87 @@ differences:
|
||||||
controlled by the element. This means that a buffer cannot be pushed to a
|
controlled by the element. This means that a buffer cannot be pushed to a
|
||||||
non-linked pad any sooner than buffers in any other stream which were received
|
non-linked pad any sooner than buffers in any other stream which were received
|
||||||
before it.
|
before it.
|
||||||
|
|
||||||
|
|
||||||
|
=====================================
|
||||||
|
Parsers, decoders and auto-plugging
|
||||||
|
=====================================
|
||||||
|
|
||||||
|
This section has DRAFT status.
|
||||||
|
|
||||||
|
Some media formats come in different "flavours" or "stream formats". These
|
||||||
|
formats differ in the way the setup data and media data is signalled and/or
|
||||||
|
packaged. An example for this is H.264 video, where there is a bytestream
|
||||||
|
format (with codec setup data signalled inline and units prefixed by a sync
|
||||||
|
code and packet length information) and a "raw" format where codec setup
|
||||||
|
data is signalled out of band (via the caps) and the chunking is implicit
|
||||||
|
in the way the buffers were muxed into a container, to mention just two of
|
||||||
|
the possible variants.
|
||||||
|
|
||||||
|
Especially on embedded platforms it is common that decoders can only
|
||||||
|
handle one particular stream format, and not all of them.
|
||||||
|
|
||||||
|
Where there are multiple stream formats, parsers are usually expected
|
||||||
|
to be able to convert between the different formats. This will, if
|
||||||
|
implemented correctly, work as expected in a static pipeline such as
|
||||||
|
|
||||||
|
... ! parser ! decoder ! sink
|
||||||
|
|
||||||
|
where the parser can query the decoder's capabilities even before
|
||||||
|
processing the first piece of data, and configure itself to convert
|
||||||
|
accordingly, if conversion is needed at all.
|
||||||
|
|
||||||
|
In an auto-plugging context this is not so straight-forward though,
|
||||||
|
because elements are plugged incrementally and not before the previous
|
||||||
|
element has processes some data and decided what it will output exactly
|
||||||
|
(unless the template caps are completely fixed, then it can continue
|
||||||
|
right away, this is not always the case here though, see below). A
|
||||||
|
parser will thus have to decide on *some* output format so auto-plugging
|
||||||
|
can continue. It doesn't know anything about the available decoders and
|
||||||
|
their capabilities though, so it's possible that it will choose a format
|
||||||
|
that is not supported by any of the available decoders, or by the preferred
|
||||||
|
decoder.
|
||||||
|
|
||||||
|
If the parser had sufficiently concise but fixed source pad template caps,
|
||||||
|
decodebin could continue to plug a decoder right away, allowing the
|
||||||
|
parser to configure itself in the same way as it would with a static
|
||||||
|
pipeline. This is not an option, unfortunately, because often the
|
||||||
|
parser needs to process some data to determine e.g. the format's profile or
|
||||||
|
other stream properties (resolution, sample rate, channel configuration, etc.),
|
||||||
|
and there may be different decoders for different profiles (e.g. DSP codec
|
||||||
|
for baseline profile, and software fallback for main/high profile; or a DSP
|
||||||
|
codec only supporting certain resolutions, with a software fallback for
|
||||||
|
unusual resolutions). So if decodebin just plugged the most highest-ranking
|
||||||
|
decoder, that decoder might not be be able to handle the actual stream later
|
||||||
|
on, which would yield in an error (this is a data flow error then which would
|
||||||
|
be hard to intercept and avoid in decodebin). In other words, we can't solve
|
||||||
|
this issue by plugging a decoder right away with the parser.
|
||||||
|
|
||||||
|
So decodebin need to communicate to the parser the set of available decoder
|
||||||
|
caps (which would contain the relevant capabilities/restrictions such as
|
||||||
|
supported profiles, resolutions, etc.), after the usual "autoplug-*" signal
|
||||||
|
filtering/sorting of course.
|
||||||
|
|
||||||
|
This could be done in multiple ways, e.g.
|
||||||
|
|
||||||
|
- plug a capsfilter element right after the parser, and construct
|
||||||
|
a set of filter caps from the list of available decoders (one
|
||||||
|
could append at the end just the name(s) of the caps structures
|
||||||
|
from the parser pad template caps to function as an 'ANY other'
|
||||||
|
caps equivalent). This would let the parser negotiate to a
|
||||||
|
supported stream format in the same way as with the static
|
||||||
|
pipeline mentioned above, but of course incur some overhead
|
||||||
|
through the additional capsfilter element.
|
||||||
|
|
||||||
|
- one could add a filter-caps equivalent property to the parsers
|
||||||
|
(and/or GstBaseParse class) (e.g. "prefered-caps" or so).
|
||||||
|
|
||||||
|
- one could add some kind of "fixate-caps" or "fixate-format"
|
||||||
|
signal to such parsers
|
||||||
|
|
||||||
|
Alternatively, one could simply make all decoders incorporate parsers, so
|
||||||
|
that always all formats are supported. This is problematic for other reasons
|
||||||
|
though (e.g. we would not be able to detect the profile in all cases then
|
||||||
|
before plugging a decoder, which would make it hard to just play the audio
|
||||||
|
part of a stream and not the video if a suitable decoder was missing, for
|
||||||
|
example).
|
||||||
|
|
Loading…
Reference in a new issue