mirror of
https://gitlab.freedesktop.org/gstreamer/gstreamer.git
synced 2024-11-13 21:01:14 +00:00
266 lines
9.9 KiB
Text
266 lines
9.9 KiB
Text
In this document we describe how we can obtain various properties
|
|
of the pipeline we are running.
|
|
|
|
we have 5 possible ways to get information, each one of these
|
|
methods focus on one particular property of the pipeline.
|
|
|
|
- caps: this is a description of the media type that flows between
|
|
2 pads.
|
|
- metadata: non essential extra (human readable) information about
|
|
the stream, like author, copyright, name etc..
|
|
- streaminfo: information about the stream as encoded into the
|
|
stream itself.
|
|
- pad/element queries: information about the stream as it is being
|
|
processed.
|
|
- pad/element convert: information about the relation between formats
|
|
in a streams as it is being processed.
|
|
|
|
note that element properties are not included in this list. Element
|
|
properties are only used to configure the codec/element.
|
|
|
|
caps, metadata, streaminfo are supposed to remain fairly static during
|
|
the stream. The queries/converts can be done on demand. The reason
|
|
for this is that caps/metadata/streaminfo is quite expensive and
|
|
could degrade pipeline performance.
|
|
|
|
|
|
Caps.
|
|
-----
|
|
|
|
Caps are automatically set on pads by the core when two pads agree
|
|
on a media type. This automatically means that the caps are fixed.
|
|
|
|
Since caps is a property of the pad and the g_object_notify() mechanism
|
|
is used to signal a change, the user can either connect a "notify"
|
|
signal handler to the pad or connect to the "deep_notify" signal on
|
|
a parent pipeline.
|
|
|
|
The caps notifications are useful if you want to know what kind of
|
|
media is passing through pads. You can, for example, report to the
|
|
user how the video or audiosink is configured or what kind of
|
|
media some plugin is producing/accepting.
|
|
|
|
So, always use the caps to find out the channels/samplerate/size
|
|
of the media.
|
|
|
|
|
|
Metadata
|
|
--------
|
|
|
|
Metadata is a GstCaps element property (named "metadata") that contains
|
|
additional information encoded into the stream that doesn't say anything
|
|
about the media type of the stream itself.
|
|
|
|
Metadata are typically human readable information like author, copyright,
|
|
title, ... and can be displayed by the application as-is.
|
|
|
|
An element with a "metadata" property is supposed to g_object_notify that
|
|
property when the metadata changes so that the app can connect a signal
|
|
handler to the property or use the "deep_notify" signal to get the
|
|
notification.
|
|
|
|
|
|
Streaminfo
|
|
----------
|
|
|
|
Streaminfo is a GstCaps element property (named "streaminfo") that contains
|
|
additional information about the stream that is not stricly required to
|
|
specify the media type of the stream.
|
|
|
|
Streaminfo is typically the length, bitrate, framerate, flags, ... of the
|
|
stream.
|
|
|
|
It is important to note that this information should be derived from the
|
|
stream itself and might not be correct for the pipeline being processed.
|
|
Let's illustrate this with an example:
|
|
|
|
- an mp3 stream has an id3 tags that contains the length of the stream
|
|
(TLEN).
|
|
- we cut the mp3 stream in half
|
|
|
|
The actual length doesn't match the stated length as encoded in the id3
|
|
tags, so the TLEN tag should be put in the streaminfo and the actual
|
|
length should be queried with a pad_query.
|
|
|
|
So, be careful when showing streaminfo as-is in an app.
|
|
|
|
Queries
|
|
-------
|
|
|
|
Queries can be performed on pads and elements and are used to get
|
|
information about the current stream. The value is a single gint64
|
|
in a specific format.
|
|
|
|
example:
|
|
|
|
- the query (GST_QUERY_TOTAL, GST_FORMAT_TIME) will return the total
|
|
amount of time this pad/element will run.
|
|
|
|
- the query (GST_QUERY_POSITION, GST_FORMAT_UNITS) will return the
|
|
current position of the stream expressed in units (units are samples,
|
|
frames, bytes, ... depending on the media type of the stream)
|
|
|
|
two methods exist to perform a query:
|
|
|
|
- gboolean gst_pad_query (GstPad *pad, GstQueryType type,
|
|
GstFormat *format, gint64 *value);
|
|
|
|
and:
|
|
|
|
- gboolean gst_element_query (GstElement *element, GstQueryType type,
|
|
GstFormat *format, gint64 *value);
|
|
|
|
if you want to get the total duration of a stream or the current position,
|
|
you need to use a pad query. A pad query can fail (method returns FALSE),
|
|
this usually means that the duration is not known or that not enough data
|
|
has been processed to report the correct value.
|
|
|
|
the possible queries that can be performed on a pad/element can be obtained
|
|
with
|
|
|
|
- const GstQueryType* gst_pad_get_query_types (GstPad *pad);
|
|
|
|
and
|
|
|
|
- const GstQueryType* gst_element_get_query_types (GstElement *element);
|
|
|
|
These functions return an array of GstQueryTypes (last element == 0). you
|
|
can loop over the array to see what is supported or do
|
|
|
|
- gboolean gst_queries_contains (const GstQueryType *types, GstQueryType type);
|
|
|
|
to see if a specific format is contained in the list.
|
|
|
|
The value reported by a pad query can be considered correct or
|
|
as-good-as-can-be. Elements that support pad queries must do the best
|
|
they can to report correct values.
|
|
|
|
|
|
Convert
|
|
-------
|
|
|
|
The convert functions are used to query an element/pad for the relationship
|
|
between two formats that it supports. For example:
|
|
|
|
suppose we want to know how many frames a particular video decoder will
|
|
produce in one second, we ask it to convert its concept of 1 SECOND into
|
|
1 UNIT (frames in the context of video). so,
|
|
|
|
GstFormat format = GST_FORMAT_UNITS;
|
|
|
|
res = gst_pad_convert (pad, GST_FORMAT_TIME, GST_SECOND,
|
|
&format, &value);
|
|
|
|
if res == TRUE, value will contain the framerate of the video, of course
|
|
this framerate will only contain the integral part. If you want more
|
|
accuracy, use 1000 * GST_SECOND and divide the result by 1000 to get
|
|
a fractional part.
|
|
|
|
All other neat things can be done too, look at the typical cdplayer
|
|
plugin for example. It defines a new format "track". Now you can
|
|
ask it to convert a TRACK to TIME like this:
|
|
|
|
|
|
GstFormat format = GST_FORMAT_TIME;
|
|
track_format = gst_format_get_by_nick ("track");
|
|
res = gst_pad_convert (pad, track_format, 1,
|
|
&format, &value);
|
|
|
|
|
|
This will convert 1 track to a time. of course we didn't (couldn't) specify
|
|
which track, but that's not a problem if you understand how the stream is
|
|
divided into different formats:
|
|
|
|
Take the total stream as containing bytes (indicated with + in figure) we
|
|
can subdivide the (byte)stream in different formats.
|
|
|
|
|
|
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ... bytes
|
|
! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ... samples (units)
|
|
! ! ! ! ... buffers
|
|
! ! ! ... time
|
|
! ! ... track
|
|
|
|
The raw bytestream can be grouped in samples for example (for 16 bit stereo
|
|
int audio, 4 bytes == 1 sample) or we can divide it into time
|
|
(44100 samples == 176400 bytes == 1 second) or into tracks
|
|
(1 track could be 17640000 bytes or 100 seconds or 4410000 samples)
|
|
|
|
It is important to know that the stream starts at position 0 (for all formats)
|
|
and ends at position X (expressed in a specific format). now take this stream
|
|
divided into the track format:
|
|
|
|
0 200 500 700 1000
|
|
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ .. bytes
|
|
! track0 ! track1 ! track2 ! track3 ! ...
|
|
|
|
if we now perform a pad convert from 100 bytes to the track format, we
|
|
get the value 0, as the region 0-100 bytes is contained in track0.
|
|
if we perform a pad convert from 600 bytes to track, we get the value
|
|
2, as track0->track2 contains the bytes 0-500.
|
|
|
|
We can also do: convert track1 to bytes, then we get 200. If we do
|
|
convert track2 to bytes, we get 500. Note that the conversions are
|
|
always performed relative to 0, so if we convert track2 to bytes, we
|
|
always get the number of bytes from track0->track2.
|
|
|
|
If we want to get the number of bytes of one particular track, we have
|
|
to substract two convert values. Look at the folowing figure to understand
|
|
this. The --- defines the region we want to convert.
|
|
|
|
|
|
0 200 500 700 1000
|
|
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ .. bytes
|
|
! track0 ! track1 ! track2 ! track3 ! ...
|
|
|
|
----------- track1 -> bytes (1) (size of track0)
|
|
|
|
---------------------- track2 -> bytes (2) (size of track0 and track1)
|
|
|
|
----------- (2) - (1) = total bytes for track 1
|
|
|
|
Another example would be to get the bitrate of a decoder plugin, consider the
|
|
following example:
|
|
|
|
(------------)
|
|
! mad !
|
|
- sink src -
|
|
(------------)
|
|
|
|
The element has a sinkpad that will take N bytes as input to produce M
|
|
samples (units) on its srcpad.
|
|
The rate at which it takes bytes is defined as the byterate of the
|
|
stream (bitrate == byterate * 8). So, we do:
|
|
|
|
GstFormat format = GST_FORMAT_BYTES;
|
|
gint64 value;
|
|
|
|
gst_pad_convert (mad->sinkpad, GST_FORMAT_TIME, GST_SECOND,
|
|
&format, &value);
|
|
|
|
..and we get the number of bytes this plugin takes in each second.
|
|
Again, note that this value is relative to 0, you can get an average
|
|
of a specific period by using the same substract trick as above.
|
|
|
|
|
|
Element Properties
|
|
------------------
|
|
|
|
Element properties are used to configure an element. They should not be
|
|
used to describe media info, metadata fields, streaminfo fields or anything
|
|
other that doesn't involve codec configuration.
|
|
|
|
Several reasons:
|
|
|
|
- metadata requires dynamic properties (one for each tag). This cannot be done
|
|
with GObject properties.
|
|
- you cannot signal a logical group of related properties (exposing stuff like
|
|
samplerate/channels/encoding/... in different element properties is not a
|
|
good idea.
|
|
- stuff like length an position depend on the pads of the element, you cannot
|
|
sanely expose a property for each pad to describe this.
|
|
- element properties can only report stuff with one type. If your property
|
|
exposes somthing like "total_length", you cannot make it both report this
|
|
in time/bytes/samples/frames...
|
|
- impossible to sanely implement convert with element properties.
|
|
|