mirror of
https://gitlab.freedesktop.org/gstreamer/gstreamer.git
synced 2024-09-08 21:28:45 +00:00
283 lines
9.9 KiB
XML
283 lines
9.9 KiB
XML
<chapter id="chapter-advanced-qos">
|
|
<title>Quality Of Service (QoS)</title>
|
|
|
|
<para>
|
|
Quality of Service in &GStreamer; is about measuring and adjusting
|
|
the real-time performance of a pipeline. The real-time performance is
|
|
always measured relative to the pipeline clock and typically happens in
|
|
the sinks when they synchronize buffers against the clock.
|
|
</para>
|
|
<para>
|
|
When buffers arrive late in the sink, i.e. when their running-time is
|
|
smaller than that of the clock, we say that the pipeline is having a
|
|
quality of service problem. These are a few possible reasons:
|
|
</para>
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>
|
|
High CPU load, there is not enough CPU power to handle the stream,
|
|
causing buffers to arrive late in the sink.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
Network problems
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
Other resource problems such as disk load, memory bottlenecks etc
|
|
</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
<para>
|
|
The measurements result in QOS events that aim to adjust the datarate
|
|
in one or more upstream elements. Two types of adjustments can be
|
|
made:
|
|
</para>
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>
|
|
Short time "emergency" corrections based on latest observation in
|
|
the sinks.
|
|
</para>
|
|
<para>
|
|
Long term rate corrections based on trends observed in the sinks.
|
|
</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
<para>
|
|
It is also possible for the application to artificially introduce delay
|
|
between synchronized buffers, this is called throttling. It can be used
|
|
to limit or reduce the framerate, for example.
|
|
</para>
|
|
|
|
<sect1 id="section-measuring">
|
|
<title>Measuring QoS</title>
|
|
<para>
|
|
Elements that synchronize buffers on the pipeline clock will usually
|
|
measure the current QoS. They will also need to keep some statistics
|
|
in order to generate the QOS event.
|
|
</para>
|
|
<para>
|
|
For each buffer that arrives in the sink, the element needs to calculate
|
|
how late or how early it was. This is called the jitter. Negative jitter
|
|
values mean that the buffer was early, positive values mean that the
|
|
buffer was late. the jitter value gives an indication of how early/late
|
|
a buffer was.
|
|
</para>
|
|
<para>
|
|
A synchronizing element will also need to calculate how much time
|
|
elapsed between receiving two consecutive buffers. We call this the
|
|
processing time because that is the amount of time it takes for the
|
|
upstream element to produce/process the buffer. We can compare this
|
|
processing time to the duration of the buffer to have a measurement
|
|
of how fast upstream can produce data, called the proportion.
|
|
If, for example, upstream can produce a buffer in 0.5 seconds of 1
|
|
second long, it is operating at twice the required speed. If, on the
|
|
other hand, it takes 2 seconds to produce a buffer with 1 seconds worth
|
|
of data, upstream is producing buffers too slow and we won't be able to
|
|
keep synchronization. Usually, a running average is kept of the
|
|
proportion.
|
|
</para>
|
|
<para>
|
|
A synchronizing element also needs to measure its own performance in
|
|
order to figure out if the performance problem is upstream of itself.
|
|
</para>
|
|
<para>
|
|
These measurements are used to construct a QOS event that is sent
|
|
upstream. Note that a QoS event is sent for each buffer that arrives
|
|
in the sink.
|
|
</para>
|
|
</sect1>
|
|
|
|
<sect1 id="section-handling">
|
|
<title>Handling QoS</title>
|
|
<para>
|
|
An element will have to install an event function on its source pads
|
|
in order to receive QOS events. Usually, the element will need to
|
|
store the value of the QOS event and use them in the data processing
|
|
function. The element will need to use a lock to protect these QoS
|
|
values as shown in the example below. Also make sure to pass the
|
|
QoS event upstream.
|
|
</para>
|
|
<programlisting>
|
|
<![CDATA[
|
|
[...]
|
|
|
|
case GST_EVENT_QOS:
|
|
{
|
|
GstQOSType type;
|
|
gdouble proportion;
|
|
GstClockTimeDiff diff;
|
|
GstClockTime timestamp;
|
|
|
|
gst_event_parse_qos (event, &type, &proportion, &diff, ×tamp);
|
|
|
|
GST_OBJECT_LOCK (decoder);
|
|
priv->qos_proportion = proportion;
|
|
priv->qos_timestamp = timestamp;
|
|
priv->qos_diff = diff;
|
|
GST_OBJECT_UNLOCK (decoder);
|
|
|
|
res = gst_pad_push_event (decoder->sinkpad, event);
|
|
break;
|
|
}
|
|
|
|
[...]
|
|
]]>
|
|
</programlisting>
|
|
<para>
|
|
With the QoS values, there are two types of corrections that an element
|
|
can do:
|
|
</para>
|
|
|
|
<sect2 id="section-handling-short">
|
|
<title>Short term correction</title>
|
|
<para>
|
|
The timestamp and the jitter value in the QOS event can be used to
|
|
perform a short term correction. If the jitter is positive, the
|
|
previous buffer arrived late and we can be sure that a buffer with
|
|
a timestamp < timestamp + jitter is also going to be late. We
|
|
can thus drop all buffers with a timestamp less than timestamp +
|
|
jitter.
|
|
</para>
|
|
<para>
|
|
If the buffer duration is known, a better estimation for the next
|
|
likely timestamp as: timestamp + 2 * jitter + duration.
|
|
</para>
|
|
<para>
|
|
A possible algorithm typically looks like this:
|
|
</para>
|
|
<programlisting>
|
|
<![CDATA[
|
|
[...]
|
|
|
|
GST_OBJECT_LOCK (dec);
|
|
qos_proportion = priv->qos_proportion;
|
|
qos_timestamp = priv->qos_timestamp;
|
|
qos_diff = priv->qos_diff;
|
|
GST_OBJECT_UNLOCK (dec);
|
|
|
|
/* calculate the earliest valid timestamp */
|
|
if (G_LIKELY (GST_CLOCK_TIME_IS_VALID (qos_timestamp))) {
|
|
if (G_UNLIKELY (qos_diff > 0)) {
|
|
earliest_time = qos_timestamp + 2 * qos_diff + frame_duration;
|
|
} else {
|
|
earliest_time = qos_timestamp + qos_diff;
|
|
}
|
|
} else {
|
|
earliest_time = GST_CLOCK_TIME_NONE;
|
|
}
|
|
|
|
/* compare earliest_time to running-time of next buffer */
|
|
if (earliest_time > timestamp)
|
|
goto drop_buffer;
|
|
|
|
[...]
|
|
]]>
|
|
</programlisting>
|
|
</sect2>
|
|
|
|
<sect2 id="section-handling-long">
|
|
<title>Long term correction</title>
|
|
<para>
|
|
Long term corrections are a bit more difficult to perform. They
|
|
rely on the value of the proportion in the QOS event. Elements should
|
|
reduce the amount of resources they consume by the proportion
|
|
field in the QoS message.
|
|
</para>
|
|
<para>
|
|
Here are some possible strategies to achieve this:
|
|
</para>
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>
|
|
Permanently dropping frames or reducing the CPU or bandwidth
|
|
requirements of the element. Some decoders might be able to
|
|
skip decoding of B frames.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
Switch to lower quality processing or reduce the algorithmic
|
|
complexity. Care should be taken that this doesn't introduce
|
|
disturbing visual or audible glitches.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
Switch to a lower quality source to reduce network bandwidth.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
Assign more CPU cycles to critical parts of the pipeline. This
|
|
could, for example, be done by increasing the thread priority.
|
|
</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
<para>
|
|
In all cases, elements should be prepared to go back to their normal
|
|
processing rate when the proportion member in the QOS event approaches
|
|
the ideal proportion of 1.0 again.
|
|
</para>
|
|
</sect2>
|
|
</sect1>
|
|
|
|
<sect1 id="section-throttle">
|
|
<title>Throttling</title>
|
|
<para>
|
|
Elements synchronizing to the clock should expose a property to configure
|
|
them in throttle mode. In throttle mode, the time distance between buffers
|
|
is kept to a configurable throttle interval. This means that effectively
|
|
the buffer rate is limited to 1 buffer per throttle interval. This can be
|
|
used to limit the framerate, for example.
|
|
</para>
|
|
<para>
|
|
When an element is configured in throttling mode (this is usually only
|
|
implemented on sinks) it should produce QoS events upstream with the jitter
|
|
field set to the throttle interval. This should instruct upstream elements to
|
|
skip or drop the remaining buffers in the configured throttle interval.
|
|
</para>
|
|
<para>
|
|
The proportion field is set to the desired slowdown needed to get the
|
|
desired throttle interval. Implementations can use the QoS Throttle type,
|
|
the proportion and the jitter member to tune their implementations.
|
|
</para>
|
|
<para>
|
|
The default sink base class, has the <quote>throttle-time</quote>
|
|
property for this feature. You can test this with:
|
|
<command>gst-launch-1.0 videotestsrc !
|
|
xvimagesink throttle-time=500000000</command>
|
|
</para>
|
|
</sect1>
|
|
|
|
<sect1 id="section-messages">
|
|
<title>QoS Messages</title>
|
|
<para>
|
|
In addition to the QOS events that are sent between elements in the
|
|
pipeline, there are also QOS messages posted on the pipeline bus to
|
|
inform the application of QoS decisions. The QOS message contains
|
|
the timestamps of when something was dropped along with the amount
|
|
of dropped vs processed items. Elements must post a QOS
|
|
message under these conditions:
|
|
</para>
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>
|
|
The element dropped a buffer because of QoS reasons.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
An element changes its processing strategy because of QoS reasons
|
|
(quality). This could include a decoder that decides to drop every
|
|
B frame to increase its processing speed or an effect element
|
|
switching to a lower quality algorithm.
|
|
</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
</sect1>
|
|
|
|
</chapter>
|