mirror of
https://gitlab.freedesktop.org/gstreamer/gstreamer.git
synced 2025-02-20 21:16:24 +00:00
design: qos: fix subsection breakdown, missing markup and unneeded scaping
This commit is contained in:
Reynaldo H. Verdejo Pinochet
parent
ab1684cc62
commit
886911017e
1 changed files with 45 additions and 39 deletions
|
|
@ -34,24 +34,24 @@ to reduce the framerate, for example.
|
|||
The QoS event is generated by an element that synchronizes against the
|
||||
clock. It travels upstream and contains the following fields:
|
||||
|
||||
* **`type`**: `GST\_TYPE\_QOS\_TYPE:` The type of the QoS event, we have the
|
||||
following types and the default type is `GST\_QOS\_TYPE\_UNDERFLOW`:
|
||||
* **`type`**: `GST_TYPE_QOS_TYPE:` The type of the QoS event, we have the
|
||||
following types and the default type is `GST_QOS_TYPE_UNDERFLOW`:
|
||||
|
||||
* GST_QOS_TYPE_OVERFLOW: an element is receiving buffers too fast and can't
|
||||
* `GST_QOS_TYPE_OVERFLOW`: an element is receiving buffers too fast and can't
|
||||
keep up processing them. Upstream should reduce the rate.
|
||||
|
||||
* GST_QOS_TYPE_UNDERFLOW: an element is receiving buffers too slowly
|
||||
* `GST_QOS_TYPE_UNDERFLOW`: an element is receiving buffers too slowly
|
||||
and has to drop them because they are too late. Upstream should
|
||||
increase the processing rate.
|
||||
|
||||
* GST_QOS_TYPE_THROTTLE: the application is asking to add extra delay
|
||||
* `GST_QOS_TYPE_THROTTLE`: the application is asking to add extra delay
|
||||
between buffers, upstream is allowed to drop buffers
|
||||
|
||||
* **`timestamp`**: G\_TYPE\_UINT64: The timestamp on the buffer that
|
||||
* **`timestamp`**: `G_TYPE_UINT64`: The timestamp on the buffer that
|
||||
generated the QoS event. These timestamps are expressed in total
|
||||
running\_time in the sink so that the value is ever increasing.
|
||||
`running_time` in the sink so that the value is ever increasing.
|
||||
|
||||
* **`jitter`**: G\_TYPE\_INT64: The difference of that timestamp against the
|
||||
* **`jitter`**: `G_TYPE_INT64`: The difference of that timestamp against the
|
||||
current clock time. Negative values mean the timestamp was on time.
|
||||
Positive values indicate the timestamp was late by that amount. When
|
||||
buffers are received in time and throttling is not enabled, the QoS
|
||||
|
|
@ -59,7 +59,7 @@ type field is set to OVERFLOW. When throttling, the jitter contains
|
|||
the throttling delay added by the application and the type is set to
|
||||
THROTTLE.
|
||||
|
||||
* **`proportion`**: G\_TYPE\_DOUBLE: Long term prediction of the ideal rate
|
||||
* **`proportion`**: `G_TYPE_DOUBLE`: Long term prediction of the ideal rate
|
||||
relative to normal rate to get optimal quality.
|
||||
|
||||
The rest of this document deals with how these values can be calculated
|
||||
|
|
@ -84,48 +84,48 @@ against the clock (live) or it could be posted by an upstream element
|
|||
that performs QoS because of QOS events received from a downstream
|
||||
element (\!live).
|
||||
|
||||
The `GST\_MESSAGE\_QOS` contains at least the following info:
|
||||
The `GST_MESSAGE_QOS` contains at least the following info:
|
||||
|
||||
* **`live`**: G\_TYPE\_BOOLEAN: If the QoS message was dropped by a live
|
||||
* **`live`**: `G_TYPE_BOOLEAN`: If the QoS message was dropped by a live
|
||||
element such as a sink or a live source. If the live property is
|
||||
FALSE, the QoS message was generated as a response to a QoS event in
|
||||
a non-live element.
|
||||
|
||||
* **`running-time`**: G\_TYPE\_UINT64: The running\_time of the buffer that
|
||||
* **`running-time`**: `G_TYPE_UINT64`: The `running_time` of the buffer that
|
||||
generated the QoS message.
|
||||
|
||||
* **`stream-time`**: G\_TYPE\_UINT64: The stream\_time of the buffer that
|
||||
* **`stream-time`**: `G_TYPE_UINT64`: The `stream_time` of the buffer that
|
||||
generated the QoS message.
|
||||
|
||||
* **`timestamp`**: G\_TYPE\_UINT64: The timestamp of the buffer that
|
||||
* **`timestamp`**: `G_TYPE_UINT64`: The timestamp of the buffer that
|
||||
generated the QoS message.
|
||||
|
||||
* **`duration`**: G\_TYPE\_UINT64: The duration of the buffer that generated
|
||||
* **`duration`**: `G_TYPE_UINT64`: The duration of the buffer that generated
|
||||
the QoS message.
|
||||
|
||||
* **`jitter`**: G\_TYPE\_INT64: The difference of the running-time against
|
||||
* **`jitter`**: `G_TYPE_INT64`: The difference of the running-time against
|
||||
the deadline. Negative values mean the timestamp was on time.
|
||||
Positive values indicate the timestamp was late (and dropped) by
|
||||
that amount. The deadline can be a realtime running\_time or an
|
||||
estimated running\_time.
|
||||
that amount. The deadline can be a realtime `running_time` or an
|
||||
estimated `running_time`.
|
||||
|
||||
* **`proportion`**: G\_TYPE\_DOUBLE: Long term prediction of the ideal rate
|
||||
* **`proportion`**: `G_TYPE_DOUBLE`: Long term prediction of the ideal rate
|
||||
relative to normal rate to get optimal quality.
|
||||
|
||||
* **`quality`**: G\_TYPE\_INT: An element dependent integer value that
|
||||
* **`quality`**: `G_TYPE_INT`: An element dependent integer value that
|
||||
specifies the current quality level of the element. The default
|
||||
maximum quality is 1000000.
|
||||
|
||||
* **`format`**: GST\_TYPE\_FORMAT Units of the *processed* and *dropped*
|
||||
fields. Video sinks and video filters will use GST\_FORMAT\_BUFFERS
|
||||
* **`format`**: `GST_TYPE_FORMAT` Units of the *processed* and *dropped*
|
||||
fields. Video sinks and video filters will use `GST_FORMAT_BUFFERS`
|
||||
(frames). Audio sinks and audio filters will likely use
|
||||
GST\_FORMAT\_DEFAULT (samples).
|
||||
`GST_FORMAT_DEFAULT` (samples).
|
||||
|
||||
* **`processed`**: G\_TYPE\_UINT64: Total number of units correctly
|
||||
* **`processed`**: `G_TYPE_UINT64`: Total number of units correctly
|
||||
processed since the last state change to READY or a flushing
|
||||
operation.
|
||||
|
||||
* **`dropped`**: G\_TYPE\_UINT64: Total number of units dropped since the
|
||||
* **`dropped`**: `G_TYPE_UINT64`: Total number of units dropped since the
|
||||
last state change to READY or a flushing operation.
|
||||
|
||||
The *running-time* and *processed* fields can be used to estimate the
|
||||
|
|
@ -144,7 +144,7 @@ J1 return value from the clock. The jitter J1 is simply calculated as
|
|||
|
||||
Where CT is the clock time when the entry arrives in the sink. This
|
||||
value is calculated inside the clock when we perform
|
||||
`gst\_clock\_id\_wait()`.
|
||||
`gst_clock_id_wait()`.
|
||||
|
||||
If the jitter is negative, the entry arrived in time and can be rendered
|
||||
after waiting for the clock to reach time B1 (which is also CT - J1).
|
||||
|
|
@ -159,22 +159,28 @@ upstream.
|
|||
Using the jitter we can calculate the time when the buffer arrived in
|
||||
the sink:
|
||||
|
||||
```
|
||||
T1 = B1 + J1. (1)
|
||||
```
|
||||
|
||||
The time the buffer leaves the sink after synchronisation is measured
|
||||
as:
|
||||
|
||||
```
|
||||
T2 = B1 + (J1 < 0 ? 0 : J1) (2)
|
||||
```
|
||||
|
||||
For buffers that arrive in time (J1 \< 0) the buffer leaves after
|
||||
synchronisation which is exactly B1. Late buffers (J1 \>= 0) leave the
|
||||
sink when they arrive, whithout any synchronisation, which is T2 = T1 =
|
||||
B1 + J1.
|
||||
sink when they arrive, whithout any synchronisation, which is `T2 = T1 =
|
||||
B1 + J1`.
|
||||
|
||||
Using a previous T0 and a new T1, we can calculate the time it took for
|
||||
upstream to generate a buffer with timestamp B1.
|
||||
|
||||
```
|
||||
PT1 = T1 - T0 (3)
|
||||
```
|
||||
|
||||
We call PT1 the processing time needed to generate buffer with timestamp
|
||||
B1.
|
||||
|
|
@ -206,7 +212,7 @@ performance problems can indeed also be caused by the element itself
|
|||
when it receives too much data it cannot process in time. The element is
|
||||
then said to be overflowed.
|
||||
|
||||
# Short term correction
|
||||
## Short term correction
|
||||
|
||||
The timestamp and jitter serve as short term correction information for
|
||||
upstream elements. Indeed, given arrival time T1 as given in (1) we can
|
||||
|
|
@ -269,7 +275,7 @@ Which yields a more accurate prediction for the next buffer given as:
|
|||
B2 = B1 + 2 * J1 + D1 (5)
|
||||
```
|
||||
|
||||
# Long term correction
|
||||
## Long term correction
|
||||
|
||||
The datarate used to calculate (5) for the short term prediction is
|
||||
based on a single observation. A more accurate datarate can be obtained
|
||||
|
|
@ -285,7 +291,7 @@ Receivers of the QoS event should permanently reduce their datarate as
|
|||
given by the proportion member. Failure to do so will certainly lead to
|
||||
more dropped frames and a generally worse QoS.
|
||||
|
||||
# Throttling
|
||||
## Throttling
|
||||
|
||||
In throttle mode, the time distance between buffers is kept to a
|
||||
configurable throttle interval. This means that effectively the buffer
|
||||
|
|
@ -303,7 +309,7 @@ desired throttle interval. Implementations can use the QoS Throttle
|
|||
type, the proportion and the jitter member to tune their
|
||||
implementations.
|
||||
|
||||
# QoS strategies
|
||||
## QoS strategies
|
||||
|
||||
Several strategies exist to reduce processing delay that might affect
|
||||
real time performance.
|
||||
|
|
@ -325,12 +331,12 @@ real time performance.
|
|||
|
||||
- assign more CPU(s) to critical pipeline parts
|
||||
|
||||
# QoS implementations
|
||||
## QoS implementations
|
||||
|
||||
Here follows a small overview of how QoS can be implemented in a range
|
||||
of different types of elements.
|
||||
|
||||
# GstBaseSink
|
||||
### GstBaseSink
|
||||
|
||||
The primary implementor of QoS is GstBaseSink. It will calculate the
|
||||
following values:
|
||||
|
|
@ -367,12 +373,12 @@ is not noticeable for the human eye.
|
|||
A QoS message is posted whenever a (part of a) buffer is dropped.
|
||||
|
||||
In throttle mode, the sink sends QoS event upstream with the timestamp
|
||||
set to the running\_time of the latest buffer and the jitter set to the
|
||||
set to the `running_time` of the latest buffer and the jitter set to the
|
||||
throttle interval. If the throttled buffer is late, the lateness is
|
||||
subtracted from the throttle interval in order to keep the desired
|
||||
throttle interval.
|
||||
|
||||
# GstBaseTransform
|
||||
### GstBaseTransform
|
||||
|
||||
Transform elements can entirely skip the transform based on the
|
||||
timestamp and jitter values of recent QoS event since these buffers will
|
||||
|
|
@ -390,7 +396,7 @@ A QoS message should be posted when a frame is dropped or when the
|
|||
quality of the filter is reduced. The quality member in the QOS message
|
||||
should reflect the quality setting of the filter.
|
||||
|
||||
# Video Decoders
|
||||
### Video Decoders
|
||||
|
||||
A video decoder can, based on the codec in use, decide to not decode
|
||||
intermediate frames. A typical codec can for example skip the decoding
|
||||
|
|
@ -416,7 +422,7 @@ frame every 18 frames. The quality member should be set to `1000000 *
|
|||
- skipping P/B frames: quality = 55555 (for I-frame spacing of 18
|
||||
frames)
|
||||
|
||||
# Demuxers
|
||||
### Demuxers
|
||||
|
||||
Demuxers usually cannot do a lot regarding QoS except for skipping
|
||||
frames to the next keyframe when a lateness QoS event arrives on a
|
||||
|
|
@ -429,7 +435,7 @@ Most demuxers that have multiple output pads might need to combine the
|
|||
QoS events on all the pads and derive an aggregated QoS event for the
|
||||
upstream element.
|
||||
|
||||
# Sources
|
||||
### Sources
|
||||
|
||||
The QoS events only apply to push based sources since pull based sources
|
||||
are entirely controlled by another downstream element.
|
||||
|
|
|
|||
Loading…
Reference in a new issue