mirrors/gstreamer
mirrors/gstreamer
1
1
Fork 0
mirror of https://gitlab.freedesktop.org/gstreamer/gstreamer.git synced 2024-06-27 02:10:41 +00:00
Code Issues 1 Projects Releases Wiki Activity

docs/design/part-qos.txt: Bring docs in line with the code. Mostly the sign of the jitter was wrong in the docs.

Browse source 
Original commit message from CVS:
* docs/design/part-qos.txt:
Bring docs in line with the code. Mostly the sign of the jitter was
wrong in the docs.
* gst/gstclock.c:
Fix the docs for the jitter.
* gst/gstevent.c: (gst_event_new_custom), (gst_event_new_tag),
(gst_event_parse_tag), (gst_event_new_buffer_size),
(gst_event_parse_buffer_size), (gst_event_parse_qos),
(gst_event_new_seek), (gst_event_parse_seek),
(gst_event_new_navigation):
Make sure the GstStructure has no parent when creating custom
events.
Add some more argument checking so that we avoid 0.0 rates.
Flesh out the docs for the QoS event some more.
This commit is contained in:
Wim Taymans 2006-08-11 13:05:30 +00:00
parent 1532848c9a
commit ff240086e0
4 changed files with 98 additions and 47 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

19
ChangeLog
View file

@ -1,3 +1,22 @@
2006-08-11 Wim Taymans <wim@fluendo.com>
* docs/design/part-qos.txt:
Bring docs in line with the code. Mostly the sign of the jitter was
wrong in the docs.
* gst/gstclock.c:
Fix the docs for the jitter.
* gst/gstevent.c: (gst_event_new_custom), (gst_event_new_tag),
(gst_event_parse_tag), (gst_event_new_buffer_size),
(gst_event_parse_buffer_size), (gst_event_parse_qos),
(gst_event_new_seek), (gst_event_parse_seek),
(gst_event_new_navigation):
Make sure the GstStructure has no parent when creating custom
events.
Add some more argument checking so that we avoid 0.0 rates.
Flesh out the docs for the QoS event some more.
2006-08-11 Wim Taymans <wim@fluendo.com>
* docs/gst/gstreamer-sections.txt:

57
docs/design/part-qos.txt
View file

@ -22,6 +22,7 @@ Sources of quality problems
- High CPU load
- Network problems
- Other resource problems such as disk load, memory bottlenecks etc.
QoS event
@ -30,14 +31,19 @@ QoS event
The QoS event travels upstream and contains the following fields:
- timestamp: The timestamp on the buffer that generated the QoS
event
- jitter: The difference of that timestamp against the clock.
event. These timestamps are expressed in total running time in
the sink so that the value is every increasing.
- jitter: The difference of that timestamp against the currentl clock time.
Negative values mean the timestamp was on time. Positive values
indicate the timestamp was late by that amount.
- proportion: Long term prediction of the ideal rate relative to
normal rate to get optimal quality.
The rest of this document deals with how these values can be calculated
in a sink.
in a sink and how the values can be used by other elements to adjust their
operations.
Collecting statistics
@ -47,30 +53,31 @@ A buffer with timestamp B1 arrives in the sink at time T1. The buffer
timestamp is then synchronized against the clock which yields a jitter J1
return value from the clock. The jitter J1 is simply calculated as
J1 = B1 - CT
J1 = CT - B1
Where CT is the clock time when the entry arrives in the sink. This value
is calculated inside the clock when we perform gst_clock_entry_wait().
If the jitter is positive, the entry arrived in time and can be rendered
after waiting for the clock to reach time B1 (which is also CT + J1).
If the jitter is negative, the entry arrived in time and can be rendered
after waiting for the clock to reach time B1 (which is also CT - J1).
If the jitter is negative however, the entry arrived too late in the sink
and should therefore be dropped. A dropped buffer should generate a QoS
event upstream. J1 is the amount of time the entry was late.
If the jitter is positive however, the entry arrived too late in the sink
and should therefore be dropped. J1 is the amount of time the entry was late.
Any buffer that arrives in the sink should generate a QoS event upstream.
Using the jitter we can calculate the time when the buffer arrived in the
sink:
T1 = B1 - J1. (1)
T1 = B1 + J1. (1)
The time the buffer leaves the sink after synchronisation is measured as:
T2 = B1 - (J1 < 0 ? J1 : 0) (2)
T2 = B1 + (J1 < 0 ? 0 : J1) (2)
For buffers that arrive in time (J1 >= 0) the buffer leaves after synchronisation
which is exactly B1. Late buffers (J1 < 0) leave the sink when they arrive,
whithout any synchronisation, which is T1 = B1 - J1.
For buffers that arrive in time (J1 < 0) the buffer leaves after synchronisation
which is exactly B1. Late buffers (J1 >= 0) leave the sink when they arrive,
whithout any synchronisation, which is T2 = T1 = B1 + J1.
Using a previous T0 and a new T1, we can calculate the time it took for
upstream to generate a buffer with timestamp B1.
@ -114,11 +121,11 @@ for upstream elements. Indeed, given arrival time T1 as given in (1)
we can be certain that buffers with a timestamp B2 < T1 will be too late
in the sink.
In case of a negative jitter we can therefore send a QoS message with
In case of a positive jitter we can therefore send a QoS message with
a timestamp B1, jitter J1 and proportion given by (4).
This allows an upstream element to not generate any data with a timestamps
B2 < T1, where the element can derive T1 as B1 - J1.
B2 < T1, where the element can derive T1 as B1 + J1.
This will effectively result in frame drops.
@ -127,7 +134,7 @@ should output.
Indeed, given the element generated a buffer with timestamp B0 that arrived
in time in the sink but then received a QoS message stating B1 arrived J1
too late. This means generating B1 took (B1 - J1) - B0 = T1 - T0 = PT1, as
too late. This means generating B1 took (B1 + J1) - B0 = T1 - T0 = PT1, as
given in (3). Given the buffer B1 had a duration D1 and assuming that
generating a new buffer B2 will take the same amount of processing time,
a better estimation for B2 would then be:
@ -136,15 +143,15 @@ a better estimation for B2 would then be:
expanding gives:
B2 = (B1 - J1) + D2 * (B1 - J1 - B0)
B2 = (B1 + J1) + D2 * (B1 + J1 - B0)
--------------
D1
assuming the durations of the frames are equal and thus D1 = D2:
B2 = (B1 - J1) + (B1 - J1 - B0)
B2 = (B1 + J1) + (B1 + J1 - B0)
B2 = 2 * (B1 - J1) - B0
B2 = 2 * (B1 + J1) - B0
also:
@ -152,11 +159,11 @@ a better estimation for B2 would then be:
so:
B2 = 2 * (B1 - J1) - (B1 - D1)
B2 = 2 * (B1 + J1) - (B1 - D1)
Which yields a more accurate prediction for the next buffer given as:
B2 = B1 - 2 * J1 + D1 (5)
B2 = B1 + 2 * J1 + D1 (5)
Long term correction
@ -226,7 +233,7 @@ A QoS message with the most current values is sent upstream for each buffer
that was received by the sink.
Normally QoS is only enabled for video pipelines. The reason being that drops
in audio is more disturbing than dropping frames. Also video requires in
in audio are more disturbing than dropping video frames. Also video requires in
general more processing than audio.
Normally there is a threshold for when buffers get dropped in a video sink. Frames
@ -259,13 +266,13 @@ the CPU usage and framerate.
If each frame is independantly decodable, any arbitrary frame can be skipped based
on the timestamp and jitter values of the latest QoS message. In addition can the
proportion member be used to skip frames.
proportion member be used to permanently skip frames.
Demuxers
--------
Demuxer usually cannot do a lot regarding QoS except for skipping frames to the next
Demuxers usually cannot do a lot regarding QoS except for skipping frames to the next
keyframe when a lateness QoS message arrives on a source pad.
A demuxer can however measure if the performance problems are upstream or downstream

11
gst/gstclock.c
View file

@ -99,7 +99,7 @@
* "window-threshold" defines the minimum number of samples before the
* calibration is performed.
*
* Last reviewed on 2006-03-09 (0.10.4)
* Last reviewed on 2006-08-11 (0.10.10)
*/
@ -347,9 +347,10 @@ gst_clock_id_get_time (GstClockID id)
* If the @jitter argument is not NULL and this function returns #GST_CLOCK_OK
* or #GST_CLOCK_EARLY, it will contain the difference
* against the clock and the time of @id when this method was
* called. Negative values indicate how late @id was relative to the clock
* called.
* Positive values indicate how late @id was relative to the clock
* (in which case this function will return #GST_CLOCK_EARLY).
* Positive values indicate how much time was spent waiting on the clock
* Negative values indicate how much time was spent waiting on the clock
* before this function returned.
*
* Returns: the result of the blocking wait. #GST_CLOCK_EARLY will be returned
@ -390,10 +391,12 @@ gst_clock_id_wait (GstClockID id, GstClockTimeDiff * jitter)
GST_CAT_DEBUG_OBJECT (GST_CAT_CLOCK, clock, "waiting on clock entry %p", id);
/* jitter is the diff against the clock when this entry is scheduled */
if (jitter) {
GstClockTime now = gst_clock_get_time (clock);
/* jitter is the diff against the clock when this entry is scheduled. Negative
* values mean that the entry was in time, a positive value means that the
* entry was too late. */
*jitter = GST_CLOCK_DIFF (requested, now);
}
res = cclass->wait (clock, entry);

58
gst/gstevent.c
View file

@ -72,7 +72,7 @@
* </programlisting>
* </example>
*
* Last reviewed on 2006-01-24 (0.10.2)
* Last reviewed on 2006-08-11 (0.10.10)
*/
@ -315,6 +315,10 @@ gst_event_new_custom (GstEventType type, GstStructure * structure)
{
GstEvent *event;
/* structure must not have a parent */
if (structure)
g_return_val_if_fail (structure->parent_refcount == NULL, NULL);
event = gst_event_new (type);
if (structure) {
gst_structure_set_parent_refcount (structure, &event->mini_object.refcount);
@ -473,7 +477,7 @@ gst_event_parse_new_segment (GstEvent * event, gboolean * update,
* The newsegment event marks the range of buffers to be processed. All
* data not within the segment range is not to be processed. This can be
* used intelligently by plugins to apply more efficient methods of skipping
* unneeded packets.
* unneeded data.
*
* The position value of the segment is used in conjunction with the start
* value to convert the buffer timestamps into the stream time. This is
@ -599,6 +603,7 @@ GstEvent *
gst_event_new_tag (GstTagList * taglist)
{
g_return_val_if_fail (taglist != NULL, NULL);
return gst_event_new_custom (GST_EVENT_TAG, (GstStructure *) taglist);
}
@ -614,6 +619,7 @@ gst_event_parse_tag (GstEvent * event, GstTagList ** taglist)
{
g_return_if_fail (GST_IS_EVENT (event));
g_return_if_fail (GST_EVENT_TYPE (event) == GST_EVENT_TAG);
if (taglist)
*taglist = (GstTagList *) event->structure;
}
@ -641,6 +647,7 @@ gst_event_new_buffer_size (GstFormat format, gint64 minsize,
"creating buffersize format %d, minsize %" G_GINT64_FORMAT
", maxsize %" G_GINT64_FORMAT ", async %d", format,
minsize, maxsize, async);
return gst_event_new_custom (GST_EVENT_BUFFERSIZE,
gst_structure_new ("GstEventBufferSize",
"format", GST_TYPE_FORMAT, format,
@ -667,6 +674,7 @@ gst_event_parse_buffer_size (GstEvent * event, GstFormat * format,
g_return_if_fail (GST_IS_EVENT (event));
g_return_if_fail (GST_EVENT_TYPE (event) == GST_EVENT_BUFFERSIZE);
structure = gst_event_get_structure (event);
if (format)
*format = g_value_get_enum (gst_structure_get_value (structure, "format"));
@ -690,24 +698,31 @@ gst_event_parse_buffer_size (GstEvent * event, GstFormat * format,
* The QOS event is generated in an element that wants an upstream
* element to either reduce or increase its rate because of
* high/low CPU load or other resource usage such as network performance.
* Typically sinks generate these events for each buffer they receive.
*
* @proportion is the requested adjustment in datarate, 1.0 is the normal
* datarate, 0.75 means increase datarate by 75%, 1.5 is 150%. Negative
* values request a slow down, so -0.75 means a decrease by 75%.
* @proportion indicates the real-time performance of the streaming in the
* element that generated the QoS event (usually the sink). The value is
* generally computed based on more long term statistics about the streams
* timestamps compared to the clock.
* A value < 1.0 indicates that the upstream element is producing data faster
* than real-time. A value > 1.0 indicates that the upstream element is not
* producing data fast enough. 1.0 is the ideal @proportion value. The
* proportion value can safely be used to lower or increase the quality of
* the element.
*
* @diff is the difference against the clock in stream time of the last
* buffer that caused the element to generate the QOS event.
* @diff is the difference against the clock in running time of the last
* buffer that caused the element to generate the QOS event. A negative value
* means that the buffer with @timestamp arrived in time. A positive value
* indicates how late the buffer with @timestamp was.
*
* @timestamp is the timestamp of the last buffer that cause the element
* to generate the QOS event.
* to generate the QOS event. It is expressed in running time and thus an ever
* increasing value.
*
* The upstream element can use the @diff and @timestamp values to decide
* whether to process more buffers. All buffers with timestamp <=
* @timestamp + @diff will certainly arrive late in the sink as well.
*
* The @proportion value is generally computed based on more long
* term statistics about the stream quality and can be used in various ways
* such as lowering or increasing processing quality.
* whether to process more buffers. For possitive @diff, all buffers with
* timestamp <= @timestamp + @diff will certainly arrive late in the sink
* as well.
*
* The application can use general event probes to intercept the QoS
* event and implement custom application specific QoS handling.
@ -737,7 +752,8 @@ gst_event_new_qos (gdouble proportion, GstClockTimeDiff diff,
* @diff: A pointer to store the diff in
* @timestamp: A pointer to store the timestamp in
*
* Get the proportion, diff and timestamp in the qos event.
* Get the proportion, diff and timestamp in the qos event. See
* gst_event_new_qos() for more information about the different QoS values.
*/
void
gst_event_parse_qos (GstEvent * event, gdouble * proportion,
@ -747,6 +763,7 @@ gst_event_parse_qos (GstEvent * event, gdouble * proportion,
g_return_if_fail (GST_IS_EVENT (event));
g_return_if_fail (GST_EVENT_TYPE (event) == GST_EVENT_QOS);
structure = gst_event_get_structure (event);
if (proportion)
*proportion =
@ -779,9 +796,10 @@ gst_event_parse_qos (GstEvent * event, gdouble * proportion,
* rate is not allowed.
*
* @cur_type and @stop_type specify how to adjust the current and stop
* time, relative or absolute. A type of #GST_SEEK_TYPE_NONE means that
* the position should not be updated. The currently configured playback
* segment can be queried with #GST_QUERY_SEGMENT.
* time, relative or absolute to the last configured positions. A type
* of #GST_SEEK_TYPE_NONE means that the position should not be updated.
* The currently configured playback segment can be queried with
* #GST_QUERY_SEGMENT.
*
* Note that updating the @cur position will actually move the current
* playback pointer to that new position. It is not possible to seek
@ -795,6 +813,8 @@ GstEvent *
gst_event_new_seek (gdouble rate, GstFormat format, GstSeekFlags flags,
GstSeekType cur_type, gint64 cur, GstSeekType stop_type, gint64 stop)
{
g_return_val_if_fail (rate != 0.0, NULL);
if (format == GST_FORMAT_TIME) {
GST_CAT_INFO (GST_CAT_EVENT,
"creating seek rate %lf, format TIME, flags %d, "
@ -842,6 +862,7 @@ gst_event_parse_seek (GstEvent * event, gdouble * rate,
g_return_if_fail (GST_IS_EVENT (event));
g_return_if_fail (GST_EVENT_TYPE (event) == GST_EVENT_SEEK);
structure = gst_event_get_structure (event);
if (rate)
*rate = g_value_get_double (gst_structure_get_value (structure, "rate"));
@ -873,5 +894,6 @@ GstEvent *
gst_event_new_navigation (GstStructure * structure)
{
g_return_val_if_fail (structure != NULL, NULL);
return gst_event_new_custom (GST_EVENT_NAVIGATION, structure);
}