design: update QoS document

Add new QoS types and talk about the new throttle QoS message.

docs/design/part-qos.txt

@@ -16,6 +16,10 @@ made:
     in the sinks.
   - long term rate corrections based on trends observed in the sinks.
 
+It is also possible for the application to artificially introduce delay
+between synchronized buffers, this is called throttling. It can be used
+to reduce the framerate, for example.
+
 
 Sources of quality problems
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -23,6 +27,7 @@ Sources of quality problems
  - High CPU load
  - Network problems
  - Other resource problems such as disk load, memory bottlenecks etc.
+ - application level throttling
 
 
 QoS event
@@ -31,6 +36,19 @@ QoS event
 The QoS event is generated by an element that synchronizes against the clock. It
 travels upstream and contains the following fields:
 
+ - type, GST_TYPE_QOS_TYPE:
+      The type of the QoS event, we have the following types and the default type
+      is GST_QOS_TYPE_UNDERFLOW:
+
+      GST_QOS_TYPE_OVERFLOW:  an element is receiving buffers too fast and can't
+                              keep up processing them. Upstream should reduce the
+                              rate.
+      GST_QOS_TYPE_UNDERFLOW: an element is receiving buffers too slowly and has
+                              to drop them because they are too late. Upstream should
+                              increase the processing rate.
+      GST_QOS_TYPE_THROTTLE:  the application is asking to add extra delay between
+                              buffers, upstream is allowed to drop buffers
+
  - timestamp, G_TYPE_UINT64:
       The timestamp on the buffer that generated the QoS event. These timestamps
       are expressed in total running_time in the sink so that the value is ever
@@ -39,7 +57,9 @@ travels upstream and contains the following fields:
  - jitter, G_TYPE_INT64:
       The difference of that timestamp against the current clock time.  Negative
       values mean the timestamp was on time. Positive values indicate the
-      timestamp was late by that amount.
+      timestamp was late by that amount. 
+      When throttling, the jitter contains the throttling delay added by the 
+      application.
 
  - proportion, G_TYPE_DOUBLE:
       Long term prediction of the ideal rate relative to normal rate to get
@@ -175,7 +195,7 @@ Values DR1 > 1.0 mean that the upstream element cannot produce buffers of
 duration D1 in real-time. It is exactly DR1 that tells the amount of speedup
 we require from upstream to regain real-time performance.
 
-An element that is not receiving enough data is said to be starved.
+An element that is not receiving enough data is said to be underflowed.
 
 
 Element measurements
@@ -185,7 +205,7 @@ In addition to the measurements of the datarate of the upstream element, a
 typical element must also measure its own performance. Global pipeline 
 performance problems can indeed also be caused by the element itself when it
 receives too much data it cannot process in time. The element is then said to
-be flooded.
+be overflowed.
 
 
 Short term correction
@@ -259,6 +279,24 @@ as given by the proportion member. Failure to do so will certainly lead to
 more dropped frames and a generally worse QoS.
 
 
+Throttling
+----------
+
+In throttle mode, the time distance between buffers is kept to a configurable
+throttle interval. This means that effectively the buffer rate is limited
+to 1 buffer per throttle interval. This can be used to limit the framerate,
+for example.
+
+When an element is configured in throttling mode (this is usually only
+implemented on sinks) it should produce QoS events upstream with the jitter
+field set to the throttle interval. This should instruct upstream elements to
+skip or drop the remaining buffers in the configured throttle interval.
+
+The proportion field is set to the desired slowdown needed to get the
+desired throttle interval. Implementations can use the QoS Throttle type,
+the proportion and the jitter member to tune their implementations. 
+
+
 QoS strategies
 --------------
 
@@ -299,7 +337,7 @@ calculate a proportion.
 
 The processing time in system time is compared to render time to decide if
 the majority of the time is spend upstream or in the sink itself. This value
-is used to decide flood or starvation.
+is used to decide overflow or underflow.
 
 The number of rendered and dropped buffers is used to query stats on the sink.
 
@@ -316,6 +354,11 @@ the human eye.
 
 A QoS message is posted whenever a (part of a) buffer is dropped.
 
+In throttle mode, the sink sends QoS event upstream with the timestamp set to
+the running_time of the latest buffer and the jitter set to the throttle interval.
+If the throttled buffer is late, the lateness is subtracted from the throttle
+interval in order to keep the desired throttle interval. 
+
 
 GstBaseTransform
 ----------------
@@ -380,7 +423,7 @@ Sources
 The QoS events only apply to push based sources since pull based sources are entirely
 controlled by another downstream element.
 
-Sources can receive a flood or starvation event that can be used to switch to
+Sources can receive a overflow or underflow event that can be used to switch to
 less demanding source material. In case of a network stream, a switch could be done
 to a lower or higher quality stream or additional enhancement layers could be used
 or ignored.