docs/design/draft-latency.txt: Fix typo.

Original commit message from CVS:
* docs/design/draft-latency.txt:
Fix typo.
* docs/design/part-buffering.txt:
Update design docs with more buffering ideas.

ChangeLog

@@ -1,3 +1,11 @@
+2008-04-04  Wim Taymans  <wim.taymans@collabora.co.uk>
+
+	* docs/design/draft-latency.txt:
+	Fix typo.
+
+	* docs/design/part-buffering.txt:
+	Update design docs with more buffering ideas.
+
 2008-04-03  Tim-Philipp Müller  <tim at centricular dot net>
 
 	* configure.ac:

docs/design/draft-latency.txt

@@ -293,7 +293,7 @@ A flush in a pipeline can happen in the following cases:
 
 When a playing sink is flushed by a FLUSH_START event, an ASYNC_START  message is
 posted by the element. As part of the message, the fact that the element got
-flushes is included. The element also goes to a pending PAUSED state and has to
+flushed is included. The element also goes to a pending PAUSED state and has to
 be set to the PLAYING state again later.
 
 The ASYNC_START message is kept by the parent bin. When the element prerolls,

docs/design/part-buffering.txt

@@ -6,26 +6,109 @@ core that can be used by plugins and applications.
 
 The purpose of buffering is to accumulate enough data in a pipeline so that
 playback can occur smoothly and without interruptions. It is typically done
-when reading from a (slow) and non-live network source.
+when reading from a (slow) and non-live network source but can also be used for
+live sources.
+
+We want to be able to implement the following features:
+
+ - buffering up to a specifc amount of data, in memory, before starting playback
+   so that network fluctuations are minimized.
+ - download of the network file to a local disk with fast seeking in the
+   downloaded data. This is similar to the quicktime/youtube players.
+ - caching of semi-live streams to a local, on disk, ringbuffer with seeking in
+   the cached area. This is similar to tivo-like timeshifting.
+ - progress report about the buffering operations
+ - easy (backward compatible) application notification of buffering
+ - the possibility for the application to do more complex buffering
 
  Some use cases:
 
-   +---------+     +--------+     +-------+
+ * Stream buffering:
-   | httpsrc |     | buffer |     | demux |
+
-   |        src - sink     src - sink     ....
+   +---------+     +---------+     +-------+
-   +---------+     +--------+     +-------+
+   | httpsrc |     | buffer  |     | demux |
+   |        src - sink      src - sink     ....
+   +---------+     +---------+     +-------+
 
   In this case we are reading from a slow network source into a buffer element
-  (such as queue). 
+  (such as queue2). 
   
   The buffer element has a low and high watermark expressed in bytes. The
   buffer uses the watermarks as follows:
   
-   - The buffer element will not produce data on the src pad until the high
+   - The buffer element will post BUFFERING messages until the high watermark
-     watermark is hit. While accumulating data in the buffer, progress is
+     is hit. This instructs the application to keep the pipeline PAUSED, which
-     reported by posting BUFFERING messages.
+     will eventually block the srcpad from pushing while data is prerolled in
-   - The source will stop to produce data on the src pad when the low watermark
+     the sinks.
-     is hit.
+   - When the high watermark is hit, a BUFFERING message with 100% will be
+     posted, which instructs the application to continue playback.
+   - When during playback, the low watermark is hit, the queue will start posting
+     BUFFERING messages again, making the application PAUSE the pipeline again
+     until the high watermark is hit again.
+   - during playback, the queue level will fluctuate between the high and the
+     low watermark as a way to compensate for network irregularities.
+
+  This buffering method is usable when the demuxer operates in push mode.
+  Seeking in the stream requires the seek to happen in the network source.
+  It is mostly desirable when the total duration of the file is not know, such
+  as in live streaming or when efficient seeking is not possible/required.
+
+ * Incremental download
+
+   +---------+     +---------+     +-------+
+   | httpsrc |     | buffer  |     | demux |
+   |        src - sink      src - sink     ....
+   +---------+     +----|----+     +-------+
+                        V
+                       file
+  
+  In this case, we know the server is streaming a fixed length file to the
+  client. The application can choose to download the file on disk. The buffer
+  element will provide a push or pull based srcpad to the demuxer to navigate in
+  the downloaded file.
+
+  This mode is only suitable when the client can determine the length of the
+  file on the server.
+
+  In this case, buffering messages will be emited as usual when the requested
+  range is not within the downloaded area + buffersize. The buffering message
+  will also contain an indication that incremental download is being performed.
+  This flag can be used to let the application control the buffering in a more
+  intelligent way, using the BUFFERING query, for example.
+
+  The application can use the BUFFERING query to get the estimated download time
+  and match this time to the current/remaining playback time to control when
+  playback should start to have a non-interupted playback experience.
+
+
+ * Timeshifting
+
+   +---------+     +---------+     +-------+
+   | httpsrc |     | buffer  |     | demux |
+   |        src - sink      src - sink     ....
+   +---------+     +----|----+     +-------+
+                        V
+                    file-ringbuffer
+
+  In this mode, a fixed size ringbuffer is kept to download the server content.
+  This allows for seeking in the buffered data. Depending on the size of the
+  buffer one can seek further back in time.
+
+  This mode is suitable for all live streams.
+
+  As with the incremental download mode, buffering messages are emited along
+  with an indication that timeshifting download is in progress. 
+
+
+ * Live buffering
+
+  In live pipelines we usually introduce some latency between the capture and
+  the playback elements. This latency can be introduced by a queue (such as a
+  jitterbuffer) or by other means (in the audiosink).
+
+  Buffering messages can be emited in those live pipelines as well and serve as
+  an indication to the user of the latency buffering. The application usually
+  does not react to these buffering messages with a state change.
 
 
 Messages
@@ -37,6 +120,46 @@ BUFFERING message is 100, buffering is done. Values less than 100 mean that
 buffering is in progress. 
 
 The BUFFERING message should be intercepted and acted upon by the application.
+The message contains at least one field that is sufficient for basic
+functionality:
+
+  "buffer-percent", G_TYPE_INT, between 0 and 100
+
+Several more clever ways of dealing with the buffering messages can be used when
+in incremental or timeshifting download mode. For this purpose additional fields
+are added to the buffering message:
+
+  "buffering-mode", GST_TYPE_BUFFERING_MODE, 
+                   enum { "none", "stream", "download", "timeshift", "live" }
+      - gives the buffering mode in use. See above for an explanation of the
+        different modes of buffering.
+
+  "avg-in-rate", G_TYPE_INT
+      - gives the average input buffering speed in bytes/second. -1 is unknown.
+
+        This is the average number of bytes per second that is received on the
+	buffering element input (sink) pads. If is a measurement of the network
+	speed in most cases.
+
+  "avg-out-rate", G_TYPE_INT
+      - gives the average consumption speed in bytes/second. -1 is unknown.
+
+        This is the average number of bytes per second that is consumed by the
+	downstream element of the buffering element. 
+
+  "buffering-left", G_TYPE_INT
+      - gives the estimated time that bufferring will take in milliseconds.
+        -1 unknown.
+
+      This is measured based on the avg-in-rate and the filled level of the
+      queue. The application can use this hint to update the gui while buffering.
+
+  "estimated-time", G_TYPE_INT
+      - gives the estimated download time in milliseconds. -1 unknown.
+
+      When the size of the downloaded file is known, this value will contain
+      the latest estimate of the remaining download time. This value is usualy
+      only filled for the "download" buffering mode.
 
 
 Application
@@ -51,29 +174,72 @@ application should not set the pipeline to PLAYING before a BUFFERING message
 with 100 percent value is received, which might only happen after the pipeline
 prerolled.
 
+An exception is made for live pipelines. The application may not change
+the state of a live pipeline when a buffering message is received. Usually these
+buffering messages contain the "buffering-mode" = "live".
+
+The buffering message can also instruct the application to switch to a periodical
+BUFFERING query instead to more precisely control the buffering process. The
+application can, for example, choose to not act on the BUFFERING message with
+100 percent fill level to resume playback but instead use the estimated download
+time to resume playback to get uninterrupted playback.
+
 
 Buffering Query
 ---------------
 
-It is possible to query the amount of buffering performed in the pipeline, which
+In addition to the BUFFERING messages posted by the buffering elements we want
-is defined as the amount of data made available at the source. This amount is
+to be able to query the same information from the application. We also want to
-expressed in some GstFormat and is usually compared to the duration or position 
+be able to present the user with information about the downloaded range in the
-of the media stream in the same GstFormat.
+file so that the GUI can react on it.
 
-The buffering query should return the following information:
+In addition to all the fields present in the buffering message, the BUFFERING
+query contains the following field, which indicate the available downloaded
+range in a specific format:
 
- - format
+  "format", GST_TYPE_FORMAT
- - position
+    - the format of the "start" and "stop" values below  
- - duration
+   
+  "start", G_TYPE_INT64, -1 unknown
+    - the start position of the available data
 
-The format is of lesser importance, the ratio of position versus duration can be
+  "stop", G_TYPE_INT64, -1 unknown
-used to calculate the percentage of available media. It should also be possible
+    - the stop position of the available data
-for the application to calculate the expected time when the complete file will
+
-be buffered. 
+For the "download" and "timeshift" buffering-modes, the start and stop positions
+specify the ranges where efficient seeking in the downloaded media is possible.
+Seeking outside of these ranges might be slow or not at all possible.
+
+For the "stream" and "live" mode the start and stop values describe the oldest
+and newest item (expressed in "format") in the buffer. 
 
 
+Defaults
+--------
 
-Incremental download
+Some defaults for common elements:
---------------------
 
+A GstBaseSrc with random access replies to the BUFFERING query with:
+
+  "buffer-percent" = 100
+  "buffering-mode" = "none"
+  "avg-in-rate" = -1
+  "avg-out-rate" = -1
+  "buffering-left" = 0
+  "estimated-time" = 0
+  "format" = GST_FORMAT_BYTES
+  "start" = 0
+  "stop" = the total filesize
+
+A GstBaseSrc in push mode replies to the BUFFERING query with:
+
+  "buffer-percent" = 100
+  "buffering-mode" = "none"
+  "avg-in-rate" = -1
+  "avg-out-rate" = -1
+  "buffering-left" = 0
+  "estimated-time" = 0
+  "format" = a valid GST_TYPE_FORMAT
+  "start" = current position
+  "stop" = current position