docs: update and improve docs

docs/design/draft-metadata.txt

@@ -96,17 +96,16 @@ http://wiki.creativecommons.org/Tracker_CC_Indexing
 Current Metadata handling
 ~~~~~~~~~~~~~~~~~~~~~~~~~
 
-When reading files, demuxers or parsers extract the metadata. It will be sent as
+When reading files, demuxers or parsers extract the metadata. It will be sent 
-a GST_MESSAGE_TAG to the GstBus and GST_EVENT_TAG to downstream elements.
+a GST_EVENT_TAG to downstream elements. When a sink element receives a tag
+event, it will post a GST_MESSAGE_TAG message on the bus with the contents of
+the tag event.
 
 Elements receiving GST_EVENT_TAG events can mangle them, mux them into the 
 buffers they send or just pass them through. Usually is muxers that will format
 the tag data into the form required by the format they mux. Such elements would
 also implement the GstTagSetter interface to receive tags from the application.
 
-   GstMessage(tag) over GstBus to application
-       ^
-       |
   +----------+
   | demux    |
  sink       src --> GstEvent(tag) over GstPad to downstream element

docs/design/part-TODO.txt

@@ -51,7 +51,7 @@ API/ABI
     and another a push, the push might be busy while the block callback is done.
   * maybe this name is overloaded. We need to look at some more use cases before
     trying to fix this.
-  FIXED in 0.11 with BLOCKING probes. Not everything is implemented yet, though.
+  FIXED in 0.11 with BLOCKING probes.
 
 - rethink the way we do upstream renegotiation. Currently it's done with
   pad_alloc but this has many issues such as only being able to suggest 1 format
@@ -61,7 +61,7 @@ API/ABI
   This could, for example, easily be triggered with a RENEGOTIATE event.
   FIXED in 0.11 with RECONFIGURE events.
 
-- Remove the result format value in queries.
+- Remove the result format value in queries. FIXED in 0.11
 
 - Try to minimize the amount of acceptcaps calls when pushing buffers around.
   The element pushing the buffer usually negotiated already and decided on the

docs/design/part-bufferpool.txt

@@ -147,7 +147,7 @@ Allocating from pool
  will block when all buffers are outstanding until a buffer is returned to the
  pool. This behaviour can be changed by specifying the
  GST_BUFFER_POOL_FLAG_DONTWAIT flag in the parameters. With this flag set,
- allocation will return GST_FLOW_UNEXPECTED when the pool is empty.
+ allocation will return GST_FLOW_EOS when the pool is empty.
 
 
 Renegotiation

docs/design/part-element-transform.txt

@@ -273,7 +273,7 @@ Allocation
 ~~~~~~~~~~
 
 After the transform element is configured with caps, a bufferpool needs to be
-negotiated to perform the allocation of buffers. We habe 2 cases:
+negotiated to perform the allocation of buffers. We have 2 cases:
 
   - The element is operating in passthrough we don't need to allocate a buffer
     in the transform element.

docs/design/part-events.txt

@@ -34,27 +34,23 @@ Different types of events exist to implement various functionalities.
 src pads
 --------
 
-A gst_pad_push_event() on a srcpad will first store the event in the sticky
+A gst_pad_push_event() on a srcpad will first store the sticky event in the
-array before sending the event to the peer pad. If there is no peer pad, the
+sticky array before sending the event to the peer pad. If there is no peer pad
-gst_pad_push_event() function returns NOT_LINKED.
+and the event was not stored in the sticky array, FALSE is returned.
 
-Note that the behaviour is not influenced by a flushing pad.
+Flushing pads will refuse the events and will not store the sticky events.
 
-FLUSH_START and FLUSH_STOP events are dropped on blocked pads.
 
 sink pads
 ---------
 
-A gst_pad_send_event() on a sinkpad will check the new event against the
+A gst_pad_send_event() on a sinkpad will call the event function on the pad. If
-existing event. If they are different, the new event is stored as a pending
+the event function returns success, the sticky event is stored in the sticky
-event. If the events are the same, nothing changes.
+event array and the event is marked for update.
 
-When the pad is flushing, the _send_event() function returns WRONG_STATE
+When the pad is flushing, the _send_event() function returns FALSE immediately.
-immediately.
 
-The event function is then called for all pending events. If the function
+When the next data item is pushed, the pending events are pushed first.
-returns success, the pending event is copied to the active events, else the
-pending event is removed and the current active event is unchanged.
 
 This ensures that the event function is never called for flushing pads and that
 the sticky array only contains events for which the event function returned
@@ -64,9 +60,9 @@ success.
 pad link
 --------
 
-When linking pads, all the sticky events from the srcpad are copied to the
+When linking pads, the srcpad sticky events are marked for update when they are
-pending array on the sinkpad. The pending events will be sent to the event
+different from the sinkpad events. The next buffer push will push the events to
-function of the sinkpad on the next event or buffer.
+the sinkpad.
 
 
 FLUSH_START/STOP
@@ -112,7 +108,7 @@ The EOS event can only be sent on a sinkpad. It is typically emited by the
 source element when it has finished sending data. This event is mainly sent
 in the streaming thread but can also be sent from the application thread.
 
-An EOS event sent on a srcpad returns GST_FLOW_UNEXPECTED.
+An EOS event sent on a srcpad returns GST_FLOW_EOS.
 
 The downstream element should forward the EOS event to its downstream peer
 elements. This way the event will eventually reach the sinks which should
@@ -131,7 +127,7 @@ GStreamer core will take the STREAM_LOCK.
 Sometimes the EOS event is generated by another element than the source, for 
 example a demuxer element can generate an EOS event before the source element.
 This is not a problem, the demuxer does not send an EOS event to the upstream
-element but returns GST_FLOW_UNEXPECTED, causing the source element to stop
+element but returns GST_FLOW_EOS, causing the source element to stop
 sending data.
 
 An element that sends EOS on a pad should stop sending data on that pad. Source

docs/design/part-latency.txt

@@ -220,6 +220,31 @@ separate async state change thread (like the one currently used for going from
 PAUSED->PLAYING in a non-live pipeline).
 
 
+Query
+~~~~~
+
+The pipeline latency is queried with the LATENCY query.
+
+ (out) "live", G_TYPE_BOOLEAN (default FALSE)
+        - if a live element is found upstream
+
+ (out) "min-latency", G_TYPE_UINT64 (default 0)
+        - the minimum latency in the pipeline
+
+ (out) "max-latency", G_TYPE_UINT64 (default 0)
+        - the maximum latency in the pipeline
+
+
+Event
+~~~~~
+
+The latency in the pipeline is configured with the LATENCY event, which contains
+the following fields:
+
+      "latency", G_TYPE_UINT64
+        - the configured latency in the pipeline
+
+
 Latency compensation
 ~~~~~~~~~~~~~~~~~~~~
 

docs/design/part-meta.txt

@@ -61,6 +61,7 @@ GstMeta
 A GstMeta is a structure as follows:
 
   struct _GstMeta {
+    GstMetaFlags       flags;
     const GstMetaInfo *info;    /* tag and info for the meta item */
   };
 
@@ -166,7 +167,8 @@ The complete buffer with metadata could, for example, look as follows:
 GstMiniObject            |     GType (GstBuffer)               |
                          |     refcount, flags, copy/disp/free |
                          +-------------------------------------+
-GstBuffer                |     caps, parent, pool              |
+GstBuffer                |     pool,pts,dts,duration,offsets   |
+                         |     <private data>                  |
                          +.....................................+
                          |     next                           ---+
                       +- |     info                           ------> GstMetaInfo

docs/design/part-negotiation.txt

@@ -8,10 +8,65 @@ pipeline that have information to those parts of the pipeline that are
 flexible, constrained by those parts of the pipeline that are not
 flexible.
 
+
+Basic rules
+~~~~~~~~~~~
+
+The simple rules must be followed:
+
+ 1) downstream suggests formats
+ 2) upstream decides on format
+
+There are 4 queries/events used in caps negotiation:
+
+ 1) GST_QUERY_CAPS        : get possible formats
+ 2) GST_QUERY_ACCEPT_CAPS : check if format is possible
+ 3) GST_EVENT_CAPS        : configure format (downstream)
+ 4) GST_EVENT_RECONFIGURE : inform upstream of possibly new caps
+
+
+Queries
+-------
+
+A pad can ask the peer pad for its supported GstCaps. It does this with
+the CAPS query. The list of supported caps can be used to choose an
+appropriate GstCaps for the data transfer.
+
+ (in) "filter", GST_TYPE_CAPS (default NULL)
+       - a GstCaps to filter the results against
+
+ (out) "caps", GST_TYPE_CAPS (default NULL)
+       - the result caps
+
+
+A pad can ask the peer pad if it supports a given caps. It does this with
+the ACCEPT_CAPS query.
+
+ (in) "caps", GST_TYPE_CAPS
+       - a GstCaps to check
+
+ (out) "result", G_TYPE_BOOLEAN (default FALSE)
+       - TRUE if the caps are accepted
+
+
+Events
+~~~~~~
+
+When a media format is negotiated, peer elements are notified of the GstCaps
+with the CAPS event. The caps must be fixed.
+
+    "caps", GST_TYPE_CAPS
+       - the negotiated GstCaps
+
+
+Operation
+~~~~~~~~~
+
 GStreamer's two scheduling modes, push mode and pull mode, lend
 themselves to different mechanisms to achieve this goal. As it is more
 common we describe push mode negotiation first.
 
+
 Push-mode negotiation
 ~~~~~~~~~~~~~~~~~~~~~
 
@@ -31,9 +86,9 @@ The basics of negotiation are as follows:
 
  - An element should reconfigure itself to the new format received as a CAPS
    event before processing the following buffers. If the data type in the
-   caps event is not acceptable, the element should refuse the buffer by 
+   caps event is not acceptable, the element should refuse the event. The
-   returning an appropriate GST_FLOW_NOT_NEGOTIATED return value from the
+   element should also refuse the next buffers by returning an appropriate
-   chain function.
+   GST_FLOW_NOT_NEGOTIATED return value from the chain function.
 
  - Downstream elements can request a format change of the stream by sending a
    RECONFIGURE event upstream. Upstream elements will renegotiate a new format
@@ -43,11 +98,18 @@ The general flow for a source pad starting the negotiation.
 
              src              sink
               |                 |
-              |  accepts?       |
+              |  querycaps?     |
-  type A      |---------------->|
+              |---------------->|
-              |      yes        |
+              |     caps        |
-              |< - - - - - - - -|
+ select caps  |< - - - - - - - -|
-              |                 |
+ from the     |                 |
+ candidates   |                 |
+              |                 |-.
+              |  accepts?       | |
+  type A      |---------------->| | optional
+              |      yes        | |
+              |< - - - - - - - -| |
+              |                 |-'
               |  send_event()   |
  send CAPS    |---------------->| Receive type A, reconfigure to
  event A      |                 | process type A.
@@ -61,9 +123,9 @@ The general flow for a source pad starting the negotiation.
  [element wants to create a buffer]
  if not format
    # see what we can do
-   ourcaps = gst_pad_get_caps (srcpad)
+   ourcaps = gst_pad_query_caps (srcpad)
    # see what the peer can do filtered against our caps
-   candidates = gst_pad_peer_get_caps (srcpad, ourcaps)
+   candidates = gst_pad_peer_query_caps (srcpad, ourcaps)
 
    foreach candidate in candidates
      # make sure the caps is fixed
@@ -100,7 +162,7 @@ The general flow for a sink pad starting a renegotiation.
   mark      .-|<----------------| send RECONFIGURE event
  renegotiate| |                 |
             '>|                 |
-              |   get_caps()    |
+              |  querycaps()    |
  renegotiate  |---------------->| 
               |  suggest B      |
               |< - - - - - - - -|
@@ -121,7 +183,7 @@ videotestsrc ! xvimagesink
 
   1) Who decides what format to use?
    - src pad always decides, by convention. sinkpad can suggest a format
-     by putting it high in the getcaps function GstCaps. 
+     by putting it high in the caps query result GstCaps. 
    - since the src decides, it can always choose something that it can do,
      so this step can only fail if the sinkpad stated it could accept
      something while later on it couldn't.
@@ -143,11 +205,11 @@ videotestsrc ! xvimagesink
    - src receives the RECONFIGURE event and marks renegotiation
    - On the next buffer push, the source renegotiates the caps and the
      bufferpool. The sink will put the new new prefered format high in the list
-     of caps it returns from its getcaps function.
+     of caps it returns from its caps query.
 
 videotestsrc ! queue ! xvimagesink
 
-  - queue proxies all accept and getcaps to the other peer pad.
+  - queue proxies all accept and caps queries to the other peer pad.
   - queue proxies the bufferpool
   - queue proxies the RECONFIGURE event
   - queue stores CAPS event in the queue. This means that the queue can contain
@@ -230,15 +292,15 @@ The sink determines that the upstream elements support pull based scheduling by
 doing a SCHEDULING query.
 
 The sink initiates the negotiation process by intersecting the results
-of gst_pad_get_caps() on its sink pad and its peer src pad. This is the
+of gst_pad_query_caps() on its sink pad and its peer src pad. This is the
 operation performed by gst_pad_get_allowed_caps(). In the simple
-passthrough case, the peer pad's getcaps() function should return the
+passthrough case, the peer pad's caps query should return the
 intersection of calling get_allowed_caps() on all of its sink pads. In
 this way the sink element knows the capabilities of the entire pipeline.
 
 The sink element then fixates the resulting caps, if necessary,
-resulting in the flow caps.  From now on, the getcaps function
+resulting in the flow caps.  From now on, the caps query of the sinkpad
-of the sinkpad will only return these fixed caps meaning that upstream elements
+will only return these fixed caps meaning that upstream elements
 will only be able to produce this format.
 
 If the sink element could not set caps on its sink pad, it should post
@@ -255,3 +317,25 @@ function. The state will commit to PAUSED when the first buffer is received in
 the sink. This is needed to provide a consistent API to the applications that
 expect ASYNC return values from sinks but it also allows us to perform the
 remainder of the negotiation outside of the context of the pulling thread.
+
+
+Patterns
+~~~~~~~~
+
+We can identify 3 patterns in negotiation:
+
+ 1) Fixed : Can't choose the output format
+      - Caps encoded in the stream
+      - A video/audio decoder
+      - usually uses gst_pad_use_fixed_caps()
+
+ 2) Passthrough
+      - Caps not modified
+      - can do caps transform based on element property
+      - videobox
+
+ 3) Dynamic : can choose output format 
+      - A converter element
+      - depends on downstream caps, needs to do a CAPS query to find
+        transform.
+      - usually prefers to use the identity transform

docs/design/part-probes.txt

@@ -15,8 +15,8 @@ identify the following types:
    This is essential to be able to implement dynamic relinking of elements
    without breaking the dataflow.
 
- - be notified when data or events are pushed or sent on a pad. It should also
+ - be notified when data, events or queries are pushed or sent on a pad. It
-   be possible to inspect and modify the data.
+   should also be possible to inspect and modify the data.
  
  - be able to drop, pass and block on data based on the result of the callback.
 
@@ -52,14 +52,17 @@ Overview
       GST_PAD_PROBE_TYPE_BLOCK            = (1 << 1),
 
       /* flags to select datatypes */
-      GST_PAD_PROBE_TYPE_BUFFER           = (1 << 2),
+      GST_PAD_PROBE_TYPE_BUFFER           = (1 << 4),
-      GST_PAD_PROBE_TYPE_BUFFER_LIST      = (1 << 3),
+      GST_PAD_PROBE_TYPE_BUFFER_LIST      = (1 << 5),
-      GST_PAD_PROBE_TYPE_EVENT_DOWNSTREAM = (1 << 4),
+      GST_PAD_PROBE_TYPE_EVENT_DOWNSTREAM = (1 << 6),
-      GST_PAD_PROBE_TYPE_EVENT_UPSTREAM   = (1 << 5),
+      GST_PAD_PROBE_TYPE_EVENT_UPSTREAM   = (1 << 7),
+      GST_PAD_PROBE_TYPE_EVENT_FLUSH      = (1 << 8),
+      GST_PAD_PROBE_TYPE_QUERY_DOWNSTREAM = (1 << 9),
+      GST_PAD_PROBE_TYPE_QUERY_UPSTREAM   = (1 << 10),
 
       /* flags to select scheduling mode */
-      GST_PAD_PROBE_TYPE_PUSH             = (1 << 6),
+      GST_PAD_PROBE_TYPE_PUSH             = (1 << 12),
-      GST_PAD_PROBE_TYPE_PULL             = (1 << 7),
+      GST_PAD_PROBE_TYPE_PULL             = (1 << 13),
 
     } GstPadProbeType;
 
@@ -111,7 +114,7 @@ Blocking probes
   the callback will be called as soon as the pad becomes idle in the streaming
   thread.
 
-  The IDLE probe in useful to perform dynamic linking, it allows to wait for for
+  The IDLE probe is useful to perform dynamic linking, it allows to wait for for
   a safe moment when an unlink/link operation can be done. Since the probe is a
   blocking probe, it will also make sure that the pad stays idle until the probe
   is removed.
@@ -279,3 +282,6 @@ Query probes have the GST_PAD_PROBE_TYPE_QUERY_* flag set in the callbacks.
         < - - - - - - O                               | 
                       |                               | 
 
+For queries, the PUSH ProbeType is set when the query is traveling to the object
+that will answer the query and the PULL type is set when the query contains the
+answer.

docs/design/part-query.txt

@@ -1,44 +1,12 @@
 Query
 -----
 
-NOTE: this is implemented as proposed
-
 Purpose
 ~~~~~~~
 
 Queries are used to get information about the stream. 
 A query is started on a specific pad and travels up or downstream.
 
-Types of queries
-~~~~~~~~~~~~~~~~
-
-  - get length of stream
-  - get position in stream
-  - get seeking capabilities
-  - get latency
-  - convert one value to another
-  - query supported formats
-  - query internal links.
-
-Current implementation
-~~~~~~~~~~~~~~~~~~~~~~
-
-  The current implementation of query requires pads to implement the 
-  following functions:
-
-     gboolean                (*GstPadConvertFunction)        (GstPad *pad,
-                                                              GstFormat src_format,  gint64  src_value,
-                                                              GstFormat *dest_format, gint64 *dest_value);
-     gboolean                (*GstPadQueryFunction)          (GstPad *pad, GstQueryType type,
-                                                              GstFormat *format, gint64  *value);
-     GList*                  (*GstPadIntLinkFunction)        (GstPad *pad);
-     const GstFormat*        (*GstPadFormatsFunction)        (GstPad *pad);
-     const GstEventMask*     (*GstPadEventMaskFunction)      (GstPad *pad);
-     const GstQueryType*     (*GstPadQueryTypeFunction)      (GstPad *pad);
-
-  Most of these functions are not very extensible in particular,
-  the queryfunction can only return one value.
-
 
 Requirements
 ~~~~~~~~~~~~
@@ -47,29 +15,23 @@ Requirements
   - one pad function to perform the query
   - extensible queries.
 
-Proposition
+Implementation
-~~~~~~~~~~~
+~~~~~~~~~~~~~~
 
-  - define GstQuery extending GstMiniObject and containing a GstStructure (see GstMessage)
+  - GstQuery extends GstMiniObject and contains a GstStructure (see GstMessage)
-  - define standard query types (see proposed types)
+  - some standard query types are defined below
-  - define methods to create a parse the results in the GstQuery.
+  - methods to create and parse the results in the GstQuery.
   - define pad method:
 
-     gboolean                (*GstPadQueryFunction)          (GstPad *pad, GstQuery *query);
+     gboolean                (*GstPadQueryFunction)          (GstPad    *pad,
+                                                              GstObject *parent,
+                                                              GstQuery  *query);
 
     pad returns result in query structure and TRUE as result or FALSE when
     query is not supported.
 
-Proposed types
+Query types
-~~~~~~~~~~~~~~
+~~~~~~~~~~~
-
- - GST_QUERY_SEEKING:
-   
-     get info on how seeking can be done 
-       - getrange, with/without offset/size
-       - ranges where seeking is efficient (for caching network sources)
-       - flags describing seeking behaviour (forward, backward, segments,
-	             play backwards, ...)
 
  - GST_QUERY_POSITION:
   
@@ -81,7 +43,19 @@ Proposed types
  
  - GST_QUERY_LATENCY:
 
-     get amount of latency introduced in the pipeline. 
+     get amount of latency introduced in the pipeline. (See part-latency.txt)
+
+ - GST_QUERY_RATE:
+
+     get the current playback rate of the pipeline
+
+ - GST_QUERY_SEEKING:
+   
+     get info on how seeking can be done 
+       - getrange, with/without offset/size
+       - ranges where seeking is efficient (for caching network sources)
+       - flags describing seeking behaviour (forward, backward, segments,
+	             play backwards, ...)
 
  - GST_QUERY_SEGMENT:
 
@@ -95,3 +69,32 @@ Proposed types
 
      return list of supported formats that can be used for GST_QUERY_CONVERT.
 
+ - GST_QUERY_BUFFERING:
+
+     query available media for efficient seeking (See part-buffering.txt)
+
+ - GST_QUERY_CUSTOM:
+
+     a custom query, the name of the query defines the properties of the query.
+
+ - GST_QUERY_URI:
+
+     query the uri of the source or sink element
+
+ - GST_QUERY_ALLOCATION:
+
+     the buffer allocation properties (See part-bufferpool.txt)
+
+ - GST_QUERY_SCHEDULING:
+
+     the scheduling properties (See part-scheduling.txt)
+
+ - GST_QUERY_ACCEPT_CAPS:
+
+     check if caps are supported (See part-negotiation.txt)
+
+ - GST_QUERY_CAPS:
+
+     get the possible caps (See part-negotiation.txt)
+    
+

docs/design/part-segments.txt

@@ -90,7 +90,7 @@ Use case: FLUSHING seek
     segment, the decoder knows it should not send the P frame.
 
     Avidemux stops sending data after pushing a frame with timestamp 5 and
-    returns GST_FLOW_UNEXPECTED from the chain function to make the upstream
+    returns GST_FLOW_EOS from the chain function to make the upstream
     elements perform the EOS logic.
 
 

docs/design/part-streams.txt

@@ -64,7 +64,7 @@ Typical stream
 
   3) EOS
      - marks the end of data, nothing is to be expected after EOS, elements
-       should refuse more data and return GST_FLOW_UNEXPECTED. A FLUSH_STOP
+       should refuse more data and return GST_FLOW_EOS. A FLUSH_STOP
        event clears the EOS state of an element.
 
 

gst/gstquery.c

@@ -566,8 +566,8 @@ gst_query_new_latency (void)
  * gst_query_set_latency:
  * @query: a #GstQuery
  * @live: if there is a live element upstream
- * @min_latency: the minimal latency of the live element
+ * @min_latency: the minimal latency of the upstream elements
- * @max_latency: the maximal latency of the live element
+ * @max_latency: the maximal latency of the upstream elements
  *
  * Answer a latency query by setting the requested values in the given format.
  *