mirrors/gstreamer
mirrors/gstreamer
1
1
Fork 0
mirror of https://gitlab.freedesktop.org/gstreamer/gstreamer.git synced 2025-02-03 21:12:26 +00:00
Code Issues 1 Projects Releases Wiki Activity

docs: update and improve docs

Browse source
This commit is contained in:
Wim Taymans 2012-01-14 19:16:01 +01:00
parent 552d726c3a
commit 3d4a1a98b5
13 changed files with 223 additions and 108 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

9
docs/design/draft-metadata.txt
View file

@ -96,17 +96,16 @@ http://wiki.creativecommons.org/Tracker_CC_Indexing
Current Metadata handling
~~~~~~~~~~~~~~~~~~~~~~~~~
When reading files, demuxers or parsers extract the metadata. It will be sent as
a GST_MESSAGE_TAG to the GstBus and GST_EVENT_TAG to downstream elements.
When reading files, demuxers or parsers extract the metadata. It will be sent
a GST_EVENT_TAG to downstream elements. When a sink element receives a tag
event, it will post a GST_MESSAGE_TAG message on the bus with the contents of
the tag event.
Elements receiving GST_EVENT_TAG events can mangle them, mux them into the
buffers they send or just pass them through. Usually is muxers that will format
the tag data into the form required by the format they mux. Such elements would
also implement the GstTagSetter interface to receive tags from the application.
GstMessage(tag) over GstBus to application
^
|
+----------+
| demux |
sink src --> GstEvent(tag) over GstPad to downstream element

4
docs/design/part-TODO.txt
View file

@ -51,7 +51,7 @@ API/ABI
and another a push, the push might be busy while the block callback is done.
* maybe this name is overloaded. We need to look at some more use cases before
trying to fix this.
FIXED in 0.11 with BLOCKING probes. Not everything is implemented yet, though.
FIXED in 0.11 with BLOCKING probes.
- rethink the way we do upstream renegotiation. Currently it's done with
pad_alloc but this has many issues such as only being able to suggest 1 format
@ -61,7 +61,7 @@ API/ABI
This could, for example, easily be triggered with a RENEGOTIATE event.
FIXED in 0.11 with RECONFIGURE events.
- Remove the result format value in queries.
- Remove the result format value in queries. FIXED in 0.11
- Try to minimize the amount of acceptcaps calls when pushing buffers around.
The element pushing the buffer usually negotiated already and decided on the

2
docs/design/part-bufferpool.txt
View file

@ -147,7 +147,7 @@ Allocating from pool
will block when all buffers are outstanding until a buffer is returned to the
pool. This behaviour can be changed by specifying the
GST_BUFFER_POOL_FLAG_DONTWAIT flag in the parameters. With this flag set,
allocation will return GST_FLOW_UNEXPECTED when the pool is empty.
allocation will return GST_FLOW_EOS when the pool is empty.
Renegotiation

2
docs/design/part-element-transform.txt
View file

@ -273,7 +273,7 @@ Allocation
~~~~~~~~~~
After the transform element is configured with caps, a bufferpool needs to be
negotiated to perform the allocation of buffers. We habe 2 cases:
negotiated to perform the allocation of buffers. We have 2 cases:
- The element is operating in passthrough we don't need to allocate a buffer
in the transform element.

32
docs/design/part-events.txt
View file

@ -34,27 +34,23 @@ Different types of events exist to implement various functionalities.
src pads
--------
A gst_pad_push_event() on a srcpad will first store the event in the sticky
array before sending the event to the peer pad. If there is no peer pad, the
gst_pad_push_event() function returns NOT_LINKED.
A gst_pad_push_event() on a srcpad will first store the sticky event in the
sticky array before sending the event to the peer pad. If there is no peer pad
and the event was not stored in the sticky array, FALSE is returned.
Note that the behaviour is not influenced by a flushing pad.
Flushing pads will refuse the events and will not store the sticky events.
FLUSH_START and FLUSH_STOP events are dropped on blocked pads.
sink pads
---------
A gst_pad_send_event() on a sinkpad will check the new event against the
existing event. If they are different, the new event is stored as a pending
event. If the events are the same, nothing changes.
A gst_pad_send_event() on a sinkpad will call the event function on the pad. If
the event function returns success, the sticky event is stored in the sticky
event array and the event is marked for update.
When the pad is flushing, the _send_event() function returns WRONG_STATE
immediately.
When the pad is flushing, the _send_event() function returns FALSE immediately.
The event function is then called for all pending events. If the function
returns success, the pending event is copied to the active events, else the
pending event is removed and the current active event is unchanged.
When the next data item is pushed, the pending events are pushed first.
This ensures that the event function is never called for flushing pads and that
the sticky array only contains events for which the event function returned
@ -64,9 +60,9 @@ success.
pad link
--------
When linking pads, all the sticky events from the srcpad are copied to the
pending array on the sinkpad. The pending events will be sent to the event
function of the sinkpad on the next event or buffer.
When linking pads, the srcpad sticky events are marked for update when they are
different from the sinkpad events. The next buffer push will push the events to
the sinkpad.
FLUSH_START/STOP
@ -112,7 +108,7 @@ The EOS event can only be sent on a sinkpad. It is typically emited by the
source element when it has finished sending data. This event is mainly sent
in the streaming thread but can also be sent from the application thread.
An EOS event sent on a srcpad returns GST_FLOW_UNEXPECTED.
An EOS event sent on a srcpad returns GST_FLOW_EOS.
The downstream element should forward the EOS event to its downstream peer
elements. This way the event will eventually reach the sinks which should
@ -131,7 +127,7 @@ GStreamer core will take the STREAM_LOCK.
Sometimes the EOS event is generated by another element than the source, for
example a demuxer element can generate an EOS event before the source element.
This is not a problem, the demuxer does not send an EOS event to the upstream
element but returns GST_FLOW_UNEXPECTED, causing the source element to stop
element but returns GST_FLOW_EOS, causing the source element to stop
sending data.
An element that sends EOS on a pad should stop sending data on that pad. Source

25
docs/design/part-latency.txt
View file

@ -220,6 +220,31 @@ separate async state change thread (like the one currently used for going from
PAUSED->PLAYING in a non-live pipeline).
Query
~~~~~
The pipeline latency is queried with the LATENCY query.
(out) "live", G_TYPE_BOOLEAN (default FALSE)
- if a live element is found upstream
(out) "min-latency", G_TYPE_UINT64 (default 0)
- the minimum latency in the pipeline
(out) "max-latency", G_TYPE_UINT64 (default 0)
- the maximum latency in the pipeline
Event
~~~~~
The latency in the pipeline is configured with the LATENCY event, which contains
the following fields:
"latency", G_TYPE_UINT64
- the configured latency in the pipeline
Latency compensation
~~~~~~~~~~~~~~~~~~~~

4
docs/design/part-meta.txt
View file

@ -61,6 +61,7 @@ GstMeta
A GstMeta is a structure as follows:
struct _GstMeta {
GstMetaFlags flags;
const GstMetaInfo *info; /* tag and info for the meta item */
};
@ -166,7 +167,8 @@ The complete buffer with metadata could, for example, look as follows:
GstMiniObject | GType (GstBuffer) |
| refcount, flags, copy/disp/free |
+-------------------------------------+
GstBuffer | caps, parent, pool |
GstBuffer | pool,pts,dts,duration,offsets |
| <private data> |
+.....................................+
| next ---+
+- | info ------> GstMetaInfo

120
docs/design/part-negotiation.txt
View file

@ -8,10 +8,65 @@ pipeline that have information to those parts of the pipeline that are
flexible, constrained by those parts of the pipeline that are not
flexible.
Basic rules
~~~~~~~~~~~
The simple rules must be followed:
1) downstream suggests formats
2) upstream decides on format
There are 4 queries/events used in caps negotiation:
1) GST_QUERY_CAPS : get possible formats
2) GST_QUERY_ACCEPT_CAPS : check if format is possible
3) GST_EVENT_CAPS : configure format (downstream)
4) GST_EVENT_RECONFIGURE : inform upstream of possibly new caps
Queries
-------
A pad can ask the peer pad for its supported GstCaps. It does this with
the CAPS query. The list of supported caps can be used to choose an
appropriate GstCaps for the data transfer.
(in) "filter", GST_TYPE_CAPS (default NULL)
- a GstCaps to filter the results against
(out) "caps", GST_TYPE_CAPS (default NULL)
- the result caps
A pad can ask the peer pad if it supports a given caps. It does this with
the ACCEPT_CAPS query.
(in) "caps", GST_TYPE_CAPS
- a GstCaps to check
(out) "result", G_TYPE_BOOLEAN (default FALSE)
- TRUE if the caps are accepted
Events
~~~~~~
When a media format is negotiated, peer elements are notified of the GstCaps
with the CAPS event. The caps must be fixed.
"caps", GST_TYPE_CAPS
- the negotiated GstCaps
Operation
~~~~~~~~~
GStreamer's two scheduling modes, push mode and pull mode, lend
themselves to different mechanisms to achieve this goal. As it is more
common we describe push mode negotiation first.
Push-mode negotiation
~~~~~~~~~~~~~~~~~~~~~
@ -31,9 +86,9 @@ The basics of negotiation are as follows:
- An element should reconfigure itself to the new format received as a CAPS
event before processing the following buffers. If the data type in the
caps event is not acceptable, the element should refuse the buffer by
returning an appropriate GST_FLOW_NOT_NEGOTIATED return value from the
chain function.
caps event is not acceptable, the element should refuse the event. The
element should also refuse the next buffers by returning an appropriate
GST_FLOW_NOT_NEGOTIATED return value from the chain function.
- Downstream elements can request a format change of the stream by sending a
RECONFIGURE event upstream. Upstream elements will renegotiate a new format
@ -43,11 +98,18 @@ The general flow for a source pad starting the negotiation.
src sink
| |
| accepts? |
type A |---------------->|
| yes |
|< - - - - - - - -|
| |
| querycaps? |
|---------------->|
| caps |
select caps |< - - - - - - - -|
from the | |
candidates | |
| |-.
| accepts? | |
type A |---------------->| | optional
| yes | |
|< - - - - - - - -| |
| |-'
| send_event() |
send CAPS |---------------->| Receive type A, reconfigure to
event A | | process type A.
@ -61,9 +123,9 @@ The general flow for a source pad starting the negotiation.
[element wants to create a buffer]
if not format
# see what we can do
ourcaps = gst_pad_get_caps (srcpad)
ourcaps = gst_pad_query_caps (srcpad)
# see what the peer can do filtered against our caps
candidates = gst_pad_peer_get_caps (srcpad, ourcaps)
candidates = gst_pad_peer_query_caps (srcpad, ourcaps)
foreach candidate in candidates
# make sure the caps is fixed
@ -100,7 +162,7 @@ The general flow for a sink pad starting a renegotiation.
mark .-|<----------------| send RECONFIGURE event
renegotiate| | |
'>| |
| get_caps() |
| querycaps() |
renegotiate |---------------->|
| suggest B |
|< - - - - - - - -|
@ -121,7 +183,7 @@ videotestsrc ! xvimagesink
1) Who decides what format to use?
- src pad always decides, by convention. sinkpad can suggest a format
by putting it high in the getcaps function GstCaps.
by putting it high in the caps query result GstCaps.
- since the src decides, it can always choose something that it can do,
so this step can only fail if the sinkpad stated it could accept
something while later on it couldn't.
@ -143,11 +205,11 @@ videotestsrc ! xvimagesink
- src receives the RECONFIGURE event and marks renegotiation
- On the next buffer push, the source renegotiates the caps and the
bufferpool. The sink will put the new new prefered format high in the list
of caps it returns from its getcaps function.
of caps it returns from its caps query.
videotestsrc ! queue ! xvimagesink
- queue proxies all accept and getcaps to the other peer pad.
- queue proxies all accept and caps queries to the other peer pad.
- queue proxies the bufferpool
- queue proxies the RECONFIGURE event
- queue stores CAPS event in the queue. This means that the queue can contain
@ -230,15 +292,15 @@ The sink determines that the upstream elements support pull based scheduling by
doing a SCHEDULING query.
The sink initiates the negotiation process by intersecting the results
of gst_pad_get_caps() on its sink pad and its peer src pad. This is the
of gst_pad_query_caps() on its sink pad and its peer src pad. This is the
operation performed by gst_pad_get_allowed_caps(). In the simple
passthrough case, the peer pad's getcaps() function should return the
passthrough case, the peer pad's caps query should return the
intersection of calling get_allowed_caps() on all of its sink pads. In
this way the sink element knows the capabilities of the entire pipeline.
The sink element then fixates the resulting caps, if necessary,
resulting in the flow caps. From now on, the getcaps function
of the sinkpad will only return these fixed caps meaning that upstream elements
resulting in the flow caps. From now on, the caps query of the sinkpad
will only return these fixed caps meaning that upstream elements
will only be able to produce this format.
If the sink element could not set caps on its sink pad, it should post
@ -255,3 +317,25 @@ function. The state will commit to PAUSED when the first buffer is received in
the sink. This is needed to provide a consistent API to the applications that
expect ASYNC return values from sinks but it also allows us to perform the
remainder of the negotiation outside of the context of the pulling thread.
Patterns
~~~~~~~~
We can identify 3 patterns in negotiation:
1) Fixed : Can't choose the output format
- Caps encoded in the stream
- A video/audio decoder
- usually uses gst_pad_use_fixed_caps()
2) Passthrough
- Caps not modified
- can do caps transform based on element property
- videobox
3) Dynamic : can choose output format
- A converter element
- depends on downstream caps, needs to do a CAPS query to find
transform.
- usually prefers to use the identity transform

24
docs/design/part-probes.txt
View file

@ -15,8 +15,8 @@ identify the following types:
This is essential to be able to implement dynamic relinking of elements
without breaking the dataflow.
- be notified when data or events are pushed or sent on a pad. It should also
be possible to inspect and modify the data.
- be notified when data, events or queries are pushed or sent on a pad. It
should also be possible to inspect and modify the data.
- be able to drop, pass and block on data based on the result of the callback.
@ -52,14 +52,17 @@ Overview
GST_PAD_PROBE_TYPE_BLOCK = (1 << 1),
/* flags to select datatypes */
GST_PAD_PROBE_TYPE_BUFFER = (1 << 2),
GST_PAD_PROBE_TYPE_BUFFER_LIST = (1 << 3),
GST_PAD_PROBE_TYPE_EVENT_DOWNSTREAM = (1 << 4),
GST_PAD_PROBE_TYPE_EVENT_UPSTREAM = (1 << 5),
GST_PAD_PROBE_TYPE_BUFFER = (1 << 4),
GST_PAD_PROBE_TYPE_BUFFER_LIST = (1 << 5),
GST_PAD_PROBE_TYPE_EVENT_DOWNSTREAM = (1 << 6),
GST_PAD_PROBE_TYPE_EVENT_UPSTREAM = (1 << 7),
GST_PAD_PROBE_TYPE_EVENT_FLUSH = (1 << 8),
GST_PAD_PROBE_TYPE_QUERY_DOWNSTREAM = (1 << 9),
GST_PAD_PROBE_TYPE_QUERY_UPSTREAM = (1 << 10),
/* flags to select scheduling mode */
GST_PAD_PROBE_TYPE_PUSH = (1 << 6),
GST_PAD_PROBE_TYPE_PULL = (1 << 7),
GST_PAD_PROBE_TYPE_PUSH = (1 << 12),
GST_PAD_PROBE_TYPE_PULL = (1 << 13),
} GstPadProbeType;
@ -111,7 +114,7 @@ Blocking probes
the callback will be called as soon as the pad becomes idle in the streaming
thread.
The IDLE probe in useful to perform dynamic linking, it allows to wait for for
The IDLE probe is useful to perform dynamic linking, it allows to wait for for
a safe moment when an unlink/link operation can be done. Since the probe is a
blocking probe, it will also make sure that the pad stays idle until the probe
is removed.
@ -279,3 +282,6 @@ Query probes have the GST_PAD_PROBE_TYPE_QUERY_* flag set in the callbacks.
< - - - - - - O |
| |
For queries, the PUSH ProbeType is set when the query is traveling to the object
that will answer the query and the PULL type is set when the query contains the
answer.

101
docs/design/part-query.txt
View file

@ -1,44 +1,12 @@
Query
-----
NOTE: this is implemented as proposed
Purpose
~~~~~~~
Queries are used to get information about the stream.
A query is started on a specific pad and travels up or downstream.
Types of queries
~~~~~~~~~~~~~~~~
- get length of stream
- get position in stream
- get seeking capabilities
- get latency
- convert one value to another
- query supported formats
- query internal links.
Current implementation
~~~~~~~~~~~~~~~~~~~~~~
The current implementation of query requires pads to implement the
following functions:
gboolean (*GstPadConvertFunction) (GstPad *pad,
GstFormat src_format, gint64 src_value,
GstFormat *dest_format, gint64 *dest_value);
gboolean (*GstPadQueryFunction) (GstPad *pad, GstQueryType type,
GstFormat *format, gint64 *value);
GList* (*GstPadIntLinkFunction) (GstPad *pad);
const GstFormat* (*GstPadFormatsFunction) (GstPad *pad);
const GstEventMask* (*GstPadEventMaskFunction) (GstPad *pad);
const GstQueryType* (*GstPadQueryTypeFunction) (GstPad *pad);
Most of these functions are not very extensible in particular,
the queryfunction can only return one value.
Requirements
~~~~~~~~~~~~
@ -47,29 +15,23 @@ Requirements
- one pad function to perform the query
- extensible queries.
Proposition
~~~~~~~~~~~
Implementation
~~~~~~~~~~~~~~
- define GstQuery extending GstMiniObject and containing a GstStructure (see GstMessage)
- define standard query types (see proposed types)
- define methods to create a parse the results in the GstQuery.
- GstQuery extends GstMiniObject and contains a GstStructure (see GstMessage)
- some standard query types are defined below
- methods to create and parse the results in the GstQuery.
- define pad method:
gboolean (*GstPadQueryFunction) (GstPad *pad, GstQuery *query);
gboolean (*GstPadQueryFunction) (GstPad *pad,
GstObject *parent,
GstQuery *query);
pad returns result in query structure and TRUE as result or FALSE when
query is not supported.
Proposed types
~~~~~~~~~~~~~~
- GST_QUERY_SEEKING:
get info on how seeking can be done
- getrange, with/without offset/size
- ranges where seeking is efficient (for caching network sources)
- flags describing seeking behaviour (forward, backward, segments,
play backwards, ...)
Query types
~~~~~~~~~~~
- GST_QUERY_POSITION:
@ -81,7 +43,19 @@ Proposed types
- GST_QUERY_LATENCY:
get amount of latency introduced in the pipeline.
get amount of latency introduced in the pipeline. (See part-latency.txt)
- GST_QUERY_RATE:
get the current playback rate of the pipeline
- GST_QUERY_SEEKING:
get info on how seeking can be done
- getrange, with/without offset/size
- ranges where seeking is efficient (for caching network sources)
- flags describing seeking behaviour (forward, backward, segments,
play backwards, ...)
- GST_QUERY_SEGMENT:
@ -95,3 +69,32 @@ Proposed types
return list of supported formats that can be used for GST_QUERY_CONVERT.
- GST_QUERY_BUFFERING:
query available media for efficient seeking (See part-buffering.txt)
- GST_QUERY_CUSTOM:
a custom query, the name of the query defines the properties of the query.
- GST_QUERY_URI:
query the uri of the source or sink element
- GST_QUERY_ALLOCATION:
the buffer allocation properties (See part-bufferpool.txt)
- GST_QUERY_SCHEDULING:
the scheduling properties (See part-scheduling.txt)
- GST_QUERY_ACCEPT_CAPS:
check if caps are supported (See part-negotiation.txt)
- GST_QUERY_CAPS:
get the possible caps (See part-negotiation.txt)

2
docs/design/part-segments.txt
View file

@ -90,7 +90,7 @@ Use case: FLUSHING seek
segment, the decoder knows it should not send the P frame.
Avidemux stops sending data after pushing a frame with timestamp 5 and
returns GST_FLOW_UNEXPECTED from the chain function to make the upstream
returns GST_FLOW_EOS from the chain function to make the upstream
elements perform the EOS logic.

2
docs/design/part-streams.txt
View file

@ -64,7 +64,7 @@ Typical stream
3) EOS
- marks the end of data, nothing is to be expected after EOS, elements
should refuse more data and return GST_FLOW_UNEXPECTED. A FLUSH_STOP
should refuse more data and return GST_FLOW_EOS. A FLUSH_STOP
event clears the EOS state of an element.

4
gst/gstquery.c
View file

@ -566,8 +566,8 @@ gst_query_new_latency (void)
* gst_query_set_latency:
* @query: a #GstQuery
* @live: if there is a live element upstream
* @min_latency: the minimal latency of the live element
* @max_latency: the maximal latency of the live element
* @min_latency: the minimal latency of the upstream elements
* @max_latency: the maximal latency of the upstream elements
*
* Answer a latency query by setting the requested values in the given format.
*