gstreamer/docs/design/part-caps.txt

Caps
----

Caps are lighweight refcounted objects describing media types.
They are composed of an array of GstStructures plus optionally
a GstCapsFeatures set for the GstStructure.

Caps are exposed on GstPadTemplates to describe all possible types a
given pad can handle. They are also stored in the registry along with
a description of the element.

Caps are exposed on the element pads via CAPS and ACCEPT_CAPS queries.

This function describes the possible types that the pad can handle or
produce (see part-pads.txt and part-negotiation.txt).

Various methods exist to work with the media types such as subtracting
or intersecting.

Operations
~~~~~~~~~~
Fixating
--------
Caps are fixed if they only contain a single structure and this
structure is fixed. A structure is fixed if none of the fields of the
structure is a unfixed types, for example a range, list or array.

For fixating caps only the first structure is kept as the order of
structures is meant to express the preferences for the different
structures.
Afterwards each unfixed field of this structure is set to a value
that makes most sense for the media format by the element or pad
implementation and afterwards every remaining unfixed fields is set to
an arbitrary value that would be a subset of the unfixed field value.

EMPTY caps are fixed caps, ANY caps are not fixed. Caps with ANY caps
features are not fixed.

Subset
------
One caps "A" is the subset of another caps "B" if for each structure in
"A" there exists a structure in "B" that is a superset of the structure
in "A".

A structure "a" is the subset of a structure "b" if it has the same 
structure name, the same caps features and each field in "b" exists
in "a" and the value of the field in "a" is a subset of the value of
the field in "b". "a" can have additional fields that are not in "b".

EMPTY caps are a subset of every other caps, every caps are a subset of
ANY caps.

Equality
--------
Caps "A" and "B" are equal if "A" is a subset of "B" and "B" is a subset
of "A". This means that both caps are expressing the same possibilities
but their structures can still be different if they contain unfixed
fields.

Intersection
------------
The intersection of caps "A" and caps "B" are the caps that contain the
intersection of all their structures with each other.

The intersection of structure "a" and structure "b" is empty if their
structure name or their caps features are not equal, or if "a" and "b"
contain the same field but the intersection of both field values is empty.
If one structure contains a field that is not existing in the other
structure it will be copied over to the intersection with the same
value.

The intersection with ANY caps is always the other caps, the
intersection with EMPTY caps is always EMPTY.

Union
-----
The union of caps "A" and caps "B" are the caps that contain the union
of all their structures with each other.

The union of structure "a" and structure "b" are the two structures "a"
and "b" if the structure names or caps features are not equal. Otherwise
the union is the structure that contains the union of each fields value.
If a field is only in one of the two structures it is not contained in
the union.

The union with ANY caps is always ANY, the union with EMPTY caps is
always the other caps.

Subtraction
-----------
The subtraction of caps "A" from caps "B" is the most generic subset
of "B" that has an empty intersection with "A" but only contains
structures with names and caps features that are existing in "B".

Basic Rules
~~~~~~~~~~~

Semantics of caps and their usage
---------------------------------
A caps can contain multiple structures, in which case any of the
structures would be acceptable. The structures are in the preferred
order of the creator of the caps, with the preferred structure being
first and during negotiation of caps this order should be considered to
select the most optimal structure.

Each of these structures has a name that specifies the media type, e.g.
"video/x-theora" to specify Theora video. Additional fields in the
structure add additional constraints and/or information about the media
type, like the width and height of a video frame, or the codec profile
that is used. These fields can be non-fixed (e.g. ranges) for non-fixed
caps but must be fixed to a fixed value during negotiation.
If a field is included in the caps returned by a pad via the CAPS query,
it imposes an additional constraint during negotiation. The caps in the
end must have this field with a value that is a subset of the non-fixed
value. Additional fields that are added in the negotiated caps give
additional information about the media but are treated as optional.
Information that can change for every buffer and are not relevant during
negotiation must not be stored inside the caps.


For each of the structures in caps it is possible to store caps
features. The caps features are expressing additional requirements
for a specific structure, and only structures with the same name _and_
equal caps features are considered compatible.
Caps features can be used to require a specific memory representation
or a specific meta to be set on buffers, for example a pad could require
for a specific structure that it is passed EGLImage memory or buffers
with the video meta.
If not caps features are provided for a structure, it is assumed that
system memory is required unless later negotiation steps (e.g. the
ALLOCATION query) detect that something else can be used. The special
ANY caps features can be used to specify that any caps feature would
be accepted, for example if the buffer memory is not touched at all.


Compatibility of caps
---------------------
Pads can be linked when the caps of both caps are compatible. This is
the case when their intersection is not empty.

For checking if a pad actually supports a fixed caps an intersection is
not enough. Instead the fixed caps must be at least a subset of the
pad's caps but pads can introduce additional constraints which would be
checked in the ACCEPT_CAPS query handler.

Data flow can only happen after pads have decided on a common, fixed
caps and these caps are distributed to both pads with the CAPS event.