mirror of
https://gitlab.freedesktop.org/gstreamer/gstreamer.git
synced 2024-12-02 22:46:35 +00:00
328 lines
10 KiB
Text
328 lines
10 KiB
Text
Caps Negotiation
|
|
================
|
|
|
|
|
|
Definitions
|
|
-----------
|
|
|
|
- GstCaps is a structure that holds a mimetype and a set of properties. The properties are
|
|
either fixed or non-fixed, they contain ranges or lists.
|
|
|
|
- filter caps: caps set by the application to constrain a link to a specific GstCaps.
|
|
|
|
- allowed caps: caps calculated when connecting elements, these are the types that are
|
|
possible on this connection. If no types are possible, the link is refused.
|
|
|
|
- pad template caps: caps put on padtemplates to describe the possible media types of
|
|
the pad.
|
|
|
|
- pad caps: the caps of the pad as it is currently negotiated.
|
|
|
|
|
|
General overview
|
|
----------------
|
|
|
|
Caps negotiation is the process of two pads agreeing on a common fixed GstCaps structure
|
|
that describes the media type that will be exchanged between those pads.
|
|
|
|
We define that caps negotiation only happens in the PLAYING state, when there is actual
|
|
dataflow and elements thus know what kind of data they are dealing with, or when
|
|
linking elements.
|
|
|
|
Caps negotiation is first performed when linking elements. If the elements can agree
|
|
on a media type at link time then negotiation will not have to be performed at runtime.
|
|
Usually this is not possible, so the pads remain unnegotiated until at runtime.
|
|
|
|
Caps negotiation is initiated by an element that wants to send data on a non-negotiated
|
|
pad, thus the source pad. The core only provides a policy and convenience methods to aid
|
|
the element in performing the negotiation. The core does not keep track of what pads
|
|
are negotiated or not, this is a task of the pads.
|
|
|
|
Caps negotiation is normally started by the source element right before it sends out
|
|
data (case 1).
|
|
Caps negotiation can also be redone by a sink element that wants to receive another
|
|
format from a downstream element (case 2).
|
|
|
|
There are two functions handling the negotiation, the link function and the do_link
|
|
function. The link function is always called first and must eventually call do_link on
|
|
the peer pad. The link function must figure out a compatible format for the connection
|
|
and then call the do_link function on the peer pad. The do_link function can only
|
|
accept or refuse the provided caps.
|
|
|
|
For autopluggers it is important to know when the pad is ready to start the negotiation.
|
|
It is also inportant to know when the negotiation failed and it must be possible to
|
|
restart the negotiation with another element. This functionality will be provided
|
|
with signals.
|
|
|
|
Pad functions
|
|
-------------
|
|
|
|
!
|
|
! const GstCaps* gst_pad_iterate_caps (GstPad *pad, gint position);
|
|
!
|
|
Returns the nth possible caps that describes the media type that can flow over
|
|
this pad. This function should not return static caps but caps that are
|
|
dependent of the media type and the peer connections of the element.
|
|
|
|
This function can be called at any time so that an autoplugger can know the
|
|
exact types of the pads at any time.
|
|
|
|
!
|
|
! gboolean gst_pad_accept_caps (GstPad *pad, GstCaps *caps);
|
|
!
|
|
Checks that a given caps is acceptable for this pad. Returns FALSE if the
|
|
pad cannot handle the caps.
|
|
|
|
!
|
|
! gboolean gst_pad_link (GstPad *pad, GstPad *peer, GstCaps *caps);
|
|
!
|
|
Tells the pad to start negotiation with the given filtercaps. The
|
|
caps need not be fixed and serves as a filter for performing the negotiation
|
|
with the peerpad.
|
|
|
|
!
|
|
! gboolean gst_pad_do_link (GstPad *pad, GstPad *peer, GstCaps *caps);
|
|
!
|
|
Configures the pad to accept data with the given caps. The caps must be fixed.
|
|
|
|
!
|
|
! const GstCaps* gst_pad_get_negotiated_caps (GstPad *pad);
|
|
!
|
|
Get the negotiated caps of a pad or NULL if the pad is not negotiated.
|
|
|
|
|
|
Linking Elements
|
|
----------------
|
|
|
|
When linking elements with the gst_pad_link function, the core will call
|
|
the link function on the srcpad. If that pad returns GST_PAD_LINK_OK,
|
|
the link is established, else the link fails.
|
|
|
|
Since the link function of the pad will call the do_link function of
|
|
the peerpad, the caps will be set on both pads.
|
|
|
|
It is not required to decide on a caps at link time, a plugin can choose to
|
|
dynamically renegotiate at runtime.
|
|
|
|
When a link fails, the core emits a signal link_failed, which the application
|
|
can catch to try to establish a new link with another element, for example.
|
|
|
|
!
|
|
! def gst_pad_link_filtered (pad1, pad2, filtercaps):
|
|
!
|
|
! ... get srcpad and sinkpad ...
|
|
! srcpad = (pad1 == GST_PAD_SRC ? pad1 : pad2);
|
|
! sinkpad = (pad1 == GST_PAD_SINK ? pad1 : pad2);
|
|
!
|
|
! ... more checks to see if the pads are of different types and
|
|
! ... that they live in the same scheduler etc...
|
|
!
|
|
! res = gst_pad_link (srcpad, sinkpad, filtercaps)
|
|
! if (res == GST_PAD_LINK_OK)
|
|
! ... perform other setup ...
|
|
! else
|
|
! res = signal_emit (srcpad, "link_failed")
|
|
!
|
|
! return res
|
|
!
|
|
|
|
Pad link is just a convenience function that passes a NULL filtercaps:
|
|
!
|
|
! def gst_pad_link (pad1, pad2):
|
|
! gst_pad_link_filtered (pad1, pad2, NULL)
|
|
!
|
|
|
|
|
|
Dynamic renegotiation
|
|
---------------------
|
|
|
|
Dynamic renegotiation happens at runtime when the element knows the exact media
|
|
type it is handling.
|
|
|
|
The element that wants to redecide on the data type of a connection just calls
|
|
gst_pad_relink on one of it's pads. The core will call unlink and link on the
|
|
pads again as if this were a new connection. Since the iterate_caps function
|
|
returns other values while linking, the new link will renegotiate to a new
|
|
format or the link will fail.
|
|
|
|
Prototype of the relink function:
|
|
!
|
|
! def gst_pad_relink_filtered (pad, filtercaps):
|
|
! gst_pad_unlink (pad, peerpad)
|
|
! gst_pad_link_filtered (pad, peerpad, filtercaps)
|
|
!
|
|
|
|
Again the relink function is a convenience function for not having to pass
|
|
filtercaps.
|
|
!
|
|
! def gst_pad_relink (pad):
|
|
! gst_pad_relink_filtered (pad, NULL)
|
|
!
|
|
|
|
The core, however, should optimize the relink function so that it does a minimal
|
|
amount of work, like not informing the scheduler about the unlink/link call in
|
|
case of success.
|
|
|
|
Error recovery
|
|
--------------
|
|
|
|
When a pad is ready to negotiate and it has no peer pad, it fires the "need-link"
|
|
signal. Autopluggers can use this signal to start plugging elements to the pad.
|
|
|
|
When a link fails because of a renegotiation, the "link-failed" signal is fired
|
|
so that autopluggers can try other elements.
|
|
|
|
|
|
Default implementations
|
|
-----------------------
|
|
|
|
!
|
|
! gst_pad_src_link_func (pad, peerpad, filtercaps)
|
|
! {
|
|
! srcpad->negotiated_caps = NULL;
|
|
! gboolean possible = FALSE;
|
|
!
|
|
! for (i=0; peercaps = gst_pad_iterate_caps (peerpad, i); i++)
|
|
! {
|
|
! for (j=0; srccaps = gst_pad_iterate_caps (srcpad, j); j++)
|
|
! {
|
|
! test = gst_caps_intersect (srccaps, peercaps, filtercaps);
|
|
! if (test == GST_CAPS_EMPTY)
|
|
! continue;
|
|
!
|
|
! /* non empty caps, something is possible */
|
|
! possible = TRUE;
|
|
!
|
|
! if (!gst_caps_is_fixed (caps))
|
|
! continue;
|
|
!
|
|
! if (gst_pad_accepts_caps (peerpad, test))
|
|
! {
|
|
! if (gst_pad_do_link (peerpad, srcpad, test))
|
|
! {
|
|
! srcpad->negotiated_caps = test;
|
|
! goto done;
|
|
! }
|
|
! else
|
|
! {
|
|
! /* weird, it accepted but did not want to link */
|
|
! }
|
|
! }
|
|
! else
|
|
! {
|
|
! /* caps are not accepted by peer, try next */
|
|
! }
|
|
! }
|
|
! }
|
|
! done:
|
|
! if (!possible)
|
|
! return GST_PAD_LINK_FAILED;
|
|
! else
|
|
! return GST_PAD_LINK_OK;
|
|
! }
|
|
!
|
|
|
|
gst_pad_iterate_caps returns caps in the following order:
|
|
|
|
1) preferred caps (if any)
|
|
2) negotiated caps (if any)
|
|
3) profile caps (if any, filtered against caps of current media type)
|
|
4) padtemplate caps (filtered against caps of current media type)
|
|
|
|
1 = a caps describing the media type that would result in optimal
|
|
processing of the current media type
|
|
2 = a caps that it is currently handling
|
|
3 = a caps describing the user configured default values
|
|
4 = a caps describing all other possibilities that this pad can
|
|
produce or consume given the current media type if any.
|
|
|
|
|
|
generic flow 1:
|
|
|
|
Linkage
|
|
|
|
src sink
|
|
| |
|
|
| <--- link(sink, A) |
|
|
check if | |
|
|
A is ok | |
|
|
or filter | |
|
|
against | |
|
|
allowed | |
|
|
src caps | |
|
|
| iterate_caps |
|
|
if A not +-------------------------> |
|
|
fixed | |
|
|
| <-------------------------+
|
|
filter | |
|
|
against A | |
|
|
to get A' | |
|
|
| can_accept(A') |
|
|
if A' fix +-------------------------> |
|
|
| | check if A' is ok
|
|
| yes |
|
|
| <-------------------------+
|
|
| |
|
|
if A' not | |
|
|
fixed, | |
|
|
A'=null | |
|
|
| do_link(src, A') |
|
|
+-------------------------> |
|
|
| ok | store src, A'
|
|
| <-------------------------+
|
|
store | |
|
|
sink, A' | |
|
|
|
|
|
|
|
|
Unlink
|
|
|
|
src sink
|
|
| |
|
|
| <--- unlink() |
|
|
| |
|
|
| unlink() |
|
|
+-------------------------> |
|
|
unref | | unref src, format
|
|
sink, fmt | |
|
|
|
|
|
|
Dynamic negotiation of B
|
|
|
|
src sink
|
|
| can_accept(B) |
|
|
+-------------------------> |
|
|
| | check if B is ok
|
|
| yes |
|
|
| <-------------------------+
|
|
| |
|
|
| do_link(B) |
|
|
+-------------------------> |
|
|
| ok | store B
|
|
| <-------------------------+
|
|
store | |
|
|
B | |
|
|
|
|
|
|
|
|
TODO
|
|
-----
|
|
|
|
Write test objects to simulate different behaviour.
|
|
|
|
|
|
tests (in python):
|
|
|
|
vts -> colorspace -> ximagsink
|
|
|
|
relink with different colorspaces while running
|
|
|
|
|
|
vts -> videoscale -> ximagesink
|
|
|
|
relink to different sizes while running, scaling vts or ximagesink.
|
|
|
|
sinesrc -> audiorate -> audiosink
|
|
|
|
where audiosink only has one possible fixed caps.
|
|
|