diff --git a/docs/pwg/advanced-negotiation.xml b/docs/pwg/advanced-negotiation.xml index 6264ec2157..ca47402c00 100644 --- a/docs/pwg/advanced-negotiation.xml +++ b/docs/pwg/advanced-negotiation.xml @@ -1,21 +1,194 @@ Caps negotiation - Caps negotiation is the process where elements configure themselves - and each other for streaming a particular media format over their pads. - Since different types of elements have different requirements for the - media formats they can negotiate to, it is important that this process - is generic and implements all those use cases correctly. - - - In this chapter, we will discuss downstream negotiation and upstream - negotiation from a pipeline perspective, implicating the responsibilities - of different types of elements in a pipeline, and we will introduce the - concept of fixed caps. + Caps negotiation is the act of finding a media format (GstCaps) between + elements that they can handle. This process in &GStreamer; can in most + cases find an optimal solution for the complete pipeline. In this section + we explain how this works. - + + Caps negotiation basics + + In &GStreamer;, negotiation of the media format always follows the + following simple rules: + + + + + A downstream element suggest a format on its sinkpad and places the + suggestion in the result of the CAPS query performed on the sinkpad. + See also . + + + + + An upstream element decides on a format. It sends the selected media + format downstream on its source pad with a CAPS event. Downstream + elements reconfigure themselves to handle the media type in the CAPS + event on the sinkpad. + + + + + An upstream element can inform downstream that it would like to + suggest a new format by sending a RECONFIGURE event upstream. The + RECONFIGURE event simply instructs an upstream element to restart + the negotiation phase. Because the element that sent out the + RECONFIGURE event is now suggesting another format, the format + in the pipeline might change. + + + + + In addition to the CAPS and RECONFIGURE event and the CAPS query, there + is an ACCEPT_CAPS query to quickly check if a certain caps can + be accepted by an element. + + + All negotiation follows these simple rules. Let's take a look at some + typical uses cases and how negotiation happens. + + + + Caps negotiation use cases + + In what follows we will look at some use cases for push-mode scheduling. + The pull-mode scheduling negotiation phase is discussed in + and is actually similar as we + will see. + + + Since the sink pads only suggest formats and the source pads need to + decide, the most complicated work is done in the source pads. + We can identify 3 caps negotiation use cases for the source pads: + + + + + Fixed negotiation. An element can output one format only. + See . + + + + + Transform negotiation. There is a (fixed) transform between the + input and output format of the element, usually based on some + element property. The caps that the element will produce depend + on the upstream caps and the caps that the element can accept + depend on the downstream caps. + See . + + + + + Dynamic negotiation. An element can output many formats. + See . + + + + + + Fixed negotiation + + In this case, the source pad can only produce a fixed format. Usually + this format is encoded inside the media. No downstream element can + ask for a different format, the only way that the source pad will + renegotiate is when the element decides to change the caps itself. + + + Elements that could implement fixed caps (on their source pads) are, + in general, all elements that are not renegotiable. Examples include: + + + + + A typefinder, since the type found is part of the actual data stream + and can thus not be re-negotiated. The typefinder will look at the + stream of bytes, figure out the type, send a CAPS event with the + caps and then push buffers of the type. + + + + + Pretty much all demuxers, since the contained elementary data + streams are defined in the file headers, and thus not + renegotiable. + + + + + Some decoders, where the format is embedded in the data stream + and not part of the peercaps and where the + decoder itself is not reconfigurable, too. + + + + + gst_pad_use_fixed_caps() is used on the source + pad with fixed caps. As long as the pad is not negotiated, the default + CAPS query will return the caps presented in the padtemplate. As soon + as the pad is negotiated, the CAPS query will return the negotiated + caps (and nothing else). These are the relevant code snippets for fixed + caps source pads. + + + + + + The fixed caps can then be set on the pad by calling + gst_pad_set_caps (). + + +, + "channels", G_TYPE_INT, , NULL); + if (!gst_pad_set_caps (pad, caps)) { + GST_ELEMENT_ERROR (element, CORE, NEGOTIATION, (NULL), + ("Some debug information here")); + return GST_FLOW_ERROR; + } +[..] +]]> + + + All other elements that need to be configured for the format should + implement full caps negotiation, which will be explained in the next + few sections. + + + + + Transform negotiation + + + + + + Dynamic negotiation + + + + + + + Pull-mode Caps negotiation + + + + + Downstream caps negotiation