diff --git a/docs/random/caps b/docs/random/caps new file mode 100644 index 0000000000..e7995dd261 --- /dev/null +++ b/docs/random/caps @@ -0,0 +1,205 @@ +During the course of a discussion on IRC, it turned out +that there are many possible ways to handle the capabilities. + +A capabilitiy is bascially a set of properties attached to a +mimetype in order to more closely describe the mimetype. +Capabiltities are supposed to be attached to pads so that the +autoplugging algorithm has more specific information to connect +compatible pads. + +We present 3 possible implementation for the capabilities. we need +to pick one of them. + +1. static capabilities +---------------------- + +When an element is created, it creates its pads like: + + mpg123->sinkpad = gst_pad_new ("sink", GST_PAD_SINK); + gst_element_add_pad (GST_ELEMENT (mpg123), mpg123->sinkpad); + + mpg123->srcpad = gst_pad_new ("src", GST_PAD_SRC); + gst_element_add_pad (GST_ELEMENT (mpg123), mpg123->srcpad); + +In the static capabilities case, it will attach a GstCaps* structure +to the pad. The GstCaps structure in the above example might look like: + + static GstCapsFactory mpg123_sink_caps = { + "audio/mp3", + "layer", GST_CAPS_INT_RANGE (1, 3), + "bitrate", GST_CAPS_INT_RANGE (8, 320), + NULL + }; + +with + +mpg123sinkcaps = gst_caps_register (mpg123_sink_caps); + +the factory can be converted into a GstCaps* structure. The +GstCaps* structure is attached to the pad with: + +gst_pad_add_caps (mpg123->sinkpad, mpg123sinkcaps); + +The GstElement would then have a sinkpad with the given +mimetype (audio/mp3) and with the capabilitities of accepting +mpeg layer 1 to 3 and a bitrate from 8 up to 320 Kbps. + +Likewise, the src pad could be set up in the same way. An +example capability factory could look like: + + static GstCapsFactory mpg123_src_caps = { + "audio/raw", + "format", GST_CAPS_BITFIELD (...), + "depth", GST_CAPS_INT (16), + "rate", GST_CAPS_INT_RANGE (4000, 96000), + "channels", GST_CAPS_INT_RANGE (1, 2), + NULL + }; + +All GstElements would present their pads with the appropriate +capabilities structure. + +The autoplugger would then proceed (once the source media type +is known with a typefind function) in finding all the elements +with compatible pads and connecting them into a pipeline. + +All elements of the complete pipeline could then be constructed +with one single pass. No new elements should be added to the +pipeline because we can figure out all the possibilities using the +pad capabilities. + +We call this the static case because the capabilities of the pads +are supposed to stay the same after creating the element. + +While the ability to completly setup the pipeline before actually +starting playback is an advantage regarding performance, one obvious +problem with this setup is that the static case may be too static in +some cases. We can illustrate this with the following setup: + + ----------) (------------ + mpg123 ! ! audiosink + src sink + ! ! + ----------) (------------ + +The mpg123 element has its src capabilities set up as mpg123_src_caps +in the above example. + +The audio renderer has its capabilities set up with the following +factory: + + static GstCapsFactory audio_sink_caps = { + "audio/raw", + "format", GST_CAPS_BITFIELD (...), + "depth", GST_CAPS_INT (16), + "rate", GST_CAPS_INT_RANGE (22000, 44000), + "channels", GST_CAPS_INT_RANGE (1, 2), + NULL + }; + +The static autoplugger has to be carefull when connecting the mpg123 +element with the audiosink because it is theoretically possible that +the mpg123 element outputs raw audio with a rate that cannot be +handled by the audiosink (ex. 4000KHz). In the absense of another +audiosink with more capabilities, the autoplugging of this simple +pipeline will not be possible and would fail. + +the autoplugging algorithm would probably select another element to +insert between the mpg123 element and the audiosink in order to handle +the (uncommon) case of a rate conversion (audioscaler). + +It is clear that this static setup might even fail or work suboptimal +for even the common case and should therefore be considered as too +restrictive. + + +2. dynamic capabilities +----------------------- + +The idea of dynamic capabilities is that the capabilities are not set +at element create time but rather while the pipeline is running. + +An element would still list its mime type using: + + gst_pad_add_type_id(mpg123->sinkpad, mp3type); + +The idea would then be that a rough draft of the pipeline would be +built afer the media type of the stream has been detected with the +typefind functions. The rough draft would consist of laying out a +global plan to reach the renderer(s). this plan would basically list +the set of conversions that have to be performed. (mime-type to +mime-type conversion). + +Elements that accept the src mime-type are tried by giving it a buffer. +If the element accepts the buffer, it will set its capabilities for +both the sink pad and the src pad. At that time other elements can be +tried and added to the src pad, until we reach the renderer. As usual +one has to be carefull to add just the minimum amount of elements to +reach the renderer. The global plan will help with that. + +Since we basically do not use the capabilities of the sink pad one has +to question the need for sink pad capabilities in the first place. + +We might also have a hard time trying different elements until we find +a compatible one that does not cause a dead end at some point. + + +3. combined setup +----------------- + +This combined setup will minimise the effort needed to try different +elements encountered by option 2 while still allowing a more dynamic +setup based on the actual media stream we are handling. + +The combined setup will list/add the sink capabilities at create time. +It will only set the mime-type of its src pads. + +As with option2, a global plan will be built. At runtime the src pads +will actually specify the capabilities they need for any element that +wants to be connected to its source pads. + +In this case we specifiy the capabilities for all the sink pads of an +element at create time. The capabilities of the src pads would only +become available when data has been processed by the element. + +The autoplugger would then be able to choose an element that can handle +the capability listed by the src pad. + +in our previous example: + + ----------) (------------ + mpg123 ! ! audiosink + src sink + ! ! + ----------) (------------ + +the audiosink element would specify its sink pad capabilities at create +time, while the mpg123 elements src pad would not yet have any capabilities +set. + +When data is handled by the mpg123 element, a capability would be added to +the mpg123 src pad. This capability might be: + + static GstCapsFactory mpg123_src_caps = { + "audio/raw", + "format", GST_CAPS_INT (S16), + "depth", GST_CAPS_INT (16), + "rate", GST_CAPS_INT (44000), + "channels", GST_CAPS_INT (2), + NULL + }; + +This capability would be compatible with the audiosinks sinkpad capabilities +and the autoplugger would therefore be able to connect the two elements. + +While allowing a more flexible setup with option3, compared to option1, we +introduce a slightly higher overhead when we need to dynamically connect +elements. This overhead will not be as big as option2 because we don't +have to 'try' elements. + +so: + + src caps: added at runtime to list the caps needed for an element that + wants to connect to this pad. + sink caps: the (static) capabilities that this sinkpad has. +