gstreamer/docs/random/autoplug2

1) The Autoplugger API
----------------------

We'll first describe how to use the autoplugger. We will provide
a use case: autoplug an mpeg1 system stream for audio/video playback.


a) creating an autoplugger
--------------------------

Before any autoplugging can be done, you'll have to create an 
autoplugger object. Autoplugger objects (autopluggers) are 
provided by plugins and are created with gst_autoplugfactor_make().

GStreamer has provisions for two types of autopluggers:

 - regular autopluggers, which act as a complex element construction
   mechanism. They usually don't create threads and operate solely on
   GstCaps* for the source and destination. The complex elements 
   created by regular autopluggers have src and sink pad compatible 
   with the requested GstCaps*.
 
 - renderer autopluggers, which are designed to create a complex
   object that can be used to playback media. Renderer autoplugged
   complex elements have no src pads, only one sink pad.

We'll create a renderer autoplugger like this:

!
!  GstAutoplug *autoplug;
!
!  autoplug = gst_autoplugfactory_make ("staticrender");
!


b) finding out the source media type.
-------------------------------------

Before we can start the autoplugger, we have to find out the 
source media type. This can be done using the typefind functions 
provided by various plugins.

We will create a little pipeline to detect the media type by connecting
a disksrc element to a typefind element. The typefind element will 
repeatedly call all registered typefind functions with the buffer it
receives on its sink pad. when a typefind function returns a non NULL
GstCaps*, that caps is set to the sink pad of the typefind element and
a signal is emitted to notify the app.

Due to caps negotiation, the disksrc will have the detected GstCaps* 
set on its src pad.

We typically use a function like below to detect the type of a media stream
on an element (typically a disksrc). The function accepts a pipeline and the
element inside the pipeline on which the typefind should be performed (passing
a GstPad* is probably a better option FIXME).

! 
! static GstCaps*
! gst_play_typefind (GstBin *bin, GstElement *element)
! {
!   GstElement *typefind;
!   GstCaps *caps = NULL;
! 
!   typefind = gst_elementfactory_make ("typefind", "typefind");
!   g_return_val_if_fail (typefind != NULL, FALSE);
! 
!   gst_pad_connect (gst_element_get_pad (element, "src"),
!                    gst_element_get_pad (typefind, "sink"));
! 
!   gst_bin_add (bin, typefind);
! 
!   gst_element_set_state (GST_ELEMENT (bin), GST_STATE_PLAYING);
! 
!   // push a buffer... the have_type signal handler will set the found flag
!   gst_bin_iterate (bin);
! 
!   gst_element_set_state (GST_ELEMENT (bin), GST_STATE_NULL);
! 
!   caps = gst_pad_get_caps (gst_element_get_pad (element, "src"));
! 
!   gst_pad_disconnect (gst_element_get_pad (element, "src"),
!                       gst_element_get_pad (typefind, "sink"));
!   gst_bin_remove (bin, typefind);
!   gst_object_unref (GST_OBJECT (typefind));
! 
!   return caps;
! }
! 

Also note that the disksrc was added to the pipeline before calling this
typefind function.

When the function returns a non-NULL pointer, the media type has been 
determined and autoplugging can commence.

Assume that in our mpeg1 use case the above function returns a GstCaps* 
like:

!
!  srccaps = GST_CAPS_NEW ("mpeg1system_typefind",
!                          "video/mpeg",
!                            "mpegversion",  GST_PROPS_INT (1),
!                            "systemstream", GST_PROPS_BOOLEAN (TRUE)
!                         );
!


c) Performing the autoplugging
------------------------------

Since we use the renderer API, we have to create the output elements
that are going to be used as the final sink elements.

!
!  osssink = gst_elementfactory_make("osssink", "play_audio");
!  videosink = gst_elementfactory_make("xvideosink", "play_video");
!  
  
We then create a complex element using the following code.

!
!  new_element = gst_autoplug_to_renderers (autoplug,
!           srccaps,
!           videosink,
!           osssink,
!           NULL);
!
!  if (!new_element) {
!    g_print ("could not autoplug, no suitable codecs found...\n");
!    exit (-1);
!  }
!

2) Autoplugging internals
-------------------------

We will now describe the internals of the above gst_autoplug_to_renderers()
function call. This code is implemented in a plugin found in:

    gst/autoplug/gststaticautoplugrender.c



a) phase1: create lists of factories.
---------------------------------------

The autoplugger will start with executing the following piece of
code:

!
! i = 0;
!
! for each sink:
! {
!   sinkpad = take the first sinkpad of the sink (HACK)
!   
!   list[i] = gst_autoplug_caps (srccaps, sinkpad->caps);
!     
!   i++;
! }
!

gst_autoplug_caps will figure out (based on the padtemplates)
which elementfactories are needed to connect srccaps to sinkpad->caps
and will return them in a list.

The element list is created by using a modified shortest path algorithm
by Dijkstra (http://www.orie.cornell.edu/~or115/handouts/handout3/handout3.html).
The nodes of the graph are the elementfactories and the weight of the
arcs is based on the pad compatibility of the padtemplates of the 
elementfactory. For incompatible elementfactories, we use a weight of 
MAX_COST (999999) and for compatible padtemplates we use 1.

ex. we have two sinks with following caps:

!
!       video/raw                   audio/raw
!	"...."                       "...."
!

gst_autoplug_caps will figure out that for the first sink the following
elements are needed:

!
!   mpeg1parse, mp1videoparse, mpeg_play
!

for the second sink the following is needed:
 
!
!   mpeg1parse, mad
!

Note that for the audio connection the element list "mpeg1parse, mp3parse,
mpg123" would also connect the srccaps to the audiosink caps. Since the
"mpeg1parse, mad" list is shorter, it it always preferred by the autoplugger.

We now have two lists of elementfactories.


b) phase2: collect common elements from the lists and add them to a bin.
------------------------------------------------------------------------

The rationale is that from the lists we have created in phase1, there
must be some element that is a splitter and that it has to come first (HACK)
We try to find that element by comparing the lists until an element differs.

We start by creating a toplevel bin that is going to be our complex element.

In our use-case we find that mpeg1parse is an element common to both lists,
so we add it to the bin. We then try to find a good ghostpad for the resulting
complex element. This is done by looping over the sink pads of the first common
element and taking the pad that is compatible with the srcaps.

We end up with a bin like this:
!
!  (----------------------)
!  ! autoplug_bin         !
!  !                      !
!  !       (------------) !
!  !       ! mpeg1parse ! !
!  !   - sink           ! !
!  !  /    (------------) !
! sink                    !
!  (----------------------)
!


c) phase3: add remaining elements
---------------------------------

now we loop over all the list and try to add the remaining elements

(HACK) we always use a new thread for the elements when there is a common
element found.

if a new thread is needed (either because the previous element is a common
element or the object flag of the next element is set to GST_SUGGEST_THREAD)
we add a queue to the bin and we add a new thread. We add the elements to
the bin and connect them using gst_pipeline_pads_autoplug. 

we finally arrive at the sink element and we're done.

ex.
    
     we have just found our mpeg1parse common element, so we start a thread.
     We add a queue to the bin and a new thread, we add the elements
     mp1videoparse and mpeg_play to the thread. We arrive at the videosink, we
     see that the SUGGEST_THREAD flag is set, we add a queue and a thread and
     add the videosink in the thread.
 
     the same procedure happens for the audio part. We are now left with the
     following pipeline:

     We will also have set a signal "new_pad" on the mpeg1parse element because
     the element mp1videoparse could not be connected to the element just yet.

  (---------------------------------------------------------------------------------------------)
  !autoplug_bin                                                                                 !
  !                                                                                             !
  !                                   (----------------------------------------) (------------) !
  !                                   !thread                                  ! ! thread     ! !
  !                         (-----)   ! (-------------)   (---------)  (-----) ! ! (---------)! !
  !			    !queue!   ! !mp1videoparse!   !mpeg_play!  !queue! ! ! !videosink!! !
  !                         sink src-sink            src-sink      src-sink src-sink         !! !
  !       (-----------)     (-----)   ! (-------------)   (---------)  (-----) ! ! (---------)! !
  !       ! mpeg1parse!               (----------------------------------------) (------------) !
  !   - sink          !                                                                         !
  !  /    (-----------)                                                                         !
 sink                                 (----------------------------------------) (------------) !
  !                                   !thread                                  ! ! thread     ! !
  !                         (-----)   ! (-------------)                (-----) ! ! (---------)! !
  !			    !queue!   ! !mad          !                !queue! ! ! !videosink!! !
  !                         sink src-sink            src ------------ sink  src-sink         !! !
  !                         (-----)   ! (-------------)                (-----) ! ! (---------)! !
  !                                   (----------------------------------------) (------------) !
  (---------------------------------------------------------------------------------------------)

   The autoplugger will return the autoplug_bin. the app will then connect the 
   disksrc to the sinkpad of the autoplugged bin.

   Then we play, create_plan happens, data is flowing and the "new_pad" signal is called
   from mpeg1parse, gst_pipeline_pad_autoplug is called and the connection between
   mpeg1parse and the videoqueue is made. same for audio.

   Et voila. same procedure for mp3/vorbis/avi/qt/mpeg2 etc...