mirrors/gstreamer
mirrors/gstreamer
1
1
Fork 0
mirror of https://gitlab.freedesktop.org/gstreamer/gstreamer.git synced 2024-11-22 17:51:16 +00:00
Code Issues 1 Projects Releases Wiki Activity

Added an explanation of the innner workings of the current autoplugger.

Browse source 
Original commit message from CVS:
Added an explanation of the innner workings of the current autoplugger.
This commit is contained in:
Wim Taymans 2001-04-30 23:20:42 +00:00
parent d51b59c57b
commit b67a5bf39d
1 changed files with 289 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

289
docs/random/autoplug2 Normal file
View file

@ -0,0 +1,289 @@
1) The Autoplugger API
----------------------
We'll first describe how to use the autoplugger. We will provide
a use case: autoplug an mpeg1 system stream for audio/video playback.
a) creating an autoplugger
--------------------------
Before any autoplugging can be done, you'll have to create an
autoplugger object. Autoplugger objects (autopluggers) are
provided by plugins and are created with gst_autoplugfactor_make().
GStreamer has provisions for two types of autopluggers:
- regular autopluggers, which act as a complex element construction
mechanism. They usually don't create threads and operate solely on
GstCaps* for the source and destination. The complex elements
created by regular autopluggers have src and sink pad compatible
with the requested GstCaps*.
- renderer autopluggers, which are designed to create a complex
object that can be used to playback media. Renderer autoplugged
complex elements have no src pads, only one sink pad.
We'll create a renderer autoplugger like this:
!
! GstAutoplug *autoplug;
!
! autoplug = gst_autoplugfactory_make ("staticrender");
!
b) finding out the source media type.
-------------------------------------
Before we can start the autoplugger, we have to find out the
source media type. This can be done using the typefind functions
provided by various plugins.
We will create a little pipeline to detect the media type by connecting
a disksrc element to a typefind element. The typefind element will
repeadedly call all registered typefind functions with the buffer it
receives on its sink pad. when a typefind function returns a non NULL
GstCaps*, that caps is set to the sink pad of the typefind element and
a signal is emitted to notify the app.
Due to caps negotiation, the disksrc will have the detected GstCaps*
set on its src pad.
We typically use a function like below to detect the type of a media stream
on an element (typically a disksrc). The function accepts a pipeline and the
element inside the pipeline on which the typefind should be performed (passing
a GstPad* is probably a better option FIXME).
!
! static GstCaps*
! gst_play_typefind (GstBin *bin, GstElement *element)
! {
! GstElement *typefind;
! GstCaps *caps = NULL;
!
! typefind = gst_elementfactory_make ("typefind", "typefind");
! g_return_val_if_fail (typefind != NULL, FALSE);
!
! gst_pad_connect (gst_element_get_pad (element, "src"),
! gst_element_get_pad (typefind, "sink"));
!
! gst_bin_add (bin, typefind);
!
! gst_element_set_state (GST_ELEMENT (bin), GST_STATE_PLAYING);
!
! // push a buffer... the have_type signal handler will set the found flag
! gst_bin_iterate (bin);
!
! gst_element_set_state (GST_ELEMENT (bin), GST_STATE_NULL);
!
! caps = gst_pad_get_caps (gst_element_get_pad (element, "src"));
!
! gst_pad_disconnect (gst_element_get_pad (element, "src"),
! gst_element_get_pad (typefind, "sink"));
! gst_bin_remove (bin, typefind);
! gst_object_unref (GST_OBJECT (typefind));
!
! return caps;
! }
!
Also note that the disksrc was added to the pipeline before calling this
typefind function.
When the function returns a non-NULL pointer, the media type has been
determined and autoplugging can commence.
Assume that in our mpeg1 use case the above function returns a GstCaps*
like:
!
! srccaps = GST_CAPS_NEW ("mpeg1system_typefind",
! "video/mpeg",
! "mpegversion", GST_PROPS_INT (1),
! "systemstream", GST_PROPS_BOOLEAN (TRUE)
! );
!
c) Performing the autoplugging
------------------------------
Since we use the renderer API, we have to create the output elements
that are going to be used as the final sink elements.
!
! osssink = gst_elementfactory_make("osssink", "play_audio");
! videosink = gst_elementfactory_make("xvideosink", "play_video");
!
We then create a complex element using the following code.
!
! new_element = gst_autoplug_to_renderers (autoplug,
! srccaps,
! videosink,
! osssink,
! NULL);
!
! if (!new_element) {
! g_print ("could not autoplug, no suitable codecs found...\n");
! exit (-1);
! }
!
2) Autoplugging internals
-------------------------
We will now describe the internals of the above gst_autoplug_to_renderers()
function call. This code is implemented in a plugin found in:
gst/autoplug/gststaticautoplugrender.c
a) phase1: create lists of factories.
---------------------------------------
The autoplugger will start with executing the following piece of
code:
!
! i = 0;
!
! for each sink:
! {
! sinkpad = take the first sinkpad of the sink (HACK)
!
! list[i] = gst_autoplug_caps (srccaps, sinkpad->caps);
!
! i++;
! }
!
gst_autoplug_caps will figure out (based on the padtemplates)
which elementfactories are needed to connect srccaps to sinkpad->caps
and will return them in a list.
The element list is created by using a modified shortest path algorithm
by Dijkstra (http://www.orie.cornell.edu/~or115/handouts/handout3/handout3.html).
The nodes of the graph are the elementfactories and the weight of the
arcs is based on the pad compatibility of the padtemplates of the
elementfactory. For incompatible elementfactories, we use a weight of
MAX_COST (999999) and for compatible padtemplates we use 1.
ex. we have two sinks with following caps:
!
! video/raw audio/raw
! "...." "...."
!
gst_autoplug_caps will figure out that for the first sink the following
elements are needed:
!
! mpeg1parse, mp1videoparse, mpeg_play
!
for the second sink the following is needed:
!
! mpeg1parse, mad
!
Note that for the audio connection the element list "mpeg1parse, mp3parse,
mpg123" would also connect the srccaps to the audiosink caps. Since the
"mpeg1parse, mad" list is shorter, it it always prefered by the autoplugger.
We now have two lists of elementfactories.
b) phase2: collect common elements from the lists and add them to a bin.
------------------------------------------------------------------------
The rationale is that from the lists we have created in phase1, there
must be some element that is a splitter and that it has to come first (HACK)
We try to find that element by comparing the lists until an element differs.
We start by creating a toplevel bin that is going to be our complex element.
In our use-case we find that mpeg1parse is an element common to both lists,
so we add it to the bin. We then try to find a good ghostpad for the resulting
complex element. This is done by looping over the sink pads of the first common
element and taking the pad that is compatible with the srcaps.
We end up with a bin like this:
!
! (----------------------)
! ! autoplug_bin !
! ! !
! ! (------------) !
! ! ! mpeg1parse ! !
! ! - sink ! !
! ! / (------------) !
! sink !
! (----------------------)
!
c) phase3: add remaining elements
---------------------------------
now we loop over all the list and try to add the remaining elements
(HACK) we always use a new thread for the elements when there is a common
element found.
if a new thread is needed (either becuase the previous element is a common
element or the object flag of the next element is set to GST_SUGGEST_THREAD)
we add a queue to the bin and we add a new thread. We add the elements to
the bin and connect them using gst_pipeline_pads_autoplug.
we finally arrive at the sink element and we're done.
ex.
we have just found our mpeg1parse common element, so we start a thread.
We add a queue to the bin and a new thread, we add the elements
mp1videoparse and mpeg_play to the thread. We arrive at the videosink, we
see that the SUGGEST_THREAD flag is set, we add a queue and a thread and
add the videosink in the thread.
the same procedure happens for the audio part. We are now left with the
following pipeline:
We will also have set a signal "new_pad" on the mpeg1parse element because
the element mp1videoparse could not be connected to the element just yet.
(---------------------------------------------------------------------------------------------)
!autoplug_bin !
! !
! (----------------------------------------) (------------) !
! !thread ! ! thread ! !
! (-----) ! (-------------) (---------) (-----) ! ! (---------)! !
! !queue! ! !mp1videoparse! !mpeg_play! !queue! ! ! !videosink!! !
! sink src-sink src-sink src-sink src-sink !! !
! (-----------) (-----) ! (-------------) (---------) (-----) ! ! (---------)! !
! ! mpeg1parse! (----------------------------------------) (------------) !
! - sink ! !
! / (-----------) !
sink (----------------------------------------) (------------) !
! !thread ! ! thread ! !
! (-----) ! (-------------) (-----) ! ! (---------)! !
! !queue! ! !mad ! !queue! ! ! !videosink!! !
! sink src-sink src ------------ sink src-sink !! !
! (-----) ! (-------------) (-----) ! ! (---------)! !
! (----------------------------------------) (------------) !
(---------------------------------------------------------------------------------------------)
The autoplugger will return the autoplug_bin. the app will then connect the
disksrc to the sinkpad of the autoplugged bin.
Then we play, create_plan happens, data is flowing and the "new_pad" signal is called
from mpeg1parse, gst_pipeline_pad_autoplug is called and the connection between
mpeg1parse and the videoqueue is made. same for audio.
Et voila. same procedure for mp3/vorbis/avi/qt/mpeg2 etc...