Element types ------------- SOURCES ------- * never chain-based * have no sinkpads 1) get based src (----------) ! fakesrc ! ! src- (get based) (----------) * no sinkpads * srcpad(s) that are get-based 2) loop based src (----------) ! fakesrc ! ! src- (loop based) (----------) * no sinkpads * element is loop-based * data is pushed on the srcpad(s) FILTERS ------- 3) chain based filter (----------) ! identity ! -sink src- (----------) * sinkpad(s) have a chain function * srcpad(s) push data 4) loop-based filter (----------) ! identity ! -sink src- (----------) * element is loop-based * data is pushed on the srcpads * data is pulled from sinkpad(s) SINKS ----- 5) chain based sink (----------) ! fakesink ! -sink ! (----------) * sinkpad(s) have a chain function * no srcpads 6) loop-based sink (----------) ! fakesink ! -sink ! (----------) * element is loop-based * data is pulled from sinkpad(s) DECOUPLED --------- 7) decoupled element (----------) ! queue ! -sink src- (----------) * sinkpad(s) have chain function * srcpad(s) have get function * never loop-based * always acts like a chain-based sink for upstream elements * always acts like a get-based src for downstream elements * are not added to a group, but marked as an entry point in case it acts as a src element Connection types ---------------- 1) get based src 2) loop based src 3) chain based filter 4) loop-based filter 5) chain based sink 6) loop-based sink 7) decoupled element ! 1 ! 2 ! 3 ! 4 ! 5 ! 6 ! 7 ! ---+-----+-----+-----+-----+-----+-----+-----+ 1 ! X ! X ! A ! C ! A ! C ! A ! ! ! ! ! ! ! ! ! 2 ! X ! X ! B ! F ! B ! F ! B ! ! ! ! ! ! ! ! ! 3 ! X ! X ! D ! E ! D ! E ! D ! ! ! ! ! ! ! ! ! 4 ! X ! X ! B ! F ! B ! F ! B ! ! ! ! ! ! ! ! ! 5 ! X ! X ! X ! X ! X ! X ! X ! ! ! ! ! ! ! ! ! 6 ! X ! X ! X ! X ! X ! X ! X ! ! ! ! ! ! ! ! ! 7 ! X ! X ! A ! C ! A ! C ! X ! ! ! ! ! ! ! ! ! A) src -> sink src -> filter src -> decoupled decoupled -> sink decoupled -> filter - get based source - chain based sink * one group * src at start of group and entry point * _get from src, push to sink (-group1---------------) ! ! ! *fakesrc -> fakesink ! (----------------------) B) src -> sink src -> filter src -> decoupled filter -> sink filter -> filter filter -> decoupled - loop based source/filter - chain based sink/filter/decoupled * one group * src/filter at start of group and entry point * loop on src, chainhandler set to chain function (-group1----------------) ! ! ! %*fakesrc -> fakesink ! (-----------------------) C) src -> sink src -> filter decoupled -> sink decoupled -> filter - get based source/decoupled - loop based sink/filter * one group * loop based element is entry point * loop on sink/filter, gethandler set to getfunction (-group1----------------) ! ! ! fakesrc -> %*fakesink ! (-----------------------) D) filter -> filter filter -> sink filter -> decoupled - chain based filter - chain based filter/sink/decoupled * one group is created to hold the two elements * no entry point * chainhandler set to peer chainfunction (-group1----------------) ! ! ! identity -> identity ! (-----------------------) E) filter -> filter filter -> sink - chain based filter - loop based filter/sink * two groups * group is created for src element if needed * chainhandler of loop based element set to loop wrapper, control is handed to the peer group * gethandler of loop based element set to get wrapper (-group1---) (-group2------) ! ! ! ! ! identity ---> %*identity ! (----------) (-------------) F) src -> filter src -> sink filter -> filter filter -> sink - loop based filter/src - loop based filter/sink * two groups * two entry points * chainhandler set to loop wrapper * gethandler set to get wrapper (-group1-----) (-group2------) ! ! ! ! ! %*identity ---> %*identity ! (------------) (-------------) Grouping -------- * a group has at most one loop based element * elements in a group are sorted, src elements first (not mandatory) * a group has one cothread * a group is created immediatly for loop based elements, all other elements are added to a group when a pad connection is made * get-based plugins are put in the same group as a peer loop based element * chain based elements are put in the same group as sink peer elements * entry point in the group is: - loopbased element - first src element if no loopbased element exists in the group Result: you end up with a group of connected elements with either: - a loop based plugin as the entry point - a get based plugin as the entry point Scheduling the group is a matter of starting the cothread and calling the loop function or doing a _get/_push on a srcpad. other examples of groups: ------------------------- % = loop based * = entry point of group . (-group1---------------) ! ! ! *fakesrc -> fakesink ! (----------------------) . (-group1---------------------------------------) ! ! ! *fakesrc -> identity -> identity -> fakesink ! (----------------------------------------------) . (-group1-----------------------------------------) ! ! ! fakesrc -> %*identity -> identity -> fakesink ! (------------------------------------------------) . (-group1---------------) (-group2-----------------) ! ! ! ! ! *fakesrc -> identity --> *%identity -> fakesink ! (----------------------) (------------------------) . (-group1------------------------------------) ! ! ! *fakesrc -> tee --> identity -> fakesink ! ! --> identity -> fakesink ! (-------------------------------------------) . (-group1------------------------------------) ! ! ! *fakesrc -> tee --> identity -> fakesink ! (--------------!----------------------------) v (-group2-----------------) ! ! ! *%identity -> fakesink ! (------------------------) . (-group1----------) (-group2-----------------) ! ! ! ! ! *fakesrc -> tee --> *%identity -> fakesink ! (--------------!--) (------------------------) v (-group3-----------------) ! ! ! *%identity -> fakesink ! (------------------------) . (-group1-----------------------) (-group2----------------------) ! ! ! ! ! filesrc -> *%mpegdemux --> queue* -> mpeg2dec -> xvideosink ! ! ! (-----------------------------) ! ! (-group3----------------------) ! ! ! ! ! --> queue* -> mad -> osssink ! (------------------------------) (-----------------------------) Chaining -------- * groups that are connected end up in the same chain * a group always belongs to a chain * updating the chain is only needed when two groups are connected with a connection of type E/F. for other connection types, the group itself is updated. * a chain is scheduled by scheduling a random group in the chain. Wrapper functions ----------------- iterating without cothreads --------------------------- A cothread for each group is the easiest way to schedule most of the pipelines. Some pipelines are however schedulable without any cothreads. Each group is schedulable without cothreads, one can call the group schedule function and be done with it. Problems arise one the group boundaries of connected elements, which are always of type E and F (chain->loop, loop->loop) We always have a producer group and a provider group in this case. chain->loop ----------- Scheduler algorithm ------------------- 1: select (random?) group in chain 2: schedule group 3: on E/F connections, the get/chain wrapper is called - get wrapper puts the peer element on the runqueue and recursively invokes the scheduler. - chain wrapper puts the buffer in the bufpen and puts the peer element in the runqueue 4: when the group is scheduled, take group from the runqueue and goto 2: 5: no more groups on the runqueue, iteration ends NOTES: - We need a GList instead of a single bufpen to hold buffers for multi-out elements. - We probably need to set a limit on the maximum number of recursions and size of the bufpen list. - elements run non-preemtively, a group is done scheduling when all elements in that group complete their chain/loop function. - can we only have a stack overflow when there is a loop in the pipeline? I think so. - putting groups twice on the runqueue is not a good idea, we need to check a flag or something, maybe give the group a higher priority? - what about starvation? We'll probably have to put the group at the end of the runqueue. - multi-out elements can introduce significant latency. consider: (-group1----------) (-group2-----------------) ! ! ! ! ! *filesrc -> mad --> *%identity -> osssink ! (-----------------) (------------------------) mad produces N output buffers, they are queued in the bufpen, when group1 is done, group2 is scheduled to empty the bufpen queue. The time it takes for the first buffer to arrive at osssink equals the time needed to decode the N-1 other buffers. Of course, given the right buffersize for filesrc N can be reduced to 1. mad can also suggest a buffersize to filesrc with the BUFFER_SIZE event.