Scheduling is, in short, a method for making sure that every element gets called once in a while to process data and prepare data for the next element. Likewise, a kernel has a scheduler for processes, and your brain is a very complex scheduler too in a way. Randomly calling elements' chain functions won't bring us far, however, so you'll understand that the schedulers in &GStreamer; are a bit more complex than this. However, as a start, it's a nice picture. So far, we have only discussed _chain ()-operating elements, i.e. elements that have a chain-function set on their sink pad and push buffers on their source pad(s). Pads (or elements) can also operate in two other scheduling modes, however. In this chapter, we will discuss what those scheduling modes are, how they can be enabled and in what cases they are useful. The other two scheduling modes are random access (_getrange ()-based) or task-runner (which means that this element is the driving force in the pipeline) mode. The stage in which &GStreamer; decides in what scheduling mode the various elements will operate, is called the pad-activation stage. In this stage, &GStreamer; will query the scheduling capabilities (i.e. it will see in what modes each particular element/pad can operate) and decide on the optimal scheduling composition for the pipeline. Next, each pad will be notified of the scheduling mode that was assigned to it, and after that the pipeline will start running. Pads can be assigned one of three modes, each mode putting several prerequisites on the pads. Pads should implement a notification function (gst_pad_set_activatepull_function () and gst_pad_set_activatepush_function ()) to be notified of the scheduling mode assignment. Also, sinkpads assigned to do pull-based scheduling mode should start and stop their task in this function. If all pads of an element are assigned to do push-based scheduling, then this means that data will be pushed by upstream elements to this element using the sinkpads _chain ()-function. Prerequisites for this scheduling mode are that a chain-function was set for each sinkpad usinggst_pad_set_chain_function () and that all downstream elements operate in the same mode. Pads are assigned to do push-based scheduling in sink-to-source element order, and within an element first sourcepads and then sinkpads. Sink elements can operate in this mode if their sinkpad is activated for push-based scheduling. Source elements cannot be chain-based. Alternatively, sinkpads can be the driving force behind a pipeline by operating in pull-based mode, while the sourcepads of the element still operate in push-based mode. In order to be the driving force, those pads start a GstTask when their pads are being activated. This task is a thread, which will call a function specified by the element. When called, this function will have random data access (through gst_pad_get_range ()) over all sinkpads, and can push data over the sourcepads, which effectively means that this element controls data flow in the pipeline. Prerequisites for this mode are that all downstream elements can act in chain-based mode, and that all upstream elements allow random access (see below). Source elements can be told to act in this mode if their sourcepads are activated in push-based fashion. Sink elements can be told to act in this mode when their sinkpads are activated in pull-mode. lastly, all pads in an element can be assigned to act in pull-mode. too. However, contrary to the above, this does not mean that they start a task on their own. Rather, it means that they are pull slave for the downstream element, and have to provide random data access to it from their _get_range ()-function. Requirements are that the a _get_range ()-function was set on this pad using the function gst_pad_set_getrange_function (). Also, if the element has any sinkpads, all those pads (and thereby their peers) need to operate in random access mode, too. Note that the element is supposed to activate those elements itself! &GStreamer; will not do that for you. In the next two sections, we will go closer into pull-based scheduling (elements/pads driving the pipeline, and elements/pads providing random access), and some specific use cases will be given. Sinkpads assigned to operate in pull-based mode, while none of its sourcepads operate in pull-based mode (or it has no sourcepads), can start a task that will drive the pipeline data flow. Within this function, those elements have random access over all of their sinkpads, and push data over their sourcepads. This can come in useful for several different kinds of elements: Demuxers, parsers and certain kinds of decoders where data comes in unparsed (such as MPEG-audio or video streams), since those will prefer byte-exact (random) access from their input. If possible, however, such elements should be prepared to operate in chain-based mode, too. Certain kind of audio outputs, which require control over their input data flow, such as the Jack sound server. In order to start this task, you will need to create it in the activation function. #include "filter.h" #include <string.h> static gboolean gst_my_filter_activate (GstPad * pad); static gboolean gst_my_filter_activate_pull (GstPad * pad, gboolean active); static void gst_my_filter_loop (GstMyFilter * filter); GST_BOILERPLATE (GstMyFilter, gst_my_filter, GstElement, GST_TYPE_ELEMENT); static void gst_my_filter_init (GstMyFilter * filter) { [..] gst_pad_set_activate_function (filter->sinkpad, gst_my_filter_activate); gst_pad_set_activatepull_function (filter->sinkpad, gst_my_filter_activate_pull); [..] } [..] static gboolean gst_my_filter_activate (GstPad * pad) { if (gst_pad_check_pull_range (pad)) { return gst_pad_activate_pull (pad, TRUE); } else { return FALSE; } } static gboolean gst_my_filter_activate_pull (GstPad *pad, gboolean active) { GstMyFilter *filter = GST_MY_FILTER (GST_OBJECT_PARENT (pad)); if (active) { filter->offset = 0; return gst_pad_start_task (pad, (GstTaskFunction) gst_my_filter_loop, filter); } else { return gst_pad_stop_task (pad); } } Once started, your task has full control over input and output. The most simple case of a task function is one that reads input and pushes that over its source pad. It's not all that useful, but provides some more flexibility than the old chain-based case that we've been looking at so far. #define BLOCKSIZE 2048 static void gst_my_filter_loop (GstMyFilter * filter) { GstFlowReturn ret; guint64 len; GstFormat fmt = GST_FORMAT_BYTES; GstBuffer *buf = NULL; if (!gst_pad_query_duration (filter->sinkpad, &fmt, &len)) { GST_DEBUG_OBJECT (filter, "failed to query duration, pausing"); goto stop; } if (filter->offset >= len) { GST_DEBUG_OBJECT (filter, "at end of input, sending EOS, pausing"); gst_pad_push_event (filter->srcpad, gst_event_new_eos ()); goto stop; } /* now, read BLOCKSIZE bytes from byte offset filter->offset */ ret = gst_pad_pull_range (filter->sinkpad, filter->offset, BLOCKSIZE, &buf); if (ret != GST_FLOW_OK) { GST_DEBUG_OBJECT (filter, "pull_range failed: %s", gst_flow_get_name (ret)); goto stop; } /* now push buffer downstream */ ret = gst_pad_push (filter->srcpad, buf); buf = NULL; /* gst_pad_push() took ownership of buffer */ if (ret != GST_FLOW_OK) { GST_DEBUG_OBJECT (filter, "pad_push failed: %s", gst_flow_get_name (ret)); goto stop; } /* everything is fine, increase offset and wait for us to be called again */ filter->offset += BLOCKSIZE; return; stop: GST_DEBUG_OBJECT (filter, "pausing task"); gst_pad_pause_task (filter->sinkpad); } In the previous section, we have talked about how elements (or pads) that are assigned to drive the pipeline using their own task, have random access over their sinkpads. This means that all elements linked to those pads (recursively) need to provide random access functions. Requesting random access is done using the function gst_pad_pull_range (), which requests a buffer of a specified size and offset. Source pads implementing and assigned to do random access will have a _get_range ()-function set using gst_pad_set_getrange_function (), and that function will be called when the peer pad requests some data. The element is then responsible for seeking to the right offset and providing the requested data. Several elements can implement random access: Data sources, such as a file source, that can provide data from any offset with reasonable low latency. Filters that would like to provide a pull-based-like scheduling mode over the whole pipeline. Note that elements assigned to do random access-based scheduling are themselves responsible for assigning this scheduling mode to their upstream peers! &GStreamer; will not do that for you. Parsers who can easily provide this by skipping a small part of their input and are thus essentially "forwarding" random access requests literally without any own processing involved. Examples include tag readers (e.g. ID3) or single output parsers, such as a WAVE parser. The following example will show how a _get_range ()-function can be implemented in a source element: #include "filter.h" static GstFlowReturn gst_my_filter_get_range (GstPad * pad, guint64 offset, guint length, GstBuffer ** buf); GST_BOILERPLATE (GstMyFilter, gst_my_filter, GstElement, GST_TYPE_ELEMENT); static void gst_my_filter_init (GstMyFilter * filter) { GstElementClass *klass = GST_ELEMENT_GET_CLASS (filter); filter->srcpad = gst_pad_new_from_template ( gst_element_class_get_pad_template (klass, "src"), "src"); gst_pad_set_getrange_function (filter->srcpad, gst_my_filter_get_range); gst_element_add_pad (GST_ELEMENT (filter), filter->srcpad); [..] } static gboolean gst_my_filter_get_range (GstPad * pad, guint64 offset, guint length, GstBuffer ** buf) { GstMyFilter *filter = GST_MY_FILTER (GST_OBJECT_PARENT (pad)); [.. here, you would fill *buf ..] return GST_FLOW_OK; } In practice, many elements that could theoretically do random access, may in practice often be assigned to do push-based scheduling anyway, since there is no downstream element able to start its own task. Therefore, in practice, those elements should implement both a _get_range ()-function and a _chain ()-function (for filters and parsers) or a _get_range ()-function and be prepared to start their own task by providing _activate_* ()-functions (for source elements), so that &GStreamer; can decide for the optimal scheduling mode and have it just work fine in practice.