mirror of
https://gitlab.freedesktop.org/gstreamer/gstreamer.git
synced 2024-12-12 19:36:38 +00:00
437 lines
17 KiB
XML
437 lines
17 KiB
XML
<chapter id="chapter-scheduling" xreflabel="Different scheduling modes">
|
|
<title>Different scheduling modes</title>
|
|
<para>
|
|
Scheduling is, in short, a method for making sure that every element gets
|
|
called once in a while to process data and prepare data for the next
|
|
element. Likewise, a kernel has a scheduler for processes, and your
|
|
brain is a very complex scheduler too in a way.
|
|
Randomly calling elements' chain functions won't bring us far, however, so
|
|
you'll understand that the schedulers in &GStreamer; are a bit more complex
|
|
than this. However, as a start, it's a nice picture.
|
|
</para>
|
|
<para>
|
|
So far, we have only discussed <function>_chain ()</function>-operating
|
|
elements, i.e. elements that have a chain-function set on their sink pad
|
|
and push buffers on their source pad(s). Pads (or elements) can also operate
|
|
in two other scheduling modes, however. In this chapter, we will discuss
|
|
what those scheduling modes are, how they can be enabled and in what
|
|
cases they are useful. The other two scheduling modes are random access
|
|
(<function>_getrange ()</function>-based) or task-runner (which means
|
|
that this element is the driving force in the pipeline) mode.
|
|
</para>
|
|
|
|
<sect1 id="section-scheduling-activation"
|
|
xreflabel="The pad actication stage">
|
|
<title>The pad activation stage</title>
|
|
<para>
|
|
The stage in which &GStreamer; decides in what scheduling mode the
|
|
various elements will operate, is called the pad-activation stage. In
|
|
this stage, &GStreamer; will query the scheduling capabilities (i.e.
|
|
it will see in what modes each particular element/pad can operate) and
|
|
decide on the optimal scheduling composition for the pipeline. Next,
|
|
each pad will be notified of the scheduling mode that was assigned to
|
|
it, and after that the pipeline will start running.
|
|
</para>
|
|
<para>
|
|
Pads can be assigned one of three modes, each mode putting several
|
|
prerequisites on the pads. Pads should implement a notification
|
|
function (<function>gst_pad_set_activatepull_function ()</function> and
|
|
<function>gst_pad_set_activatepush_function ()</function>) to be
|
|
notified of the scheduling mode assignment. Also, sinkpads assigned
|
|
to do pull-based scheduling mode should start and stop their task
|
|
in this function.
|
|
</para>
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>
|
|
If all pads of an element are assigned to do
|
|
<quote>push</quote>-based scheduling, then this means that data
|
|
will be pushed by upstream elements to this element using the
|
|
sinkpads <function>_chain ()</function>-function. Prerequisites
|
|
for this scheduling mode are that a chain-function was set for
|
|
each sinkpad using<function>gst_pad_set_chain_function ()</function>
|
|
and that all downstream elements operate in the same mode. Pads are
|
|
assigned to do push-based scheduling in sink-to-source element
|
|
order, and within an element first sourcepads and then sinkpads.
|
|
Sink elements can operate in this mode if their sinkpad is activated
|
|
for push-based scheduling. Source elements cannot be chain-based.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
Alternatively, sinkpads can be the driving force behind a pipeline
|
|
by operating in <quote>pull</quote>-based mode, while the sourcepads
|
|
of the element still operate in push-based mode. In order to be the
|
|
driving force, those pads start a <classname>GstTask</classname>
|
|
when they are activated. This task is a thread, which
|
|
will call a function specified by the element. When called, this
|
|
function will have random data access (through
|
|
<function>gst_pad_get_range ()</function>) over all sinkpads, and
|
|
can push data over the sourcepads, which effectively means that
|
|
this element controls data flow in the pipeline. Prerequisites for
|
|
this mode are that all downstream elements can act in chain-based
|
|
mode, and that all upstream elements allow random access (see below).
|
|
Source elements can be told to act in this mode if their sourcepads
|
|
are activated in push-based fashion. Sink elements can be told to
|
|
act in this mode when their sinkpads are activated in pull-mode.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
lastly, all pads in an element can be assigned to act in pull-mode.
|
|
too. However, contrary to the above, this does not mean that they
|
|
start a task on their own. Rather, it means that they are pull
|
|
slave for the downstream element, and have to provide random data
|
|
access to it from their <function>_get_range ()</function>-function.
|
|
Requirements are that the a <function>_get_range
|
|
()</function>-function was set on this pad using the function
|
|
<function>gst_pad_set_getrange_function ()</function>. Also, if
|
|
the element has any sinkpads, all those pads (and thereby their
|
|
peers) need to operate in random access mode, too. Note that the
|
|
element is supposed to activate those elements itself! &GStreamer;
|
|
will not do that for you.
|
|
</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
<para>
|
|
In the next two sections, we will go closer into pull-based scheduling
|
|
(elements/pads driving the pipeline, and elements/pads providing random
|
|
access), and some specific use cases will be given.
|
|
</para>
|
|
</sect1>
|
|
|
|
<sect1 id="section-scheduling-loop" xreflabel="Pads driving the pipeline">
|
|
<title>Pads driving the pipeline</title>
|
|
<para>
|
|
Sinkpads assigned to operate in pull-based mode, while none of its
|
|
sourcepads operate in pull-based mode (or it has no sourcepads), can
|
|
start a task that will drive the pipeline data flow. Within this
|
|
function, those elements have random access over all of their sinkpads,
|
|
and push data over their sourcepads. This can come in useful for
|
|
several different kinds of elements:
|
|
</para>
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>
|
|
Demuxers, parsers and certain kinds of decoders where data comes
|
|
in unparsed (such as MPEG-audio or video streams), since those will
|
|
prefer byte-exact (random) access from their input. If possible,
|
|
however, such elements should be prepared to operate in chain-based
|
|
mode, too.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
Certain kind of audio outputs, which require control over their
|
|
input data flow, such as the Jack sound server.
|
|
</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
<para>
|
|
In order to start this task, you will need to create it in the
|
|
activation function.
|
|
</para>
|
|
<programlisting><!-- example-begin task.c a -->
|
|
#include "filter.h"
|
|
#include <string.h>
|
|
|
|
static gboolean gst_my_filter_activate (GstPad * pad);
|
|
static gboolean gst_my_filter_activate_pull (GstPad * pad,
|
|
gboolean active);
|
|
static void gst_my_filter_loop (GstMyFilter * filter);
|
|
|
|
GST_BOILERPLATE (GstMyFilter, gst_my_filter, GstElement, GST_TYPE_ELEMENT);
|
|
<!-- example-end task.c a -->
|
|
<!-- example-begin task.c b --><!--
|
|
static gboolean gst_my_filter_setcaps (GstPad *pad,
|
|
GstCaps *caps);
|
|
static GstCaps *gst_my_filter_getcaps (GstPad *pad);
|
|
|
|
static void
|
|
gst_my_filter_base_init (gpointer klass)
|
|
{
|
|
GstElementClass *element_class = GST_ELEMENT_CLASS (klass);
|
|
static GstElementDetails my_filter_details = {
|
|
"An example plugin",
|
|
"Example/FirstExample",
|
|
"Shows the basic structure of a plugin",
|
|
"your name <your.name@your.isp>"
|
|
};
|
|
static GstStaticPadTemplate sink_factory =
|
|
GST_STATIC_PAD_TEMPLATE (
|
|
"sink",
|
|
GST_PAD_SINK,
|
|
GST_PAD_ALWAYS,
|
|
GST_STATIC_CAPS ("ANY")
|
|
);
|
|
static GstStaticPadTemplate src_factory =
|
|
GST_STATIC_PAD_TEMPLATE (
|
|
"src",
|
|
GST_PAD_SRC,
|
|
GST_PAD_ALWAYS,
|
|
GST_STATIC_CAPS ("ANY")
|
|
);
|
|
|
|
gst_element_class_set_details (element_class, &my_filter_details);
|
|
gst_element_class_add_pad_template (element_class,
|
|
gst_static_pad_template_get (&src_factory));
|
|
gst_element_class_add_pad_template (element_class,
|
|
gst_static_pad_template_get (&sink_factory));
|
|
}
|
|
|
|
static void
|
|
gst_my_filter_class_init (GstMyFilterClass * klass)
|
|
{
|
|
}
|
|
--><!-- example-begin task.c b -->
|
|
<!-- example-begin task.c c -->
|
|
static void
|
|
gst_my_filter_init (GstMyFilter * filter)
|
|
{
|
|
<!-- example-end task.c c -->
|
|
[..]<!-- example-begin task.c d --><!--
|
|
GstElementClass *klass = GST_ELEMENT_GET_CLASS (filter);
|
|
|
|
filter->sinkpad = gst_pad_new_from_template (
|
|
gst_element_class_get_pad_template (klass, "sink"), "sink");
|
|
gst_pad_set_setcaps_function (filter->sinkpad, gst_my_filter_setcaps);
|
|
gst_pad_set_getcaps_function (filter->sinkpad, gst_my_filter_getcaps);
|
|
--><!-- example-end task.c d -->
|
|
<!-- example-begin task.c e -->
|
|
gst_pad_set_activate_function (filter->sinkpad, gst_my_filter_activate);
|
|
gst_pad_set_activatepull_function (filter->sinkpad,
|
|
gst_my_filter_activate_pull);
|
|
<!-- example-end task.c e -->
|
|
<!-- example-begin task.c f --><!--
|
|
gst_element_add_pad (GST_ELEMENT (filter), filter->sinkpad);
|
|
|
|
filter->srcpad = gst_pad_new_from_template (
|
|
gst_element_class_get_pad_template (klass, "src"), "src");
|
|
gst_element_add_pad (GST_ELEMENT (filter), filter->srcpad);
|
|
--><!-- example-end task.c f -->
|
|
[..]<!-- example-begin task.c g -->
|
|
}
|
|
<!-- example-end task.c g -->
|
|
[..]<!-- example-begin task.c h --><!--
|
|
#include "caps.func"
|
|
--><!-- example-end task.c h -->
|
|
<!-- example-begin task.c i -->
|
|
static gboolean
|
|
gst_my_filter_activate (GstPad * pad)
|
|
{
|
|
if (gst_pad_check_pull_range (pad)) {
|
|
return gst_pad_activate_pull (pad, TRUE);
|
|
} else {
|
|
return FALSE;
|
|
}
|
|
}
|
|
|
|
static gboolean
|
|
gst_my_filter_activate_pull (GstPad *pad,
|
|
gboolean active)
|
|
{
|
|
GstMyFilter *filter = GST_MY_FILTER (GST_OBJECT_PARENT (pad));
|
|
|
|
if (active) {
|
|
filter->offset = 0;
|
|
return gst_pad_start_task (pad,
|
|
(GstTaskFunction) gst_my_filter_loop, filter);
|
|
} else {
|
|
return gst_pad_stop_task (pad);
|
|
}
|
|
}
|
|
<!-- example-end task.c i --></programlisting>
|
|
<para>
|
|
Once started, your task has full control over input and output. The
|
|
most simple case of a task function is one that reads input and pushes
|
|
that over its source pad. It's not all that useful, but provides some
|
|
more flexibility than the old chain-based case that we've been looking
|
|
at so far.
|
|
</para>
|
|
<programlisting><!-- example-begin task.c j -->
|
|
#define BLOCKSIZE 2048
|
|
|
|
static void
|
|
gst_my_filter_loop (GstMyFilter * filter)
|
|
{
|
|
GstFlowReturn ret;
|
|
guint64 len;
|
|
GstFormat fmt = GST_FORMAT_BYTES;
|
|
GstBuffer *buf = NULL;
|
|
|
|
if (!gst_pad_query_duration (filter->sinkpad, &fmt, &len)) {
|
|
GST_DEBUG_OBJECT (filter, "failed to query duration, pausing");
|
|
goto stop;
|
|
}
|
|
|
|
if (filter->offset >= len) {
|
|
GST_DEBUG_OBJECT (filter, "at end of input, sending EOS, pausing");
|
|
gst_pad_push_event (filter->srcpad, gst_event_new_eos ());
|
|
goto stop;
|
|
}
|
|
|
|
/* now, read BLOCKSIZE bytes from byte offset filter->offset */
|
|
ret = gst_pad_pull_range (filter->sinkpad, filter->offset,
|
|
BLOCKSIZE, &buf);
|
|
|
|
if (ret != GST_FLOW_OK) {
|
|
GST_DEBUG_OBJECT (filter, "pull_range failed: %s", gst_flow_get_name (ret));
|
|
goto stop;
|
|
}
|
|
|
|
/* now push buffer downstream */
|
|
ret = gst_pad_push (filter->srcpad, buf);
|
|
|
|
buf = NULL; /* gst_pad_push() took ownership of buffer */
|
|
|
|
if (ret != GST_FLOW_OK) {
|
|
GST_DEBUG_OBJECT (filter, "pad_push failed: %s", gst_flow_get_name (ret));
|
|
goto stop;
|
|
}
|
|
|
|
/* everything is fine, increase offset and wait for us to be called again */
|
|
filter->offset += BLOCKSIZE;
|
|
return;
|
|
|
|
stop:
|
|
GST_DEBUG_OBJECT (filter, "pausing task");
|
|
gst_pad_pause_task (filter->sinkpad);
|
|
}
|
|
<!-- example-end task.c j -->
|
|
<!-- example-begin task.c k --><!--
|
|
#include "register.func"
|
|
--><!-- example-end task.c k --></programlisting>
|
|
</sect1>
|
|
|
|
<sect1 id="section-scheduling-randomxs" xreflabel="Providing random access">
|
|
<title>Providing random access</title>
|
|
<para>
|
|
In the previous section, we have talked about how elements (or pads)
|
|
that are assigned to drive the pipeline using their own task, have
|
|
random access over their sinkpads. This means that all elements linked
|
|
to those pads (recursively) need to provide random access functions.
|
|
Requesting random access is done using the function
|
|
<function>gst_pad_pull_range ()</function>, which requests a buffer of
|
|
a specified size and offset. Source pads implementing and assigned to
|
|
do random access will have a <function>_get_range ()</function>-function
|
|
set using <function>gst_pad_set_getrange_function ()</function>, and
|
|
that function will be called when the peer pad requests some data. The
|
|
element is then responsible for seeking to the right offset and
|
|
providing the requested data. Several elements can implement random
|
|
access:
|
|
</para>
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>
|
|
Data sources, such as a file source, that can provide data from any
|
|
offset with reasonable low latency.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
Filters that would like to provide a pull-based-like scheduling
|
|
mode over the whole pipeline. Note that elements assigned to do
|
|
random access-based scheduling are themselves responsible for
|
|
assigning this scheduling mode to their upstream peers! &GStreamer;
|
|
will not do that for you.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
Parsers who can easily provide this by skipping a small part of
|
|
their input and are thus essentially "forwarding" random access
|
|
requests literally without any own processing involved. Examples
|
|
include tag readers (e.g. ID3) or single output parsers, such as
|
|
a WAVE parser.
|
|
</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
<para>
|
|
The following example will show how a <function>_get_range
|
|
()</function>-function can be implemented in a source element:
|
|
</para>
|
|
<programlisting><!-- example-begin range.c a -->
|
|
#include "filter.h"
|
|
static GstFlowReturn
|
|
gst_my_filter_get_range (GstPad * pad,
|
|
guint64 offset,
|
|
guint length,
|
|
GstBuffer ** buf);
|
|
|
|
GST_BOILERPLATE (GstMyFilter, gst_my_filter, GstElement, GST_TYPE_ELEMENT);
|
|
<!-- example-end range.c a -->
|
|
<!-- example-begin range.c b --><!--
|
|
static void
|
|
gst_my_filter_base_init (gpointer klass)
|
|
{
|
|
GstElementClass *element_class = GST_ELEMENT_CLASS (klass);
|
|
static GstElementDetails my_filter_details = {
|
|
"An example plugin",
|
|
"Example/FirstExample",
|
|
"Shows the basic structure of a plugin",
|
|
"your name <your.name@your.isp>"
|
|
};
|
|
static GstStaticPadTemplate src_factory =
|
|
GST_STATIC_PAD_TEMPLATE (
|
|
"src",
|
|
GST_PAD_SRC,
|
|
GST_PAD_ALWAYS,
|
|
GST_STATIC_CAPS ("ANY")
|
|
);
|
|
|
|
gst_element_class_set_details (element_class, &my_filter_details);
|
|
gst_element_class_add_pad_template (element_class,
|
|
gst_static_pad_template_get (&src_factory));
|
|
}
|
|
|
|
static void
|
|
gst_my_filter_class_init (GstMyFilterClass * klass)
|
|
{
|
|
}
|
|
--><!-- example-begin range.c b -->
|
|
<!-- example-begin range.c c -->
|
|
static void
|
|
gst_my_filter_init (GstMyFilter * filter)
|
|
{
|
|
GstElementClass *klass = GST_ELEMENT_GET_CLASS (filter);
|
|
|
|
filter->srcpad = gst_pad_new_from_template (
|
|
gst_element_class_get_pad_template (klass, "src"), "src");
|
|
gst_pad_set_getrange_function (filter->srcpad,
|
|
gst_my_filter_get_range);
|
|
gst_element_add_pad (GST_ELEMENT (filter), filter->srcpad);
|
|
<!-- example-end range.c c -->
|
|
[..]<!-- example-begin range.c d -->
|
|
}
|
|
|
|
static gboolean
|
|
gst_my_filter_get_range (GstPad * pad,
|
|
guint64 offset,
|
|
guint length,
|
|
GstBuffer ** buf)
|
|
{
|
|
<!-- example-end range.c d -->
|
|
GstMyFilter *filter = GST_MY_FILTER (GST_OBJECT_PARENT (pad));
|
|
|
|
[.. here, you would fill *buf ..]
|
|
<!-- example-begin range.c e -->
|
|
return GST_FLOW_OK;
|
|
}
|
|
<!-- example-end range.c e -->
|
|
<!-- example-begin range.c f --><!--
|
|
#include "register.func"
|
|
--><!-- example-end range.c f --></programlisting>
|
|
<para>
|
|
In practice, many elements that could theoretically do random access,
|
|
may in practice often be assigned to do push-based scheduling anyway,
|
|
since there is no downstream element able to start its own task.
|
|
Therefore, in practice, those elements should implement both a
|
|
<function>_get_range ()</function>-function and a <function>_chain
|
|
()</function>-function (for filters and parsers) or a <function>_get_range
|
|
()</function>-function and be prepared to start their own task by
|
|
providing <function>_activate_* ()</function>-functions (for
|
|
source elements), so that &GStreamer; can decide for the optimal
|
|
scheduling mode and have it just work fine in practice.
|
|
</para>
|
|
</sect1>
|
|
</chapter>
|