pwg: rework scheduling docs

This commit is contained in:
Wim Taymans 2012-09-28 13:25:30 +02:00
parent f32c9d8572
commit 723c2a48b9

View file

@ -1,85 +1,93 @@
<chapter id="chapter-scheduling" xreflabel="Different scheduling modes"> <chapter id="chapter-scheduling" xreflabel="Different scheduling modes">
<title>Different scheduling modes</title> <title>Different scheduling modes</title>
<para> <para>
Scheduling is, in short, a method for making sure that every element gets The scheduling mode of a pad defines how data is retrieved from (source)
called once in a while to process data and prepare data for the next or given to (sink) pads. &GStreamer; can operate in two scheduling
element. Likewise, a kernel has a scheduler for processes, and your mode, called push- and pull-mode. &GStreamer; supports elements with pads
brain is a very complex scheduler too in a way. in any of the scheduling modes where not all pads need to be operating
Randomly calling elements' chain functions won't bring us far, however, so in the same mode.
you'll understand that the schedulers in &GStreamer; are a bit more complex
than this. However, as a start, it's a nice picture.
</para> </para>
<para> <para>
So far, we have only discussed <function>_chain ()</function>-operating So far, we have only discussed <function>_chain ()</function>-operating
elements, i.e. elements that have a chain-function set on their sink pad elements, i.e. elements that have a chain-function set on their sink pad
and push buffers on their source pad(s). Pads (or elements) can also operate and push buffers on their source pad(s). We call this the push-mode
in two other scheduling modes, however. In this chapter, we will discuss because a peer element will use <function>gst_pad_push ()</function> on
what those scheduling modes are, how they can be enabled and in what a srcpad, which will cause our <function>_chain ()</function>-function
cases they are useful. The other two scheduling modes are random access to be called, which in turn causes our element to push out a buffer on
(<function>_getrange ()</function>-based) or task-runner (which means the source pad. The initiative to start the dataflow happens somewhere
that this element is the driving force in the pipeline) mode. upstream when it pushes out a buffer and all downstream elements get
scheduled when their <function>_chain ()</function>-functions are
called in turn.
</para>
<para>
Before we explain pull-mode scheduling, let's first understand how the
different scheduling modes are selected and activated on a pad.
</para> </para>
<sect1 id="section-scheduling-activation" <sect1 id="section-scheduling-activation"
xreflabel="The pad actication stage"> xreflabel="The pad activation stage">
<title>The pad activation stage</title> <title>The pad activation stage</title>
<para> <para>
The stage in which &GStreamer; decides in what scheduling mode the During the element state change of READY->PAUSED, the pads of an
various elements will operate, is called the pad-activation stage. In element will be activated. This happens first on the source pads and
this stage, &GStreamer; will query the scheduling capabilities (i.e. then on the sink pads of the element. &GStreamer; calls the
it will see in what modes each particular element/pad can operate) and <function>_activate ()</function> of a pad. By default this function
decide on the optimal scheduling composition for the pipeline. Next, will activate the pad in push-mode by calling
each pad will be notified of the scheduling mode that was assigned to <function>gst_pad_activate_mode ()</function> with the GST_PAD_MODE_PUSH
it, and after that the pipeline will start running. scheduling mode.
It is possible to override the <function>_activate ()</function> of a pad
and decide on a different scheduling mode. You can know in what
scheduling mode a pad is activated by overriding the
<function>_activate_mode ()</function>-function.
</para> </para>
<para> <para>
Pads can be assigned one of three modes, each mode putting several &GStreamer; allows the different pads of an element to operate in
prerequisites on the pads. Pads should implement a notification different scheduling modes. This allows for many different possible
function (<function>gst_pad_set_activatepull_function ()</function> and use-cases. What follows is an overview of some typical use-cases.
<function>gst_pad_set_activatepush_function ()</function>) to be
notified of the scheduling mode assignment. Also, sinkpads assigned
to do pull-based scheduling mode should start and stop their task
in this function.
</para> </para>
<itemizedlist> <itemizedlist>
<listitem> <listitem>
<para> <para>
If all pads of an element are assigned to do If all pads of an element are activated in push-mode scheduling,
<quote>push</quote>-based scheduling, then this means that data the element as a whole is operating in push-mode.
will be pushed by upstream elements to this element using the For source elements this means that they will have to start a
sinkpads <function>_chain ()</function>-function. Prerequisites task that pushes out buffers on the source pad to the downstream
for this scheduling mode are that a chain-function was set for elements.
each sinkpad using<function>gst_pad_set_chain_function ()</function> Downstream elements will have data pushed to them by upstream elements
and that all downstream elements operate in the same mode. Pads are using the sinkpads <function>_chain ()</function>-function which will
assigned to do push-based scheduling in sink-to-source element push out buffers on the source pads.
order, and within an element first sourcepads and then sinkpads. Prerequisites for this scheduling mode are that a chain-function was
Sink elements can operate in this mode if their sinkpad is activated set for each sinkpad using <function>gst_pad_set_chain_function ()</function>
for push-based scheduling. Source elements cannot be chain-based. and that all downstream elements operate in the same mode.
</para> </para>
</listitem> </listitem>
<listitem> <listitem>
<para> <para>
Alternatively, sinkpads can be the driving force behind a pipeline Alternatively, sinkpads can be the driving force behind a pipeline
by operating in <quote>pull</quote>-based mode, while the sourcepads by operating in pull-mode, while the sourcepads
of the element still operate in push-based mode. In order to be the of the element still operate in push-mode. In order to be the
driving force, those pads start a <classname>GstTask</classname> driving force, those pads start a <classname>GstTask</classname>
when they are activated. This task is a thread, which when they are activated. This task is a thread, which
will call a function specified by the element. When called, this will call a function specified by the element. When called, this
function will have random data access (through function will have random data access (through
<function>gst_pad_get_range ()</function>) over all sinkpads, and <function>gst_pad_pull_range ()</function>) over all sinkpads, and
can push data over the sourcepads, which effectively means that can push data over the sourcepads, which effectively means that
this element controls data flow in the pipeline. Prerequisites for this element controls data flow in the pipeline. Prerequisites for
this mode are that all downstream elements can act in chain-based this mode are that all downstream elements can act in push
mode, and that all upstream elements allow random access (see below). mode, and that all upstream elements operate in pull-mode (see below).
Source elements can be told to act in this mode if their sourcepads </para>
are activated in push-based fashion. Sink elements can be told to <para>
act in this mode when their sinkpads are activated in pull-mode. Source pads can be activated in PULL mode by a downstream element
when they return GST_PAD_MODE_PULL from the GST_QUERY_SCHEDULING
query. Prerequisites for this scheduling mode are that a
getrange-function was set for the source pad using
<function>gst_pad_set_getrange_function ()</function>.
</para> </para>
</listitem> </listitem>
<listitem> <listitem>
<para> <para>
lastly, all pads in an element can be assigned to act in pull-mode. Lastly, all pads in an element can be activated in PULL-mode.
too. However, contrary to the above, this does not mean that they However, contrary to the above, this does not mean that they
start a task on their own. Rather, it means that they are pull start a task on their own. Rather, it means that they are pull
slave for the downstream element, and have to provide random data slave for the downstream element, and have to provide random data
access to it from their <function>_get_range ()</function>-function. access to it from their <function>_get_range ()</function>-function.
@ -87,14 +95,18 @@
()</function>-function was set on this pad using the function ()</function>-function was set on this pad using the function
<function>gst_pad_set_getrange_function ()</function>. Also, if <function>gst_pad_set_getrange_function ()</function>. Also, if
the element has any sinkpads, all those pads (and thereby their the element has any sinkpads, all those pads (and thereby their
peers) need to operate in random access mode, too. Note that the peers) need to operate in PULL access mode, too.
element is supposed to activate those elements itself! &GStreamer; </para>
will not do that for you. <para>
When a sink element is activated in PULL mode, it should start a
task that calls <function>gst_pad_pull_range ()</function> on its
sinkpad. It can only do this when the upstream SCHEDULING query
returns support for the GST_PAD_MODE_PULL scheduling mode.
</para> </para>
</listitem> </listitem>
</itemizedlist> </itemizedlist>
<para> <para>
In the next two sections, we will go closer into pull-based scheduling In the next two sections, we will go closer into pull-mode scheduling
(elements/pads driving the pipeline, and elements/pads providing random (elements/pads driving the pipeline, and elements/pads providing random
access), and some specific use cases will be given. access), and some specific use cases will be given.
</para> </para>
@ -103,12 +115,12 @@
<sect1 id="section-scheduling-loop" xreflabel="Pads driving the pipeline"> <sect1 id="section-scheduling-loop" xreflabel="Pads driving the pipeline">
<title>Pads driving the pipeline</title> <title>Pads driving the pipeline</title>
<para> <para>
Sinkpads assigned to operate in pull-based mode, while none of its Sinkpads operating in pull-mode, with the sourcepads operating in
sourcepads operate in pull-based mode (or it has no sourcepads), can push-mode (or it has no sourcepads when it is a sink), can start a task
start a task that will drive the pipeline data flow. Within this that will drive the pipeline data flow.
function, those elements have random access over all of their sinkpads, Within this task function, you have random access over all of the sinkpads,
and push data over their sourcepads. This can come in useful for and push data over the sourcepads.
several different kinds of elements: This can come in useful for several different kinds of elements:
</para> </para>
<itemizedlist> <itemizedlist>
<listitem> <listitem>
@ -116,7 +128,7 @@
Demuxers, parsers and certain kinds of decoders where data comes Demuxers, parsers and certain kinds of decoders where data comes
in unparsed (such as MPEG-audio or video streams), since those will in unparsed (such as MPEG-audio or video streams), since those will
prefer byte-exact (random) access from their input. If possible, prefer byte-exact (random) access from their input. If possible,
however, such elements should be prepared to operate in chain-based however, such elements should be prepared to operate in push-mode
mode, too. mode, too.
</para> </para>
</listitem> </listitem>
@ -128,85 +140,40 @@
</listitem> </listitem>
</itemizedlist> </itemizedlist>
<para> <para>
In order to start this task, you will need to create it in the First you need to perform a SCHEDULING query to check if the upstream
activation function. element(s) support pull-mode scheduling. If that is possible, you
can activate the sinkpad in pull-mode. Inside the activate_mode
function you can then start the task.
</para> </para>
<programlisting><!-- example-begin task.c a --> <programlisting><!-- example-begin task.c a -->
#include "filter.h" #include "filter.h"
#include &lt;string.h&gt; #include &lt;string.h&gt;
static gboolean gst_my_filter_activate (GstPad * pad); static gboolean gst_my_filter_activate (GstPad * pad,
static gboolean gst_my_filter_activate_pull (GstPad * pad, GstObject * parent);
gboolean active); static gboolean gst_my_filter_activate_mode (GstPad * pad,
static void gst_my_filter_loop (GstMyFilter * filter); GstObject * parent,
GstPadMode mode
gboolean active);
static void gst_my_filter_loop (GstMyFilter * filter);
GST_BOILERPLATE (GstMyFilter, gst_my_filter, GstElement, GST_TYPE_ELEMENT); G_DEFINE_TYPE (GstMyFilter, gst_my_filter, GST_TYPE_ELEMENT);
<!-- example-end task.c a --> <!-- example-end task.c a -->
<!-- example-begin task.c b --><!--
static gboolean gst_my_filter_setcaps (GstPad *pad,
GstCaps *caps);
static GstCaps *gst_my_filter_getcaps (GstPad *pad);
static void
gst_my_filter_base_init (gpointer klass)
{
GstElementClass *element_class = GST_ELEMENT_CLASS (klass);
static GstElementDetails my_filter_details = {
"An example plugin",
"Example/FirstExample",
"Shows the basic structure of a plugin",
"your name <your.name@your.isp>"
};
static GstStaticPadTemplate sink_factory =
GST_STATIC_PAD_TEMPLATE (
"sink",
GST_PAD_SINK,
GST_PAD_ALWAYS,
GST_STATIC_CAPS ("ANY")
);
static GstStaticPadTemplate src_factory =
GST_STATIC_PAD_TEMPLATE (
"src",
GST_PAD_SRC,
GST_PAD_ALWAYS,
GST_STATIC_CAPS ("ANY")
);
gst_element_class_set_details (element_class, &my_filter_details);
gst_element_class_add_pad_template (element_class,
gst_static_pad_template_get (&src_factory));
gst_element_class_add_pad_template (element_class,
gst_static_pad_template_get (&sink_factory));
}
static void
gst_my_filter_class_init (GstMyFilterClass * klass)
{
}
--><!-- example-begin task.c b -->
<!-- example-begin task.c c --> <!-- example-begin task.c c -->
static void static void
gst_my_filter_init (GstMyFilter * filter) gst_my_filter_init (GstMyFilter * filter)
{ {
<!-- example-end task.c c --> <!-- example-end task.c c -->
[..]<!-- example-begin task.c d --><!-- [..]<!-- example-begin task.c d --><!--
GstElementClass *klass = GST_ELEMENT_GET_CLASS (filter);
filter-&gt;sinkpad = gst_pad_new_from_template (
gst_element_class_get_pad_template (klass, "sink"), "sink");
gst_pad_set_setcaps_function (filter-&gt;sinkpad, gst_my_filter_setcaps);
gst_pad_set_getcaps_function (filter-&gt;sinkpad, gst_my_filter_getcaps);
--><!-- example-end task.c d --> --><!-- example-end task.c d -->
<!-- example-begin task.c e --> <!-- example-begin task.c e -->
gst_pad_set_activate_function (filter-&gt;sinkpad, gst_my_filter_activate); gst_pad_set_activate_function (filter-&gt;sinkpad, gst_my_filter_activate);
gst_pad_set_activatepull_function (filter-&gt;sinkpad, gst_pad_set_activatemode_function (filter-&gt;sinkpad,
gst_my_filter_activate_pull); gst_my_filter_activate_mode);
<!-- example-end task.c e --> <!-- example-end task.c e -->
<!-- example-begin task.c f --><!-- <!-- example-begin task.c f --><!--
gst_element_add_pad (GST_ELEMENT (filter), filter-&gt;sinkpad); gst_element_add_pad (GST_ELEMENT (filter), filter-&gt;sinkpad);
filter-&gt;srcpad = gst_pad_new_from_template (
gst_element_class_get_pad_template (klass, "src"), "src");
gst_element_add_pad (GST_ELEMENT (filter), filter-&gt;srcpad); gst_element_add_pad (GST_ELEMENT (filter), filter-&gt;srcpad);
--><!-- example-end task.c f --> --><!-- example-end task.c f -->
[..]<!-- example-begin task.c g --> [..]<!-- example-begin task.c g -->
@ -217,35 +184,73 @@ gst_my_filter_init (GstMyFilter * filter)
--><!-- example-end task.c h --> --><!-- example-end task.c h -->
<!-- example-begin task.c i --> <!-- example-begin task.c i -->
static gboolean static gboolean
gst_my_filter_activate (GstPad * pad) gst_my_filter_activate (GstPad * pad, GstObject * parent)
{ {
if (gst_pad_check_pull_range (pad)) { GstQuery *query;
return gst_pad_activate_pull (pad, TRUE); gboolean pull_mode;
} else {
return FALSE; /* first check what upstream scheduling is supported */
query = gst_query_new_scheduling ();
if (!gst_pad_peer_query (pad, query)) {
gst_query_unref (query);
goto activate_push;
}
/* see if pull-mode is supported */
pull_mode = gst_query_has_scheduling_mode_with_flags (query,
GST_PAD_MODE_PULL, GST_SCHEDULING_FLAG_SEEKABLE);
gst_query_unref (query);
if (!pull_mode)
goto activate_push;
/* now we can activate in pull-mode. GStreamer will also
* activate the upstream peer in pull-mode */
return gst_pad_activate_mode (pad, GST_PAD_MODE_PULL, TRUE);
activate_push:
{
/* something not right, we fallback to push-mode */
return gst_pad_activate_mode (pad, GST_PAD_MODE_PUSH, TRUE);
} }
} }
static gboolean static gboolean
gst_my_filter_activate_pull (GstPad *pad, gst_my_filter_activate_pull (GstPad * pad,
gboolean active) GstObject * parent,
GstPadMode mode,
gboolean active)
{ {
GstMyFilter *filter = GST_MY_FILTER (GST_OBJECT_PARENT (pad)); gboolean res;
GstMyFilter *filter = GST_MY_FILTER (parent);
if (active) { switch (mode) {
filter->offset = 0; case GST_PAD_MODE_PUSH:
return gst_pad_start_task (pad, res = TRUE;
(GstTaskFunction) gst_my_filter_loop, filter); break;
} else { case GST_PAD_MODE_PULL:
return gst_pad_stop_task (pad); if (active) {
filter->offset = 0;
res = gst_pad_start_task (pad,
(GstTaskFunction) gst_my_filter_loop, filter, NULL);
} else {
res = gst_pad_stop_task (pad);
}
break;
default:
/* unknown scheduling mode */
res = FALSE;
break;
} }
return res;
} }
<!-- example-end task.c i --></programlisting> <!-- example-end task.c i --></programlisting>
<para> <para>
Once started, your task has full control over input and output. The Once started, your task has full control over input and output. The
most simple case of a task function is one that reads input and pushes most simple case of a task function is one that reads input and pushes
that over its source pad. It's not all that useful, but provides some that over its source pad. It's not all that useful, but provides some
more flexibility than the old chain-based case that we've been looking more flexibility than the old push-mode case that we've been looking
at so far. at so far.
</para> </para>
<programlisting><!-- example-begin task.c j --> <programlisting><!-- example-begin task.c j -->
@ -259,7 +264,7 @@ gst_my_filter_loop (GstMyFilter * filter)
GstFormat fmt = GST_FORMAT_BYTES; GstFormat fmt = GST_FORMAT_BYTES;
GstBuffer *buf = NULL; GstBuffer *buf = NULL;
if (!gst_pad_query_duration (filter-&gt;sinkpad, &amp;fmt, &amp;len)) { if (!gst_pad_query_duration (filter-&gt;sinkpad, fmt, &amp;len)) {
GST_DEBUG_OBJECT (filter, "failed to query duration, pausing"); GST_DEBUG_OBJECT (filter, "failed to query duration, pausing");
goto stop; goto stop;
} }
@ -307,16 +312,15 @@ stop:
<title>Providing random access</title> <title>Providing random access</title>
<para> <para>
In the previous section, we have talked about how elements (or pads) In the previous section, we have talked about how elements (or pads)
that are assigned to drive the pipeline using their own task, have that are activated to drive the pipeline using their own task, must use
random access over their sinkpads. This means that all elements linked pull-mode scheduling on their sinkpads. This means that all pads linked
to those pads (recursively) need to provide random access functions. to those pads need to be activated in pull-mode.
Requesting random access is done using the function Source pads activated in pull-mode must implement a
<function>gst_pad_pull_range ()</function>, which requests a buffer of <function>_get_range ()</function>-function set using
a specified size and offset. Source pads implementing and assigned to <function>gst_pad_set_getrange_function ()</function>, and
do random access will have a <function>_get_range ()</function>-function that function will be called when the peer pad requests some data with
set using <function>gst_pad_set_getrange_function ()</function>, and <function>gst_pad_pull_range ()</function>.
that function will be called when the peer pad requests some data. The The element is then responsible for seeking to the right offset and
element is then responsible for seeking to the right offset and
providing the requested data. Several elements can implement random providing the requested data. Several elements can implement random
access: access:
</para> </para>
@ -329,17 +333,14 @@ stop:
</listitem> </listitem>
<listitem> <listitem>
<para> <para>
Filters that would like to provide a pull-based-like scheduling Filters that would like to provide a pull-mode scheduling
mode over the whole pipeline. Note that elements assigned to do over the whole pipeline.
random access-based scheduling are themselves responsible for
assigning this scheduling mode to their upstream peers! &GStreamer;
will not do that for you.
</para> </para>
</listitem> </listitem>
<listitem> <listitem>
<para> <para>
Parsers who can easily provide this by skipping a small part of Parsers who can easily provide this by skipping a small part of
their input and are thus essentially "forwarding" random access their input and are thus essentially "forwarding" getrange
requests literally without any own processing involved. Examples requests literally without any own processing involved. Examples
include tag readers (e.g. ID3) or single output parsers, such as include tag readers (e.g. ID3) or single output parsers, such as
a WAVE parser. a WAVE parser.
@ -354,15 +355,16 @@ stop:
#include "filter.h" #include "filter.h"
static GstFlowReturn static GstFlowReturn
gst_my_filter_get_range (GstPad * pad, gst_my_filter_get_range (GstPad * pad,
GstObject * parent,
guint64 offset, guint64 offset,
guint length, guint length,
GstBuffer ** buf); GstBuffer ** buf);
GST_BOILERPLATE (GstMyFilter, gst_my_filter, GstElement, GST_TYPE_ELEMENT); G_DEFINE_TYPE (GstMyFilter, gst_my_filter, GST_TYPE_ELEMENT);
<!-- example-end range.c a --> <!-- example-end range.c a -->
<!-- example-begin range.c b --><!-- <!-- example-begin range.c b --><!--
static void static void
gst_my_filter_base_init (gpointer klass) gst_my_filter_class_init (gpointer klass)
{ {
GstElementClass *element_class = GST_ELEMENT_CLASS (klass); GstElementClass *element_class = GST_ELEMENT_CLASS (klass);
static GstElementDetails my_filter_details = { static GstElementDetails my_filter_details = {
@ -393,25 +395,32 @@ gst_my_filter_class_init (GstMyFilterClass * klass)
static void static void
gst_my_filter_init (GstMyFilter * filter) gst_my_filter_init (GstMyFilter * filter)
{ {
<!-- example-end task.c c --><!--
GstElementClass *klass = GST_ELEMENT_GET_CLASS (filter); GstElementClass *klass = GST_ELEMENT_GET_CLASS (filter);
filter-&gt;srcpad = gst_pad_new_from_template ( filter-&gt;srcpad = gst_pad_new_from_template (
gst_element_class_get_pad_template (klass, "src"), "src"); gst_element_class_get_pad_template (klass, "src"), "src");
-->
[..]<!-- example-begin task.c d --><!--
--><!-- example-end task.c d -->
<!-- example-begin task.c e -->
gst_pad_set_getrange_function (filter-&gt;srcpad, gst_pad_set_getrange_function (filter-&gt;srcpad,
gst_my_filter_get_range); gst_my_filter_get_range);
<!-- example-end range.c c --><!--
gst_element_add_pad (GST_ELEMENT (filter), filter-&gt;srcpad); gst_element_add_pad (GST_ELEMENT (filter), filter-&gt;srcpad);
<!-- example-end range.c c --> -->
[..]<!-- example-begin range.c d --> [..]<!-- example-begin range.c d -->
} }
static gboolean static GstFlowReturn
gst_my_filter_get_range (GstPad * pad, gst_my_filter_get_range (GstPad * pad,
GstObject * parent,
guint64 offset, guint64 offset,
guint length, guint length,
GstBuffer ** buf) GstBuffer ** buf)
{ {
<!-- example-end range.c d --> <!-- example-end range.c d -->
GstMyFilter *filter = GST_MY_FILTER (GST_OBJECT_PARENT (pad)); GstMyFilter *filter = GST_MY_FILTER (parent);
[.. here, you would fill *buf ..] [.. here, you would fill *buf ..]
<!-- example-begin range.c e --> <!-- example-begin range.c e -->
@ -423,15 +432,14 @@ gst_my_filter_get_range (GstPad * pad,
--><!-- example-end range.c f --></programlisting> --><!-- example-end range.c f --></programlisting>
<para> <para>
In practice, many elements that could theoretically do random access, In practice, many elements that could theoretically do random access,
may in practice often be assigned to do push-based scheduling anyway, may in practice often be activated in push-mode scheduling anyway,
since there is no downstream element able to start its own task. since there is no downstream element able to start its own task.
Therefore, in practice, those elements should implement both a Therefore, in practice, those elements should implement both a
<function>_get_range ()</function>-function and a <function>_chain <function>_get_range ()</function>-function and a <function>_chain
()</function>-function (for filters and parsers) or a <function>_get_range ()</function>-function (for filters and parsers) or a <function>_get_range
()</function>-function and be prepared to start their own task by ()</function>-function and be prepared to start their own task by
providing <function>_activate_* ()</function>-functions (for providing <function>_activate_* ()</function>-functions (for
source elements), so that &GStreamer; can decide for the optimal source elements).
scheduling mode and have it just work fine in practice.
</para> </para>
</sect1> </sect1>
</chapter> </chapter>