mirrors/gstreamer
mirrors/gstreamer
1
1
Fork 0
mirror of https://gitlab.freedesktop.org/gstreamer/gstreamer.git synced 2025-01-24 08:08:22 +00:00
Code Issues 1 Projects Releases Wiki Activity

pwg: rework scheduling docs

Browse source
This commit is contained in:
Wim Taymans 2012-09-28 13:25:30 +02:00
parent f32c9d8572
commit 723c2a48b9
1 changed files with 167 additions and 159 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

312
docs/pwg/advanced-scheduling.xml
View file

@ -1,85 +1,93 @@
<chapter id="chapter-scheduling" xreflabel="Different scheduling modes">
<title>Different scheduling modes</title>
<para>
Scheduling is, in short, a method for making sure that every element gets
called once in a while to process data and prepare data for the next
element. Likewise, a kernel has a scheduler for processes, and your
brain is a very complex scheduler too in a way.
Randomly calling elements' chain functions won't bring us far, however, so
you'll understand that the schedulers in &GStreamer; are a bit more complex
than this. However, as a start, it's a nice picture.
The scheduling mode of a pad defines how data is retrieved from (source)
or given to (sink) pads. &GStreamer; can operate in two scheduling
mode, called push- and pull-mode. &GStreamer; supports elements with pads
in any of the scheduling modes where not all pads need to be operating
in the same mode.
</para>
<para>
So far, we have only discussed <function>_chain ()</function>-operating
elements, i.e. elements that have a chain-function set on their sink pad
and push buffers on their source pad(s). Pads (or elements) can also operate
in two other scheduling modes, however. In this chapter, we will discuss
what those scheduling modes are, how they can be enabled and in what
cases they are useful. The other two scheduling modes are random access
(<function>_getrange ()</function>-based) or task-runner (which means
that this element is the driving force in the pipeline) mode.
and push buffers on their source pad(s). We call this the push-mode
because a peer element will use <function>gst_pad_push ()</function> on
a srcpad, which will cause our <function>_chain ()</function>-function
to be called, which in turn causes our element to push out a buffer on
the source pad. The initiative to start the dataflow happens somewhere
upstream when it pushes out a buffer and all downstream elements get
scheduled when their <function>_chain ()</function>-functions are
called in turn.
</para>
<para>
Before we explain pull-mode scheduling, let's first understand how the
different scheduling modes are selected and activated on a pad.
</para>
<sect1 id="section-scheduling-activation"
xreflabel="The pad actication stage">
xreflabel="The pad activation stage">
<title>The pad activation stage</title>
<para>
The stage in which &GStreamer; decides in what scheduling mode the
various elements will operate, is called the pad-activation stage. In
this stage, &GStreamer; will query the scheduling capabilities (i.e.
it will see in what modes each particular element/pad can operate) and
decide on the optimal scheduling composition for the pipeline. Next,
each pad will be notified of the scheduling mode that was assigned to
it, and after that the pipeline will start running.
During the element state change of READY->PAUSED, the pads of an
element will be activated. This happens first on the source pads and
then on the sink pads of the element. &GStreamer; calls the
<function>_activate ()</function> of a pad. By default this function
will activate the pad in push-mode by calling
<function>gst_pad_activate_mode ()</function> with the GST_PAD_MODE_PUSH
scheduling mode.
It is possible to override the <function>_activate ()</function> of a pad
and decide on a different scheduling mode. You can know in what
scheduling mode a pad is activated by overriding the
<function>_activate_mode ()</function>-function.
</para>
<para>
Pads can be assigned one of three modes, each mode putting several
prerequisites on the pads. Pads should implement a notification
function (<function>gst_pad_set_activatepull_function ()</function> and
<function>gst_pad_set_activatepush_function ()</function>) to be
notified of the scheduling mode assignment. Also, sinkpads assigned
to do pull-based scheduling mode should start and stop their task
in this function.
&GStreamer; allows the different pads of an element to operate in
different scheduling modes. This allows for many different possible
use-cases. What follows is an overview of some typical use-cases.
</para>
<itemizedlist>
<listitem>
<para>
If all pads of an element are assigned to do
<quote>push</quote>-based scheduling, then this means that data
will be pushed by upstream elements to this element using the
sinkpads <function>_chain ()</function>-function. Prerequisites
for this scheduling mode are that a chain-function was set for
each sinkpad using<function>gst_pad_set_chain_function ()</function>
and that all downstream elements operate in the same mode. Pads are
assigned to do push-based scheduling in sink-to-source element
order, and within an element first sourcepads and then sinkpads.
Sink elements can operate in this mode if their sinkpad is activated
for push-based scheduling. Source elements cannot be chain-based.
If all pads of an element are activated in push-mode scheduling,
the element as a whole is operating in push-mode.
For source elements this means that they will have to start a
task that pushes out buffers on the source pad to the downstream
elements.
Downstream elements will have data pushed to them by upstream elements
using the sinkpads <function>_chain ()</function>-function which will
push out buffers on the source pads.
Prerequisites for this scheduling mode are that a chain-function was
set for each sinkpad using <function>gst_pad_set_chain_function ()</function>
and that all downstream elements operate in the same mode.
</para>
</listitem>
<listitem>
<para>
Alternatively, sinkpads can be the driving force behind a pipeline
by operating in <quote>pull</quote>-based mode, while the sourcepads
of the element still operate in push-based mode. In order to be the
by operating in pull-mode, while the sourcepads
of the element still operate in push-mode. In order to be the
driving force, those pads start a <classname>GstTask</classname>
when they are activated. This task is a thread, which
will call a function specified by the element. When called, this
function will have random data access (through
<function>gst_pad_get_range ()</function>) over all sinkpads, and
<function>gst_pad_pull_range ()</function>) over all sinkpads, and
can push data over the sourcepads, which effectively means that
this element controls data flow in the pipeline. Prerequisites for
this mode are that all downstream elements can act in chain-based
mode, and that all upstream elements allow random access (see below).
Source elements can be told to act in this mode if their sourcepads
are activated in push-based fashion. Sink elements can be told to
act in this mode when their sinkpads are activated in pull-mode.
this mode are that all downstream elements can act in push
mode, and that all upstream elements operate in pull-mode (see below).
</para>
<para>
Source pads can be activated in PULL mode by a downstream element
when they return GST_PAD_MODE_PULL from the GST_QUERY_SCHEDULING
query. Prerequisites for this scheduling mode are that a
getrange-function was set for the source pad using
<function>gst_pad_set_getrange_function ()</function>.
</para>
</listitem>
<listitem>
<para>
lastly, all pads in an element can be assigned to act in pull-mode.
too. However, contrary to the above, this does not mean that they
Lastly, all pads in an element can be activated in PULL-mode.
However, contrary to the above, this does not mean that they
start a task on their own. Rather, it means that they are pull
slave for the downstream element, and have to provide random data
access to it from their <function>_get_range ()</function>-function.
@ -87,14 +95,18 @@
()</function>-function was set on this pad using the function
<function>gst_pad_set_getrange_function ()</function>. Also, if
the element has any sinkpads, all those pads (and thereby their
peers) need to operate in random access mode, too. Note that the
element is supposed to activate those elements itself! &GStreamer;
will not do that for you.
peers) need to operate in PULL access mode, too.
</para>
<para>
When a sink element is activated in PULL mode, it should start a
task that calls <function>gst_pad_pull_range ()</function> on its
sinkpad. It can only do this when the upstream SCHEDULING query
returns support for the GST_PAD_MODE_PULL scheduling mode.
</para>
</listitem>
</itemizedlist>
<para>
In the next two sections, we will go closer into pull-based scheduling
In the next two sections, we will go closer into pull-mode scheduling
(elements/pads driving the pipeline, and elements/pads providing random
access), and some specific use cases will be given.
</para>
@ -103,12 +115,12 @@
<sect1 id="section-scheduling-loop" xreflabel="Pads driving the pipeline">
<title>Pads driving the pipeline</title>
<para>
Sinkpads assigned to operate in pull-based mode, while none of its
sourcepads operate in pull-based mode (or it has no sourcepads), can
start a task that will drive the pipeline data flow. Within this
function, those elements have random access over all of their sinkpads,
and push data over their sourcepads. This can come in useful for
several different kinds of elements:
Sinkpads operating in pull-mode, with the sourcepads operating in
push-mode (or it has no sourcepads when it is a sink), can start a task
that will drive the pipeline data flow.
Within this task function, you have random access over all of the sinkpads,
and push data over the sourcepads.
This can come in useful for several different kinds of elements:
</para>
<itemizedlist>
<listitem>
@ -116,7 +128,7 @@
Demuxers, parsers and certain kinds of decoders where data comes
in unparsed (such as MPEG-audio or video streams), since those will
prefer byte-exact (random) access from their input. If possible,
however, such elements should be prepared to operate in chain-based
however, such elements should be prepared to operate in push-mode
mode, too.
</para>
</listitem>
@ -128,85 +140,40 @@
</listitem>
</itemizedlist>
<para>
In order to start this task, you will need to create it in the
activation function.
First you need to perform a SCHEDULING query to check if the upstream
element(s) support pull-mode scheduling. If that is possible, you
can activate the sinkpad in pull-mode. Inside the activate_mode
function you can then start the task.
</para>
<programlisting><!-- example-begin task.c a -->
#include "filter.h"
#include &lt;string.h&gt;
static gboolean gst_my_filter_activate (GstPad * pad);
static gboolean gst_my_filter_activate_pull (GstPad * pad,
static gboolean gst_my_filter_activate (GstPad * pad,
GstObject * parent);
static gboolean gst_my_filter_activate_mode (GstPad * pad,
GstObject * parent,
GstPadMode mode
gboolean active);
static void gst_my_filter_loop (GstMyFilter * filter);
GST_BOILERPLATE (GstMyFilter, gst_my_filter, GstElement, GST_TYPE_ELEMENT);
G_DEFINE_TYPE (GstMyFilter, gst_my_filter, GST_TYPE_ELEMENT);
<!-- example-end task.c a -->
<!-- example-begin task.c b --><!--
static gboolean gst_my_filter_setcaps (GstPad *pad,
GstCaps *caps);
static GstCaps *gst_my_filter_getcaps (GstPad *pad);
static void
gst_my_filter_base_init (gpointer klass)
{
GstElementClass *element_class = GST_ELEMENT_CLASS (klass);
static GstElementDetails my_filter_details = {
"An example plugin",
"Example/FirstExample",
"Shows the basic structure of a plugin",
"your name <your.name@your.isp>"
};
static GstStaticPadTemplate sink_factory =
GST_STATIC_PAD_TEMPLATE (
"sink",
GST_PAD_SINK,
GST_PAD_ALWAYS,
GST_STATIC_CAPS ("ANY")
);
static GstStaticPadTemplate src_factory =
GST_STATIC_PAD_TEMPLATE (
"src",
GST_PAD_SRC,
GST_PAD_ALWAYS,
GST_STATIC_CAPS ("ANY")
);
gst_element_class_set_details (element_class, &my_filter_details);
gst_element_class_add_pad_template (element_class,
gst_static_pad_template_get (&src_factory));
gst_element_class_add_pad_template (element_class,
gst_static_pad_template_get (&sink_factory));
}
static void
gst_my_filter_class_init (GstMyFilterClass * klass)
{
}
--><!-- example-begin task.c b -->
<!-- example-begin task.c c -->
static void
gst_my_filter_init (GstMyFilter * filter)
{
<!-- example-end task.c c -->
[..]<!-- example-begin task.c d --><!--
GstElementClass *klass = GST_ELEMENT_GET_CLASS (filter);
filter-&gt;sinkpad = gst_pad_new_from_template (
gst_element_class_get_pad_template (klass, "sink"), "sink");
gst_pad_set_setcaps_function (filter-&gt;sinkpad, gst_my_filter_setcaps);
gst_pad_set_getcaps_function (filter-&gt;sinkpad, gst_my_filter_getcaps);
--><!-- example-end task.c d -->
<!-- example-begin task.c e -->
gst_pad_set_activate_function (filter-&gt;sinkpad, gst_my_filter_activate);
gst_pad_set_activatepull_function (filter-&gt;sinkpad,
gst_my_filter_activate_pull);
gst_pad_set_activatemode_function (filter-&gt;sinkpad,
gst_my_filter_activate_mode);
<!-- example-end task.c e -->
<!-- example-begin task.c f --><!--
gst_element_add_pad (GST_ELEMENT (filter), filter-&gt;sinkpad);
filter-&gt;srcpad = gst_pad_new_from_template (
gst_element_class_get_pad_template (klass, "src"), "src");
gst_element_add_pad (GST_ELEMENT (filter), filter-&gt;srcpad);
--><!-- example-end task.c f -->
[..]<!-- example-begin task.c g -->
@ -217,35 +184,73 @@ gst_my_filter_init (GstMyFilter * filter)
--><!-- example-end task.c h -->
<!-- example-begin task.c i -->
static gboolean
gst_my_filter_activate (GstPad * pad)
gst_my_filter_activate (GstPad * pad, GstObject * parent)
{
if (gst_pad_check_pull_range (pad)) {
return gst_pad_activate_pull (pad, TRUE);
} else {
return FALSE;
GstQuery *query;
gboolean pull_mode;
/* first check what upstream scheduling is supported */
query = gst_query_new_scheduling ();
if (!gst_pad_peer_query (pad, query)) {
gst_query_unref (query);
goto activate_push;
}
/* see if pull-mode is supported */
pull_mode = gst_query_has_scheduling_mode_with_flags (query,
GST_PAD_MODE_PULL, GST_SCHEDULING_FLAG_SEEKABLE);
gst_query_unref (query);
if (!pull_mode)
goto activate_push;
/* now we can activate in pull-mode. GStreamer will also
* activate the upstream peer in pull-mode */
return gst_pad_activate_mode (pad, GST_PAD_MODE_PULL, TRUE);
activate_push:
{
/* something not right, we fallback to push-mode */
return gst_pad_activate_mode (pad, GST_PAD_MODE_PUSH, TRUE);
}
}
static gboolean
gst_my_filter_activate_pull (GstPad * pad,
GstObject * parent,
GstPadMode mode,
gboolean active)
{
GstMyFilter *filter = GST_MY_FILTER (GST_OBJECT_PARENT (pad));
gboolean res;
GstMyFilter *filter = GST_MY_FILTER (parent);
switch (mode) {
case GST_PAD_MODE_PUSH:
res = TRUE;
break;
case GST_PAD_MODE_PULL:
if (active) {
filter->offset = 0;
return gst_pad_start_task (pad,
(GstTaskFunction) gst_my_filter_loop, filter);
res = gst_pad_start_task (pad,
(GstTaskFunction) gst_my_filter_loop, filter, NULL);
} else {
return gst_pad_stop_task (pad);
res = gst_pad_stop_task (pad);
}
break;
default:
/* unknown scheduling mode */
res = FALSE;
break;
}
return res;
}
<!-- example-end task.c i --></programlisting>
<para>
Once started, your task has full control over input and output. The
most simple case of a task function is one that reads input and pushes
that over its source pad. It's not all that useful, but provides some
more flexibility than the old chain-based case that we've been looking
more flexibility than the old push-mode case that we've been looking
at so far.
</para>
<programlisting><!-- example-begin task.c j -->
@ -259,7 +264,7 @@ gst_my_filter_loop (GstMyFilter * filter)
GstFormat fmt = GST_FORMAT_BYTES;
GstBuffer *buf = NULL;
if (!gst_pad_query_duration (filter-&gt;sinkpad, &amp;fmt, &amp;len)) {
if (!gst_pad_query_duration (filter-&gt;sinkpad, fmt, &amp;len)) {
GST_DEBUG_OBJECT (filter, "failed to query duration, pausing");
goto stop;
}
@ -307,16 +312,15 @@ stop:
<title>Providing random access</title>
<para>
In the previous section, we have talked about how elements (or pads)
that are assigned to drive the pipeline using their own task, have
random access over their sinkpads. This means that all elements linked
to those pads (recursively) need to provide random access functions.
Requesting random access is done using the function
<function>gst_pad_pull_range ()</function>, which requests a buffer of
a specified size and offset. Source pads implementing and assigned to
do random access will have a <function>_get_range ()</function>-function
set using <function>gst_pad_set_getrange_function ()</function>, and
that function will be called when the peer pad requests some data. The
element is then responsible for seeking to the right offset and
that are activated to drive the pipeline using their own task, must use
pull-mode scheduling on their sinkpads. This means that all pads linked
to those pads need to be activated in pull-mode.
Source pads activated in pull-mode must implement a
<function>_get_range ()</function>-function set using
<function>gst_pad_set_getrange_function ()</function>, and
that function will be called when the peer pad requests some data with
<function>gst_pad_pull_range ()</function>.
The element is then responsible for seeking to the right offset and
providing the requested data. Several elements can implement random
access:
</para>
@ -329,17 +333,14 @@ stop:
</listitem>
<listitem>
<para>
Filters that would like to provide a pull-based-like scheduling
mode over the whole pipeline. Note that elements assigned to do
random access-based scheduling are themselves responsible for
assigning this scheduling mode to their upstream peers! &GStreamer;
will not do that for you.
Filters that would like to provide a pull-mode scheduling
over the whole pipeline.
</para>
</listitem>
<listitem>
<para>
Parsers who can easily provide this by skipping a small part of
their input and are thus essentially "forwarding" random access
their input and are thus essentially "forwarding" getrange
requests literally without any own processing involved. Examples
include tag readers (e.g. ID3) or single output parsers, such as
a WAVE parser.
@ -354,15 +355,16 @@ stop:
#include "filter.h"
static GstFlowReturn
gst_my_filter_get_range (GstPad * pad,
GstObject * parent,
guint64 offset,
guint length,
GstBuffer ** buf);
GST_BOILERPLATE (GstMyFilter, gst_my_filter, GstElement, GST_TYPE_ELEMENT);
G_DEFINE_TYPE (GstMyFilter, gst_my_filter, GST_TYPE_ELEMENT);
<!-- example-end range.c a -->
<!-- example-begin range.c b --><!--
static void
gst_my_filter_base_init (gpointer klass)
gst_my_filter_class_init (gpointer klass)
{
GstElementClass *element_class = GST_ELEMENT_CLASS (klass);
static GstElementDetails my_filter_details = {
@ -393,25 +395,32 @@ gst_my_filter_class_init (GstMyFilterClass * klass)
static void
gst_my_filter_init (GstMyFilter * filter)
{
<!-- example-end task.c c --><!--
GstElementClass *klass = GST_ELEMENT_GET_CLASS (filter);
filter-&gt;srcpad = gst_pad_new_from_template (
gst_element_class_get_pad_template (klass, "src"), "src");
-->
[..]<!-- example-begin task.c d --><!--
--><!-- example-end task.c d -->
<!-- example-begin task.c e -->
gst_pad_set_getrange_function (filter-&gt;srcpad,
gst_my_filter_get_range);
<!-- example-end range.c c --><!--
gst_element_add_pad (GST_ELEMENT (filter), filter-&gt;srcpad);
<!-- example-end range.c c -->
-->
[..]<!-- example-begin range.c d -->
}
static gboolean
static GstFlowReturn
gst_my_filter_get_range (GstPad * pad,
GstObject * parent,
guint64 offset,
guint length,
GstBuffer ** buf)
{
<!-- example-end range.c d -->
GstMyFilter *filter = GST_MY_FILTER (GST_OBJECT_PARENT (pad));
GstMyFilter *filter = GST_MY_FILTER (parent);
[.. here, you would fill *buf ..]
<!-- example-begin range.c e -->
@ -423,15 +432,14 @@ gst_my_filter_get_range (GstPad * pad,
--><!-- example-end range.c f --></programlisting>
<para>
In practice, many elements that could theoretically do random access,
may in practice often be assigned to do push-based scheduling anyway,
may in practice often be activated in push-mode scheduling anyway,
since there is no downstream element able to start its own task.
Therefore, in practice, those elements should implement both a
<function>_get_range ()</function>-function and a <function>_chain
()</function>-function (for filters and parsers) or a <function>_get_range
()</function>-function and be prepared to start their own task by
providing <function>_activate_* ()</function>-functions (for
source elements), so that &GStreamer; can decide for the optimal
scheduling mode and have it just work fine in practice.
source elements).
</para>
</sect1>
</chapter>