mirror of
https://gitlab.freedesktop.org/gstreamer/gstreamer.git
synced 2024-09-08 21:28:45 +00:00
https://bugzilla.gnome.org/show_bug.cgi?id=747267 https://bugzilla.gnome.org/show_bug.cgi?id=747266
445 lines
17 KiB
XML
445 lines
17 KiB
XML
<chapter id="chapter-scheduling" xreflabel="Different scheduling modes">
|
|
<title>Different scheduling modes</title>
|
|
<para>
|
|
The scheduling mode of a pad defines how data is retrieved from (source)
|
|
or given to (sink) pads. &GStreamer; can operate in two scheduling
|
|
mode, called push- and pull-mode. &GStreamer; supports elements with pads
|
|
in any of the scheduling modes where not all pads need to be operating
|
|
in the same mode.
|
|
</para>
|
|
<para>
|
|
So far, we have only discussed <function>_chain ()</function>-operating
|
|
elements, i.e. elements that have a chain-function set on their sink pad
|
|
and push buffers on their source pad(s). We call this the push-mode
|
|
because a peer element will use <function>gst_pad_push ()</function> on
|
|
a srcpad, which will cause our <function>_chain ()</function>-function
|
|
to be called, which in turn causes our element to push out a buffer on
|
|
the source pad. The initiative to start the dataflow happens somewhere
|
|
upstream when it pushes out a buffer and all downstream elements get
|
|
scheduled when their <function>_chain ()</function>-functions are
|
|
called in turn.
|
|
</para>
|
|
<para>
|
|
Before we explain pull-mode scheduling, let's first understand how the
|
|
different scheduling modes are selected and activated on a pad.
|
|
</para>
|
|
|
|
<sect1 id="section-scheduling-activation"
|
|
xreflabel="The pad activation stage">
|
|
<title>The pad activation stage</title>
|
|
<para>
|
|
During the element state change of READY->PAUSED, the pads of an
|
|
element will be activated. This happens first on the source pads and
|
|
then on the sink pads of the element. &GStreamer; calls the
|
|
<function>_activate ()</function> of a pad. By default this function
|
|
will activate the pad in push-mode by calling
|
|
<function>gst_pad_activate_mode ()</function> with the GST_PAD_MODE_PUSH
|
|
scheduling mode.
|
|
It is possible to override the <function>_activate ()</function> of a pad
|
|
and decide on a different scheduling mode. You can know in what
|
|
scheduling mode a pad is activated by overriding the
|
|
<function>_activate_mode ()</function>-function.
|
|
</para>
|
|
<para>
|
|
&GStreamer; allows the different pads of an element to operate in
|
|
different scheduling modes. This allows for many different possible
|
|
use-cases. What follows is an overview of some typical use-cases.
|
|
</para>
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>
|
|
If all pads of an element are activated in push-mode scheduling,
|
|
the element as a whole is operating in push-mode.
|
|
For source elements this means that they will have to start a
|
|
task that pushes out buffers on the source pad to the downstream
|
|
elements.
|
|
Downstream elements will have data pushed to them by upstream elements
|
|
using the sinkpads <function>_chain ()</function>-function which will
|
|
push out buffers on the source pads.
|
|
Prerequisites for this scheduling mode are that a chain-function was
|
|
set for each sinkpad using <function>gst_pad_set_chain_function ()</function>
|
|
and that all downstream elements operate in the same mode.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
Alternatively, sinkpads can be the driving force behind a pipeline
|
|
by operating in pull-mode, while the sourcepads
|
|
of the element still operate in push-mode. In order to be the
|
|
driving force, those pads start a <classname>GstTask</classname>
|
|
when they are activated. This task is a thread, which
|
|
will call a function specified by the element. When called, this
|
|
function will have random data access (through
|
|
<function>gst_pad_pull_range ()</function>) over all sinkpads, and
|
|
can push data over the sourcepads, which effectively means that
|
|
this element controls data flow in the pipeline. Prerequisites for
|
|
this mode are that all downstream elements can act in push
|
|
mode, and that all upstream elements operate in pull-mode (see below).
|
|
</para>
|
|
<para>
|
|
Source pads can be activated in PULL mode by a downstream element
|
|
when they return GST_PAD_MODE_PULL from the GST_QUERY_SCHEDULING
|
|
query. Prerequisites for this scheduling mode are that a
|
|
getrange-function was set for the source pad using
|
|
<function>gst_pad_set_getrange_function ()</function>.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
Lastly, all pads in an element can be activated in PULL-mode.
|
|
However, contrary to the above, this does not mean that they
|
|
start a task on their own. Rather, it means that they are pull
|
|
slave for the downstream element, and have to provide random data
|
|
access to it from their <function>_get_range ()</function>-function.
|
|
Requirements are that the a <function>_get_range
|
|
()</function>-function was set on this pad using the function
|
|
<function>gst_pad_set_getrange_function ()</function>. Also, if
|
|
the element has any sinkpads, all those pads (and thereby their
|
|
peers) need to operate in PULL access mode, too.
|
|
</para>
|
|
<para>
|
|
When a sink element is activated in PULL mode, it should start a
|
|
task that calls <function>gst_pad_pull_range ()</function> on its
|
|
sinkpad. It can only do this when the upstream SCHEDULING query
|
|
returns support for the GST_PAD_MODE_PULL scheduling mode.
|
|
</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
<para>
|
|
In the next two sections, we will go closer into pull-mode scheduling
|
|
(elements/pads driving the pipeline, and elements/pads providing random
|
|
access), and some specific use cases will be given.
|
|
</para>
|
|
</sect1>
|
|
|
|
<sect1 id="section-scheduling-loop" xreflabel="Pads driving the pipeline">
|
|
<title>Pads driving the pipeline</title>
|
|
<para>
|
|
Sinkpads operating in pull-mode, with the sourcepads operating in
|
|
push-mode (or it has no sourcepads when it is a sink), can start a task
|
|
that will drive the pipeline data flow.
|
|
Within this task function, you have random access over all of the sinkpads,
|
|
and push data over the sourcepads.
|
|
This can come in useful for several different kinds of elements:
|
|
</para>
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>
|
|
Demuxers, parsers and certain kinds of decoders where data comes
|
|
in unparsed (such as MPEG-audio or video streams), since those will
|
|
prefer byte-exact (random) access from their input. If possible,
|
|
however, such elements should be prepared to operate in push-mode
|
|
mode, too.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
Certain kind of audio outputs, which require control over their
|
|
input data flow, such as the Jack sound server.
|
|
</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
<para>
|
|
First you need to perform a SCHEDULING query to check if the upstream
|
|
element(s) support pull-mode scheduling. If that is possible, you
|
|
can activate the sinkpad in pull-mode. Inside the activate_mode
|
|
function you can then start the task.
|
|
</para>
|
|
<programlisting><!-- example-begin task.c a -->
|
|
#include "filter.h"
|
|
#include <string.h>
|
|
|
|
static gboolean gst_my_filter_activate (GstPad * pad,
|
|
GstObject * parent);
|
|
static gboolean gst_my_filter_activate_mode (GstPad * pad,
|
|
GstObject * parent,
|
|
GstPadMode mode,
|
|
gboolean active);
|
|
static void gst_my_filter_loop (GstMyFilter * filter);
|
|
|
|
G_DEFINE_TYPE (GstMyFilter, gst_my_filter, GST_TYPE_ELEMENT);
|
|
<!-- example-end task.c a -->
|
|
<!-- example-begin task.c c -->
|
|
static void
|
|
gst_my_filter_init (GstMyFilter * filter)
|
|
{
|
|
<!-- example-end task.c c -->
|
|
[..]<!-- example-begin task.c d --><!--
|
|
--><!-- example-end task.c d -->
|
|
<!-- example-begin task.c e -->
|
|
gst_pad_set_activate_function (filter->sinkpad, gst_my_filter_activate);
|
|
gst_pad_set_activatemode_function (filter->sinkpad,
|
|
gst_my_filter_activate_mode);
|
|
<!-- example-end task.c e -->
|
|
<!-- example-begin task.c f --><!--
|
|
gst_element_add_pad (GST_ELEMENT (filter), filter->sinkpad);
|
|
|
|
gst_element_add_pad (GST_ELEMENT (filter), filter->srcpad);
|
|
--><!-- example-end task.c f -->
|
|
[..]<!-- example-begin task.c g -->
|
|
}
|
|
<!-- example-end task.c g -->
|
|
[..]<!-- example-begin task.c h --><!--
|
|
#include "caps.func"
|
|
--><!-- example-end task.c h -->
|
|
<!-- example-begin task.c i -->
|
|
static gboolean
|
|
gst_my_filter_activate (GstPad * pad, GstObject * parent)
|
|
{
|
|
GstQuery *query;
|
|
gboolean pull_mode;
|
|
|
|
/* first check what upstream scheduling is supported */
|
|
query = gst_query_new_scheduling ();
|
|
|
|
if (!gst_pad_peer_query (pad, query)) {
|
|
gst_query_unref (query);
|
|
goto activate_push;
|
|
}
|
|
|
|
/* see if pull-mode is supported */
|
|
pull_mode = gst_query_has_scheduling_mode_with_flags (query,
|
|
GST_PAD_MODE_PULL, GST_SCHEDULING_FLAG_SEEKABLE);
|
|
gst_query_unref (query);
|
|
|
|
if (!pull_mode)
|
|
goto activate_push;
|
|
|
|
/* now we can activate in pull-mode. GStreamer will also
|
|
* activate the upstream peer in pull-mode */
|
|
return gst_pad_activate_mode (pad, GST_PAD_MODE_PULL, TRUE);
|
|
|
|
activate_push:
|
|
{
|
|
/* something not right, we fallback to push-mode */
|
|
return gst_pad_activate_mode (pad, GST_PAD_MODE_PUSH, TRUE);
|
|
}
|
|
}
|
|
|
|
static gboolean
|
|
gst_my_filter_activate_pull (GstPad * pad,
|
|
GstObject * parent,
|
|
GstPadMode mode,
|
|
gboolean active)
|
|
{
|
|
gboolean res;
|
|
GstMyFilter *filter = GST_MY_FILTER (parent);
|
|
|
|
switch (mode) {
|
|
case GST_PAD_MODE_PUSH:
|
|
res = TRUE;
|
|
break;
|
|
case GST_PAD_MODE_PULL:
|
|
if (active) {
|
|
filter->offset = 0;
|
|
res = gst_pad_start_task (pad,
|
|
(GstTaskFunction) gst_my_filter_loop, filter, NULL);
|
|
} else {
|
|
res = gst_pad_stop_task (pad);
|
|
}
|
|
break;
|
|
default:
|
|
/* unknown scheduling mode */
|
|
res = FALSE;
|
|
break;
|
|
}
|
|
return res;
|
|
}
|
|
<!-- example-end task.c i --></programlisting>
|
|
<para>
|
|
Once started, your task has full control over input and output. The
|
|
most simple case of a task function is one that reads input and pushes
|
|
that over its source pad. It's not all that useful, but provides some
|
|
more flexibility than the old push-mode case that we've been looking
|
|
at so far.
|
|
</para>
|
|
<programlisting><!-- example-begin task.c j -->
|
|
#define BLOCKSIZE 2048
|
|
|
|
static void
|
|
gst_my_filter_loop (GstMyFilter * filter)
|
|
{
|
|
GstFlowReturn ret;
|
|
guint64 len;
|
|
GstFormat fmt = GST_FORMAT_BYTES;
|
|
GstBuffer *buf = NULL;
|
|
|
|
if (!gst_pad_query_duration (filter->sinkpad, fmt, &len)) {
|
|
GST_DEBUG_OBJECT (filter, "failed to query duration, pausing");
|
|
goto stop;
|
|
}
|
|
|
|
if (filter->offset >= len) {
|
|
GST_DEBUG_OBJECT (filter, "at end of input, sending EOS, pausing");
|
|
gst_pad_push_event (filter->srcpad, gst_event_new_eos ());
|
|
goto stop;
|
|
}
|
|
|
|
/* now, read BLOCKSIZE bytes from byte offset filter->offset */
|
|
ret = gst_pad_pull_range (filter->sinkpad, filter->offset,
|
|
BLOCKSIZE, &buf);
|
|
|
|
if (ret != GST_FLOW_OK) {
|
|
GST_DEBUG_OBJECT (filter, "pull_range failed: %s", gst_flow_get_name (ret));
|
|
goto stop;
|
|
}
|
|
|
|
/* now push buffer downstream */
|
|
ret = gst_pad_push (filter->srcpad, buf);
|
|
|
|
buf = NULL; /* gst_pad_push() took ownership of buffer */
|
|
|
|
if (ret != GST_FLOW_OK) {
|
|
GST_DEBUG_OBJECT (filter, "pad_push failed: %s", gst_flow_get_name (ret));
|
|
goto stop;
|
|
}
|
|
|
|
/* everything is fine, increase offset and wait for us to be called again */
|
|
filter->offset += BLOCKSIZE;
|
|
return;
|
|
|
|
stop:
|
|
GST_DEBUG_OBJECT (filter, "pausing task");
|
|
gst_pad_pause_task (filter->sinkpad);
|
|
}
|
|
<!-- example-end task.c j -->
|
|
<!-- example-begin task.c k --><!--
|
|
#include "register.func"
|
|
--><!-- example-end task.c k --></programlisting>
|
|
</sect1>
|
|
|
|
<sect1 id="section-scheduling-randomxs" xreflabel="Providing random access">
|
|
<title>Providing random access</title>
|
|
<para>
|
|
In the previous section, we have talked about how elements (or pads)
|
|
that are activated to drive the pipeline using their own task, must use
|
|
pull-mode scheduling on their sinkpads. This means that all pads linked
|
|
to those pads need to be activated in pull-mode.
|
|
Source pads activated in pull-mode must implement a
|
|
<function>_get_range ()</function>-function set using
|
|
<function>gst_pad_set_getrange_function ()</function>, and
|
|
that function will be called when the peer pad requests some data with
|
|
<function>gst_pad_pull_range ()</function>.
|
|
The element is then responsible for seeking to the right offset and
|
|
providing the requested data. Several elements can implement random
|
|
access:
|
|
</para>
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>
|
|
Data sources, such as a file source, that can provide data from any
|
|
offset with reasonable low latency.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
Filters that would like to provide a pull-mode scheduling
|
|
over the whole pipeline.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
Parsers who can easily provide this by skipping a small part of
|
|
their input and are thus essentially "forwarding" getrange
|
|
requests literally without any own processing involved. Examples
|
|
include tag readers (e.g. ID3) or single output parsers, such as
|
|
a WAVE parser.
|
|
</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
<para>
|
|
The following example will show how a <function>_get_range
|
|
()</function>-function can be implemented in a source element:
|
|
</para>
|
|
<programlisting><!-- example-begin range.c a -->
|
|
#include "filter.h"
|
|
static GstFlowReturn
|
|
gst_my_filter_get_range (GstPad * pad,
|
|
GstObject * parent,
|
|
guint64 offset,
|
|
guint length,
|
|
GstBuffer ** buf);
|
|
|
|
G_DEFINE_TYPE (GstMyFilter, gst_my_filter, GST_TYPE_ELEMENT);
|
|
<!-- example-end range.c a -->
|
|
<!-- example-begin range.c b --><!--
|
|
static void
|
|
gst_my_filter_class_init (gpointer klass)
|
|
{
|
|
GstElementClass *element_class = GST_ELEMENT_CLASS (klass);
|
|
static GstElementDetails my_filter_details = {
|
|
"An example plugin",
|
|
"Example/FirstExample",
|
|
"Shows the basic structure of a plugin",
|
|
"your name <your.name@your.isp>"
|
|
};
|
|
static GstStaticPadTemplate src_factory =
|
|
GST_STATIC_PAD_TEMPLATE (
|
|
"src",
|
|
GST_PAD_SRC,
|
|
GST_PAD_ALWAYS,
|
|
GST_STATIC_CAPS ("ANY")
|
|
);
|
|
|
|
gst_element_class_set_details (element_class, &my_filter_details);
|
|
gst_element_class_add_pad_template (element_class,
|
|
gst_static_pad_template_get (&src_factory));
|
|
}
|
|
|
|
static void
|
|
gst_my_filter_class_init (GstMyFilterClass * klass)
|
|
{
|
|
}
|
|
--><!-- example-begin range.c b -->
|
|
<!-- example-begin range.c c -->
|
|
static void
|
|
gst_my_filter_init (GstMyFilter * filter)
|
|
{
|
|
<!-- example-end task.c c --><!--
|
|
GstElementClass *klass = GST_ELEMENT_GET_CLASS (filter);
|
|
|
|
filter->srcpad = gst_pad_new_from_template (
|
|
gst_element_class_get_pad_template (klass, "src"), "src");
|
|
-->
|
|
[..]<!-- example-begin task.c d --><!--
|
|
--><!-- example-end task.c d -->
|
|
<!-- example-begin task.c e -->
|
|
gst_pad_set_getrange_function (filter->srcpad,
|
|
gst_my_filter_get_range);
|
|
<!-- example-end range.c c --><!--
|
|
gst_element_add_pad (GST_ELEMENT (filter), filter->srcpad);
|
|
-->
|
|
[..]<!-- example-begin range.c d -->
|
|
}
|
|
|
|
static GstFlowReturn
|
|
gst_my_filter_get_range (GstPad * pad,
|
|
GstObject * parent,
|
|
guint64 offset,
|
|
guint length,
|
|
GstBuffer ** buf)
|
|
{
|
|
<!-- example-end range.c d -->
|
|
GstMyFilter *filter = GST_MY_FILTER (parent);
|
|
|
|
[.. here, you would fill *buf ..]
|
|
<!-- example-begin range.c e -->
|
|
return GST_FLOW_OK;
|
|
}
|
|
<!-- example-end range.c e -->
|
|
<!-- example-begin range.c f --><!--
|
|
#include "register.func"
|
|
--><!-- example-end range.c f --></programlisting>
|
|
<para>
|
|
In practice, many elements that could theoretically do random access,
|
|
may in practice often be activated in push-mode scheduling anyway,
|
|
since there is no downstream element able to start its own task.
|
|
Therefore, in practice, those elements should implement both a
|
|
<function>_get_range ()</function>-function and a <function>_chain
|
|
()</function>-function (for filters and parsers) or a <function>_get_range
|
|
()</function>-function and be prepared to start their own task by
|
|
providing <function>_activate_* ()</function>-functions (for
|
|
source elements).
|
|
</para>
|
|
</sect1>
|
|
</chapter>
|