gstreamer/docs/pwg/advanced-dparams.xml

<!-- ############ chapter ############# -->

<chapter id="cha-dparams">
  <title>Supporting Dynamic Parameters</title>
  <para>
    Sometimes object properties are not powerful enough to control the
    parameters that affect the behaviour of your element. When this is the case
    you can expose these parameters as Dynamic Parameters which can be
    manipulated by any Dynamic Parameters aware application.
  </para>
  <para>
    Throughout this section, the term <emphasis>dparams</emphasis> will be used
    as an abbreviation for "Dynamic Parameters".
  </para>

  <sect1 id="sect-dparams-compare">
    <title>Comparing Dynamic Parameters with GObject Properties</title>
    <para>
      Your first exposure to dparams may be to convert an existing element from
      using object properties to using dparams. The following table gives an
      overview of the difference between these approaches. The significance of
      these differences should become apparent later on.
    </para>
    <informaltable frame="all">
      <tgroup cols="3">
      <thead>
        <row>
          <entry></entry>
          <entry>Object Properties</entry>
          <entry>Dynamic Parameters</entry>
        </row>
      </thead>
      <tbody>
      <row>
        <entry><emphasis>Parameter definition</emphasis></entry>
        <entry>Class level at compile time</entry>
        <entry>Any level at run time</entry>
      </row>
      <row>
        <entry><emphasis>Getting and setting</emphasis></entry>
        <entry>Implemented by element subclass as functions</entry>
        <entry>Handled entirely by dparams subsystem</entry>
      </row>
      <row>
        <entry><emphasis>Extra objects required</emphasis></entry>
        <entry>None - all functionality is derived from base GObject</entry>
        <entry>Element needs to create and store a <filename>GstDParamManager</filename> at object creation</entry>
      </row>
      <row>
        <entry><emphasis>Frequency and resolution of updates</emphasis></entry>
        <entry>Object properties will only be updated between calls to _get, _chain or _loop</entry>
        <entry>dparams can be updated at any rate independant of calls to _get, _chain or _loop up to sample-level accuracy</entry>
      </row>
      </tbody>
      </tgroup>
    </informaltable>
  </sect1>

<sect1 id="sect-dparam-start">
  <title>Getting Started</title>

  <para>
    The dparams subsystem is contained within the
    <filename>gstcontrol</filename> library. You need to include the header in
    your element's source file:
  </para>
  <programlisting>
  #include &lt;gst/control/control.h&gt;
  </programlisting>

  <para>
    Even though the <filename>gstcontrol</filename> library may be linked into
    the host application, you should make sure it is loaded in your
    <filename>plugin_init</filename> function:
  </para>
  <programlisting>
  static gboolean
  plugin_init (GModule *module, GstPlugin *plugin)
  {
    ...

    /* load dparam support library */
    if (!gst_library_load ("gstcontrol"))
    {
      gst_info ("example: could not load support library: 'gstcontrol'\n");
      return FALSE;
    }

    ...
  }
  </programlisting>

  <para>
    You need to store an instance of <filename>GstDParamManager</filename> in
    your element's struct:
  </para>
  <programlisting>
  struct _GstExample {
    GstElement element;
    ...

    GstDParamManager *dpman;

    ...
  };
  </programlisting>

  <para>
    The <filename>GstDParamManager</filename> can be initialised in your
    element's init function:
  </para>
  <programlisting>
  static void
  gst_example_init (GstExample *example)
  {
    ...

    example-&gt;dpman = gst_dpman_new ("example_dpman", GST_ELEMENT(example));

    ...
  }
  </programlisting>
</sect1>

<sect1 id="sect-dparam-define">
  <title>Defining Parameter Specificiations</title>
  <para>
    You can define the dparams you need anywhere within your element but will
    usually need to do so in only a couple of places:
    <itemizedlist>
      <listitem>
        <para>
        In the element <filename>init</filename> function, just after the call
        to <filename>gst_dpman_new</filename>
        </para>
      </listitem>
      <listitem>
        <para>
        Whenever a new pad is created so that parameters can affect data going
        into or out of a specific pad. An example of this would be a mixer
        element where a seperate volume parameter is needed on every pad.
        </para>
      </listitem>
    </itemizedlist>
  </para>
  <para>
    There are three different ways the dparams subsystem can pass parameters
    into your element. Which one you use will depend on how that parameter is
    used within your element. Each of these methods has its own function to
    define a required dparam:
    <itemizedlist>
<!-- FIXME: are we sure we need to use filename for function calls ??? -->
      <listitem><para><filename>gst_dpman_add_required_dparam_direct</filename></para></listitem>
      <listitem><para><filename>gst_dpman_add_required_dparam_callback</filename></para></listitem>
      <listitem><para><filename>gst_dpman_add_required_dparam_array</filename></para></listitem>
    </itemizedlist>
    These functions will return TRUE if the required dparam was added
    successfully.
    </para>
    <para>
    The following function will be used as an example.
    <programlisting>
  gboolean
  gst_dpman_add_required_dparam_direct (GstDParamManager *dpman,
                                        GParamSpec *param_spec,
                                        gboolean is_log,
                                        gboolean is_rate,
                                        gpointer update_data)
    </programlisting>
    The common parameters to these functions are:
    <itemizedlist>
      <listitem>
        <para>
        <filename>GstDParamManager *dpman</filename> the element's dparam
        manager
        </para>
      </listitem>
      <listitem>
        <para>
        <filename>GParamSpec *param_spec</filename> the param spec which defines
        the required dparam
        </para>
      </listitem>
      <listitem>
        <para>
        <filename>gboolean is_log</filename> whether this dparam value should be
        interpreted on a log scale (such as a frequency or a decibel value)
        </para>
      </listitem>
      <listitem>
        <para>
        <filename>gboolean is_rate</filename> whether this dparam value is a
        proportion of the sample rate. For example with a sample rate of 44100,
        0.5 would be 22050 Hz and 0.25 would be 11025 Hz.
        </para>
      </listitem>
    </itemizedlist>
  </para>
  <sect2 id="sect-dparam-direct">
    <title>Direct Method</title>
    <para>
      This method is the simplest and has the lowest overhead for parameters
      which change less frequently than the sample rate. First you need
      somewhere to store the parameter - this will usually be in your element's
      stuct.
    </para>
    <programlisting>
  struct _GstExample {
    GstElement element;
    ...

    GstDParamManager *dpman;
    gfloat volume;
    ...
  };
    </programlisting>
    <para>
      Then to define the required dparam just call
      <filename>gst_dpman_add_required_dparam_direct</filename> and pass in the
      location of the parameter to change. In this case the location is
      <filename>&amp;(example-&gt;volume)</filename>.
    </para>
    <programlisting>
  gst_dpman_add_required_dparam_direct (
    example-&gt;dpman,
    g_param_spec_float("volume","Volume","Volume of the audio",
                       0.0, 1.0, 0.8, G_PARAM_READWRITE),
    FALSE,
    FALSE,
    &amp;(example-&gt;volume)
  );
    </programlisting>
    <para>
      You can now use <filename>example-&gt;volume</filename> anywhere in your
      element knowing that it will always contain the correct value to use.
    </para>
  </sect2>
  <sect2 id="sect-dparam-callback">
    <title>Callback Method</title>
    <para>
      This should be used if the you have other values to calculate whenever a
      parameter changes. If you used the direct method you wouldn't know if a
      parameter had changed so you would have to recalculate the other values
      every time you needed them. By using the callback method, other values
      only have to be recalculated when the dparam value actually changes.
    </para>
    <para>
      The following code illustrates an instance where you might want to use the
      callback method. If you had a volume dparam which was represented by a
      gfloat number, your element may only deal with integer arithmatic. The
      callback could be used to calculate the integer scaler when the volume
      changes. First you will need somewhere to store these values.
    </para>
    <programlisting>
  struct _GstExample {
    GstElement element;
    ...

    GstDParamManager *dpman;
    gfloat volume_f;
    gint   volume_i;
    ...
  };
    </programlisting>
    <para>
      When the required dparam is defined, the callback function
      <filename>gst_example_update_volume</filename> and some user data (which
      in this case is our element instance) is passed in to the call to
      <filename>gst_dpman_add_required_dparam_callback</filename>.
    </para>
    <programlisting>
  gst_dpman_add_required_dparam_callback (
    example-&gt;dpman,
    g_param_spec_float("volume","Volume","Volume of the audio",
                       0.0, 1.0, 0.8, G_PARAM_READWRITE),
    FALSE,
    FALSE,
    gst_example_update_volume,
    example
  );
    </programlisting>
    <para>
      The callback function needs to conform to this signiture
    </para>
    <programlisting>
typedef void (*GstDPMUpdateFunction) (GValue *value, gpointer data);
    </programlisting>
    <para>
      In our example the callback function looks like this
    </para>
    <programlisting>
static void
gst_example_update_volume(GValue *value, gpointer data)
{
  GstExample *example = (GstExample*)data;
  g_return_if_fail(GST_IS_EXAMPLE(example));

  example-&gt;volume_f = g_value_get_float(value);
  example-&gt;volume_i = example-&gt;volume_f * 8192;
}
    </programlisting>
    <para>
      Now <filename>example-&gt;volume_i</filename> can be used elsewhere and it
      will always contain the correct value.
    </para>
  </sect2>
  <sect2 id="sect-dparam-array">
    <title>Array Method</title>
    <para>
      This method is quite different from the other two. It could be thought of
      as a specialised method which should only be used if you need the
      advantages that it provides. Instead of giving the element a single value
      it provides an array of values where each item in the array corresponds to
      a sample of audio in your buffer. There are a couple of reasons why this
      might be useful.
    </para>

    <itemizedlist>
      <listitem>
        <para>
        Certain optimisations may be possible since you can iterate over your
        dparams array and your buffer data together.
        </para>
      </listitem>
      <listitem>
        <para>
        Some dparams may be able to interpolate changing values at the sample
        rate. This would allow the array to contain very smoothly changing
        values which may be required for the stability and quality of some DSP
        algorithms.
        </para>
      </listitem>
    </itemizedlist>
    <para>
      The array method is currently the least mature of the three methods and is
      not yet ready to be used in elements, but plugin writers should be aware
      of its existance for the future.
    </para>
  </sect2>
</sect1>

<sect1 id="cha-dparam-loop">
  <title>The Data Processing Loop</title>
  <para>
    This is the most critical aspect of the dparams subsystem as it relates to
    elements. In a traditional audio processing loop, a <filename>for</filename>
    loop will usually iterate over each sample in the buffer, processing one
    sample at a time until the buffer is finished. A simplified loop with no
    error checking might look something like this.
  </para>
  <programlisting>
static void
example_chain (GstPad *pad, GstBuffer *buf)
{
  ...
  gfloat *float_data;
  int j;
  GstExample *example = GST_EXAMPLE(GST_OBJECT_PARENT (pad));
  int num_samples = GST_BUFFER_SIZE(buf)/sizeof(gfloat);
  float_data = (gfloat *)GST_BUFFER_DATA(buf);
  ...
  for (j = 0; j &lt; num_samples; j++) {
    float_data[j] *= example-&gt;volume;
  }
  ...
}
  </programlisting>
  <para>
    To make this dparams aware, a couple of changes are needed.
  </para>
  <programlisting>
static void
example_chain (GstPad *pad, GstBuffer *buf)
{
  ...
  int j = 0;
  GstExample *example = GST_EXAMPLE(GST_OBJECT_PARENT (pad));
  int num_samples = GST_BUFFER_SIZE(buf)/sizeof(gfloat);
  gfloat *float_data = (gfloat *)GST_BUFFER_DATA(buf);
  int frame_countdown = GST_DPMAN_PREPROCESS(example-&gt;dpman, num_samples, GST_BUFFER_TIMESTAMP(buf));
  ...
  while (GST_DPMAN_PROCESS_COUNTDOWN(example-&gt;dpman, frame_countdown, j)) {
    float_data[j++] *= example-&gt;volume;
  }
  ...
}
  </programlisting>
  <para>
    The biggest changes here are 2 new macros,
    <filename>GST_DPMAN_PREPROCESS</filename> and
    <filename>GST_DPMAN_PROCESS_COUNTDOWN</filename>. You will also notice that
    the for loop has become a while loop.
    <filename>GST_DPMAN_PROCESS_COUNTDOWN</filename> is called as the condition
    for the while loop so that any required dparams can be updated in the middle
    of a buffer if required. This is because one of the required behaviours of
    dparams is that they can be <emphasis>sample accurate</emphasis>. This means
    that parameters change at the exact timestamp that they are supposed to -
    not after the buffer has finished being processed.
  </para>
  <para>
    It may be alarming to see a macro as the condition for a while loop, but it
    is actually very efficient. The macro expands to the following.
  </para>
  <programlisting>
#define GST_DPMAN_PROCESS_COUNTDOWN(dpman, frame_countdown, frame_count) \
				(frame_countdown-- || \
				(frame_countdown = GST_DPMAN_PROCESS(dpman, frame_count)))
  </programlisting>
  <para>
    So as long as <filename>frame_countdown</filename> is greater than 0,
    <filename>GST_DPMAN_PROCESS</filename> will not be called at all. Also in
    many cases, <filename>GST_DPMAN_PROCESS</filename> will do nothing and
    simply return 0, meaning that there is no more data in the buffer to
    process.
  </para>
  <para>
    The macro <filename>GST_DPMAN_PREPROCESS</filename> will do the following:
    <itemizedlist>
      <listitem>
        <para>
        Update any dparams which are due to be updated.
        </para>
      </listitem>
      <listitem>
        <para>
        Calculate how many samples should be processed before the next required
        update
        </para>
      </listitem>
      <listitem>
        <para>
        Return the number of samples until next update, or the number of samples
        in the buffer - whichever is less.
        </para>
      </listitem>
    </itemizedlist>
    In fact <filename>GST_DPMAN_PROCESS</filename> may do the same things as
    <filename>GST_DPMAN_PREPROCESS</filename> depending on the mode that the
    dparam manager is running in (see below).
  </para>
  <sect2 id="sect-dparam-modes">
    <title>DParam Manager Modes</title>
    <para>
      A brief explanation of dparam manager modes might be useful here even
      though it doesn't generally affect the way your element is written. There
      are different ways media applications will be used which require that an
      element's parameters be updated in differently. These include:
      <itemizedlist>
        <listitem>
          <para>
          <emphasis>Timelined</emphasis> - all parameter changes are known in
          advance before the pipeline is run.
          </para>
        </listitem>
        <listitem>
          <para>
          <emphasis>Realtime low-latency</emphasis> - Nothing is known ahead of
          time about when a parameter might change. Changes need to be
          propagated to the element as soon as possible.
          </para>
        </listitem>
      </itemizedlist>
      When a dparam-aware application gets the dparam manager for an element,
      the first thing it will do is set the dparam manager mode. Current modes
      are <filename>"synchronous"</filename> and
      <filename>"asynchronous"</filename>.
    </para>
    <para>
      If you are in a realtime low-latency situation then the
      <filename>"synchronous"</filename> mode is appropriate. During
      <filename>GST_DPMAN_PREPROCESS</filename> this mode will poll all dparams
      for required updates and propagate them.
      <filename>GST_DPMAN_PROCESS</filename> will do nothing in this mode. To
      then achieve the desired latency, the size of the buffers needs to be
      reduced so that the dparams will be polled for updates at the desired
      frequency.
    </para>
    <para>
      In a timelined situation, the <filename>"asynchronous"</filename> mode
      will be required. This mode hasn't actually been implemented yet but will
      be described anyway. The <filename>GST_DPMAN_PREPROCESS</filename> call
      will precalculate when and how often each dparam needs to update for the
      duration of the current buffer. From then on
      <filename>GST_DPMAN_PROCESS</filename> will propagate the calculated
      updates each time it is called until end of the buffer. If the application
      is rendering to disk in non-realtime, the render could be sped up by
      increasing the buffer size. In the <filename>"asynchronous"</filename>
      mode this could be done without affecting the sample accuracy of the
      parameter updates
    </para>
  </sect2>
  <sect2 id="sect-dparam-audio-video">
    <title>DParam Manager Modes</title>
    <para>
      All of the explanation so far has presumed that the buffer contains audio
      data with many samples. Video should be regarded differently since a video
      buffer often contains only 1 frame. In this case some of the complexity of
      dparams isn't required but the other benefits still make it useful for
      video parameters. If a buffer only contains one frame of video, only a
      single call to <filename>GST_DPMAN_PREPROCESS</filename> should be
      required. For more than one frame per buffer, treat it the same as the
      audio case.
    </para>
  </sect2>
</sect1>
</chapter>