gstreamer/docs/pwg/advanced-events.xml

<chapter id="chapter-advanced-events">
  <title>Events: Seeking, Navigation and More</title>
  <para>
    There are many different event types but only two ways they can travel in
    the pipeline: downstream or upstream. It is very important to understand
    how both of these methods work because if one element in the pipeline is not
    handling them correctly the whole event system of the pipeline is broken.
    We will try to explain here how these methods work and how elements are 
    supposed to implement them.
  </para>
  <sect1 id="section-events-downstream" xreflabel="Downstream events">
    <title>Downstream events</title>
    <para>
      Downstream events are received through the sink pad's event handler,
      as set using <function>gst_pad_set_event_function ()</function> when
      the pad was created.
    </para>
    <para>
      Downstream events can travel in two ways: they can be in-band (serialised
      with the buffer flow) or out-of-band (travelling through the pipeline
      instantly, possibly not in the same thread as the streaming thread that
      is processing the buffers, skipping ahead of buffers being processed
      or queued in the pipeline). The most common downstream events
      (SEGMENT, CAPS, TAG, EOS) are all serialised with the buffer flow.
    </para>
    <para>
      Here is a typical event function:
    </para>
    <programlisting>
static gboolean
gst_my_filter_sink_event (GstPad  *pad, GstObject * parent, GstEvent * event)
{
  GstMyFilter *filter;
  gboolean ret;

  filter = GST_MY_FILTER (parent);
  ...

  switch (GST_EVENT_TYPE (event)) {
    case GST_EVENT_SEGMENT:
      /* maybe save and/or update the current segment (e.g. for output
       * clipping) or convert the event into one in a different format
       * (e.g. BYTES to TIME) or drop it and set a flag to send a segment
       * event in a different format later */
      ret = gst_pad_push_event (filter-&gt;src_pad, event);
      break;
    case GST_EVENT_EOS:
      /* end-of-stream, we should close down all stream leftovers here */
      gst_my_filter_stop_processing (filter);
      ret = gst_pad_push_event (filter-&gt;src_pad, event);
      break;
    case GST_EVENT_FLUSH_STOP:
      gst_my_filter_clear_temporary_buffers (filter);
      ret = gst_pad_push_event (filter-&gt;src_pad, event);
      break;      
    default:
      ret = gst_pad_event_default (pad, parent, event);
      break;
  }

  ...
  return ret;
}
    </programlisting>
  <para>
    If your element is chain-based, you will almost always have to implement
    a sink event function, since that is how you are notified about
    segments, caps and the end of the stream.
  </para>
  <para>
    If your element is exclusively loop-based, you may or may not want a
    sink event function (since the element is driving the pipeline it will
    know the length of the stream in advance or be notified by the flow
    return value of <function>gst_pad_pull_range()</function>. In some cases
    even loop-based element may receive events from upstream though (for
    example audio decoders with an id3demux or apedemux element in front of
    them, or demuxers that are being fed input from sources that send
    additional information about the stream in custom events, as DVD sources
    do).
  </para>
  </sect1>
  <sect1 id="section-events-upstream" xreflabel="Upstream events">
    <title>Upstream events</title>
    <para>
      Upstream events are generated by an element somewhere downstream in
      the pipeline (example: a video sink may generate navigation
      events that informs upstream elements about the current position of
      the mouse pointer). This may also happen indirectly on request of the
      application, for example when the application executes a seek on a
      pipeline this seek request will be passed on to a sink element which
      will then in turn generate an upstream seek event.
    </para>
    <para>
      The most common upstream events are seek events, Quality-of-Service
      (QoS) and reconfigure events.
    </para>
    <para>
      An upstream event can be sent using the
      <function>gst_pad_send_event</function> function. This
      function simply call the default event handler of that pad. The default
      event handler of pads is <function>gst_pad_event_default</function>, and
      it basically sends the event to the peer of the internally linked pad.
      So upstream events always arrive on the src pad of your element and are
      handled by the default event handler except if you override that handler
      to handle it yourself. There are some specific cases where you have to
      do that :
    </para>
    <itemizedlist mark="opencircle">
      <listitem>
        <para>
          If you have multiple sink pads in your element. In that case you will
          have to decide which one of the sink pads you will send the event to
          (if not all of them).
        </para>
      </listitem>
      <listitem>
        <para>
          If you need to handle that event locally. For example a navigation 
          event that you will want to convert before sending it upstream, or
          a QoS event that you want to handle.
        </para>
      </listitem>
    </itemizedlist>
    <para>
      The processing you will do in that event handler does not really matter
      but there are important rules you have to absolutely respect because
      one broken element event handler is breaking the whole pipeline event 
      handling. Here they are :
    </para>
    <itemizedlist mark="opencircle">
      <listitem>
        <para>
          Always handle events you won't handle using the default 
          <function>gst_pad_event_default</function> method. This method will
          depending on the event, forward the event or drop it. 
        </para>
      </listitem>
      <listitem>
        <para>
          If you are generating some new event based on the one you received
          don't forget to gst_event_unref the event you received.
        </para>
      </listitem>
      <listitem>
        <para>
          Event handler function are supposed to return TRUE or FALSE indicating
          if the event has been handled or not. Never simply return TRUE/FALSE
          in that handler except if you really know that you have handled that 
          event.
        </para>
      </listitem>
      <listitem>
        <para>
          Remember that the event handler might be called from a different
          thread than the streaming thread, so make sure you use
          appropriate locking everywhere.
        </para>
      </listitem>
    </itemizedlist>
  </sect1>

  <sect1 id="section-events-definitions" xreflabel="All Events Together">
    <title>All Events Together</title>
    <para>
      In this chapter follows a list of all defined events that are currently
      being used, plus how they should be used/interpreted. You can check the
      what type a certain event is using the GST_EVENT_TYPE macro (or if you
      need a string for debugging purposes you can use GST_EVENT_TYPE_NAME).
    </para>
    <para>
      In this chapter, we will discuss the following events:
    </para>
    <itemizedlist>
      <listitem><para><xref linkend="section-events-stream-start"/></para></listitem>
      <listitem><para><xref linkend="section-events-caps"/></para></listitem>
      <listitem><para><xref linkend="section-events-segment"/></para></listitem>
      <listitem><para><xref linkend="section-events-tag"/></para></listitem>
      <listitem><para><xref linkend="section-events-eos"/></para></listitem>
      <listitem><para><xref linkend="section-events-toc"/></para></listitem>
      <listitem><para><xref linkend="section-events-gap"/></para></listitem>
      <listitem><para><xref linkend="section-events-flush-start"/></para></listitem>
      <listitem><para><xref linkend="section-events-flush-stop"/></para></listitem>
      <listitem><para><xref linkend="section-events-qos"/></para></listitem>
      <listitem><para><xref linkend="section-events-seek"/></para></listitem>
      <listitem><para><xref linkend="section-events-nav"/></para></listitem>
    </itemizedlist>
    <para>
      For more comprehensive information about events and how they should be
      used correctly in various circumstances please consult the GStreamer
      design documentation. This section only gives a general overview.      
    </para>

    <sect2 id="section-events-stream-start" xreflabel="Stream Start">
      <title>Stream Start</title>
      <para>
        WRITEME
      </para>
    </sect2>

    <sect2 id="section-events-caps" xreflabel="Caps">
      <title>Caps</title>
      <para>
        The CAPS event contains the format description of the following
        buffers. See <xref linkend="chapter-negotiation"/> for more
        information about negotiation.
      </para>
    </sect2>

    <sect2 id="section-events-segment" xreflabel="Segment">
      <title>Segment</title>
      <para>
        A segment event is sent downstream to announce the range of valid
        timestamps in the stream and how they should be transformed into
        running-time and stream-time. A segment event must always be sent
        before the first buffer of data and after a flush (see above).
      </para>
      <para>
        The first segment event is created by the element driving the 
        pipeline, like a source operating in push-mode or a demuxer/decoder
        operating pull-based. This segment event then travels down the
        pipeline and may be transformed on the way (a decoder, for example,
        might receive a segment event in BYTES format and might transform
        this into a segment event in TIMES format based on the average
        bitrate).
      </para>
      <para>
        Depending on the element type, the event can simply be forwarded using
        <function>gst_pad_event_default ()</function>, or it should be parsed
        and a modified event should be sent on. The last is true for demuxers,
        which generally have a byte-to-time conversion concept. Their input
        is usually byte-based, so the incoming event will have an offset in
        byte units (<symbol>GST_FORMAT_BYTES</symbol>), too. Elements
        downstream, however, expect segment events in time units, so that
        it can be used to synchronize against the pipeline clock. Therefore,
        demuxers and similar elements should not forward the event, but parse
        it, free it and send a segment event (in time units,
        <symbol>GST_FORMAT_TIME</symbol>) further downstream.
      </para>
      <para>
        The segment event is created using the function
        <function>gst_event_new_segment ()</function>. See the API
        reference and design document for details about its parameters. 
      </para>
      <para>
        Elements parsing this event can use gst_event_parse_segment()
        to extract the event details. Elements may find the GstSegment
        API useful to keep track of the current segment (if they want to use
        it for output clipping, for example).
      </para>
    </sect2>

    <sect2 id="section-events-tag" xreflabel="Tag (metadata)">
      <title>Tag (metadata)</title>
      <para>
        Tagging events are being sent downstream to indicate the tags as parsed
        from the stream data. This is currently used to preserve tags during
        stream transcoding from one format to the other. Tags are discussed
        extensively in <xref linkend="chapter-advanced-tagging"/>. Most
        elements will simply forward the event by calling
        <function>gst_pad_event_default ()</function>.
      </para>
      <para>
        The tag event is created using the function
        <function>gst_event_new_tag ()</function>, but more often elements will
        send a tag event downstream that will be converted into a message
        on the bus by sink elements.
        All of these functions require a filled-in taglist as
        argument, which they will take ownership of.
      </para>
      <para>
        Elements parsing this event can use the function
        <function>gst_event_parse_tag ()</function> to acquire the
        taglist that the event contains.
      </para>
    </sect2>

    <sect2 id="section-events-eos" xreflabel="End of Stream (EOS)">
      <title>End of Stream (EOS)</title>
      <para>
        End-of-stream events are sent if the stream that an element sends out
        is finished. An element receiving this event (from upstream, so it
        receives it on its sinkpad) will generally just process any buffered
        data (if there is any) and then forward the event further downstream.
        The <function>gst_pad_event_default ()</function> takes care of all
        this, so most elements do not need to support this event. Exceptions are
        elements that explicitly need to close a resource down on EOS, and
        N-to-1 elements. Note that the stream itself is <emphasis>not</emphasis>
        a resource that should be closed down on EOS! Applications might seek
        back to a point before EOS and continue playing again.
      </para>
      <para>
        The EOS event has no properties, which makes it one of the simplest
        events in &GStreamer;. It is created using the 
        <function>gst_event_new_eos()</function> function.
      </para>
      <para>
        It is important to note that <emphasis>only elements driving the
        pipeline should ever send an EOS event</emphasis>. If your element
        is chain-based, it is not driving the pipeline. Chain-based elements
        should just return GST_FLOW_EOS from their chain function at
        the end of the stream (or the configured segment), the upstream
        element that is driving the pipeline will then take care of
        sending the EOS event (or alternatively post a SEGMENT_DONE message
        on the bus depending on the mode of operation). If you are implementing
        your own source element, you also do not need to ever manually send
        an EOS event, you should also just return GST_FLOW_EOS in
        your create or fill function (assuming your element derives from
        GstBaseSrc or GstPushSrc).
      </para>
    </sect2>

    <sect2 id="section-events-toc" xreflabel="Table Of Contents">
      <title>Table Of Contents</title>
      <para>
        WRITEME
      </para>
    </sect2>

    <sect2 id="section-events-gap" xreflabel="Gap">
      <title>Gap</title>
      <para>
        WRITEME
      </para>
    </sect2>

    <sect2 id="section-events-flush-start" xreflabel="Flush Start">
      <title>Flush Start</title>
      <para>
        The flush start event is sent downstream (in push mode) or upstream
        (in pull mode) if all buffers and caches in the pipeline should be
        emptied. <quote>Queue</quote> elements will
        empty their internal list of buffers when they receive this event, for
        example. File sink elements (e.g. <quote>filesink</quote>) will flush
        the kernel-to-disk cache (<function>fdatasync ()</function> or
        <function>fflush ()</function>) when they receive this event. Normally,
        elements receiving this event will simply just forward it, since most
        filter or filter-like elements don't have an internal cache of data.
        <function>gst_pad_event_default ()</function> does just that, so for
        most elements, it is enough to forward the event using the default
        event handler.
      </para>
      <para>
        As a side-effect of flushing all data from the pipeline, this event
        unblocks the streaming thread by making all pads reject data until
        they receive a <xref linkend="section-events-flush-stop"/> signal
        (elements trying to push data will get a FLUSHING flow return
         and stop processing data).
      </para>
      <para>
        The flush-start event is created with the
        <function>gst_event_new_flush_start ()</function>.
        Like the EOS event, it has no properties. This event is usually
        only created by elements driving the pipeline, like source elements
        operating in push-mode or pull-range based demuxers/decoders.
      </para>
    </sect2>

    <sect2 id="section-events-flush-stop" xreflabel="Flush Stop">
      <title>Flush Stop</title>
      <para>
        The flush-stop event is sent by an element driving the pipeline
        after a flush-start and tells pads and elements downstream that
        they should accept events and buffers again (there will be at
        least a SEGMENT event before any buffers first though).
      </para>
      <para>
        If your element keeps temporary caches of stream data, it should
        clear them when it receives a FLUSH-STOP event (and also whenever
        its chain function receives a buffer with the DISCONT flag set).
      </para>
      <para>
        The flush-stop event is created with
        <function>gst_event_new_flush_stop ()</function>. It has one
        parameter that controls if the running-time of the pipeline should
        be reset to 0 or not. Normally after a flushing seek, the
        running_time is set back to 0.
      </para>
    </sect2>

    <sect2 id="section-events-qos" xreflabel="Quality Of Service (QOS)">
      <title>Quality Of Service (QOS)</title>
      <para>
        The QOS event contains a report about the current real-time
        performance of the stream. See more info in 
        <xref linkend="chapter-advanced-qos"/>.
      </para>
    </sect2>

    <sect2 id="section-events-seek" xreflabel="Seek Request">
      <title>Seek Request</title>
      <para>
        Seek events are meant to request a new stream position to elements.
        This new position can be set in several formats (time, bytes or
        <quote>default units</quote> [a term indicating frames for video,
        channel-independent samples for audio, etc.]). Seeking can be done with
        respect to the end-of-file or start-of-file, and
        usually happens in upstream direction (downstream seeking is done by
        sending a SEGMENT event with the appropriate offsets for elements
        that support that, like filesink).
      </para>
      <para>
        Elements receiving seek events should, depending on the element type,
        either just forward it upstream (filters, decoders), change the
        format in which the event is given and then forward it (demuxers),
        or handle the event by changing the file pointer in their internal
        stream resource (file sources, demuxers/decoders driving the pipeline
        in pull-mode) or something else.
      </para>
      <para>
        Seek events are built up using positions in specified formats (time,
        bytes, units). They are created using the function
        <function>gst_event_new_seek ()</function>. Note that many plugins do
        not support seeking from the end of the stream.
        An element not driving the pipeline and forwarding a seek
        request should not assume that the seek succeeded or actually happened,
        it should operate based on the SEGMENT events it receives.
      </para>
      <para>
        Elements parsing this event can do this using
        <function>gst_event_parse_seek()</function>.
      </para>
    </sect2>

    <sect2 id="section-events-nav" xreflabel="Navigation">
      <title>Navigation</title>
      <para>
        Navigation events are sent upstream by video sinks to inform upstream
        elements of where the mouse pointer is, if and where mouse pointer
        clicks have happened, or if keys have been pressed or released.
      </para>
      <para>
        All this information is contained in the event structure which can
        be obtained with <function>gst_event_get_structure ()</function>.
      </para>
      <para>
        Check out the navigationtest element in gst-plugins-good for an idea
        how to extract navigation information from this event.
      </para>
    </sect2>

  </sect1>
</chapter>