mirror of
https://gitlab.freedesktop.org/gstreamer/gstreamer.git
synced 2024-12-12 03:16:33 +00:00
404 lines
19 KiB
XML
404 lines
19 KiB
XML
<chapter id="chapter-advanced-events">
|
|
<title>Events: Seeking, Navigation and More</title>
|
|
<para>
|
|
There are many different event types but only two ways they can travel in
|
|
the pipeline: downstream or upstream. It is very important to understand
|
|
how both of these methods work because if one element in the pipeline is not
|
|
handling them correctly the whole event system of the pipeline is broken.
|
|
We will try to explain here how these methods work and how elements are
|
|
supposed to implement them.
|
|
</para>
|
|
<sect1 id="section-events-downstream" xreflabel="Downstream events">
|
|
<title>Downstream events</title>
|
|
<para>
|
|
Downstream events are received through the sink pad's event handler,
|
|
as set using <function>gst_pad_set_event_function ()</function> when
|
|
the pad was created.
|
|
</para>
|
|
<para>
|
|
Downstream events can travel in two ways: they can be in-band (serialised
|
|
with the buffer flow) or out-of-band (travelling through the pipeline
|
|
instantly, possibly not in the same thread as the streaming thread that
|
|
is processing the buffers, skipping ahead of buffers being processed
|
|
or queued in the pipeline). The most common downstream events
|
|
(NEWSEGMENT, EOS, TAG) are all serialised with the buffer flow.
|
|
</para>
|
|
<para>
|
|
Here is a typical event function:
|
|
</para>
|
|
<programlisting>
|
|
static gboolean
|
|
gst_my_filter_sink_event (GstPad *pad, GstEvent * event)
|
|
{
|
|
GstMyFilter *filter;
|
|
gboolean ret;
|
|
|
|
filter = GST_MY_FILTER (gst_pad_get_parent (pad));
|
|
...
|
|
|
|
switch (GST_EVENT_TYPE (event)) {
|
|
case GST_EVENT_NEWSEGMENT:
|
|
/* maybe save and/or update the current segment (e.g. for output
|
|
* clipping) or convert the event into one in a different format
|
|
* (e.g. BYTES to TIME) or drop it and set a flag to send a newsegment
|
|
* event in a different format later */
|
|
ret = gst_pad_push_event (filter->src_pad, event);
|
|
break;
|
|
case GST_EVENT_EOS:
|
|
/* end-of-stream, we should close down all stream leftovers here */
|
|
gst_my_filter_stop_processing (filter);
|
|
ret = gst_pad_push_event (filter->src_pad, event);
|
|
break;
|
|
case GST_EVENT_FLUSH_STOP:
|
|
gst_my_filter_clear_temporary_buffers (filter);
|
|
ret = gst_pad_push_event (filter->src_pad, event);
|
|
break;
|
|
default:
|
|
ret = gst_pad_event_default (pad, event);
|
|
break;
|
|
}
|
|
|
|
...
|
|
gst_object_unref (filter);
|
|
return ret;
|
|
}
|
|
</programlisting>
|
|
<para>
|
|
If your element is chain-based, you will almost always have to implement
|
|
a sink event function, since that is how you are notified about
|
|
new segments and the end of the stream.
|
|
</para>
|
|
<para>
|
|
If your element is exclusively loop-based, you may or may not want a
|
|
sink event function (since the element is driving the pipeline it will
|
|
know the length of the stream in advance or be notified by the flow
|
|
return value of <function>gst_pad_pull_range()</function>. In some cases
|
|
even loop-based element may receive events from upstream though (for
|
|
example audio decoders with an id3demux or apedemux element in front of
|
|
them, or demuxers that are being fed input from sources that send
|
|
additional information about the stream in custom events, as DVD sources
|
|
do).
|
|
</para>
|
|
</sect1>
|
|
<sect1 id="section-events-upstream" xreflabel="Upstream events">
|
|
<title>Upstream events</title>
|
|
<para>
|
|
Upstream events are generated by an element somewhere downstream in
|
|
the pipeline (example: a video sink may generate navigation
|
|
events that informs upstream elements about the current position of
|
|
the mouse pointer). This may also happen indirectly on request of the
|
|
application, for example when the application executes a seek on a
|
|
pipeline this seek request will be passed on to a sink element which
|
|
will then in turn generate an upstream seek event.
|
|
</para>
|
|
<para>
|
|
The most common upstream events are seek events and Quality-of-Service
|
|
(QoS) events.
|
|
</para>
|
|
<para>
|
|
An upstream event can be sent using the
|
|
<function>gst_pad_send_event</function> function. This
|
|
function simply call the default event handler of that pad. The default
|
|
event handler of pads is <function>gst_pad_event_default</function>, and
|
|
it basically sends the event to the peer pad. So upstream events always
|
|
arrive on the src pad of your element and are handled by the default event
|
|
handler except if you override that handler to handle it yourself. There
|
|
are some specific cases where you have to do that :
|
|
</para>
|
|
<itemizedlist mark="opencircle">
|
|
<listitem>
|
|
<para>
|
|
If you have multiple sink pads in your element. In that case you will
|
|
have to decide which one of the sink pads you will send the event to
|
|
(if not all of them).
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
If you need to handle that event locally. For example a navigation
|
|
event that you will want to convert before sending it upstream, or
|
|
a QoS event that you want to handle.
|
|
</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
<para>
|
|
The processing you will do in that event handler does not really matter
|
|
but there are important rules you have to absolutely respect because
|
|
one broken element event handler is breaking the whole pipeline event
|
|
handling. Here they are :
|
|
</para>
|
|
<itemizedlist mark="opencircle">
|
|
<listitem>
|
|
<para>
|
|
Always forward events you won't handle upstream using the default
|
|
<function>gst_pad_event_default</function> method.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
If you are generating some new event based on the one you received
|
|
don't forget to gst_event_unref the event you received.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
Event handler function are supposed to return TRUE or FALSE indicating
|
|
if the event has been handled or not. Never simply return TRUE/FALSE
|
|
in that handler except if you really know that you have handled that
|
|
event.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
Remember that the event handler might be called from a different
|
|
thread than the streaming thread, so make sure you use
|
|
appropriate locking everywhere and at the beginning of the function
|
|
obtain a reference to your element via the
|
|
<function>gst_pad_get_parent()</function> (and release it again at
|
|
the end of the function with <function>gst_object_unref ()</function>.
|
|
</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
</sect1>
|
|
|
|
<sect1 id="section-events-definitions" xreflabel="All Events Together">
|
|
<title>All Events Together</title>
|
|
<para>
|
|
In this chapter follows a list of all defined events that are currently
|
|
being used, plus how they should be used/interpreted. You can check the
|
|
what type a certain event is using the GST_EVENT_TYPE macro (or if you
|
|
need a string for debugging purposes you can use GST_EVENT_TYPE_NAME).
|
|
</para>
|
|
<para>
|
|
In this chapter, we will discuss the following events:
|
|
</para>
|
|
<itemizedlist>
|
|
<listitem><para><xref linkend="section-events-eos"/></para></listitem>
|
|
<listitem><para><xref linkend="section-events-flush-start"/></para></listitem>
|
|
<listitem><para><xref linkend="section-events-flush-stop"/></para></listitem>
|
|
<listitem><para><xref linkend="section-events-newsegment"/></para></listitem>
|
|
<listitem><para><xref linkend="section-events-seek"/></para></listitem>
|
|
<listitem><para><xref linkend="section-events-nav"/></para></listitem>
|
|
<listitem><para><xref linkend="section-events-tag"/></para></listitem>
|
|
</itemizedlist>
|
|
<para>
|
|
For more comprehensive information about events and how they should be
|
|
used correctly in various circumstances please consult the GStreamer
|
|
design documentation. This section only gives a general overview.
|
|
</para>
|
|
|
|
<sect2 id="section-events-eos" xreflabel="End of Stream (EOS)">
|
|
<title>End of Stream (EOS)</title>
|
|
<para>
|
|
End-of-stream events are sent if the stream that an element sends out
|
|
is finished. An element receiving this event (from upstream, so it
|
|
receives it on its sinkpad) will generally just process any buffered
|
|
data (if there is any) and then forward the event further downstream.
|
|
The <function>gst_pad_event_default ()</function> takes care of all
|
|
this, so most elements do not need to support this event. Exceptions are
|
|
elements that explicitly need to close a resource down on EOS, and
|
|
N-to-1 elements. Note that the stream itself is <emphasis>not</emphasis>
|
|
a resource that should be closed down on EOS! Applications might seek
|
|
back to a point before EOS and continue playing again.
|
|
</para>
|
|
<para>
|
|
The EOS event has no properties, which makes it one of the simplest
|
|
events in &GStreamer;. It is created using the
|
|
<function>gst_event_new_eos()</function> function.
|
|
</para>
|
|
<para>
|
|
It is important to note that <emphasis>only elements driving the
|
|
pipeline should ever send an EOS event</emphasis>. If your element
|
|
is chain-based, it is not driving the pipeline. Chain-based elements
|
|
should just return GST_FLOW_EOS from their chain function at
|
|
the end of the stream (or the configured segment), the upstream
|
|
element that is driving the pipeline will then take care of
|
|
sending the EOS event (or alternatively post a SEGMENT_DONE message
|
|
on the bus depending on the mode of operation). If you are implementing
|
|
your own source element, you also do not need to ever manually send
|
|
an EOS event, you should also just return GST_FLOW_EOS in
|
|
your create function (assuming your element derives from GstBaseSrc
|
|
or GstPushSrc).
|
|
</para>
|
|
</sect2>
|
|
|
|
<sect2 id="section-events-flush-start" xreflabel="Flush Start">
|
|
<title>Flush Start</title>
|
|
<para>
|
|
The flush start event is sent downstream if all buffers and caches
|
|
in the pipeline should be emptied. <quote>Queue</quote> elements will
|
|
empty their internal list of buffers when they receive this event, for
|
|
example. File sink elements (e.g. <quote>filesink</quote>) will flush
|
|
the kernel-to-disk cache (<function>fdatasync ()</function> or
|
|
<function>fflush ()</function>) when they receive this event. Normally,
|
|
elements receiving this event will simply just forward it, since most
|
|
filter or filter-like elements don't have an internal cache of data.
|
|
<function>gst_pad_event_default ()</function> does just that, so for
|
|
most elements, it is enough to forward the event using the default
|
|
event handler.
|
|
</para>
|
|
<para>
|
|
As a side-effect of flushing all data from the pipeline, this event
|
|
unblocks the streaming thread by making all pads reject data until
|
|
they receive a <xref linkend="section-events-flush-stop"/> signal
|
|
(elements trying to push data will get a WRONG_STATE flow return
|
|
and stop processing data).
|
|
</para>
|
|
<para>
|
|
The flush-start event is created with the
|
|
<function>gst_event_new_flush_start ()</function>.
|
|
Like the EOS event, it has no properties. This event is usually
|
|
only created by elements driving the pipeline, like source elements
|
|
operating in push-mode or pull-range based demuxers/decoders.
|
|
</para>
|
|
</sect2>
|
|
|
|
<sect2 id="section-events-flush-stop" xreflabel="Flush Stop">
|
|
<title>Flush Stop</title>
|
|
<para>
|
|
The flush-stop event is sent by an element driving the pipeline
|
|
after a flush-start and tells pads and elements downstream that
|
|
they should accept events and buffers again (there will be at
|
|
least a NEWSEGMENT event before any buffers first though).
|
|
</para>
|
|
<para>
|
|
If your element keeps temporary caches of stream data, it should
|
|
clear them when it receives a FLUSH-STOP event (and also whenever
|
|
its chain function receives a buffer with the DISCONT flag set).
|
|
</para>
|
|
<para>
|
|
The flush-stop event is created with
|
|
<function>gst_event_new_flush_stop ()</function>. Like the EOS event,
|
|
it has no properties.
|
|
</para>
|
|
</sect2>
|
|
|
|
<sect2 id="section-events-newsegment" xreflabel="New Segment">
|
|
<title>New Segment</title>
|
|
<para>
|
|
A new segment event is sent downstream to either announce a new
|
|
segment of data in the data stream or to update the current segment
|
|
with new values. A new segment event must always be sent before the
|
|
first buffer of data and after a flush (see above).
|
|
</para>
|
|
<para>
|
|
The first new segment event is created by the element driving the
|
|
pipeline, like a source operating in push-mode or a demuxer/decoder
|
|
operating pull-based. This new segment event then travels down the
|
|
pipeline and may be transformed on the way (a decoder, for example,
|
|
might receive a new-segment event in BYTES format and might transform
|
|
this into a new-segment event in TIMES format based on the average
|
|
bitrate).
|
|
</para>
|
|
<para>
|
|
New segment events may also be used to indicate 'gaps' in the stream,
|
|
like in a subtitle stream for example where there may not be any
|
|
data at all for a considerable amount of (stream) time. This is done
|
|
by updating the segment start of the current segment (see the design
|
|
documentation for more details).
|
|
</para>
|
|
<para>
|
|
Depending on the element type, the event can simply be forwarded using
|
|
<function>gst_pad_event_default ()</function>, or it should be parsed
|
|
and a modified event should be sent on. The last is true for demuxers,
|
|
which generally have a byte-to-time conversion concept. Their input
|
|
is usually byte-based, so the incoming event will have an offset in
|
|
byte units (<symbol>GST_FORMAT_BYTES</symbol>), too. Elements
|
|
downstream, however, expect new segment events in time units, so that
|
|
it can be used to update the pipeline clock. Therefore, demuxers and
|
|
similar elements should not forward the event, but parse it, free it
|
|
and send a new newsegment event (in time units,
|
|
<symbol>GST_FORMAT_TIME</symbol>) further downstream.
|
|
</para>
|
|
<para>
|
|
The newsegment event is created using the function
|
|
<function>gst_event_new_new_segment ()</function>. See the API
|
|
reference and design document for details about its parameters.
|
|
</para>
|
|
<para>
|
|
Elements parsing this event can use gst_event_parse_new_segment_full()
|
|
to extract the event details. Elements may find the GstSegment
|
|
API useful to keep track of the current segment (if they want to use
|
|
it for output clipping, for example).
|
|
</para>
|
|
</sect2>
|
|
|
|
<sect2 id="section-events-seek" xreflabel="Seek Request">
|
|
<title>Seek Request</title>
|
|
<para>
|
|
Seek events are meant to request a new stream position to elements.
|
|
This new position can be set in several formats (time, bytes or
|
|
<quote>default units</quote> [a term indicating frames for video,
|
|
channel-independent samples for audio, etc.]). Seeking can be done with
|
|
respect to the end-of-file, start-of-file or current position, and
|
|
usually happens in upstream direction (downstream seeking is done by
|
|
sending a NEWSEGMENT event with the appropriate offsets for elements
|
|
that support that, like filesink).
|
|
</para>
|
|
<para>
|
|
Elements receiving seek events should, depending on the element type,
|
|
either just forward it upstream (filters, decoders), change the
|
|
format in which the event is given and then forward it (demuxers),
|
|
or handle the event by changing the file pointer in their internal
|
|
stream resource (file sources, demuxers/decoders driving the pipeline
|
|
in pull-mode) or something else.
|
|
</para>
|
|
<para>
|
|
Seek events are built up using positions in specified formats (time,
|
|
bytes, units). They are created using the function
|
|
<function>gst_event_new_seek ()</function>. Note that many plugins do
|
|
not support seeking from the end of the stream or from the current
|
|
position. An element not driving the pipeline and forwarding a seek
|
|
request should not assume that the seek succeeded or actually happened,
|
|
it should operate based on the NEWSEGMENT events it receives.
|
|
</para>
|
|
<para>
|
|
Elements parsing this event can do this using
|
|
<function>gst_event_parse_seek()</function>.
|
|
</para>
|
|
</sect2>
|
|
|
|
<sect2 id="section-events-nav" xreflabel="Navigation">
|
|
<title>Navigation</title>
|
|
<para>
|
|
Navigation events are sent upstream by video sinks to inform upstream
|
|
elements of where the mouse pointer is, if and where mouse pointer
|
|
clicks have happened, or if keys have been pressed or released.
|
|
</para>
|
|
<para>
|
|
All this information is contained in the event structure which can
|
|
be obtained with <function>gst_event_get_structure ()</function>.
|
|
</para>
|
|
<para>
|
|
Check out the navigationtest element in gst-plugins-good for an idea
|
|
how to extract navigation information from this event.
|
|
</para>
|
|
</sect2>
|
|
|
|
<sect2 id="section-events-tag" xreflabel="Tag (metadata)">
|
|
<title>Tag (metadata)</title>
|
|
<para>
|
|
Tagging events are being sent downstream to indicate the tags as parsed
|
|
from the stream data. This is currently used to preserve tags during
|
|
stream transcoding from one format to the other. Tags are discussed
|
|
extensively in <xref linkend="chapter-advanced-tagging"/>. Most
|
|
elements will simply forward the event by calling
|
|
<function>gst_pad_event_default ()</function>.
|
|
</para>
|
|
<para>
|
|
The tag event is created using the function
|
|
<function>gst_event_new_tag ()</function>, but more often elements will
|
|
use either the <function>gst_element_found_tags ()</function> function
|
|
or the <function>gst_element_found_tags_for_pad ()</function>, which
|
|
will do both: post a tag message on the bus and send a tag event
|
|
downstream. All of these functions require a filled-in taglist as
|
|
argument, which they will take ownership of.
|
|
</para>
|
|
<para>
|
|
Elements parsing this event can use the function
|
|
<function>gst_event_parse_tag ()</function> to acquire the
|
|
taglist that the event contains.
|
|
</para>
|
|
</sect2>
|
|
</sect1>
|
|
</chapter>
|