docs/design/part-events.txt: Remove mention of FILLER events.

Original commit message from CVS:
* docs/design/part-events.txt:
Remove mention of FILLER events.
Add DRAIN event.

* docs/design/part-sparsestreams.txt:
Write some things about using NEWSEGMENT to keep sparse streams
flowing.
This commit is contained in:
Jan Schmidt 2006-01-18 18:05:43 +00:00
parent c33121174b
commit 483c800475
2 changed files with 111 additions and 5 deletions

View file

@ -15,11 +15,11 @@ Different types of events exist to implement various functionalities.
GST_EVENT_EOS: no more data is to be expected on a pad. GST_EVENT_EOS: no more data is to be expected on a pad.
GST_EVENT_NEWSEGMENT: A new group of buffers with common start time GST_EVENT_NEWSEGMENT: A new group of buffers with common start time
GST_EVENT_TAG: Stream metadata. GST_EVENT_TAG: Stream metadata.
GST_EVENT_FILLER: Filler for sparse data streams
GST_EVENT_BUFFERSIZE: Buffer size requirements GST_EVENT_BUFFERSIZE: Buffer size requirements
GST_EVENT_QOS: A notification of the quality of service of the stream GST_EVENT_QOS: A notification of the quality of service of the stream
GST_EVENT_SEEK: A seek should be performed to a new position in the stream GST_EVENT_SEEK: A seek should be performed to a new position in the stream
GST_EVENT_NAVIGATION: A navigation event. GST_EVENT_NAVIGATION: A navigation event.
GST_EVENT_DRAIN: Play all data downstream before returning.
FLUSH_START/STOP FLUSH_START/STOP
@ -142,10 +142,6 @@ tags in a media file. Encoders can use this event to adjust their tagging
system. A tag is serialized with buffers. system. A tag is serialized with buffers.
FILLER
------
BUFFERSIZE BUFFERSIZE
---------- ----------
@ -226,3 +222,17 @@ of a navigation event such as a mouse movement or button click.
Navigation events travel upstream. Navigation events travel upstream.
DRAIN
-----
Drain event indicates that upstream is about to perform a real-time event, such
as pausing to present an interactive menu or such, and needs to wait for all
data it has sent to be played-out in the sink.
Drain should only be used by live elements, as it may otherwise occur during
prerolling.
Usually after draining the pipeline, an element either needs to modify timestamps,
or FLUSH to prevent subsequent data being discarded at the sinks for arriving
late (only applies during playback scenarios).

View file

@ -0,0 +1,96 @@
DRAFT Sparse Streams
--------------------
Introduction
------------
In 0.8, there was some support for Sparse Streams through the use of
FILLER events. These were used to mark gaps between buffers so that downstream
elements could know not to expect any more data for that gap.
In 0.10, segment information conveyed through NEWSEGMENT events can be used
for the same purpose.
Use cases
---------
1) Sub-title streams
Sub-title information from muxed formats such as Matroska or MPEG consist of
irregular buffers spaced far apart compared to the other streams
(audio and video). Since these usually only appear when someone speaks or
some other action in the video/audio needs describing, they can be anywhere
from 1-2 seconds to several minutes apart.
Downstream elements that want to mix sub-titles and video (and muxers)
have no way of knowing whether to process a video packet or wait a moment
for a corresponding sub-title to be delivered on another pad.
2) Still frame/menu support
In DVDs (and other formats), there are still-frame regions where the current
video frame should be retained and no audio played for a period. In DVD,
these are described either as a fixed duration, or infinite duration still
frame.
3) Avoiding processing silence from audio generators
Imagine a source that from time to time produces empty buffers (silence
or blank images). If the pipeline has many elements next, it is better to
optimise the obsolete data processing in this case. Examples for such sources
are sound-generators (simsyn in gst-buzztard) or a source in a voip
application that uses noise-gating (to save bandwith).
Details
-------
1) Sub-title streams
The main requirement here is to avoid stalling the pipeline between sub-title
packets, and is effectively updating the minimum-timestamp for that stream.
A demuxer can do this by sending an 'update' NEWSEGMENT with a new start time
to the subtitle pad. For example, every time the SCR in the MPEG data
advances more than 0.5 seconds, the MPEG demuxer can issue a NEWSEGMENT with
(update=TRUE, start=SCR ). Downstream elements can then be aware not to
expect any data older than the new start time.
This technique can also be used, for example, to represent a stream of
MIDI events spaced to a clock period. When there is no event present for
a clock time, a NEWSEGMENT update can be sent in its place.
2) Still frame/menu support
Still frames in DVD menus are not the same, in that they do not introduce
a gap in the timestamps of the data. Instead, they represent a pause in the
presentation of a stream. Correctly performing the wait requires some
synchronisation with downstream elements.
In this scenario, an upstream element that wants to execute a still frame
performs the following steps:
* Send all data before the still frame wait
* Send a DRAIN event to ensure that all data has been played downstream.
* wait on the clock for the required duration, possibly interrupting
if necessary due to an intervening activity (such as a user navigation)
* FLUSH the pipeline using a normal flush sequence (FLUSH_START,
chain-lock, FLUSH_STOP)
* Send a NEWSEGMENT to restart playback with the next timestamp in the
stream.
The upstream element performing the wait MUST be marked as LIVE element, to
prevent GStreamer from attempting to preroll the element during a still frame
wait.
DRAIN is a new event that will block on the pad until all data downstream has
been played out.
Flushing after completing the still wait is to ensure that data after the wait
is played correctly. Without it, sinks will consider the first buffers
(x seconds, where x is the duration of the wait that occurred) to be
arriving late at the sink, and they will be discarded instead of played.
3) For audio, 3) is the same case as 1) - there is a 'gap' in the audio data
that needs to be presented, and this can be done by sending a NEWSEGMENT
update that moves the start time of the segment to the next timestamp when
data will be sent.
For video, however it is slightly different. Video frames are typically
treated at the moment as continuing to be displayed after their indicated
duration if no new frame arrives. In 3), it is desired to display a blank
frame instead, in which case at least one blank frame should be sent before
updating the start time of the segment.