From 483c800475f91cd3862862b80bb9cd7853d7c2a9 Mon Sep 17 00:00:00 2001 From: Jan Schmidt Date: Wed, 18 Jan 2006 18:05:43 +0000 Subject: [PATCH] docs/design/part-events.txt: Remove mention of FILLER events. Original commit message from CVS: * docs/design/part-events.txt: Remove mention of FILLER events. Add DRAIN event. * docs/design/part-sparsestreams.txt: Write some things about using NEWSEGMENT to keep sparse streams flowing. --- docs/design/part-events.txt | 20 +++++-- docs/design/part-sparsestreams.txt | 96 ++++++++++++++++++++++++++++++ 2 files changed, 111 insertions(+), 5 deletions(-) create mode 100644 docs/design/part-sparsestreams.txt diff --git a/docs/design/part-events.txt b/docs/design/part-events.txt index 6bf31ec1df..a03bdb7cb1 100644 --- a/docs/design/part-events.txt +++ b/docs/design/part-events.txt @@ -15,11 +15,11 @@ Different types of events exist to implement various functionalities. GST_EVENT_EOS: no more data is to be expected on a pad. GST_EVENT_NEWSEGMENT: A new group of buffers with common start time GST_EVENT_TAG: Stream metadata. - GST_EVENT_FILLER: Filler for sparse data streams GST_EVENT_BUFFERSIZE: Buffer size requirements GST_EVENT_QOS: A notification of the quality of service of the stream GST_EVENT_SEEK: A seek should be performed to a new position in the stream GST_EVENT_NAVIGATION: A navigation event. + GST_EVENT_DRAIN: Play all data downstream before returning. FLUSH_START/STOP @@ -142,10 +142,6 @@ tags in a media file. Encoders can use this event to adjust their tagging system. A tag is serialized with buffers. -FILLER ------- - - BUFFERSIZE ---------- @@ -226,3 +222,17 @@ of a navigation event such as a mouse movement or button click. Navigation events travel upstream. +DRAIN +----- + +Drain event indicates that upstream is about to perform a real-time event, such +as pausing to present an interactive menu or such, and needs to wait for all +data it has sent to be played-out in the sink. + +Drain should only be used by live elements, as it may otherwise occur during +prerolling. + +Usually after draining the pipeline, an element either needs to modify timestamps, +or FLUSH to prevent subsequent data being discarded at the sinks for arriving +late (only applies during playback scenarios). + diff --git a/docs/design/part-sparsestreams.txt b/docs/design/part-sparsestreams.txt new file mode 100644 index 0000000000..83922e10a8 --- /dev/null +++ b/docs/design/part-sparsestreams.txt @@ -0,0 +1,96 @@ +DRAFT Sparse Streams +-------------------- + +Introduction +------------ +In 0.8, there was some support for Sparse Streams through the use of +FILLER events. These were used to mark gaps between buffers so that downstream +elements could know not to expect any more data for that gap. + +In 0.10, segment information conveyed through NEWSEGMENT events can be used +for the same purpose. + +Use cases +--------- +1) Sub-title streams + Sub-title information from muxed formats such as Matroska or MPEG consist of + irregular buffers spaced far apart compared to the other streams + (audio and video). Since these usually only appear when someone speaks or + some other action in the video/audio needs describing, they can be anywhere + from 1-2 seconds to several minutes apart. + + Downstream elements that want to mix sub-titles and video (and muxers) + have no way of knowing whether to process a video packet or wait a moment + for a corresponding sub-title to be delivered on another pad. + +2) Still frame/menu support + In DVDs (and other formats), there are still-frame regions where the current + video frame should be retained and no audio played for a period. In DVD, + these are described either as a fixed duration, or infinite duration still + frame. + +3) Avoiding processing silence from audio generators + Imagine a source that from time to time produces empty buffers (silence + or blank images). If the pipeline has many elements next, it is better to + optimise the obsolete data processing in this case. Examples for such sources + are sound-generators (simsyn in gst-buzztard) or a source in a voip + application that uses noise-gating (to save bandwith). + +Details +------- +1) Sub-title streams + The main requirement here is to avoid stalling the pipeline between sub-title + packets, and is effectively updating the minimum-timestamp for that stream. + + A demuxer can do this by sending an 'update' NEWSEGMENT with a new start time + to the subtitle pad. For example, every time the SCR in the MPEG data + advances more than 0.5 seconds, the MPEG demuxer can issue a NEWSEGMENT with + (update=TRUE, start=SCR ). Downstream elements can then be aware not to + expect any data older than the new start time. + + This technique can also be used, for example, to represent a stream of + MIDI events spaced to a clock period. When there is no event present for + a clock time, a NEWSEGMENT update can be sent in its place. + +2) Still frame/menu support + Still frames in DVD menus are not the same, in that they do not introduce + a gap in the timestamps of the data. Instead, they represent a pause in the + presentation of a stream. Correctly performing the wait requires some + synchronisation with downstream elements. + + In this scenario, an upstream element that wants to execute a still frame + performs the following steps: + + * Send all data before the still frame wait + * Send a DRAIN event to ensure that all data has been played downstream. + * wait on the clock for the required duration, possibly interrupting + if necessary due to an intervening activity (such as a user navigation) + * FLUSH the pipeline using a normal flush sequence (FLUSH_START, + chain-lock, FLUSH_STOP) + * Send a NEWSEGMENT to restart playback with the next timestamp in the + stream. + + The upstream element performing the wait MUST be marked as LIVE element, to + prevent GStreamer from attempting to preroll the element during a still frame + wait. + + DRAIN is a new event that will block on the pad until all data downstream has + been played out. + + Flushing after completing the still wait is to ensure that data after the wait + is played correctly. Without it, sinks will consider the first buffers + (x seconds, where x is the duration of the wait that occurred) to be + arriving late at the sink, and they will be discarded instead of played. + +3) For audio, 3) is the same case as 1) - there is a 'gap' in the audio data + that needs to be presented, and this can be done by sending a NEWSEGMENT + update that moves the start time of the segment to the next timestamp when + data will be sent. + + For video, however it is slightly different. Video frames are typically + treated at the moment as continuing to be displayed after their indicated + duration if no new frame arrives. In 3), it is desired to display a blank + frame instead, in which case at least one blank frame should be sent before + updating the start time of the segment. + +