mirrors/gstreamer
mirrors/gstreamer
1
1
Fork 0
mirror of https://gitlab.freedesktop.org/gstreamer/gstreamer.git synced 2025-01-12 02:15:31 +00:00
Code Issues 1 Projects Releases Wiki Activity

design: sparsestreams: section breakdown, markup, readability and grammar fixes

Browse source
This commit is contained in:
Reynaldo H. Verdejo Pinochet 2016-12-29 23:16:25 -08:00
parent 27addf2094
commit 8e28295ede
1 changed files with 30 additions and 26 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

56
markdown/design/sparsestreams.md
View file

@ -2,19 +2,21 @@
## Introduction ## Introduction
In 0.8, there was some support for Sparse Streams through the use of In 0.8, there was some support for sparse streams through the use of
FILLER events. These were used to mark gaps between buffers so that `FILLER` events. These were used to mark gaps between buffers so that
downstream elements could know not to expect any more data for that gap. downstream elements could know not to expect any more data for that gap.
In 0.10, segment information conveyed through SEGMENT events can be used In 0.10, segment information conveyed through `SEGMENT` events can be used
for the same purpose. for the same purpose.
In 1.0, there is a GAP event that works in a similar fashion as the In 1.0, there is a `GAP` event that works in a similar fashion as the
FILLER event in 0.8. `FILLER` event in 0.8.
## Use cases ## Use cases
1) Sub-title streams Sub-title information from muxed formats such as ### Sub-title streams
Sub-title information from muxed formats such as
Matroska or MPEG consist of irregular buffers spaced far apart compared Matroska or MPEG consist of irregular buffers spaced far apart compared
to the other streams (audio and video). Since these usually only appear to the other streams (audio and video). Since these usually only appear
when someone speaks or some other action in the video/audio needs when someone speaks or some other action in the video/audio needs
@ -23,16 +25,19 @@ apart. Downstream elements that want to mix sub-titles and video (and muxers)
have no way of knowing whether to process a video packet or wait a moment have no way of knowing whether to process a video packet or wait a moment
for a corresponding sub-title to be delivered on another pad. for a corresponding sub-title to be delivered on another pad.
2) Still frame/menu support In DVDs (and other formats), there are ### Still frame/DVD menues
still-frame regions where the current video frame should be retained and
no audio played for a period. In DVD, these are described either as a
fixed duration, or infinite duration still frame.
3) Avoiding processing silence from audio generators Imagine a source In DVDs and other formats, there are still-frame regions where the current
that from time to time produces empty buffers (silence or blank images). video frame should be retained and no audio played for a period. In DVD,
If the pipeline has many elements next, it is better to optimise the these are described either as a fixed duration, or infinite duration still
obsolete data processing in this case. Examples for such sources are frame.
sound-generators (simsyn in gst-buzztard) or a source in a voip
### Avoiding processing silence from audio generators
Imagine a source that, from time to time, produces empty buffers (silence or
blank images). If the pipeline has many elements next, it is better to
optimise the absolute data processing in this case. Examples for such sources
are sound-generators (simsyn in gst-buzztard) or a source in a voip
application that uses noise-gating (to save bandwith). application that uses noise-gating (to save bandwith).
## Details ## Details
@ -60,10 +65,9 @@ a clock time, a SEGMENT update can be sent in its place.
### Still frame/menu support ### Still frame/menu support
Still frames in DVD menus are not the same, Still frames in DVD menus are different because they do not introduce a gap
in that they do not introduce a gap in the timestamps of the data. in the data timestamps. Instead, they represent a pause in the presentation
Instead, they represent a pause in the presentation of a stream. of a stream. Correctly performing the wait requires some synchronisation with
Correctly performing the wait requires some synchronisation with
downstream elements. downstream elements.
In this scenario, an upstream element that wants to execute a still frame In this scenario, an upstream element that wants to execute a still frame
@ -71,24 +75,24 @@ performs the following steps:
- Send all data before the still frame wait - Send all data before the still frame wait
- Send a DRAIN event to ensure that all data has been played - Send a `DRAIN` event to ensure that all data has been played
downstream. downstream.
- wait on the clock for the required duration, possibly interrupting - wait on the clock for the required duration, possibly interrupting
if necessary due to an intervening activity (such as a user if necessary due to an intervening activity (such as a user
navigation) navigation)
- FLUSH the pipeline using a normal flush sequence (FLUSH\_START, - FLUSH the pipeline using a normal flush sequence (`FLUSH_START`,
chain-lock, FLUSH\_STOP) chain-lock, `FLUSH_STOP`)
- Send a SEGMENT to restart playback with the next timestamp in the - Send a SEGMENT to restart playback with the next timestamp in the
stream. stream.
The upstream element performing the wait must only do so when in the PLAYING The upstream element performing the wait must only do so when in the `PLAYING`
state. During PAUSED, the clock will not be running, and may not even have state. During `PAUSED`, the clock will not be running, and may not even have
been distributed to the element yet. been distributed to the element yet.
DRAIN is a new event that will block on a src pad until all data downstream `DRAIN` is a new event that will block on a src pad until all data downstream
has been played out. has been played out.
Flushing after completing the still wait is to ensure that data after the wait Flushing after completing the still wait is to ensure that data after the wait
@ -103,7 +107,7 @@ data that needs to be presented, and this can be done by sending a
SEGMENT update that moves the start time of the segment to the next SEGMENT update that moves the start time of the segment to the next
timestamp when data will be sent. timestamp when data will be sent.
For video, however it is slightly different. Video frames are typically For video, however, it is slightly different. Video frames are typically
treated at the moment as continuing to be displayed after their indicated treated at the moment as continuing to be displayed after their indicated
duration if no new frame arrives. Here, it is desired to display a blank duration if no new frame arrives. Here, it is desired to display a blank
frame instead, in which case at least one blank frame should be sent before frame instead, in which case at least one blank frame should be sent before