mirrors/gstreamer
mirrors/gstreamer
1
1
Fork 0
mirror of https://gitlab.freedesktop.org/gstreamer/gstreamer.git synced 2024-12-23 08:46:40 +00:00
Code Issues 1 Projects Releases Wiki Activity

docs/design/part-trickmodes.txt: More documentation on trickmodes.

Browse source 
Original commit message from CVS:
* docs/design/part-trickmodes.txt:
More documentation on trickmodes.
This commit is contained in:
Wim Taymans 2005-12-20 12:50:56 +00:00
parent bd97cc0541
commit eb6b59a158
2 changed files with 168 additions and 4 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
ChangeLog
View file

@ -1,3 +1,8 @@
2005-12-20 Wim Taymans <wim@fluendo.com>
* docs/design/part-trickmodes.txt:
More documentation on trickmodes.
2005-12-20 Edward Hervey <edward@fluendo.com>
* gst/gstcaps.c: (gst_static_caps_get_type):

167
docs/design/part-trickmodes.txt
View file

@ -1,13 +1,172 @@
Trickmodes
==========
GStreamer provides API for performing various trickmode playback. This includes:
forward playback
----------------
- server side trickmodes
- client side fast/slow forward playback
- client side fast/slow backwards playback
Server side trickmodes mean that a source (network source) can provide a
stream with different playback speed and direction. The client does not have to
perform any special algorithms to decode this stream.
Client side trickmodes mean that the decoding client (GStreamer) performs the
needed algorithms to change the direction and speed of the media file.
Seeking can both be done in a playback pipeline and a transcoding pipeline.
General seeking overview
------------------------
Consider a typical playback pipeline:
.---------. .------.
.-------. | decoder |->| sink |
.--------. | |-->'---------' '------'
| source |->| demux |
'--------' | |-->.---------. .------.
'-------' | decoder |->| sink |
'---------' '------'
The pipeline is initially configured to play back at speed 1.0 starting from
position 0 and stopping at the total duration of the file.
When performing a seek, the following steps have to be taken by the application:
1) Create a seek event:
The seek event contains:
- various flags describing:
- where to seek to (KEY_UNIT)
- how accurate the seek should be (ACCURATE)
- how to perform the seek (FLUSH)
- what to do when the stop position is reached (SEGMENT).
- a format to seek in, this can be time, bytes, units (frames, samples), ...
- a playback rate, 1.0 is normal playback speed, positive values bigger than 1.0
mean fast playback. negative values mean reverse playback. A playback speed of
0.0 is not allowed.
- a start position, this value has to be between 0 and the total duration of the
file. It can also be relative to the previously configured start value.
- a stop position, this value has to be between 0 and the total duration. It can
also be relative to the previously configured stop value.
See also gst_event_new_seek().
2) Send the seek event to the pipeline with gst_element_send_event()
By default the pipeline will send the event to all sink elements.
By default an element will forward the event upstream on all sinkpads.
Elements can modify the format of the seek event. The most common format is
GST_FORMAT_TIME.
One element will actually perform the seek, this is usually the demuxer or
source element. For more information on how to perform the different seek
types see part-seeking.txt.
For client side trickmode a NEW_SEGMENT event will be sent downstream with
the new rate and start/stop positions. All elements prepare themselves to
handle the rate (see below).
for server side trick mode a NEW_SEGMENT event is sent downstream with a
rate of 1.0 and the start/stop positions. The elements will configure themselves
for normal playback speed since the server will perform the rate conversions.
When the seek succeeds, the _send_event() function will return TRUE.
Server side trickmode
---------------------
The source element operates in push mode. It can reopen a server connection requesting
a new byte or time position and a new playback speed. The capabilities can be queried
from the server when the connection is opened.
We assume the source element is derived from the GstPushSrc base class. The base source
should be configured with gst_base_src_set_format (src, GST_FORMAT_TIME).
The do_seek method will be called on the push src subclass with the seek information
passed in the GstSegment argument.
The rate value in the segment should be used to reopen the connection to the server
requesting data at the new speed and possibly a new playback position.
When the server connection was successfully reopened, set the rate of the segment
to 1.0 so that the client side trickmode is not enabled. Alternatively a combination
of client side and serverside trickmode can be used, for example if the server does
not support certain rates, the client can perform rate conversion for the remainder.
source server
do_seek | |
----------->| |
| reopen connection |
|-------------------->|
| .
| success .
|<--------------------|
modify | |
rate to 1.0 | |
| |
return | |
TRUE | |
| |
After performing the seek, the source will inform the downstream elements of the
new segment that is to be played back. Since the segment will have a rate of 1.0,
no client side trick modes are enabled.
client side forward trickmodes
------------------------------
The seek happens as stated above. a NEW_SEGMENT event is sent downstream with a rate
different from 1.0. Plugins receiving the NEW_SEGMENT can decide to perform the
rate conversion of the media data (retimestamp video frames, resample audio, ...).
A plugin can also decide to drop frames in the case of fast playback or use a more
efficient decoding algorithm (skip B frames, ...). If a plugin decides to resample
or retimestamp, it should modify the NEW_SEGMENT with a rate of 1.0 so that
downstream elements don't resample again.
The GStreamer base audio and video sinks will resample automatically if they receive
a NEW_SEGMENT event with a rate different from 1.0.
client side backwards trickmode
-------------------------------
For backwards playback the following rules apply:
- the rate in the NEW_SEGMENT is less than 0.0.
- the NEW_SEGMENT start position is less than the stop position, playback will
however happen from stop to start in reverse.
For plugins the following rules apply:
- A source plugin sends data in chunks starting from the last chunk of the file.
The actual bytes are not reversed.
- A demuxer accumulates the chunks. As soon as a keyframe is found, everything
starting from the keyframe up to the accumulated data is sent downstream.
Timestamps on the buffers are set starting from the stop position to start,
effectively going backwards.
- A decoder decodes and accumulates all frames. If a new keyframe is received,
all accumulated frames are sent backwards.
- A sink reverses (for audio) and retimestamps (audio, video) the buffers before
playing them back. Retimestamping occurs relative to the stop position, making
the timestamps increase again and suitable for synchronizing against the clock.
- for transcoding, audio and video resamplers can be used to reverse, resample and
retimestamp the buffers.
Notes:
- The clock keeps running forward.
- backwards playback potentially uses a lot of memory as frames and undecoded data
get buffered.
backwards playback
------------------