mirror of
https://gitlab.freedesktop.org/gstreamer/gstreamer.git
synced 2024-09-08 21:28:45 +00:00
TRUE is 1, but every other non-zero value is also considered true. Comparing for equality with TRUE would only consider 1 but not the others. Also normalize booleans in a few places.
450 lines
15 KiB
XML
450 lines
15 KiB
XML
<chapter id="chapter-buffering">
|
|
<title>Buffering</title>
|
|
<para>
|
|
The purpose of buffering is to accumulate enough data in a pipeline so that
|
|
playback can occur smoothly and without interruptions. It is typically done
|
|
when reading from a (slow) and non-live network source but can also be
|
|
used for live sources.
|
|
</para>
|
|
<para>
|
|
&GStreamer; provides support for the following use cases:
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>
|
|
Buffering up to a specific amount of data, in memory, before starting
|
|
playback so that network fluctuations are minimized.
|
|
See <xref linkend="section-buffering-stream"/>.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
Download of the network file to a local disk with fast seeking in the
|
|
downloaded data. This is similar to the quicktime/youtube players.
|
|
See <xref linkend="section-buffering-download"/>.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
Caching of (semi)-live streams to a local, on disk, ringbuffer with
|
|
seeking in the cached area. This is similar to tivo-like timeshifting.
|
|
See <xref linkend="section-buffering-timeshift"/>.
|
|
</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
</para>
|
|
<para>
|
|
&GStreamer; can provide the application with progress reports about the
|
|
current buffering state as well as let the application decide on how
|
|
to buffer and when the buffering stops.
|
|
</para>
|
|
<para>
|
|
In the most simple case, the application has to listen for BUFFERING
|
|
messages on the bus. If the percent indicator inside the BUFFERING message
|
|
is smaller than 100, the pipeline is buffering. When a message is
|
|
received with 100 percent, buffering is complete. In the buffering state,
|
|
the application should keep the pipeline in the PAUSED state. When buffering
|
|
completes, it can put the pipeline (back) in the PLAYING state.
|
|
</para>
|
|
<para>
|
|
What follows is an example of how the message handler could deal with
|
|
the BUFFERING messages. We will see more advanced methods in
|
|
<xref linkend="section-buffering-strategies"/>.
|
|
</para>
|
|
<programlisting>
|
|
<![CDATA[
|
|
[...]
|
|
|
|
switch (GST_MESSAGE_TYPE (message)) {
|
|
case GST_MESSAGE_BUFFERING:{
|
|
gint percent;
|
|
|
|
/* no state management needed for live pipelines */
|
|
if (is_live)
|
|
break;
|
|
|
|
gst_message_parse_buffering (message, &percent);
|
|
|
|
if (percent == 100) {
|
|
/* a 100% message means buffering is done */
|
|
buffering = FALSE;
|
|
/* if the desired state is playing, go back */
|
|
if (target_state == GST_STATE_PLAYING) {
|
|
gst_element_set_state (pipeline, GST_STATE_PLAYING);
|
|
}
|
|
} else {
|
|
/* buffering busy */
|
|
if (!buffering && target_state == GST_STATE_PLAYING) {
|
|
/* we were not buffering but PLAYING, PAUSE the pipeline. */
|
|
gst_element_set_state (pipeline, GST_STATE_PAUSED);
|
|
}
|
|
buffering = TRUE;
|
|
}
|
|
break;
|
|
case ...
|
|
|
|
[...]
|
|
]]>
|
|
</programlisting>
|
|
|
|
<sect1 id="section-buffering-stream">
|
|
<title>Stream buffering </title>
|
|
<programlisting>
|
|
+---------+ +---------+ +-------+
|
|
| httpsrc | | buffer | | demux |
|
|
| src - sink src - sink ....
|
|
+---------+ +---------+ +-------+
|
|
</programlisting>
|
|
<para>
|
|
In this case we are reading from a slow network source into a buffer
|
|
element (such as queue2).
|
|
</para>
|
|
<para>
|
|
The buffer element has a low and high watermark expressed in bytes. The
|
|
buffer uses the watermarks as follows:
|
|
</para>
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>
|
|
The buffer element will post BUFFERING messages until the high
|
|
watermark is hit. This instructs the application to keep the pipeline
|
|
PAUSED, which will eventually block the srcpad from pushing while
|
|
data is prerolled in the sinks.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
When the high watermark is hit, a BUFFERING message with 100% will be
|
|
posted, which instructs the application to continue playback.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
When during playback, the low watermark is hit, the queue will start
|
|
posting BUFFERING messages again, making the application PAUSE the
|
|
pipeline again until the high watermark is hit again. This is called
|
|
the rebuffering stage.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
During playback, the queue level will fluctuate between the high and
|
|
the low watermark as a way to compensate for network irregularities.
|
|
</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
<para>
|
|
This buffering method is usable when the demuxer operates in push mode.
|
|
Seeking in the stream requires the seek to happen in the network source.
|
|
It is mostly desirable when the total duration of the file is not known,
|
|
such as in live streaming or when efficient seeking is not
|
|
possible/required.
|
|
</para>
|
|
<para>
|
|
The problem is configuring a good low and high watermark. Here are some
|
|
ideas:
|
|
</para>
|
|
<itemizedlist>
|
|
<listitem>
|
|
<para>
|
|
It is possible to measure the network bandwidth and configure the
|
|
low/high watermarks in such a way that buffering takes a fixed
|
|
amount of time.
|
|
</para>
|
|
<para>
|
|
The queue2 element in &GStreamer; core has the max-size-time property
|
|
that, together with the use-rate-estimate property, does exactly
|
|
that. Also the playbin buffer-duration property uses the rate estimate
|
|
to scale the amount of data that is buffered.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
Based on the codec bitrate, it is also possible to set the watermarks
|
|
in such a way that a fixed amount of data is buffered before playback
|
|
starts. Normally, the buffering element doesn't know about the
|
|
bitrate of the stream but it can get this with a query.
|
|
</para>
|
|
</listitem>
|
|
<listitem>
|
|
<para>
|
|
Start with a fixed amount of bytes, measure the time between
|
|
rebuffering and increase the queue size until the time between
|
|
rebuffering is within the application's chosen limits.
|
|
</para>
|
|
</listitem>
|
|
</itemizedlist>
|
|
<para>
|
|
The buffering element can be inserted anywhere in the pipeline. You could,
|
|
for example, insert the buffering element before a decoder. This would
|
|
make it possible to set the low/high watermarks based on time.
|
|
</para>
|
|
<para>
|
|
The buffering flag on playbin, performs buffering on the parsed data.
|
|
Another advantage of doing the buffering at a later stage is that you can
|
|
let the demuxer operate in pull mode. When reading data from a slow
|
|
network drive (with filesrc) this can be an interesting way to buffer.
|
|
</para>
|
|
</sect1>
|
|
|
|
<sect1 id="section-buffering-download">
|
|
<title>Download buffering </title>
|
|
<programlisting>
|
|
+---------+ +---------+ +-------+
|
|
| httpsrc | | buffer | | demux |
|
|
| src - sink src - sink ....
|
|
+---------+ +----|----+ +-------+
|
|
V
|
|
file
|
|
</programlisting>
|
|
<para>
|
|
If we know the server is streaming a fixed length file to the client,
|
|
the application can choose to download the entire file on disk. The
|
|
buffer element will provide a push or pull based srcpad to the demuxer
|
|
to navigate in the downloaded file.
|
|
</para>
|
|
<para>
|
|
This mode is only suitable when the client can determine the length of
|
|
the file on the server.
|
|
</para>
|
|
<para>
|
|
In this case, buffering messages will be emitted as usual when the
|
|
requested range is not within the downloaded area + buffersize. The
|
|
buffering message will also contain an indication that incremental
|
|
download is being performed. This flag can be used to let the application
|
|
control the buffering in a more intelligent way, using the BUFFERING
|
|
query, for example. See <xref linkend="section-buffering-strategies"/>.
|
|
</para>
|
|
</sect1>
|
|
|
|
<sect1 id="section-buffering-timeshift">
|
|
<title>Timeshift buffering </title>
|
|
<programlisting>
|
|
+---------+ +---------+ +-------+
|
|
| httpsrc | | buffer | | demux |
|
|
| src - sink src - sink ....
|
|
+---------+ +----|----+ +-------+
|
|
V
|
|
file-ringbuffer
|
|
</programlisting>
|
|
<para>
|
|
In this mode, a fixed size ringbuffer is kept to download the server
|
|
content. This allows for seeking in the buffered data. Depending on the
|
|
size of the ringbuffer one can seek further back in time.
|
|
</para>
|
|
<para>
|
|
This mode is suitable for all live streams. As with the incremental
|
|
download mode, buffering messages are emitted along with an indication
|
|
that timeshifting download is in progress.
|
|
</para>
|
|
</sect1>
|
|
|
|
<sect1 id="section-buffering-live">
|
|
<title>Live buffering </title>
|
|
<para>
|
|
In live pipelines we usually introduce some fixed latency between the
|
|
capture and the playback elements. This latency can be introduced by
|
|
a queue (such as a jitterbuffer) or by other means (in the audiosink).
|
|
</para>
|
|
<para>
|
|
Buffering messages can be emitted in those live pipelines as well and
|
|
serve as an indication to the user of the latency buffering. The
|
|
application usually does not react to these buffering messages with a
|
|
state change.
|
|
</para>
|
|
</sect1>
|
|
|
|
<sect1 id="section-buffering-strategies">
|
|
<title>Buffering strategies </title>
|
|
<para>
|
|
What follows are some ideas for implementing different buffering
|
|
strategies based on the buffering messages and buffering query.
|
|
</para>
|
|
|
|
<sect2 id="section-buffering-norebuffer">
|
|
<title>No-rebuffer strategy </title>
|
|
<para>
|
|
We would like to buffer enough data in the pipeline so that playback
|
|
continues without interruptions. What we need to know to implement
|
|
this is know the total remaining playback time in the file and the
|
|
total remaining download time. If the buffering time is less than the
|
|
playback time, we can start playback without interruptions.
|
|
</para>
|
|
<para>
|
|
We have all this information available with the DURATION, POSITION and
|
|
BUFFERING queries. We need to periodically execute the buffering query
|
|
to get the current buffering status. We also need to have a large
|
|
enough buffer to hold the complete file, worst case. It is best to
|
|
use this buffering strategy with download buffering (see
|
|
<xref linkend="section-buffering-download"/>).
|
|
</para>
|
|
<para>
|
|
This is what the code would look like:
|
|
</para>
|
|
<programlisting>
|
|
<!-- example-begin norebuffer.c a -->
|
|
<![CDATA[
|
|
#include <gst/gst.h>
|
|
|
|
GstState target_state;
|
|
static gboolean is_live;
|
|
static gboolean is_buffering;
|
|
|
|
static gboolean
|
|
buffer_timeout (gpointer data)
|
|
{
|
|
GstElement *pipeline = data;
|
|
GstQuery *query;
|
|
gboolean busy;
|
|
gint percent;
|
|
gint64 estimated_total;
|
|
gint64 position, duration;
|
|
guint64 play_left;
|
|
|
|
query = gst_query_new_buffering (GST_FORMAT_TIME);
|
|
|
|
if (!gst_element_query (pipeline, query))
|
|
return TRUE;
|
|
|
|
gst_query_parse_buffering_percent (query, &busy, &percent);
|
|
gst_query_parse_buffering_range (query, NULL, NULL, NULL, &estimated_total);
|
|
|
|
if (estimated_total == -1)
|
|
estimated_total = 0;
|
|
|
|
/* calculate the remaining playback time */
|
|
if (!gst_element_query_position (pipeline, GST_FORMAT_TIME, &position))
|
|
position = -1;
|
|
if (!gst_element_query_duration (pipeline, GST_FORMAT_TIME, &duration))
|
|
duration = -1;
|
|
|
|
if (duration != -1 && position != -1)
|
|
play_left = GST_TIME_AS_MSECONDS (duration - position);
|
|
else
|
|
play_left = 0;
|
|
|
|
g_message ("play_left %" G_GUINT64_FORMAT", estimated_total %" G_GUINT64_FORMAT
|
|
", percent %d", play_left, estimated_total, percent);
|
|
|
|
/* we are buffering or the estimated download time is bigger than the
|
|
* remaining playback time. We keep buffering. */
|
|
is_buffering = (busy || estimated_total * 1.1 > play_left);
|
|
|
|
if (!is_buffering)
|
|
gst_element_set_state (pipeline, target_state);
|
|
|
|
return is_buffering;
|
|
}
|
|
|
|
static void
|
|
on_message_buffering (GstBus *bus, GstMessage *message, gpointer user_data)
|
|
{
|
|
GstElement *pipeline = user_data;
|
|
gint percent;
|
|
|
|
/* no state management needed for live pipelines */
|
|
if (is_live)
|
|
return;
|
|
|
|
gst_message_parse_buffering (message, &percent);
|
|
|
|
if (percent < 100) {
|
|
/* buffering busy */
|
|
if (!is_buffering) {
|
|
is_buffering = TRUE;
|
|
if (target_state == GST_STATE_PLAYING) {
|
|
/* we were not buffering but PLAYING, PAUSE the pipeline. */
|
|
gst_element_set_state (pipeline, GST_STATE_PAUSED);
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
static void
|
|
on_message_async_done (GstBus *bus, GstMessage *message, gpointer user_data)
|
|
{
|
|
GstElement *pipeline = user_data;
|
|
|
|
if (!is_buffering)
|
|
gst_element_set_state (pipeline, target_state);
|
|
else
|
|
g_timeout_add (500, buffer_timeout, pipeline);
|
|
}
|
|
|
|
gint
|
|
main (gint argc,
|
|
gchar *argv[])
|
|
{
|
|
GstElement *pipeline;
|
|
GMainLoop *loop;
|
|
GstBus *bus;
|
|
GstStateChangeReturn ret;
|
|
|
|
/* init GStreamer */
|
|
gst_init (&argc, &argv);
|
|
loop = g_main_loop_new (NULL, FALSE);
|
|
|
|
/* make sure we have a URI */
|
|
if (argc != 2) {
|
|
g_print ("Usage: %s <URI>\n", argv[0]);
|
|
return -1;
|
|
}
|
|
|
|
/* set up */
|
|
pipeline = gst_element_factory_make ("playbin", "pipeline");
|
|
g_object_set (G_OBJECT (pipeline), "uri", argv[1], NULL);
|
|
g_object_set (G_OBJECT (pipeline), "flags", 0x697 , NULL);
|
|
|
|
bus = gst_pipeline_get_bus (GST_PIPELINE (pipeline));
|
|
gst_bus_add_signal_watch (bus);
|
|
|
|
g_signal_connect (bus, "message::buffering",
|
|
(GCallback) on_message_buffering, pipeline);
|
|
g_signal_connect (bus, "message::async-done",
|
|
(GCallback) on_message_async_done, pipeline);
|
|
gst_object_unref (bus);
|
|
|
|
is_buffering = FALSE;
|
|
target_state = GST_STATE_PLAYING;
|
|
ret = gst_element_set_state (pipeline, GST_STATE_PAUSED);
|
|
|
|
switch (ret) {
|
|
case GST_STATE_CHANGE_SUCCESS:
|
|
is_live = FALSE;
|
|
break;
|
|
|
|
case GST_STATE_CHANGE_FAILURE:
|
|
g_warning ("failed to PAUSE");
|
|
return -1;
|
|
|
|
case GST_STATE_CHANGE_NO_PREROLL:
|
|
is_live = TRUE;
|
|
break;
|
|
|
|
default:
|
|
break;
|
|
}
|
|
|
|
/* now run */
|
|
g_main_loop_run (loop);
|
|
|
|
/* also clean up */
|
|
gst_element_set_state (pipeline, GST_STATE_NULL);
|
|
gst_object_unref (GST_OBJECT (pipeline));
|
|
g_main_loop_unref (loop);
|
|
|
|
return 0;
|
|
}
|
|
]]>
|
|
<!-- example-end norebuffer.c a -->
|
|
</programlisting>
|
|
<para>
|
|
See how we set the pipeline to the PAUSED state first. We will receive
|
|
buffering messages during the preroll state when buffering is needed.
|
|
When we are prerolled (on_message_async_done) we see if buffering is
|
|
going on, if not, we start playback. If buffering was going on, we start
|
|
a timeout to poll the buffering state. If the estimated time to download
|
|
is less than the remaining playback time, we start playback.
|
|
</para>
|
|
</sect2>
|
|
</sect1>
|
|
</chapter>
|