mirrors/gstreamer
mirrors/gstreamer
1
1
Fork 0
mirror of https://gitlab.freedesktop.org/gstreamer/gstreamer.git synced 2025-04-26 06:54:49 +00:00
Code Issues 1 Projects Releases Wiki Activity

application-development: clocks: improve format & content

Browse source
This commit is contained in:
Reynaldo H. Verdejo Pinochet 2017-09-07 15:30:38 -07:00
parent 68e470dd67
commit b71e9787b7
1 changed files with 38 additions and 35 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

73
markdown/application-development/advanced/clocks.md
View file

@ -21,7 +21,7 @@ GStreamer provides support for the following use cases:
- Streaming from (slow) network streams with buffering. This is the - Streaming from (slow) network streams with buffering. This is the
typical web streaming case where you access content from a streaming typical web streaming case where you access content from a streaming
server with http. server using HTTP.
- Capture from live source and and playback to live source with - Capture from live source and and playback to live source with
configurable latency. This is used when, for example, capture from a configurable latency. This is used when, for example, capture from a
@ -33,33 +33,36 @@ GStreamer provides support for the following use cases:
recorded audio and record new samples, the purpose is to have the recorded audio and record new samples, the purpose is to have the
new audio perfectly in sync with the previously recorded data. new audio perfectly in sync with the previously recorded data.
GStreamer uses a `GstClock` object, buffer timestamps and a SEGMENT GStreamer uses a `GstClock` object, buffer timestamps and a `SEGMENT`
event to synchronize streams in a pipeline as we will see in the next event to synchronize streams in a pipeline as we will see in the next
sections. sections.
## Clock running-time ## Clock running-time
In a typical computer, there are many sources that can be used as a time In a typical computer, there are many sources that can be used as a time
source, e.g., the system time, soundcards, CPU performance counters, ... source, e.g., the system time, soundcards, CPU performance counters, etc.
For this reason, there are many `GstClock` implementations available in For this reason, GStreamer has many `GstClock` implementations available.
GStreamer. The clock time doesn't always start from 0 or from some known Note that clock time doesn't have to start from 0 or any other known
value. Some clocks start counting from some known start date, other value. Some clocks start counting from a particular start date, others
clocks start counting since last reboot, etc... from the last reboot, etc.
A `GstClock` returns the **absolute-time** according to that clock with A `GstClock` returns the **absolute-time** according to that clock with
`gst_clock_get_time ()`. The absolute-time (or clock time) of a clock is `gst_clock_get_time ()`. The **absolute-time** (or clock time) of a clock
monotonically increasing. From the absolute-time is a **running-time** is monotonically increasing.
calculated, which is simply the difference between a previous snapshot
of the absolute-time called the **base-time**. So:
A **running-time** is the difference between a previous snapshot of the
**absolute-time** called the **base-time**, and any other **absolute-time**.
```
running-time = absolute-time - base-time running-time = absolute-time - base-time
```
A GStreamer `GstPipeline` object maintains a `GstClock` object and a A GStreamer `GstPipeline` object maintains a `GstClock` object and a
base-time when it goes to the PLAYING state. The pipeline gives a handle base-time when it goes to the `PLAYING` state. The pipeline gives a handle
to the selected `GstClock` to each element in the pipeline along with to the selected `GstClock` to each element in the pipeline along with
selected base-time. The pipeline will select a base-time in such a way selected base-time. The pipeline will select a base-time in such a way
that the running-time reflects the total time spent in the PLAYING that the running-time reflects the total time spent in the `PLAYING`
state. As a result, when the pipeline is PAUSED, the running-time stands state. As a result, when the pipeline is `PAUSED`, the running-time stands
still. still.
Because all objects in the pipeline have the same clock and base-time, Because all objects in the pipeline have the same clock and base-time,
@ -69,16 +72,17 @@ clock.
## Buffer running-time ## Buffer running-time
To calculate a buffer running-time, we need a buffer timestamp and the To calculate a buffer running-time, we need a buffer timestamp and the
SEGMENT event that preceeded the buffer. First we can convert the `SEGMENT` event that preceeded the buffer. First we can convert the
SEGMENT event into a `GstSegment` object and then we can use the `SEGMENT` event into a `GstSegment` object and then we can use the
`gst_segment_to_running_time ()` function to perform the calculation of `gst_segment_to_running_time ()` function to perform the calculation of
the buffer running-time. the buffer running-time.
Synchronization is now a matter of making sure that a buffer with a Synchronization is now a matter of making sure that a buffer with a
certain running-time is played when the clock reaches the same certain running-time is played when the clock reaches the same
running-time. Usually this task is done by sink elements. Sink also have running-time. Usually, this task is performed by sink elements. These
to take into account the latency configured in the pipeline and add this elements also have to take into account the configured pipeline's latency
to the buffer running-time before synchronizing to the pipeline clock. and add it to the buffer running-time before synchronizing to the pipeline
clock.
Non-live sources timestamp buffers with a running-time starting from 0. Non-live sources timestamp buffers with a running-time starting from 0.
After a flushing seek, they will produce buffers again from a After a flushing seek, they will produce buffers again from a
@ -90,13 +94,12 @@ pipeline running-time when the first byte of the buffer was captured.
## Buffer stream-time ## Buffer stream-time
The buffer stream-time, also known as the position in the stream, is The buffer stream-time, also known as the position in the stream, is
calculated from the buffer timestamps and the preceding SEGMENT event. a value between 0 and the total duration of the media and it's calculated
It represents the time inside the media as a value between 0 and the from the buffer timestamps and the preceding `SEGMENT` event.
total duration of the media.
The stream-time is used in: The stream-time is used in:
- Report the current position in the stream with the POSITION query. - Report the current position in the stream with the `POSITION` query.
- The position used in the seek events and queries. - The position used in the seek events and queries.
@ -124,8 +127,8 @@ repeating.
A clock provider is an element in the pipeline that can provide a A clock provider is an element in the pipeline that can provide a
`GstClock` object. The clock object needs to report an absolute-time `GstClock` object. The clock object needs to report an absolute-time
that is monotonically increasing when the element is in the PLAYING that is monotonically increasing when the element is in the `PLAYING`
state. It is allowed to pause the clock while the element is PAUSED. state. It is allowed to pause the clock while the element is `PAUSED`.
Clock providers exist because they play back media at some rate, and Clock providers exist because they play back media at some rate, and
this rate is not necessarily the same as the system clock rate. For this rate is not necessarily the same as the system clock rate. For
@ -146,7 +149,7 @@ schedule playback. This can be both faster and more accurate. Therefore,
generally, elements with an internal clock like audio input or output generally, elements with an internal clock like audio input or output
devices will be a clock provider for the pipeline. devices will be a clock provider for the pipeline.
When the pipeline goes to the PLAYING state, it will go over all When the pipeline goes to the `PLAYING` state, it will go over all
elements in the pipeline from sink to source and ask each element if elements in the pipeline from sink to source and ask each element if
they can provide a clock. The last element that can provide a clock will they can provide a clock. The last element that can provide a clock will
be used as the clock provider in the pipeline. This algorithm prefers a be used as the clock provider in the pipeline. This algorithm prefers a
@ -155,9 +158,9 @@ source elements in a typical capture pipeline.
There exist some bus messages to let you know about the clock and clock There exist some bus messages to let you know about the clock and clock
providers in the pipeline. You can see what clock is selected in the providers in the pipeline. You can see what clock is selected in the
pipeline by looking at the NEW\_CLOCK message on the bus. When a clock pipeline by looking at the `NEW_CLOCK` message on the bus. When a clock
provider is removed from the pipeline, a CLOCK\_LOST message is posted provider is removed from the pipeline, a `CLOCK_LOST` message is posted
and the application should go to PAUSED and back to PLAYING to select a and the application should go to `PAUSED` and back to `PLAYING` to select a
new clock. new clock.
## Latency ## Latency
@ -173,19 +176,19 @@ of the way a live source works. Consider an audio source, it will start
capturing the first sample at time 0. If the source pushes buffers with capturing the first sample at time 0. If the source pushes buffers with
44100 samples at a time at 44100Hz it will have collected the buffer at 44100 samples at a time at 44100Hz it will have collected the buffer at
second 1. Since the timestamp of the buffer is 0 and the time of the second 1. Since the timestamp of the buffer is 0 and the time of the
clock is now \>= 1 second, the sink will drop this buffer because it is clock is now `>= 1` second, the sink will drop this buffer because it is
too late. Without any latency compensation in the sink, all buffers will too late. Without any latency compensation in the sink, all buffers will
be dropped. be dropped.
### Latency compensation ### Latency compensation
Before the pipeline goes to the PLAYING state, it will, in addition to Before the pipeline goes to the `PLAYING` state, it will, in addition to
selecting a clock and calculating a base-time, calculate the latency in selecting a clock and calculating a base-time, calculate the latency in
the pipeline. It does this by doing a LATENCY query on all the sinks in the pipeline. It does this by doing a `LATENCY` query on all the sinks in
the pipeline. The pipeline then selects the maximum latency in the the pipeline. The pipeline then selects the maximum latency in the
pipeline and configures this with a LATENCY event. pipeline and configures this with a `LATENCY` event.
All sink elements will delay playback by the value in the LATENCY event. All sink elements will delay playback by the value in the `LATENCY` event.
Since all sinks delay with the same amount of time, they will be Since all sinks delay with the same amount of time, they will be
relative in sync. relative in sync.
@ -193,7 +196,7 @@ relative in sync.
Adding/removing elements to/from a pipeline or changing element Adding/removing elements to/from a pipeline or changing element
properties can change the latency in a pipeline. An element can request properties can change the latency in a pipeline. An element can request
a latency change in the pipeline by posting a LATENCY message on the a latency change in the pipeline by posting a `LATENCY` message on the
bus. The application can then decide to query and redistribute a new bus. The application can then decide to query and redistribute a new
latency or not. Changing the latency in a pipeline might cause visual or latency or not. Changing the latency in a pipeline might cause visual or
audible glitches and should therefore only be done by the application audible glitches and should therefore only be done by the application