mirror of
https://gitlab.freedesktop.org/gstreamer/gstreamer.git
synced 2024-07-02 04:35:55 +00:00
152 lines
5.5 KiB
Plaintext
152 lines
5.5 KiB
Plaintext
tsdemux/tsparse TODO
|
|
--------------------
|
|
|
|
* clock for live streams
|
|
In order for playback to happen at the same rate as on the producer,
|
|
we need to estimate the remote clock based on capture time and PCR
|
|
values.
|
|
For this estimation to be as accurate as possible, the capture time
|
|
needs to happen on the sources.
|
|
=> Ensure live sources actually timestamp their buffers
|
|
Once we have accurate timestamps, we can use an algorithm to
|
|
calculate the PCR/local-clock skew.
|
|
=> Use the EPTLA algorithm as used in -good/rtp/rtpmanager/
|
|
gstrtpjitterbuffer
|
|
|
|
* Seeking
|
|
=> Split out in a separate file/object. It is polluting tsdemux for
|
|
code readability/clarity.
|
|
|
|
* Perfomance : Creation/Destruction of buffers is slow
|
|
* => This is due to g_type_instance_create using a dogslow rwlock
|
|
which take up to 50% of gst_adapter_take_buffer()
|
|
=> Bugzilla #585375 (performance and contention problems)
|
|
|
|
* mpegtspacketizer
|
|
* offset/timestamp of incoming buffers need to be carried on to the
|
|
sub-buffers in order for several demuxer features to work correctly.
|
|
|
|
* mpegtsparser
|
|
* SERIOUS room for improvement performance-wise (see callgrind)
|
|
|
|
|
|
|
|
|
|
Synchronization, Scheduling and Timestamping
|
|
--------------------------------------------
|
|
|
|
A mpeg-ts demuxer can be used in a variety of situations:
|
|
* lives streaming over DVB, UDP, RTP,..
|
|
* play-as-you-download like HTTP Live Streaming or UPNP/DLNA
|
|
* random-access local playback, file, Bluray, ...
|
|
|
|
Those use-cases can be categorized in 3 different categories:
|
|
* Push-based scheduling with live sources [0]
|
|
* Push-based scheduling with non-live sources
|
|
* Pull-based scheduling with fast random-access
|
|
|
|
Due to the nature of timing within the mpeg-ts format, we need to
|
|
pay extra attention to the outgoing NEWSEGMENT event and buffer
|
|
timestamps in order to guarantee proper playback and synchronization
|
|
of the stream.
|
|
|
|
|
|
1) Live push-based scheduling
|
|
|
|
The NEWSEGMENT event will be in time format and is forwarded as is,
|
|
and the values are cached locally.
|
|
|
|
Since the clock is running when the upstream buffers are captured,
|
|
the outgoing buffer timestamps need to correspond to the incoming
|
|
buffer timestamp values.
|
|
|
|
=> A delta, DTS_delta between incoming buffer timestamp and
|
|
DTS/PTS needs to be computed.
|
|
|
|
=> The outgoing buffers will be timestamped with their PTS values
|
|
(overflow corrected) offseted by that initial DTS_delta.
|
|
|
|
A latency is introduced between the time the buffer containing the
|
|
first bit of a Access Unit is received in the demuxer and the moment
|
|
the demuxer pushed out the buffer corresponding to that Access Unit.
|
|
|
|
=> That latency needs to be reported. It corresponds to the
|
|
biggest Access Unit spacing, in this case 1/video-framerate.
|
|
|
|
According to the ISO/IEC 13818-1:2007 specifications, D.0.1 Timing
|
|
mode, the "coded audio and video that represent sound and pictures
|
|
that are to be presented simultaneously may be separated in time
|
|
within the coded bit stream by ==>as much as one second<=="
|
|
|
|
=> The demuxer will therefore report an added latency of 1s to
|
|
handle this interleave.
|
|
|
|
|
|
2) Non-live push-based scheduling
|
|
|
|
If the upstream NEWSEGMENT is in time format, the NEWSEGMENT event
|
|
is forwarded as is, and the values are cached locally.
|
|
|
|
If upstream does provide a NEWSEGMENT in another format, we need to
|
|
compute one by taking the default values:
|
|
start : 0
|
|
stop : GST_CLOCK_TIME_NONE
|
|
time : 0
|
|
|
|
Since no prerolling is happening downstream and the incoming buffers
|
|
do not have capture timestamps, we need to ensure the first buffer
|
|
we push out corresponds to the base segment start runing time.
|
|
|
|
=> A delta between the first DTS to output and the segment start
|
|
position needs to be computed.
|
|
|
|
=> The outgoing buffers will be timestamped with their PTS values
|
|
(overflow corrected) offseted by that initial delta.
|
|
|
|
Latency is reported just as with the live use-case.
|
|
|
|
|
|
3) Random access pull-mode
|
|
|
|
We do not get a NEWSEGMENT event from upstream, we therefore need to
|
|
compute the outgoing values.
|
|
|
|
The base stream/running time corresponds to the DTS of the first
|
|
buffer we will output. The DTS_delta becomes that earliest DTS.
|
|
|
|
=> FILLME
|
|
|
|
X) General notes
|
|
|
|
It is assumed that PTS/DTS rollovers are detected and corrected such
|
|
as the outgoing timestamps never rollover. This can be easily
|
|
handled by correcting the DTS_delta when such rollovers are
|
|
detected. The maximum value of a GstClockTimeDiff is almost 3
|
|
centuries, we therefore have enough margin to handle a decent number
|
|
of rollovers.
|
|
|
|
The generic equation for calculating outgoing buffer timestamps
|
|
therefore becomes:
|
|
|
|
D = DTS_delta, with rollover corrections
|
|
PTS = PTS of the buffer we are going to push out
|
|
TS = Timestamp of the outgoing buffer
|
|
|
|
==> TS = PTS + D
|
|
|
|
If seeking is handled upstream for push-based cases, whether live or
|
|
not, no extra modification is required.
|
|
|
|
If seeking is handled by the demuxer in the non-live push-based
|
|
cases (converting from TIME to BYTES), the demuxer will need to
|
|
set the segment start/time values to the requested seek position.
|
|
The DTS_delta will also have to be recomputed to take into account
|
|
the seek position.
|
|
|
|
|
|
[0] When talking about live sources, we mean this in the GStreamer
|
|
definition of live sources, which is to say sources where if we miss
|
|
the capture, we will miss the data to be captured. Sources which do
|
|
internal buffering (like TCP connections or file descriptors) are
|
|
*NOT* live sources.
|