Commit graph

348 commits

Author SHA1 Message Date
Edward Hervey
dd425fe0fd adaptivedemux: Store QoS values on the element
Storing it per-stream requires taking the manifest lock which can apparenly be
hold for aeons. And since the QoS event comes from the video rendering thread
we *really* do not want to do that.

Storing it as-is in the element is fine, the important part is knowing the
earliest time downstream.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1021>
2020-11-11 20:18:11 +00:00
Stéphane Cerveau
ac74b042ec dash: Generate an XML content from object.
Add mpd node base class to provide
xml generation facilities for child
objects.
2020-01-03 20:50:27 +00:00
Stéphane Cerveau
c28e7d928d dash: move parser nodes/types to separated files
Rename GstMpdClient to GstMPDClient and use GObject model.

Move nodes to file from gstmpdparser.c:
- GstMPDRootNode
- GstMPDBaseURLNode
- GstMPDUTCTimingNode
- GstMPDMetricsNode
- GstMPDMetricsRangeNode
- GstMPDSNode
- GstMPDSegmentTimelineNode
- GstSegmentTemplateNode
- GstMPDSegmentURLNode
- GstMPDSegmentListNode
- GstMPDPeriodNode
- GstMPDRepresentationNode
- GstMPDsubRepresentationNode
- GstMPDAdaptationSetNode
- GstMPDContentComponentNode
- GstMPDSubsetNode
- GstMPDProgramInformationNode

Move types to gstmpdhelper from gstmpdparser.c:

- GstURLType
- GstDescriptorType
- GstSegmentBaseType
- GstMPDMultSegmentBaseType
- GstMPDRepresentationBaseType

Cleanup naming when possible.
2019-12-05 09:06:37 +00:00
Stéphane Cerveau
86b251b7d1 dash: split mpdparser, mpdclient and xmlhelper
provide a separate namespace for mpd helper
for xml parsing and the real mpd parsing.
2019-12-05 09:06:37 +00:00
Aaron Boxer
6d3429af34 documentation: fixed a heap o' typos 2019-11-05 09:11:25 -05:00
Nicolas Dufresne
694d9dcde5 dashdemux: Always create the adapter
Worst case it will be empty. This fixes a crash when the base class
calls data_received() when the stream is neither is_isobmff or
has_isoff_ondemand_profile.

https://bugzilla.gnome.org/show_bug.cgi?id=796745
2018-08-16 15:59:42 -04:00
Seungha Yang
9956f22537 dashdemux: Fix sync of updated manifest from previous one
_get_next_fragment_timestamp() returns relative timestamp to period start.
But gst_mpd_client_stream_seek() uses absolute MPD timeline.

https://bugzilla.gnome.org/show_bug.cgi?id=781183
2018-05-12 09:27:46 +02:00
Thiago Santos
ead63a2686 dashdemux: update manifest streams correctly if pads aren't exposed
In some cases, it is possible that we need to update the manifest before
pads have been exposed at all. If there are no current pads, just expose
the next prepared streams. This doesn't handle the case where a manifest
update would happen while a live streams is changing periods, which is a
type of use case that we're unaware of real usages yet.

https://bugzilla.gnome.org/show_bug.cgi?id=783028
2017-07-13 07:42:58 +02:00
Edward Hervey
2b401de91c dashdemux: Implement a default presentation delay
Have a fallback presentation delay is great (if not present in the
manifest), having an actual default value is better.

https://bugzilla.gnome.org/show_bug.cgi?id=783244
2017-06-01 15:27:07 +02:00
Edward Hervey
34d8ddd92d dashdemux: Remove wrong assertion
This is wrong because:
* If the rate is negative we should check for the *previous* period
* adaptivedemux already does the proper checks before calling this
  method
2017-05-31 11:28:08 +02:00
Sebastian Dröge
be465a139b dashdemux: Use correct (sub) fragment start when clipping chunk size
If we didn't download anything yet, we shouldn't use fragment.start but
the start position of the current sidx entry.
2017-05-18 19:04:57 +02:00
Sebastian Dröge
d5eee31918 dashdemux: Make sure to pass the next buffer after a seek with DISCONT flag
When we manually seek for skipping ahead in keyunit-only trickmode, we
have to enforce that ourselves as adaptivedemux does not know about the
seeks.
2017-05-18 19:04:57 +02:00
Sebastian Dröge
d884ec2904 dashdemux: Ensure that we never ever download over the current SIDX entry
Even if downloading more than needed to try to get the moof and first
keyframe both together.
2017-05-18 19:04:57 +02:00
Sebastian Dröge
71259d9ad4 dashdemux: Take SIDX entry timestamp/duration into account for keyunit-only mode
And not the whole segment's timestamp/duration
2017-05-18 19:04:57 +02:00
Sebastian Dröge
c3fd905f11 dashdemux: Reset parsing state correctly in various places 2017-05-18 19:04:57 +02:00
Sebastian Dröge
927c158303 dashdemux: Reset previous target_time when seeking 2017-05-18 19:04:57 +02:00
Sebastian Dröge
e2a6573c7d dashdemux: Smooth out skip distances in keyframe-only trick modes
This ensures smoother playback. It looks weird if we first do a big
jump, then play a couple of consecutive frames, just to again skip ahead
quite a bit because we ran late again.
2017-05-18 19:04:57 +02:00
Sebastian Dröge
ddef518701 dashdemux: Take configure max bitrate/framerate into account for keyframe skipping
And by default only produce up to 10 fps.
2017-05-18 19:04:57 +02:00
Sebastian Dröge
6a037a78de dashdemux: Try harder to not run into a loop over the same fragment over and over
While still making sure to not jump ahead one fragment further than
needed.
2017-05-18 19:04:57 +02:00
Sebastian Dröge
ef8f194855 dashdemux: Add some debug output to the target time selection 2017-05-18 19:04:57 +02:00
Sebastian Dröge
6b6c7382c3 dashdemux: Use the current clock running time in addition to the QoS earliest time 2017-05-18 19:04:57 +02:00
Sebastian Dröge
133d1e86bd dashdemux: Download keyframes from the current position if we're far enough ahead
Far enough here means more than 500ms or 4 times the average keyframe
download time. There is no need to jump ahead by one average keyframe
download time in this case.

This makes playback smooth if the network is fast enough.
2017-05-18 19:04:57 +02:00
Sebastian Dröge
4efa3185bb dashdemux: Fix sync sample selection based on target time in reverse playback mode 2017-05-18 19:04:57 +02:00
Sebastian Dröge
b3e2657ac1 dashdemux: Snap-seek for skipping ahead and use actual keyframe distance in the current fragment if applicable 2017-05-18 19:04:57 +02:00
Sebastian Dröge
b7f87e2518 dashdemux: Don't adjust for fragment duration twice when seeking ahead in KEYUNITS mode 2017-05-18 19:04:57 +02:00
Sebastian Dröge
5d09bc04a2 dashdemux: Add current fragment duration in reverse playback mode to the position
We play from the end of the fragment to the beginning, not from the
beginning backwards.
2017-05-18 19:04:57 +02:00
Sebastian Dröge
9267b4a725 dashdemux: Never take more than the current fragment duration for estimations inside the fragment
Taking the average might give us from results.
2017-05-18 19:04:57 +02:00
Sebastian Dröge
df420cc21f dashdemux: Don't increase current position if we just downloaded the moof in KEYUNIT mode 2017-05-18 19:04:57 +02:00
Edward Hervey
245d73e516 dashdemux: Improve key-unit trick mode downloading
When dealing with key-unit trick mode downloads, the goal is to
provide the best "Quality of Experience". This is achieved by:
1) maximizing the number of frames displayed per second
2) avoiding "stalling" as much as possible (i.e. not downloading and
  decoding frames fast enough)

This implementation achives this by:
1) Knowing very precisely the current keyframe being download (i.e
  more accurate than at the fragment level which might contain more
  than one keyfram). This is the new "actual_position" variable
  introduced by this commit
2) Knowing the position of downstream (provided by QoS and stored
  in the adaptivedemuxstream qos_earliest_time variable)
3) Knowing how long it takes to request and fully download a keyframe
  (the average_download_time variable)

Taking those 3 variables into account, whenever a keyframe has been
pushed downstream we calculate a "target time" (target_time variable)
which is the ideal next keyframe time to request so that:
1) It will be requested/downloaded/demuxed/decoded in time to be
  displayed without being too late
2) It will not be too far ahead that it would cause too few frames
  per second to be displayed.

How far ahead we will request is inversily proportional to how close
the actual position (actual_position) is from the downstream
position (qos_earliest_time). The more is buffered between the source
and the sink, the "closer" the target time will be, and therefore
the more frames per seconds will be displayed (up to the limit
of keyframes_per_second * absolute_rate).
2017-05-18 19:04:57 +02:00
Edward Hervey
04224ad440 dashdemux: Store average download time
This will be used to bound the download rate when working in
keyframe-only trick mode
2017-05-18 19:04:57 +02:00
Sebastian Dröge
2379dc3f5b dashdemux: Collect keyunit distance and size statistics 2017-05-18 19:04:57 +02:00
Edward Hervey
d8d1fc8bc4 dashdemux: Disable bitrate switching in key-unit trick mode
This creates too much havoc for now
2017-05-18 19:04:57 +02:00
Edward Hervey
8f400c1343 dashdemux: Use new adaptivedemux trickmode macro
Reduces the lines of code, and makes it a bit more readable
2017-05-18 19:04:57 +02:00
Seungha Yang
1c6cd54477 dashdemux: Initialize sidx_position to GST_CLOCK_TIME_NONE
If a manifest has non-zero presentation time offset
(i.e., earliest presentation time specified by sidx box is not zero),
the initial sidx position shouldn't be zero. Since we cannot define
exact sidx position until parsing sidx box, set the value to unknown.

https://bugzilla.gnome.org/show_bug.cgi?id=782693
2017-05-16 17:44:40 +03:00
Sebastian Dröge
db57a3b04f dashdemux: Rewrite ISOBMFF & SIDX handling
The previous code was handling both as separate steps and then tried to
combine the results, but this resulted in all kinds of bugs which showed
themselves as failures during seeking and offset tracking getting wrong.
This also showed itself with gst-validate on the sample stream.

The rewritten code now parses everything in one go and tracks the
current offset only once, and as a side effect simplifies the code a
lot.

Also added is detection of SIDX that point to other SIDX instead of
actual media segments, e.g. with this stream:
  http://dash.akamaized.net/dash264/TestCases/1a/sony/SNE_DASH_SD_CASE1A_REVISED.mpd
Support for this will have to be added at some point but that should
also be easier with the rewritten code.

https://bugzilla.gnome.org/show_bug.cgi?id=781233
2017-04-18 13:49:08 +02:00
Sebastian Dröge
e0e1db212f Revert "dashdemux: Fix issue when manifest update sets slow start without passing necessary header & caps changes downstream"
This reverts commit c9fbf3459a.

The representation ID comparision here was wrong and triggering always
if the ID did *not* change, causing needless redownloading of the
header. The sample stream provided in the bug does not exist anymore.
2017-04-12 20:11:16 +03:00
Thibault Saunier
78022a6e0c docs: Port all docstring to gtk-doc markdown 2017-04-12 12:57:57 -03:00
Sebastian Dröge
67778ac55e dashdemux: Handle current SIDX before current position in normal mode like in keyunit-only trick mode
Otherwise we'll get into an infinite loop here. Now this is still not
correct and will cause a clean error, but at least it won't hang forever
anymore.
2017-04-10 18:46:01 +03:00
Seungha Yang
41996ad9c0 dashdemux: Fix SEEK of multi-period On-Demand profile
For each period, media presentation is the relative to the
period-start time. So SIDX seek position should be target seek
position minus period-start. Also, if presentationTimeOffset
is defined, the value should be compensated

https://bugzilla.gnome.org/show_bug.cgi?id=780397
2017-04-09 10:54:01 +03:00
Luis de Bethencourt
2be93e47ec dashdemux: fix typo in implementation notes
s/enveloppe/envelope
2017-03-28 17:08:15 +02:00
Sebastian Dröge
b8b2946e03 dashdemux: In reverse playback mode, if seeking to the beginning of a fragment, start with the previous one instead
There's no point to start downloading a fragment just to output 1ns.
2017-03-22 12:06:33 +02:00
Sebastian Dröge
0cc777b3b0 dashdemux: Don't use ABS() for calculating differences
We know which number is bigger.
2017-03-21 20:38:09 +02:00
Sebastian Dröge
ffa63c5933 dashdemux: Fix snap SIDX seeking and report if we're going outside the index
Instead of just going to the first or last fragment, report if we're
going outside the index. This should never happen unless there's a bug
or the stream is broken.

Allow some possibility for inaccuracies here though.
2017-03-21 20:38:09 +02:00
Sebastian Dröge
7e21168d4d dashdemux: Just reset SIDX parser if the index is invalid for the current segment
https://bugzilla.gnome.org/show_bug.cgi?id=780108
2017-03-16 12:35:40 +02:00
Sebastian Dröge
141f3b2f4d dashdemux: If a SIDX seek failed although we're in the correct segment, disable the SIDX usage for this segment
The SIDX apparently does not contain information about the current
segment, so better stop using it instead of using incorrect values.
2017-03-15 13:27:23 +02:00
Sebastian Dröge
b76a2200de dashdemux: Keep track of the PTS, not the SIDX index position between representations
There is no guarantee that the index positions are the same between
representations, and assuming this easily causes us to get into invalid
index positions.
2017-03-15 13:27:23 +02:00
Seungha Yang
644192aafe dashdemux: Drain buffer at the end of subfragment
Some of streams such as below have tailing boxes at the end of subfragment.
http://dash.akamaized.net/dash264/TestCases/1a/netflix/exMPD_BIP_TC1.mpd

https://bugzilla.gnome.org/show_bug.cgi?id=776200
2017-03-15 13:27:23 +02:00
Seungha Yang
42a38b6498 dashdemux: Do not advace subfragment after pending SIDX seek done
If a MPD is On-Demand profile and no index described, demux will terminate
download loop after parsing inband SIDX with flow return custom-success.
At this moment, SIDX index is excat target position, but finish_fragment()
might cause re-advancing subfragment depending on MPD structure.

https://bugzilla.gnome.org/show_bug.cgi?id=776200
2017-03-15 13:27:23 +02:00
Seungha Yang
b65e6db8d4 dashdemux: Reset SIDX related variables per fragment
SIDX's base offset (i.e., byte offset of SIDX + sidx.first_offset)
mostly vary as per fragment. Also, target SIDX index must be zero for the
new fragment.

https://bugzilla.gnome.org/show_bug.cgi?id=776200
2017-03-15 13:27:23 +02:00
Seungha Yang
a599067786 dashdemux: Two depth seeking for On-Demand profile
Try to find fragment using MPD first, then do refinement to find
target subframgnet using SIDX if possible. Note that, if target fragment
was moved from the previously activated one, we should assume that
the last SIDX is invalid for new fragment.

https://bugzilla.gnome.org/show_bug.cgi?id=776200
2017-03-15 13:27:23 +02:00