When dealing with key-unit trick mode downloads, the goal is to
provide the best "Quality of Experience". This is achieved by:
1) maximizing the number of frames displayed per second
2) avoiding "stalling" as much as possible (i.e. not downloading and
decoding frames fast enough)
This implementation achives this by:
1) Knowing very precisely the current keyframe being download (i.e
more accurate than at the fragment level which might contain more
than one keyfram). This is the new "actual_position" variable
introduced by this commit
2) Knowing the position of downstream (provided by QoS and stored
in the adaptivedemuxstream qos_earliest_time variable)
3) Knowing how long it takes to request and fully download a keyframe
(the average_download_time variable)
Taking those 3 variables into account, whenever a keyframe has been
pushed downstream we calculate a "target time" (target_time variable)
which is the ideal next keyframe time to request so that:
1) It will be requested/downloaded/demuxed/decoded in time to be
displayed without being too late
2) It will not be too far ahead that it would cause too few frames
per second to be displayed.
How far ahead we will request is inversily proportional to how close
the actual position (actual_position) is from the downstream
position (qos_earliest_time). The more is buffered between the source
and the sink, the "closer" the target time will be, and therefore
the more frames per seconds will be displayed (up to the limit
of keyframes_per_second * absolute_rate).
If a manifest has non-zero presentation time offset
(i.e., earliest presentation time specified by sidx box is not zero),
the initial sidx position shouldn't be zero. Since we cannot define
exact sidx position until parsing sidx box, set the value to unknown.
https://bugzilla.gnome.org/show_bug.cgi?id=782693
This embeds the muxer inside the sink and accepts elementary streams
while the old HLS sink required the muxer outside. Apart from that the
interface is the same as before.
Currently only mpegtsmux is supported, but support for other muxers is
just a matter of adding a property.
The advantage of the new sink is that it reduces complexity a lot and
properly handles pre-encoded streams with appropriately spaced
keyframes.
https://bugzilla.gnome.org/show_bug.cgi?id=781496
This patch bumps the required meson to 0.40.1 as gstreamer core just
did, and cleanup some code to use a feature from 0.37 that allow
specifying version range when checking dependency.
https://bugzilla.gnome.org/show_bug.cgi?id=780654
A common subtitling use case is live-generated subtitles, in which each
new word is contained in its own span, and the spans are displayed
sequentially, with the effect that lines of displayed subtitles are
built up word-by-word.
This can, however, cause problems when the number of words in a block is
greater than the number of allowed GstMemorys in a GstBuffer.
Since in this use case each span will have the same styling as adjacent
spans, we can join adjacent spans (and other inline elements, such as
breaks) into a single element containing the concatenated text of each,
thus avoiding the limit of GstMemorys in a GstBuffer and also reducing
the amount of styling/layout metadata that is attached to each buffer.
https://bugzilla.gnome.org/show_bug.cgi?id=781725
The parser stores the text from each inline element of a scene in its
own GstMemory, which is inserted in the GstBuffer containing the scene
data. However, GstBuffers can contain only a limited number of
GstMemorys. Therefore, don't add more than the maximum number of
GstMemorys to each buffer, and warn if this is attempted.
https://bugzilla.gnome.org/show_bug.cgi?id=781725
When parsing <br> elements, store an actual newline in the text field of
the created TtmlElement. They then don't need to be treated as a
separate case from anon-span elements when being processed.
https://bugzilla.gnome.org/show_bug.cgi?id=781725
Encapsulates in a function the code that warns of an illegally
positioned element, rather than repeating the same code multiple times.
Also frees a string allocated by ttml_get_element_type_string, which was
previously being leaked.
https://bugzilla.gnome.org/show_bug.cgi?id=781725
../subprojects/gst-plugins-bad/ext/smoothstreaming/gstmssdemux.c: In function ‘gst_mss_demux_requires_periodical_playlist_update’:
../subprojects/gst-plugins-bad/ext/smoothstreaming/gstmssdemux.c:729:16: error: unused variable ‘mssdemux’ [-Werror=unused-variable]
GstMssDemux *mssdemux = GST_MSS_DEMUX_CAST (demux);
^~~~~~~~
cc1: all warnings being treated as errors
Without this, for streams where the content is stored indefinitely and
can be seeked on, the duration would never increase when in paused or,
until we reached near the end of the currently advertised stream (where
the internal fragment parser would see descriptions of new fragments).
The TTML spec has an issue in which tab (U+0009) characters that are
first in a sequence of whitespace characters are not suppressed at the
start and end of line areas. This issue was reported in [1] and the
editor of the TTML specs confirmed that this was not the intention
behind the spec.
The editor has created an issue to fix this in both the TTML1 and TTML2
specs [2], giving a proposal of what the spec should say. This patch
updates ttmlparse to implement the intended behaviour as proposed, in
which tabs in the input are converted to spaces before processing.
[1] https://github.com/w3c/imsc/issues/224
[2] https://github.com/w3c/ttml1/issues/235https://bugzilla.gnome.org/show_bug.cgi?id=781539
If multiple styles/regions with the same ID are present in the input
(which is not allowed in TTML), use the last and give a warning.
Fixes CID #1405134.
Clang's static analyser found potential code paths in which variables
were being used in comparisons when uninitialised. Fix by properly
handling out-of-range value returned by gst_ttml_get_element_index.
The previous code was handling both as separate steps and then tried to
combine the results, but this resulted in all kinds of bugs which showed
themselves as failures during seeking and offset tracking getting wrong.
This also showed itself with gst-validate on the sample stream.
The rewritten code now parses everything in one go and tracks the
current offset only once, and as a side effect simplifies the code a
lot.
Also added is detection of SIDX that point to other SIDX instead of
actual media segments, e.g. with this stream:
http://dash.akamaized.net/dash264/TestCases/1a/sony/SNE_DASH_SD_CASE1A_REVISED.mpd
Support for this will have to be added at some point but that should
also be easier with the rewritten code.
https://bugzilla.gnome.org/show_bug.cgi?id=781233
Spec 5.3.9.2 is saying about the existence of duration and SegmentTimeline
only for Representation level. Other level such as Period or AdaptationSet
might not have the attributes.
https://bugzilla.gnome.org/show_bug.cgi?id=780570
Allow 1 extra char in the tmp buffer where the motion cell
snippets are generated, so that it doesn't leave off a comma
when dealing with cells that have 2 numerals in both indices
Don't hide build behind --enable-experimental. Our goal is to not
autoplug it for now, so let's just always build it if the dependencies
are there and hide autoplugging enablement behind an env var.
This reverts commit c9fbf3459a.
The representation ID comparision here was wrong and triggering always
if the ID did *not* change, causing needless redownloading of the
header. The sample stream provided in the bug does not exist anymore.
Otherwise we'll get into an infinite loop here. Now this is still not
correct and will cause a clean error, but at least it won't hang forever
anymore.
For each period, media presentation is the relative to the
period-start time. So SIDX seek position should be target seek
position minus period-start. Also, if presentationTimeOffset
is defined, the value should be compensated
https://bugzilla.gnome.org/show_bug.cgi?id=780397
Significant whitespace in elements that don't have begin/end values
should inherit timing from its parent, or if no its parents have no
timing, from the document's Root Temporal Extent. Currently, such
whitespace is removed, which is not spec-compliant. Fix this by
retaining whitespace in content nodes, and assigning a Root Temporal
Extent of 24 hours to any significant whitespace whose parents have no
associated timing.
https://bugzilla.gnome.org/show_bug.cgi?id=781027
The specified behaviour in TTML when lineHeight is "normal" is different
from the behaviour when a percentage is given. In the former case, the
line height is a percentage (the TTML spec recommends 125%) of the largest
font size that is applied to the spans within the block; in the latter
case, the line height is the given percentage of the font size that is
applied to the block itself.
The code doesn't correctly implement this behaviour; this patch fixes
that.
https://bugzilla.gnome.org/show_bug.cgi?id=780402
In TTML, the height of every line in a block is determined by lineHeight
and fontSize style attributes, and should be the same for each line in
that block, regardless of whether different sized text appears on
different lines. Currently, a single PangoLayout is used to lay out all
the text in a block; however, pango will vary the line height in a
layout depending on the size of text used in each line, which is not
compliant with TTML.
This patch makes ttmlrender lay out the lines in a block itself, rather
than using a PangoLayout to do the work. The code still uses a
PangoLayout to render the text of each element, but the overall layout
of the text in a block is now controlled by ttmlrender itself. By doing
this, ttmlrender is able to ensure that the height of each line in a
block is correct.
https://bugzilla.gnome.org/show_bug.cgi?id=780402
The element now exposes properties to enable and configure
voice activity detection, and posts "voice-activity" messages
when the return value of stream_has_voice () changes.
https://bugzilla.gnome.org/show_bug.cgi?id=779138
A live manifest may have a set (> LookAheadFragmentCount) of fragments
that have already been served and are stored on the server, maybe
indefinitely. Adding the parsed live fragments after the manifest
fragments breaks duration reporting and the seekable range.
Fix by only adding parsed fragments outside the list of fragments which
assumes that the fragment list in the manifest is accurate enough to not
stray too far off what's in the retrieved data.
https://bugzilla.gnome.org/show_bug.cgi?id=779447
Instead of just going to the first or last fragment, report if we're
going outside the index. This should never happen unless there's a bug
or the stream is broken.
Allow some possibility for inaccuracies here though.
There is no guarantee that the index positions are the same between
representations, and assuming this easily causes us to get into invalid
index positions.
If a MPD is On-Demand profile and no index described, demux will terminate
download loop after parsing inband SIDX with flow return custom-success.
At this moment, SIDX index is excat target position, but finish_fragment()
might cause re-advancing subfragment depending on MPD structure.
https://bugzilla.gnome.org/show_bug.cgi?id=776200
SIDX's base offset (i.e., byte offset of SIDX + sidx.first_offset)
mostly vary as per fragment. Also, target SIDX index must be zero for the
new fragment.
https://bugzilla.gnome.org/show_bug.cgi?id=776200
Try to find fragment using MPD first, then do refinement to find
target subframgnet using SIDX if possible. Note that, if target fragment
was moved from the previously activated one, we should assume that
the last SIDX is invalid for new fragment.
https://bugzilla.gnome.org/show_bug.cgi?id=776200
SIDX based playback is not restricted to SegmentBase, but it possible
with SegmentList/SegmentTemplate. In the latter case, each fragment
has its own SIDX box and might be subdivided into subfragment.
So, demux should not assume that the end of subfragment is the end
of stream. Moreover, should try advance subfragment only if there
are remaining subfragments.
With additional fixes by Sebastian Dröge <sebastian@centricular.com>
https://bugzilla.gnome.org/show_bug.cgi?id=776200
All code interacting with Objective-C objects should now use Automated
Reference Counting rather than manual memory management or Garbage
Collection. Because ARC prohibits C-structs from containing
references to Objective-C objects, all such fields are now typed
'gpointer'. Setting and gettings Objective-C fields on such a
struct now uses explicit __bridge_* calls to tell ARC about
object lifetimes.
https://bugzilla.gnome.org/show_bug.cgi?id=777847
hlsdemux tries to find type if given buffer size is large enought to
find type (currently the threshold is 2KB), or EOS in some cases.
However, since there can be small byte fragments such as WebVTT,
demux should try to find type at the end of a fragment
https://bugzilla.gnome.org/show_bug.cgi?id=779011
This appears to be the internal limit of voaacenc, higher
bitrates will be ignored and 128 kbps output will be produced
instead. Therefore, we might just as well limit the allowed
property values, so that people who try to set higher bitrates
get a big fat warning instead of silently a much lower bitrate.
The PCR_flag and PCR value is in adaptation_field, not in payload.
The MSB of adaptation_field_control is used as whether adaptation_
field is exist or not.
For the case(PCR in only adaptation_field without payload), we modify
checking condition about adaptation_field_control field.
https://bugzilla.gnome.org/show_bug.cgi?id=778731
When MPD@suggestedPresentationDelay is not present in the MPD,
dashdemux can provide default suggestedPresentationDelay. However
when applying default value of suggestedPresentationDelay, the value
should be subtracted from current time, not added to it. When streams
setup is performed and live point is calculated, we have to go to the
wall clock (current time) minus suggestedPresentationDelay, if we tried
to start with current time plus suggestedPresentationDelay, we would
be asking for future stream, which has not yet been recorded. Also
the value needs to be converted from ms to us.
https://bugzilla.gnome.org/show_bug.cgi?id=764726
For duration queries on live streams, adaptivedemux ignores the query.
The problem then is that the query is answered by the downstream
qtdemux element, with the duration of the currently passing fragment.
This commit changes the behaviour of adaptivedemux to answer the duration
queries for live streams, returning GST_CLOCK_TIME_NONE.
https://bugzilla.gnome.org/show_bug.cgi?id=753879
The same symbol also exists in libgstgl, although marked as private and
internal. This has no effect when doing static linking and there's a
symbol conflict.
When the sink receives a buffer that is neither a wl_shm one nor a
dmabuf one, this buffer is copied to an internal wl_shm buffer before
being sent to the display.
In that case, the actual size of the received buffer (which may differ
from the one negotiated in the caps) must be used.
https://bugzilla.gnome.org/show_bug.cgi?id=777841
Unless the video format has an alpha component (ARGB, ...), set the
video_surface opaque.
In the usual case where the black area_surface has the same size as the
video_surface (eg: run gst-play-1.0 video.mp4), this makes the black
surface totally occluded which makes weston compositor's life easier
since it can ignore that surface.
Also unconditionally set the black area_surface opaque.
https://bugzilla.gnome.org/show_bug.cgi?id=778078
Otherwise the destructor will never be called, and we would have to call
the constructors manually at some point... which we never did, so
calling release() on it before it got otherwise initialized caused a
crash. For example when running gst-inspect-1.0.
Vulkan handles are either pointers (64-bit) or 64-bit integer
handles (32-bit). Cast the type in failure cases.
vkwindow.c:229:561: error: return makes integer from pointer without a cast [-Werror=int-conversion]
vkwindow.c:231:194: error: return makes integer from pointer without a cast [-Werror=int-conversion]
https://bugzilla.gnome.org/show_bug.cgi?id=777979
hls live starts playback from the allowed latest fragment,
but its "sequence position" is set to zero, and so stream
time is also set to zero.
This does not make sense, because hls live allows seeking to past position,
and it's negative stream time from downstream element's point of view.
Note that, allowed seekable range (and seeking query) is
from the first fragment of playlist to the allowed latest fragment.
https://bugzilla.gnome.org/show_bug.cgi?id=777682
HLS spec 6.3.3 is saying that
"the client SHOULD NOT choose a segment which starts less than
three target durations from the end of the Playlist file."
To ensure above statement, the third fragment from the end of playlist
should be excluded from seekable range and also from available starting fragment.
(i.e., the fourth fragment from end of playlist is the starting fragment).
https://bugzilla.gnome.org/show_bug.cgi?id=777682
During live playback, the first fragment in a updated
playlist can be advanced from that of startup playlist.
Meanwhile, since hlsdemux finds target seek position
by just accumulating fragment's duration, the base should
be adjusted to the updated first fragment's timestamp.
https://bugzilla.gnome.org/show_bug.cgi?id=777682
vkimagememory.c:64:14: warning: implicit conversion from enumeration type 'GstVideoGLTextureType' to different enumeration type 'VkFormat' (aka 'enum VkFormat') [-Wenum-conversion]
return GST_VIDEO_GL_TEXTURE_TYPE_RGB16;
https://bugzilla.gnome.org/show_bug.cgi?id=777760
gstmpeg2enc.cc:224:5: warning: variable 'n' is incremented both in the loop header and in the loop body [-Wfor-loop-analysis]
n++;
^
gstmpeg2enc.cc:221:29: note: incremented here
for (n = 0; fpss[n] != 0; n++) {
^
Without failing, we would play back random parts of the stream which is
arguably a worse user experience, and failing is also recommended by the
spec here.
And also handle live streams without any media sequence numbers at all
properly, that is, make sure the sequence numbers are increasing instead
of starting again at 0 every time.
https://bugzilla.gnome.org/show_bug.cgi?id=775665
