Use a single lock for all streams instead of having separate locks.
This makes maintenance easier and at most points we would need
a single lock before iterating on all streams data. So not much
is gained from individual locks.
Make dash playlists with multiple periods work again by waiting
to switch the periods when all streams have reached the end of
the current period. The stream_loop is responsible for advancing
the period, but the download loops will already start downloading
data for the next period as soon as possible.
Handle multiple languages by using the not-linked return to stop
the download task for that stream. It can be reactivated when
a reconfigure event is received. Stopping the unused streams is
relevant to save network bandwidth
Instead of having a single download task for all streams, this
commit makes each stream have its own download loop, allowing
parallel download of fragments.
always expose all streams instead of only exposing one of each type.
This is more aligned with gstreamer's way of working. Allows the user
to select the stream that it wants to use by linking its pad and leaving
the unused ones as unlinked.
As streams now flow independently, the GstSegment needs to be put
on each stream so they can track the position of each one correctly
instead of being mixed in a single segment
Download and push from the same task, makes code a lot simpler
to maintain. Also pushing from separate threads avoids deadlocking
when gst_pad_push blocks due to downstream queues being full
When a stream gets a not-linked return, it will be marked as so and
won't download any more new fragments until a reconfigure event
is received. This will make mssdemux expose all pads, but only download
fragments for the streams that are actually being used.
Relying on the pads being linked/unlinked isn't enough in this scenario
as there might be an input-selector downstream that is actually discarding
buffers for a given linked pad.
When streams are switching, the old active stream can be blocked because
input-selector will block not-linked streams. In case the mssdemux's
stream loop is blocked pushing a buffer to a full queue downstream it will
never unblock as the queue will not drain (input-selector is blocking).
In this scenario, stream switching will deadlock as input-selector is
waiting for the newly active stream data and the stream_loop that would
push this data is blocked waiting for input-selector.
To solve this issue, whenever an stream is reactivated on a reconfigure
it will enter into the 'catch up mode', in this mode it can push buffers
from its download thread until it reaches the currrent GstSegment's position.
This works because this timestamp will always be behind or equal to the maximum
timestamp pushed for all streams, after pushing data for this timestamp,
the stream will go back to default and be pushed sequentially from the main
streaming thread. By this time, the input-selector should have already
released the thread.
https://bugzilla.gnome.org/show_bug.cgi?id=711849
* ext/srtp/gstsrtp.[ch]: added GST_SRTP_CIPHER_AES_256_ICM to
GstSrtpCipherType and new function cipher_key_size.
* ext/srtp/gstsrtpenc.c: maximum key size is now 46 characters (14 for
the salt plus the key). If different ciphers are chosen for RTP and
RTCP the maximum needed key size is expected.
* ext/srtp/gstsrtpdec.c: minor documentation updates.
https://bugzilla.gnome.org/show_bug.cgi?id=720434
Alternates between 33 and 32 byte frames, but must start
with a 33 byte frame. This has been broken for ages since
the element was ported to the audio decoder base class.
https://bugzilla.gnome.org/show_bug.cgi?id=709416
This currently converts from ARGB64_F16 (16 bit float per component)
to ARGB64 by clipping. We should add support for the F16 format and
implement a conversion filter element that can apply gamma curves,
change exposure, etc.
It only gets the sink flag set when it adds the multifilesink, that
happens in null->ready and it might be too late. Set the flag
explicitly on the constructor.
https://bugzilla.gnome.org/show_bug.cgi?id=711086
This patch fixes three memory leaks in hlsdemux, one that occurs
during normal operation and two that occur during error conditions.
The gst_hls_demux_get_next_fragment function calls
gst_fragment_get_buffer which increments the reference count
on the buffer but gst_hls_demux_get_next_fragment never calls unref on
the buffer. This means that the reference count for each downloaded
fragment never gets to zero and so its memory is never released.
This patch adds a call to gst_buffer_unref after the flags have been
updated on the buffer.
There is a leak-on-error in gst_hls_demux_decrypt_fragment if it fails
to download the key file. If the key fails to download, null is
returned without doing an unref on the encrypted fragment. The
semantics of gst_hls_demux_decrypt_fragment is that it takes ownership
of the encrypted fragment and releases it before returning.
There is a leak-on-error in gst_hls_src_buf_to_utf8_playlist in the
unlikely event that the gst_buffer_map fails. In the "happy path"
operation of gst_hls_src_buf_to_utf8_playlist the buffer gets an unref
before the function returns, therefore the error condition must do the
same.
https://bugzilla.gnome.org/show_bug.cgi?id=710881
Fixed up the error-handling code when downloading fragments.
Modifed the error-handling code to use positive logic when
testing for cancellation of the download loop.
https://bugzilla.gnome.org/show_bug.cgi?id=701404
There is an issue for live streams where download_loop will keep
downloading segments until it gets a 404 error for a segment
that has not yet been published. This is a problem because this
request for a segment that doesn't exist will propagate all the
way back to the origin server(s). This means that dashdemux causes
extra load on the origin server(s) for segments that aren't yet
available.
This patch uses availabilityStartTime, period
and the host's idea of UTC to decide if a fragment is available to
be requested from an HTTP server and filter out requests for fragments
that are not yet available.
https://bugzilla.gnome.org/show_bug.cgi?id=701404
On some live HLS streams, gst_hls_demux_switch_playlist causes
assertion failures because it tried to dereference a NULL fragment.
This is because g_queue_peek_tail sometimes was returning NULL and
this case was not being checked.
This patch does two things:
* move the g_queue_peek_tail inside the semaphore protection
* check if q_queue_peek_tail returns NULL
https://bugzilla.gnome.org/show_bug.cgi?id=708849
gstdashdemux.c:1753: warning: format '%llu' expects type 'long long unsigned int', but argument 8 has type 'long unsigned int'
gstdashdemux.c:2224: warning: format '%llu' expects type 'long long unsigned int', but argument 9 has type 'guint64'
gstdashdemux.c:2224: warning: format '%llu' expects type 'long long unsigned int', but argument 10 has type 'guint64'
gstmpdparser.h:530: warning: type qualifiers ignored on function return type
gstmpdparser.c:4177: warning: type qualifiers ignored on function return type
including the following supports and fixes:
* Create DirectFB surfaces from GstBufferPool
* Add NV12 pixel format support
* Don't use the cursor in the exclusive mode
- EnableCusor() can be only used when the administrative mode is set
in DirectFB 1.6.0 and later.
* Support multiple plane rendering for planar color formats
- This accommodates the chroma plane offsets of the framebuffer
in planar formats.
* Invoke SetConfiguration regardless of video mode setting in setcaps()
- SetConfiguration() method should be invoked regardless of
the result of gst_dfbvideosink_get_best_vmode(), since the two are
unrelated.
* Disable DirectFB signal handler
- "--dfb:no-sighandler" option is passed to DirectFBInit().
This prevents DirectFB from trying to kill the process and allows
GStreamer's termination sequence to proceed normally.
https://bugzilla.gnome.org/show_bug.cgi?id=703520
For SegmentTemplate elements containing a startNumber attribute, the
`number' member of GstMediaSegments should be offset by the value of
startNumber; however, this is not currently the case. As a result, the
first URI(s) requested by the download loop will be wrong.
This commit ensures that segment numbers will be offset by startNumber
when one is present in a SegmentTemplate element.
https://bugzilla.gnome.org/show_bug.cgi?id=705661
When using a SegmentTemplate element, the timestamps of the buffers
output by dashdemux are incorrect, causing problems downstream.
The reason is that GstMediaSegment start times are calculated (in
gst_mpdparser_get_chunk_by_index) by multiplying segment index by
segment duration and then scaling the result according the `timebase'
attribute from the MPD. However, the segment duration is already a
GstClockTime (i.e., it has already been scaled according to the timebase
from the MPD and converted to a nanosecond value), so multiplying it by
the segment index will give the correct timestamp without the need for
any further scaling.
https://bugzilla.gnome.org/show_bug.cgi?id=705679
This prevents locking on startup when a stream only has a single buffer
for one of the streams and mssdemux decides to push an EOS event right
after it.
This prevents deadlocks on startup on files that have only a very
large buffer for a stream and the queue is filled and will lock on
the eos event that is pushed after the buffer. As no buffers have yet
been pushed to other streams, the pipeline locks on preroll
Every encrypted fragment will be a multiple of 128 bits, the last byte
contains the number of bytes that were added as padding in the end
and should be removed.
https://bugzilla.gnome.org/show_bug.cgi?id=701673
When using an HLS encrypted stream, an assertion failure is thrown:
(gst-launch-1.0:31028): GLib-GObject-WARNING **: cannot register
existing type `GstFragment'
(gst-launch-1.0:31028): GLib-CRITICAL **: g_once_init_leave: assertion
`result != 0' failed
Eventually tracked this down to the call gst_fragment_new()
in function gst_hls_demux_decrypt_fragment.
The GstFragment class is defined in ext/hls/gstfragment.c and in
gst-libs/gst/uridownloader/gstfragment.c. Having two class definitions
with the same name causes the assert failure when trying to allocate
GstFragment. Deleting the version from hls and editing the
Makefile.am solves this assert failure.
https://bugzilla.gnome.org/show_bug.cgi?id=704555
During a live stream it is possible for dashdemux to lag behind on a
slow connection or to rush ahead of the connection os too fast.
For the first case it is necessary to jump some segments ahead to be able to
continue playback as old segments are usually deleted from the server.
For the later, dashdemux should wait a little before attempting another
download do give time to the server to produce a new segment
When using a template based segment list, do not try to
contruct a finite segment list for the limits of the available periods.
We might not know when the period ends (for live streams) and we can
always create the segment on demand when requested by dashdemux,
avoiding use of some memory and cpu when re-creating this list.
Replaces the 2 likely larger lists with more appropriate structures
to improve performance.
Replaces S nodes GList for a GQueue, this reduces latency to startup
because of traversing the list just append an element.
Replaces the processed media segments GList for a GPtrArray as it is
constantly acessed by index during playback.
Duration from segment being unknown is a issue from the MPD and not
a programming issue, so the assert isn't useful here. Instead check
and return an error code so the caller can fallback to alternatives
When dashdemux selects its first fragment, it always selects the
first fragment listed in the manifest. For on-demand content,
this is the correct behaviour. However for live content, this
behaviour is undesirable because the first fragment listed in the
manifest might be some considerable time behind "now".
The commit uses the host's idea of UTC and tries to find the
oldest fragment that contains samples for this time of day.
https://bugzilla.gnome.org/show_bug.cgi?id=701509
According to the MPEG-DASH spec, certain elements (i.e.
SegmentBase, SegmentTemplate, and SegmentList) should inherit
attributes from the same elements in the containing AdaptationSet
or Period.
Updated the SegmentBase, SegmentTemplate, and SegmentList parsers
to properly inherit attributes from the corresponding elements in
AdaptationSet and/or Period.
https://bugzilla.gnome.org/show_bug.cgi?id=702677
Convert all xml attribute/content parsing functions to return a
boolean value indicating whether or not the attribute/content was
present. We need this finer-grained control in order to properly
implement the inheritance policies described in the spec
Also fixed several memory leak conditions when handling errors in
the xml attribute/content parsing functions.
https://bugzilla.gnome.org/show_bug.cgi?id=702677
Ensure that g_free/xmlFree is used correctly based on how the
memory was allocated.
When deallocating GLists, there were many places that were using
g_list_foreach and g_list_free. Converted these occurrences to
call g_list_free_full.
Add NULL checks to all xmlFree calls since the documentation does
not guarantee that passing NULL is safe
In places where we are strdup'ing memory allocated by libxml2,
changed those calls to use xmlMemStrdup().
There were several places where we were missing g_slice_free when
deallocating a top-level node structure.
https://bugzilla.gnome.org/show_bug.cgi?id=702837
Wayland interface could offer two buffers pixels formats: WL_SHM_FORMAT_XRGB8888 and WL_SHM_FORMAT_ARGB8888.
Update waylandsink to support them and check if the format is really available.
https://bugzilla.gnome.org/show_bug.cgi?id=702112
Fixes:
In file included from gstsegmentation.h:51:0,
from gstopencv.c:42:
/usr/include/opencv2/video/background_segm.hpp:47:16: fatal error: list:
No such file or directory
#include <list>
^
compilation terminated.
https://bugzilla.gnome.org/show_bug.cgi?id=702297
It was not properly divided by GST_SECONDS. Also fix issue with
max-buffering-time being multiplied by GST_SECONDS every time the
property is retrieved.
https://bugzilla.gnome.org/show_bug.cgi?id=700487
Split the introspection and registration part. This way we only need to open all
plugins when updating the registry. When reading the registry we can register
the elements entierly from the cache.
Add colour image enhancement element based on Retinex algorithm. Two types
exist, namely basic and multiscale; both are described in this article:
Rahman, Zia-ur, Daniel J. Jobson, and Glenn A. Woodell. "Multi-scale retinex
for color image enhancement." Image Processing, 1996. Proceedings.,
International Conference on. Vol. 3. IEEE, 1996
Visually speaking the result looks a bit funny, but is pretty invariable to
lightning changes, which is good for some applications, like image
segmentation.
https://bugzilla.gnome.org/show_bug.cgi?id=700977
WmaPro is actually wmaversion 3, and can also be found by the
WMAP fourcc.
Some manifests also contain the block_align field as "PacketSize"
in the audio track description, the libav decoders require it
to be present in caps.
Fixes#699921
Detect when the eagl surface changed its dimension (when the user rotates
the device for example) and adapt the egl internals to draw to that,
preventing that ios resizes the image again when drawing.
This is particularly harmful when eagl would scale down a image
to draw and the ios screen would scale it back up because the
surface is now bigger than when the element was configured.
wma v2 expects block_align, channels and rate fields set to its caps.
This isn't present direclty on the manifests, so mssdemux should parse
it from the waveformatex structure
https://bugzilla.gnome.org/show_bug.cgi?id=699924
bitrate info is always present on the QualityLevel xml node as part
of the adaptive selection processing, put it into caps as some
decoders require it (avdec_wmav2 for example)
https://bugzilla.gnome.org/show_bug.cgi?id=699924
It's not developed any more and replaced by the
libschroedinger-based elements in gst-plugins-good.
(The libschroedinger 1.0.9 release notes state "This
is an exciting release: most of the encoding tools in
dirac-research have been ported over to Schrödinger, so
now schro has the same or better compression efficiency
as dirac-research.")
TRM IDs are MusicBrainz' old audio fingerprinting system from
Relatable, they were phased out in favour of MusicIPs PUIDs.
https://wiki.musicbrainz.org/History:TRM
In some scenarios, for example in QtWebKit, might be difficult to obtain full
control on the egl display and it might be only accessible indirectly via
eglGetCurrentDisplay().
https://bugzilla.gnome.org/show_bug.cgi?id=700058
We only want to adjust the timestamps so that they start from 0 for live
streams. Non-live streams already start from 0 and after a seek we actually want
to timestamp to be the position we seek to.
Non-live streams should timestamp buffers with a running-time starting from
0. Since we already push a 0 -> -1 segment, bring the timestamps to 0
by subtracting the initial timestamp.
The xmlCleanupParser function seems to cleanup all statically
allocated libxml variables, making it unusable. We can't guarantee
that dashdemux won't need it anymore, so better not call it.
Manifest updates should be done periodically for live streams,
this patch makes the demuxer create a new manifest client for
the new version and transfers the stream position to the new
one, discarding the old one afterwards.
A small struct that keeps a short history of fragment download bitrates
to have an average measure of N last fragments instead of using only
the last downloaded bitrate
Do not use a global bitrate as the sizes of the fragments matter
when calculating the download rate as the connection setup time is
also being taken into the download duration, a smaller fragment
will have a lower bitrate than a larger one.
This avoids switching the bitrates for streams frequently because
of bitrate mismatches
Instead of downloading 1 fragment per stream per download loop,
select the stream with the earlier timestamp and get a fragment
only for that one.
The old algorithm would lead to problems when the fragment durations
were too different for streams.
dashdemux shouldn't emit the buffering message as that can pause
the pipeline. It has no proper knowledge of the downstream buffering
status so it can pause the pipeline when it isn't necessary. It should
have an internal buffer for downloading the streams ahead of playback,
but that shouldn't make it able to stop the pipeline for buffering.
A particular case in which this is bad is when a pad switch happens
(changing bitrates for example), the new pads dashdemux creates
will get linked to demuxers and new queues will be created,
these queues are initially empty and dashdemux will quickly
drain its buffers by pushing them to those queues. So it
would have no more buffers internally and would emit a
buffering message with a low ratio, causing the pipeline
to pause when it wouldn't be necessary.
Put EOS on the streams queues after the last fragment from the
last period for each stream. This way we keep it serialized
with the buffers and it will work when streams have different
ending times
The smallest queue should be used to prevent blocking the download
thread when a stream has too much data buffered, leaving the other
streams starving from fragments
Each stream has its own durations and timestamps, the fragment number
is different for each stream when seeking, so the seek has to be done
for all streams, rather than on a single stream and propagated to
others
GstDataQueue has proper locking and provides functions to limit the
size of the queue. Also has blocking calls that are useful to
our multithread scenario in Dash.
Store the buffers separately for each stream, this is clearer than
having a queue with a list of buffers. It also allows easier selection
of buffers to push in later refactors
Fragments should be pushed ASAP as downstream should be responsible for
doing the syncrhonization and proper buffering.
This has the great side effect of fixing most of the seeking A/V sync issues.
- the MPD file is updated in the download loop (only if we have a "dynamic" MPD and minimumUpdatePeriod is valid);
- properly LOCK/UNLOCK the GstMpdClient;
This fixes conflicts with the HLS plugin, which is also named
fragmented.
When building its registry, gstreamer was picking one or the other
between hls and dashdemux.
This fixes build that has been broken by commit
fb9aeac6552021b176a4c4bd07265e02a0b70e0f.
gst_mpd_client_get_target_duration has been removed, and
gst_mpd_client_get_next_fragment_duration should be used instead.
This was necessary to support variable-duration Fragments.
in the new API:
- gst_mpd_client_get_current_position returns the timestamp of the NEXT fragment to download;
- gst_mpd_client_get_next_fragment_duration returns the duration of the next fragment to download;
- gst_mpd_client_get_media_presentation_duration returns the mediaPresentationDuration from the MPD file;
also there is a new internal parser function:
- gst_mpd_client_get_segment_duration extracts the constant segment duration from the MPD file
(only used when there is no SegmentTimeline syntax element in the current representation)
In gst_mpd_client_get_next_fragment, we set the timestamp/duration of the fragment just downloaded
copying the values from the corresponding GstMediaSegment.
TODO: rework SEEKING to support seeking across different Periods.
- Periods are played in sequence, from PeriodStart to PeriodEnd
- seamless switching from one Period to the next one works fine;
- the 'new-segment' generation is broken, so if we need to switch pads for a new Period there is a crash;
- build a list of the available Periods with their start and duration time
- add the list of GstStreamPeriod in the GstMpdClient data struct
- remove cur_period from GstMpdClient and introduce an API to get the current GstStreamPeriod
- several API clean-ups
build the list of segments to be played using the SegmentTimeline syntax, if present
bugfixes:
- for dynamic MPD files, when mediaPresentationDuration is not present use minimumUpdatePeriod instead
- do not add a spurious '$' when building an URL from a template like "$Bandwidth$/init.mp4v"
- introduce gst_mpd_client_add_media_segment() to avoid code duplication
other fixes:
- fixed a buffering bug: now we stop buffering when we reach the end of manifest
- now gst_mpd_client_get_target_duration() always returns a valid duration
(in case of single-segment streams, we return either Period duration or mediaPresentation duration)
TODO: support SegmentTimeline
SegmentList nodes are allowed into Period, AdaptationSet or Representation nodes
and there is at most 1 element, so no need to keep a list;
Period nodes cannot have any Represention elements, as AdaptationSet nodes are mandatory;
this breaks compatibility with some legacy DASH test sequences.
gstmpdparser.c: In function ‘gst_mpdparser_get_list_and_nb_of_audio_language’:
gstmpdparser.c:2891: warning: ‘return’ with no value, in function returning non-void
g_ascii_strtoull() returns a long long integer, but we need to
pass a normal int to gst_structure_set() for fields of G_TYPE_INT,
so cast appropriately.
The buffer parameter wasn't being used, it was only to signal if
a buffer was downloaded and advance to the next fragment in the
manifest.
Replace the buffer with a boolean that has the same effect and is
safer
connection setup times seem to matter when measuring the download
rate of different streams. Streams with longer fragments have a
*relatively* lower connection setup time and achieve higher bitrates.
Using the average seems unfair here, so use each stream's measured bitrate
to select its best quality option.
We need to cancel the downloader for each stream before joining the main download task, otherwise
the download task will block until all the stream tasks finish.
When the codec is AAC-LC, some server implementation (e.g. Microsoft IIS) doesn't add the CodecPrivateData attribute. The element needs to re-create the codec data from the Quality Level attributes (channels and sampling rate).
There is no way to know if a live stream is really finished, so try to reload the manifest and check if there are more fragments to download. Else just let know it's the EOS.
Live streams force the demuxer to keep reloading the Manifest from
time to time, as the new fragments are being added as they are recorded.
The demuxer should also try to keep up and detect when it had to skip
fragments, marking the discont flag when that happens.
Curiously, the spec doesn't seem to mention when/how a live stream is supposed
to end, so keep trying downloads until the demuxer errors out.
Use pad tasks to download data and an extra task that gets the earlier
buffer (with the smallest timestamp) and pushes on the corresponding
pad.
This prevents that the audio stream rushes ahead on buffers as its
fragments should be smaller
When connection-speed changes, signal that we might need a bitrate
switch. During the switch, a new pad group is added and the old one
is drained and removed.
New pads also need to push newsegments before starting to stream
This speed limits the maximum bitrate of streams. Currently it
is only read when starting the pipeline, but it should be used
for switching bitrates during playback to adapt to network
changes.
mssdemux should set the streams it has exposed as active so that
the manifest won't use the non-active streams to compute total bitrates
or providing fragments
