Commit graph

25 commits

Author SHA1 Message Date
Edward Hervey
f4ba43b15d multiqueue: Add a pad property to "group" streams
When syncing by running time, multiqueue will throttle unlinked streams
based on a global "high-time" and the pending "next_time" of a stream.

The idea is that we don't want unlinked streams to be "behind" the global
running time of linked streams, so that if/when they get linked (like when
switching tracks) decoding/playback can resume from the same position as
the other streams.

The problem is that it assumes elements downstream will have a more or less
equal buffering/latency ... which isn't the case for streams of different
type. Video decoders tend to have higher latency (and therefore consume more
from upstream to output a given decoded frame) compared to audio ones, resulting
in the computed "high_time" being at the position of the video stream,
much further than the audio streams.

This means the unlinked audio streams end up being quite a bit after the linked
audio streams, resulting in gaps when switching streams.

In order to mitigate this issue, this patch adds a new "group-id" pad property
which allows users to "group" streams together. Calculating the high-time will
now be done not only globally, but also per group. This ensures that within
a given group unlinked streams will be throttled by that group's high-time
instead.

This fixes gaps when switching downstream elements (like switching audio tracks).
2016-06-30 14:45:10 +02:00
Edward Hervey
4d96e5b834 multiqueue: Use signed clock values for running time calculation
This improves the accuracy of queue levels and when to push buffers
for buffers falling outside of the segment

https://bugzilla.gnome.org/show_bug.cgi?id=757193
2015-12-02 16:03:20 +01:00
Edward Hervey
c51d2b2a37 multiqueue: Add an extra cache time for unlinked streams
When synchronizing the output by time, there are some use-cases (like
allowing gapless playback downstream) where we want the unlinked streams
to stay slightly behind the linked streams.

The "unlinked-cache-time" property allows the user to specify by how
much time the unlinked streams should wait before pushing again.
2015-12-02 16:03:20 +01:00
Edward Hervey
72c31876aa multiqueue: Optimize multiqueue sizes based on interleave
Multiqueue should only be used to cope with:
* decoupling upstream and dowstream threading (i.e. having separate threads
  for elementary streams).
* Ensuring individual queues have enough space to cope with upstream interleave
  (distance in stream time between co-located samples). This is to guarantee
  that we have enough room in each individual queues to provide new data in
  each, without being blocked.
* Limit the queue sizes to that interleave distance (and an extra minimal
  buffering size). This is to ensure we don't consume too much memory.

Based on that, multiqueue now continuously calculates the input interleave
(per incoming streaming thread). Based on that, it calculates a target
interleave (currently 1.5 x real_interleave + 250ms padding).

If the target interleave is greater than the current max_size.time, it will
update it accordingly (to allow enough margin to not block).
If the target interleave goes down by more than 50%, we re-adjust it once
we know we have gone past a safe distance (2 x current max_size.time).

This mode can only be used for incoming streams that are guaranteed to be
properly timestamped.

Furthermore, we ignore sparse streams when calculating interleave and maximum
size of queues.

For the simplest of use-cases (single stream), multiqueue acts as a single
queue with a time limit of 250ms.
If there are multiple inputs, but each come from a different streaming thread,
the maximum time limit will also end up being 250ms.

On regular files (more than one input stream from the same upstream streaming
thread), it can reduce the total memory used as much as 10x, ending up with
max_size.time around 500ms.

Due to the adaptive nature, it can also cope with changing interleave (which
can happen commonly on some files at startup/pre-roll time)
2015-12-02 16:03:20 +01:00
Thiago Santos
3f83ca9c43 multiqueue: do not post messages holding the lock
It might cause deadlocks to post messages while holding the multiqueue
lock. To avoid this a new boolean flag is set whenever a new buffering percent
is found. The message is posted after the lock can be released.

To make sure the buffering messages are posted in the right order, messages
are posted holding another lock. This prevents 2 threads trying to post
messages at the same time.

https://bugzilla.gnome.org/show_bug.cgi?id=736295
2014-09-16 16:56:53 -03:00
Tim-Philipp Müller
666c8c11c6 Fix FSF address
https://bugzilla.gnome.org/show_bug.cgi?id=687520
2012-11-03 20:44:48 +00:00
Sebastian Dröge
09982c3c13 dataqueue/queuearray: Make public API again
These are actually used outside of coreelements nowadays.
Also hide lots of internals and add padding and documentation.
2012-10-25 14:03:01 +02:00
Tim-Philipp Müller
14632db174 plugins: sprinkle some more G_GNUC_INTERNAL 2012-07-14 20:05:53 +01:00
Tim-Philipp Müller
58a4d806d1 multiqueue: fix size of atomic counter 2012-01-22 01:27:42 +00:00
Wim Taymans
252327f87a Update for new gthread API 2012-01-19 09:27:04 +01:00
Tim-Philipp Müller
434e7042d8 base: make GstDataQueue private API for multiqueue
There's no code that uses it other than multiqueue, so make it private
to multiqueue for now. That way we can also do optimisations that
require API/ABI breaks. If anyone ever wants to use it, we can still
make it public again.
2011-10-07 13:54:37 +01:00
Sebastian Dröge
9f83109706 multiqueue: Add mode to synchronize deactivated/not-linked streams by the running time
Fixes bug #645107, #600648.
2011-05-14 11:39:35 +02:00
Sebastian Dröge
65fbc9398a multiqueue: Really remove unused variable 2011-03-21 17:54:10 +01:00
Sebastian Dröge
383cac91b1 multiqueue: Increment unique item counter with atomic operations
Before it was only protected by the stream lock but every pad
has its own stream lock, making the protection rather useless.
2011-03-21 17:52:59 +01:00
Wim Taymans
b852da60d4 multiqueue: make sure percent increases
Keep track of the last posted percent message and make sure the next percent
messages are strictly increasing.
2009-10-27 14:40:17 +01:00
Wim Taymans
99dd9830a9 multiqueue: add buffering support
Add support for buffering mode where we post BUFFERING messages based on the
level of the queues. It currently operates on the first queue that goes over or
under the high/low thresholds.
2009-10-27 14:40:12 +01:00
Wim Taymans
6ac9191527 multiqueue: hook up low/high percent
Hook up the low/high percent properties for the buffering mode.
2009-10-27 14:40:07 +01:00
Wim Taymans
b9d4f6e550 multiqueue: hook up property for buffering 2009-10-27 14:40:02 +01:00
Wim Taymans
b95cc5031b multiqueue: avoid lock for taking the counter
The counter for incomming data is already protected with the STREAM_LOCK so we
don't need to add another lock around it.
2009-10-21 13:38:57 -04:00
Sebastian Dröge
269d3581c5 multiqueue: Use iterate_internal_links instead of deprecated get_internal_links 2009-08-18 14:57:18 +02:00
Thijs Vermeir
6d4dc1a807 plugins/elements/gstmultiqueue.*: revert extra-size-buffers stuff, caused some race conditions and extra-size-buffers...
Original commit message from CVS:
* plugins/elements/gstmultiqueue.c:
* plugins/elements/gstmultiqueue.h:
revert extra-size-buffers stuff, caused some race conditions
and extra-size-buffers is not used anymore. Docs needs some updates
2008-06-26 20:27:00 +00:00
Thijs Vermeir
e25fd35125 plugins/elements/gstmultiqueue.*: Fix dead-lock in underrun_cb
Original commit message from CVS:
* plugins/elements/gstmultiqueue.c:
* plugins/elements/gstmultiqueue.h:
Fix dead-lock in underrun_cb
2008-06-25 15:39:02 +00:00
Jan Schmidt
afebd394fa plugins/elements/gstmultiqueue.*: Take the multiqueue lock when updating the fill level so we don't get confused.
Original commit message from CVS:
* plugins/elements/gstmultiqueue.c: (gst_multi_queue_init),
(gst_single_queue_flush), (apply_segment), (apply_buffer),
(gst_single_queue_push_one), (gst_multi_queue_loop),
(gst_multi_queue_sink_activate_push), (gst_multi_queue_sink_event),
(gst_multi_queue_src_activate_push), (wake_up_next_non_linked),
(compute_high_id), (gst_single_queue_new):
* plugins/elements/gstmultiqueue.h:
Take the multiqueue lock when updating the fill level so we don't get
confused.
After applying a buffer or event on the src pad segment, make sure to
call gst_data_queue_limits_changed() to get the data queue to unblock
and check the filled state again.
Rework the not-linked pad handling so the logic is that not-linked
pads can push as fast as they like, but only so they never get
ahead of any linked pads.
* tests/check/elements/multiqueue.c: (mq_sinkpad_to_srcpad),
(mq_dummypad_getcaps), (mq_dummypad_chain), (mq_dummypad_event),
(run_output_order_test), (GST_START_TEST), (multiqueue_suite):
Add a test to check that not-linked pads always stay behind
linked pads.
2007-06-26 14:45:15 +00:00
Tim-Philipp Müller
5c9c202e47 plugins/elements/: These are not installed headers, no need for padding.
Original commit message from CVS:
* plugins/elements/Makefile.am:
* plugins/elements/gstmultiqueue.h:
* plugins/elements/gstqueue.h:
These are not installed headers, no need for padding.
2007-05-24 09:41:51 +00:00
Edward Hervey
fdb51251dd libs/gst/base/: New GstDataQueue object for threadsafe queueing. Most useful for elements that need some queueing fun...
Original commit message from CVS:
* libs/gst/base/Makefile.am:
* libs/gst/base/gstdataqueue.c: (gst_data_queue_get_type),
(gst_data_queue_base_init), (gst_data_queue_class_init),
(gst_data_queue_init), (gst_data_queue_new),
(gst_data_queue_cleanup), (gst_data_queue_finalize),
(gst_data_queue_locked_flush), (gst_data_queue_locked_is_empty),
(gst_data_queue_locked_is_full), (gst_data_queue_flush),
(gst_data_queue_is_empty), (gst_data_queue_is_full),
(gst_data_queue_set_flushing), (gst_data_queue_push),
(gst_data_queue_pop), (gst_data_queue_drop_head),
(gst_data_queue_set_property), (gst_data_queue_get_property):
* libs/gst/base/gstdataqueue.h:
New GstDataQueue object for threadsafe queueing. Most useful for
elements that need some queueing functionnality.
* docs/libs/gstreamer-libs-docs.sgml:
* docs/libs/gstreamer-libs-sections.txt:
Insert documentation for GstDataQueue
* plugins/elements/Makefile.am:
* plugins/elements/gstelements.c:
* plugins/elements/gstmultiqueue.c: (gst_multi_queue_base_init),
(gst_multi_queue_class_init), (gst_multi_queue_init),
(gst_multi_queue_finalize), (gst_multi_queue_set_property),
(gst_multi_queue_get_property), (gst_multi_queue_request_new_pad),
(gst_multi_queue_release_pad), (gst_single_queue_push_one),
(gst_multi_queue_item_destroy), (gst_multi_queue_item_new),
(gst_multi_queue_loop), (gst_multi_queue_chain),
(gst_multi_queue_sink_activate_push), (gst_multi_queue_sink_event),
(gst_multi_queue_getcaps), (gst_multi_queue_bufferalloc),
(gst_multi_queue_src_activate_push), (gst_multi_queue_acceptcaps),
(gst_multi_queue_src_event), (gst_multi_queue_src_query),
(wake_up_next_non_linked), (compute_next_non_linked),
(single_queue_overrun_cb), (single_queue_underrun_cb),
(single_queue_check_full), (gst_single_queue_new):
* plugins/elements/gstmultiqueue.h:
New multiqueue element, using GstDataQueue. Used for queuing multiple
streams.
Closes #344639 and #347785
2006-11-28 12:07:06 +00:00