Commit graph

240 commits

Author SHA1 Message Date
Aaron Boxer
509f6201e1 documentation: fix a number of typos 2019-10-06 11:12:11 -04:00
Niels De Graef
4812c4087f Don't pass default GLib marshallers for signals
By passing NULL to `g_signal_new` instead of a marshaller, GLib will
actually internally optimize the signal (if the marshaller is available
in GLib itself) by also setting the valist marshaller. This makes the
signal emission a bit more performant than the regular marshalling,
which still needs to box into `GValue` and call libffi in case of a
generic marshaller.

Note that for custom marshallers, one would use
`g_signal_set_va_marshaller()` with the valist marshaller instead.
2019-09-09 12:16:24 +00:00
Thibault Saunier
7a2d32abda multiqueue: Fix possible NULL pointer dereferencing
In the hotdoc inspector for example, pads are instantiated with
g_object_new, other code paths to get/set properties already make
that check.

And update doc cache
2019-07-02 17:27:32 -04:00
Thibault Saunier
27fbaf9d44 multiqueue: Hold weak references to pads/multiqueue in SingleQueue
Without holding a ref we have no guarantees that the SingleQueue
doesn't have dangling pointers on those objects during its destruction.
2019-06-28 15:30:54 -04:00
Thibault Saunier
6a20fcc97a multiqueue: Stop using the gst_pad_element_private API
There was a race where we could still get the pad event function
called when its private member were already unset, leading to
a segfault in the event handler:

```
0  gst_multi_queue_src_event (pad=<optimized out>, parent=<optimized out>, event=0x7f3ff0007600) at ../subprojects/gstreamer/plugins/elements/gstmultiqueue.c:2534
2534          ret = gst_pad_push_event (sq->sinkpad, event);
[Current thread is 1 (Thread 0x7f406c0258c0 (LWP 21925))]
(gdb) bt
0  0x00007f4062ec1399 in gst_multi_queue_src_event (pad=<optimized out>, parent=<optimized out>, event=0x7f3ff0007600 [GstEvent]) at ../subprojects/gstreamer/plugins/elements/gstmultiqueue.c:2534
1  0x00007f406b40f46d in gst_validate_pad_monitor_src_event_check (handler=0x7f4062ec1360 <gst_multi_queue_src_event>, event=0x7f3ff0007600 [GstEvent], parent=0x7f3fcc01f090 [GstMultiQueue|multiqueue167], pad_monitor=0x7f3fe809e7c0 [GstValidatePadMonitor|validatepadmonitor2213]) at ../subprojects/gst-devtools/validate/gst/validate/gst-validate-pad-monitor.c:2101
2  0x00007f406b40f46d in gst_validate_pad_monitor_src_event_func (pad=<optimized out>, parent=0x7f3fcc01f090 [GstMultiQueue|multiqueue167], event=0x7f3ff0007600 [GstEvent]) at ../subprojects/gst-devtools/validate/gst/validate/gst-validate-pad-monitor.c:2374
3  0x00007f406b904387 in gst_pad_send_event_unchecked (pad=pad@entry=0x7f3fdc027650 [GstPad|src_0], event=event@entry=0x7f3ff0007600 [GstEvent], type=<optimized out>, type@entry=GST_PAD_PROBE_TYPE_EVENT_UPSTREAM) at ../subprojects/gstreamer/gst/gstpad.c:5772
4  0x00007f406b90481b in gst_pad_push_event_unchecked (pad=pad@entry=0x7f4058182fc0 [GstPad|sink], event=event@entry=0x7f3ff0007600 [GstEvent], type=type@entry=GST_PAD_PROBE_TYPE_EVENT_UPSTREAM) at ../subprojects/gstreamer/gst/gstpad.c:5417
5  0x00007f406b90f016 in gst_pad_push_event (pad=0x7f4058182fc0 [GstPad|sink], event=event@entry=0x7f3ff0007600 [GstEvent]) at ../subprojects/gstreamer/gst/gstpad.c:5554
6  0x00007f406a1c99ba in gst_video_decoder_src_event_default (decoder=0x7f3fe81c6060 [GstTheoraDec|theoradec46], event=<optimized out>) at ../subprojects/gst-plugins-base/gst-libs/gst/video/gstvideodecoder.c:1532
7  0x00007f406b40f46d in gst_validate_pad_monitor_src_event_check (handler=0x7f406a1ca270 <gst_video_decoder_src_event>, event=0x7f3ff0007600 [GstEvent], parent=0x7f3fe81c6060 [GstTheoraDec|theoradec46], pad_monitor=0x7f4028163aa0 [GstValidatePadMonitor|validatepadmonitor2216]) at ../subprojects/gst-devtools/validate/gst/validate/gst-validate-pad-monitor.c:2101
8  0x00007f406b40f46d in gst_validate_pad_monitor_src_event_func (pad=<optimized out>, parent=0x7f3fe81c6060 [GstTheoraDec|theoradec46], event=0x7f3ff0007600 [GstEvent]) at ../subprojects/gst-devtools/validate/gst/validate/gst-validate-pad-monitor.c:2374
```

This make the GstSingleQueue a MiniObject, mainly so it is properly
refcounted.

This also make use of the GstMultiQueuePad class for srcpads which
is totally valid as srcpads and sinkpads share the same SingleQueue
object.
2019-06-25 12:53:05 -04:00
Thibault Saunier
eaa39398cc multiqueue: never unref queries we do not own
The `query` argument of gst_pad_query is "transfer none".

Query objects are "borrowed" by the pad query handlers and those
should never unref them.

This was leading to double freed queries in a very racy way with nested
GESTimelines.
2019-06-20 00:19:24 -04:00
Edward Hervey
127e211004 multiqueue: Don't clamp running times for position calculation
Since we use full signed running times, we no longer need to clamp
the buffer time.

This avoids having the position of single queues not advancing for
buffers that are out of segment and never waking up non-linked
streams (resulting in an apparent "deadlock").
2018-10-22 15:28:45 +02:00
luz.paz
4a402c1c7d Fix typos in comments and docs
Found via `codespell`

https://bugzilla.gnome.org/show_bug.cgi?id=795610
2018-05-01 11:18:03 +01:00
Tim-Philipp Müller
21decc643b multiqueue, inputselector: show pad properties in gst-inspect-1.0 2018-04-17 18:57:20 +01:00
Jan Alexander Steffens (heftig)
b17d03cb0a multiqueue: Don't start new pads until parented
Start task on new source pads added at runtime after they
have been added to the element, not during activation.

This ensures the pads can post their CREATE stream-status
messages and the application can set thread priorities.

https://bugzilla.gnome.org/show_bug.cgi?id=756867
2017-12-24 11:51:06 +01:00
Jan Alexander Steffens (heftig)
10c3a7bd55 multiqueue: Split task handling from gst_single_queue_flush
https://bugzilla.gnome.org/show_bug.cgi?id=756867
2017-12-24 11:51:06 +01:00
Mathieu Duponchelle
053f1523ad Multiqueue: don't allow dropping SEGMENT_DONE events
https://bugzilla.gnome.org/show_bug.cgi?id=780795
2017-09-18 14:12:06 +02:00
Edward Hervey
e2f8dc0aef plugins: *queue* elements: Handle STREAM_START in EOS situation
When queue-like elements are in "EOS" situation (received GST_FLOW_EOS
from downstream or EOS was pushed), they drain buffers/events that
wouldn't be processed anyway and let through events that might
modify the EOS situation.

Previously only GST_EVENT_EOS and GST_EVENT_SEGMENT events were let
through, but we also need to allow GST_EVENT_STREAM_START to go
through since it resets the EOS state of pads since 1.6

https://bugzilla.gnome.org/show_bug.cgi?id=786034
2017-08-09 10:54:17 +02:00
Edward Hervey
78ee0dfad1 multiqueue: Fix access to NULL pointer
sq can be NULL.

Also fix commit message (it's the queue we are iterating over that we
are logging, not the one passed as argument).

CID #1415569
2017-07-20 14:18:51 +02:00
Seungha Yang
1b0059e0c5 multiqueue: Calculate interleave only within each streaming thread
... and use the biggest interleave value among streaming threads.
This is to optimize multiqueue size adaptation on adaptive streaming
use case with "use-interleave" property.

https://bugzilla.gnome.org/show_bug.cgi?id=784448
2017-07-19 17:19:14 +02:00
Jan Schmidt
9b0a6ca7ab multiqueue: Make min-interleave-time a configurable property
Remove a FIXME about making the minimum interleave
buffering a configurable property
2017-03-09 12:09:57 +11:00
Thibault Saunier
a87b4551a6 Port gtk-doc comments to their equivalent markdown syntax
Modernizing our documentation and preparing a possible move to hotdoc.
This commits also adds missing @title metadatas to all SECTIONs
2017-01-27 16:36:38 -03:00
Sebastian Dröge
47f9223750 multiqueue: Protect against spurious wakeups of the condition variable 2016-12-18 13:59:12 +02:00
Seungha Yang
ce11ddeb0c multiqueue: Fix overflow on get_buffering_level()
guint64 denominator factor for gst_util_uint64_scale_int() can cause overflow

https://bugzilla.gnome.org/show_bug.cgi?id=775921
2016-12-11 13:30:52 +02:00
Jan Schmidt
fa3eac2d28 multiqueue: Make sure not-linked streams get woken up
When running in sync-by-running-time mode, pad groups
that have exactly 1 pad and it's not-linked might never
wake up after computing a high time, as the per-pad-group
high time was only recomputed when a pad in the group
advances.

Wake those up using the global multiqueue high-time across
all other groups instead.

https://bugzilla.gnome.org/show_bug.cgi?id=774322
2016-11-17 00:46:31 +11:00
Carlos Rafael Giani
5988095f90 multiqueue: Add higher-resolution low/high-watermark properties
low/high-watermark are of type double, and given in range 0.0-1.0. This
makes it possible to set low/high watermarks with greater resolution,
which is useful with large multiqueue max sizes and watermarks like 0.5%.

Also adding a test to check the fill and watermark level behavior.

https://bugzilla.gnome.org/show_bug.cgi?id=770628
2016-08-31 12:56:19 +03:00
Carlos Rafael Giani
67874ea86d multiqueue: Distinguish between buffering percentage and buffering level
To make the code clearer, and to facilitate future improvements, introduce
a distinction between the buffering level and the buffering percentage.

Buffering level: the queue's current fill level. The low/high watermarks
are in this range.

Buffering percentage: percentage relative to the low/high watermarks
(0% = low watermark, 100% = high watermark).

To that end, get_percentage() is renamed to get_buffering_level(). Also,
low/high_percent are renamed to low/high_watermark to avoid confusion.
mq->buffering_percent values are now normalized in the 0..100 range for
buffering messages inside update_buffering(), and not just before sending
the buffering message. Finally the buffering level range is parameterized
by adding a new constant called MAX_BUFFERING_LEVEL.

https://bugzilla.gnome.org/show_bug.cgi?id=770628
2016-08-31 12:56:19 +03:00
Carlos Rafael Giani
e83412b4fd multiqueue: Rename percent/percent_changed to buffering_percent(_changed)
This is a prerequisite for subsequent commits, and makes queue2 and
multiqueue code a little more consistent.

https://bugzilla.gnome.org/show_bug.cgi?id=770628
2016-08-31 12:56:19 +03:00
Edward Hervey
3117525cb6 multiqueue: Fix high_time wakeup logic
When calculating the high_time, cache the group value in each singlequeue.

This fixes the issue by which wake_up_next_non_linked() would use the global
high-time to decide whether to wake-up a waiting thread, instead of the group
one, resulting in those threads constantly spinning.

Tidy up a bit the waiting logic while we're at it.

With this patch, we go from 212% playing a 8 audio / 8 video file down to less
than 10% (most of it being the video decoding).

https://bugzilla.gnome.org/show_bug.cgi?id=770225
2016-08-30 12:28:40 +03:00
Thibault Saunier
4714ef2f8e Make use of the new GST_ELEMENT_FLOW_ERROR API all around.
https://bugzilla.gnome.org/show_bug.cgi?id=770158
2016-08-27 09:33:20 -03:00
Josep Torra
5f021759b3 multiqueue: removed redundant call to g_thread_self
Remove an unneeded call to g_thread_self and minor coding style fix.
2016-08-08 16:42:06 +02:00
Edward Hervey
530001661d multiqueue: Fix behaviour with not-linked and eos pads
This is an update on c9b6848885
multiqueue: Fix not-linked pad handling at EOS

While that commit did fix the behaviour if upstream sent a GST_EVENT_EOS,
it would break the same issue when *downstream* returns GST_FLOW_EOS
(which can happen for example when downstream decoders receive data
from after the segment stop).

GST_PAD_IS_EOS() is only TRUE when a GST_EVENT_EOS has flown through it
and not when a GST_EVENT_EOS has gone through it.

In order to handle both cases, also take into account the last flow
return.

https://bugzilla.gnome.org/show_bug.cgi?id=763770
2016-07-01 09:47:31 +02:00
Tim-Philipp Müller
715939cce7 multiqueue: add gtk-doc blurb for new pad property 2016-06-30 14:37:17 +01:00
Edward Hervey
f4ba43b15d multiqueue: Add a pad property to "group" streams
When syncing by running time, multiqueue will throttle unlinked streams
based on a global "high-time" and the pending "next_time" of a stream.

The idea is that we don't want unlinked streams to be "behind" the global
running time of linked streams, so that if/when they get linked (like when
switching tracks) decoding/playback can resume from the same position as
the other streams.

The problem is that it assumes elements downstream will have a more or less
equal buffering/latency ... which isn't the case for streams of different
type. Video decoders tend to have higher latency (and therefore consume more
from upstream to output a given decoded frame) compared to audio ones, resulting
in the computed "high_time" being at the position of the video stream,
much further than the audio streams.

This means the unlinked audio streams end up being quite a bit after the linked
audio streams, resulting in gaps when switching streams.

In order to mitigate this issue, this patch adds a new "group-id" pad property
which allows users to "group" streams together. Calculating the high-time will
now be done not only globally, but also per group. This ensures that within
a given group unlinked streams will be throttled by that group's high-time
instead.

This fixes gaps when switching downstream elements (like switching audio tracks).
2016-06-30 14:45:10 +02:00
Matej Knopp
efb1955ffb multiqueue: Ignore time when determining whether sparse stream limits have been reached
Basically, sq->max_size.visible is never increased for sparse streams in
overruncb when empty queue has been found;

If the queue is sparse it just skip the entire logic determining whether
max_size.visible should be increased, deadlocking the demuxer.

What should be done instead is that when determining if limits have been
reached, to ignore time for sparse streams, as the buffer may be far in the
future.

https://bugzilla.gnome.org/show_bug.cgi?id=765736
2016-04-29 16:09:23 +03:00
Carlos Rafael Giani
3bd5aeac52 multiqueue: Recheck buffering status after changing low threshold
https://bugzilla.gnome.org/show_bug.cgi?id=763757
2016-04-15 16:04:54 +03:00
Carlos Rafael Giani
a1d2f387c6 multiqueue: Recalculate fill level after changing high-threshold
This ensures the following special case is handled properly:

1. Queue is empty
2. Data is pushed, fill level is below the current high-threshold
3. high-threshold is set to a level that is below the current fill level

Since mq->percent wasn't being recalculated in step #3 properly, this
caused the multiqueue to switch off its buffering state when new data is
pushed in, and never post a 100% buffering message. The application will
have received a <100% buffering message from step #2, but will never see
100%.

Fix this by recalculating the current fill level percentage during
high-threshold property changes in the same manner as it is done when
use-buffering is modified.

https://bugzilla.gnome.org/show_bug.cgi?id=763757
2016-04-15 16:04:54 +03:00
Vincent Penquerc'h
53a7a991e2 multiqueue: catch errors and flushing case after lock
This ensures we can not get into an indefinite wait on the
following cond var wait.

https://bugzilla.gnome.org/show_bug.cgi?id=764999
2016-04-14 09:12:27 +01:00
Jan Schmidt
c9b6848885 multiqueue: Fix not-linked pad handling at EOS
Ensure that not-linked pads will drain out at EOS by
correctly detecting the EOS condition based on the EOS
pad flag (which indicates we actually pushed an EOS),
and make sure that not-linked pads are woken when doing
EOS processing on linked pads.

https://bugzilla.gnome.org/show_bug.cgi?id=763770
2016-03-18 21:21:44 +11:00
Tim-Philipp Müller
cf0680017e elements: use new gst_element_class_add_static_pad_template()
https://bugzilla.gnome.org/show_bug.cgi?id=762778
2016-02-28 13:10:07 +00:00
Tim-Philipp Müller
469475f06b multiqueue: two small fixes for when an existing pad is requested
Unlock when returning NULL from gst_single_queue_new(), and don't
crash with debug logging enabled if NULL is returned.

Spotted by Steven Hoving.
2016-01-20 10:06:14 +00:00
Sebastian Dröge
33ded76ab8 queue/multiqueue: Don't special-case CAPS events in the event handlers
For CAPS events we will never ever have a FALSE return value here, so just
remove the dead code instead of causing future confusion.
2015-12-14 11:20:43 +01:00
Edward Hervey
5f1764232f multiqueue: Don't use random segment.position from upstream
segment.position is meant for internal usage only, but the various
GST_EVENT_SEGMENT creationg/parsing functions won't clear that field.

Use the appropriate segment boundary as an initial value instead
2015-12-09 17:41:49 +01:00
Edward Hervey
906a50b995 multiqueue: Fix set/get property
Blame it on the rebasing :)
2015-12-02 16:28:58 +01:00
Edward Hervey
56d4650789 pad: Implement GstPadEventFullFunction
API: GstPadEventFullFunction

Returns a GstFlowReturn, allows asynchronous elements to properly
propagate flow returns

https://bugzilla.gnome.org/show_bug.cgi?id=757821
2015-12-02 16:04:03 +01:00
Edward Hervey
4d96e5b834 multiqueue: Use signed clock values for running time calculation
This improves the accuracy of queue levels and when to push buffers
for buffers falling outside of the segment

https://bugzilla.gnome.org/show_bug.cgi?id=757193
2015-12-02 16:03:20 +01:00
Edward Hervey
c51d2b2a37 multiqueue: Add an extra cache time for unlinked streams
When synchronizing the output by time, there are some use-cases (like
allowing gapless playback downstream) where we want the unlinked streams
to stay slightly behind the linked streams.

The "unlinked-cache-time" property allows the user to specify by how
much time the unlinked streams should wait before pushing again.
2015-12-02 16:03:20 +01:00
Edward Hervey
72c31876aa multiqueue: Optimize multiqueue sizes based on interleave
Multiqueue should only be used to cope with:
* decoupling upstream and dowstream threading (i.e. having separate threads
  for elementary streams).
* Ensuring individual queues have enough space to cope with upstream interleave
  (distance in stream time between co-located samples). This is to guarantee
  that we have enough room in each individual queues to provide new data in
  each, without being blocked.
* Limit the queue sizes to that interleave distance (and an extra minimal
  buffering size). This is to ensure we don't consume too much memory.

Based on that, multiqueue now continuously calculates the input interleave
(per incoming streaming thread). Based on that, it calculates a target
interleave (currently 1.5 x real_interleave + 250ms padding).

If the target interleave is greater than the current max_size.time, it will
update it accordingly (to allow enough margin to not block).
If the target interleave goes down by more than 50%, we re-adjust it once
we know we have gone past a safe distance (2 x current max_size.time).

This mode can only be used for incoming streams that are guaranteed to be
properly timestamped.

Furthermore, we ignore sparse streams when calculating interleave and maximum
size of queues.

For the simplest of use-cases (single stream), multiqueue acts as a single
queue with a time limit of 250ms.
If there are multiple inputs, but each come from a different streaming thread,
the maximum time limit will also end up being 250ms.

On regular files (more than one input stream from the same upstream streaming
thread), it can reduce the total memory used as much as 10x, ending up with
max_size.time around 500ms.

Due to the adaptive nature, it can also cope with changing interleave (which
can happen commonly on some files at startup/pre-roll time)
2015-12-02 16:03:20 +01:00
Vincent Penquerc'h
f6069c2c72 multiqueue: use new stream-start event sparse flag to avoid overreading subtitles
This will mean a much lower delay before a subtitles track changes take
effect. Also avoids excessive memory usage in many cases.

This will also consider sparse streams as (individually) never full, so
as to avoid blocking all playback due to one sparse stream.

https://bugzilla.gnome.org/show_bug.cgi?id=600648
2015-12-02 16:03:20 +01:00
Edward Hervey
785edec270 multiqueue: Fix high_time computation
* Avoid the computation completely if we know we don't need it (not in
  sync time mode)
* Make sure we don't override highest time with GST_CLOCK_TIME_NONE on
  unlinked pads
* Ensure the high_time gets properly updated if all pads are not linked
* Fix the comparision in the loop whether the target high time is the same
  as the current time
* Split wake_up_next_non_linked method to avoid useless calculation

https://bugzilla.gnome.org/show_bug.cgi?id=757353
2015-12-02 16:03:16 +01:00
Edward Hervey
692c0dc060 multiqueue: Use buffer DTS if present, else PTS
In order to accurately determine the amount (in time) of data
travelling in queues, we should use an increasing value.

If buffers are encoded and potentially reordered, we should be
using their DTS (increasing) and not PTS (reordered)

https://bugzilla.gnome.org/show_bug.cgi?id=756507
2015-10-27 08:38:42 +01:00
Edward Hervey
7c8f76a39f multiqueue: Improve incoming SEGMENT handling
Previously this code was just blindly setting the cached flow return
of downstream to GST_FLOW_OK when we get a SEGMENT.

The problem is that this can not be done blindly. If downstream was
not linked, the corresponding sinqlequeue source pad thread might be
waiting for the next ID to be woken up upon.

By blindly setting the cached return value to GST_FLOW_OK, and if that
stream was the only one that was NOT_LINKED, then the next time we
check (from any other thread) to see if we need to wake up a source pad
thread ... we won't even try, because none of the cached flow return
are equal to GST_FLOW_NOT_LINKED.

This would result in that thread never being woken up

https://bugzilla.gnome.org/show_bug.cgi?id=756645
2015-10-19 14:31:05 +02:00
Edward Hervey
14ad763698 multiqueue: Accept STREAM_START after EOS
In the same way core now allows STREAM_START to remove the flushing
state from pads, we need to do the same thing in multiqueue
2015-10-15 17:21:40 +02:00
Duncan Palmer
0ad06434da multiqueue: Don't automatically enter the buffering state when use-buffering is set.
There is no reason I can see to set mq->buffering = TRUE when
use_buffering is set; the code here also calls update_buffering(), which
will set mq->buffering = TRUE if this is warranted because of low buffer
levels.

https://bugzilla.gnome.org/show_bug.cgi?id=745937
2015-04-10 10:40:00 -03:00
Sebastian Dröge
5fc2a3fd66 multiqueue: Don't grow queue infinitely if only one pad is linked
This was introduced by
https://bugzilla.gnome.org/show_bug.cgi?id=719893
https://bugzilla.gnome.org/show_bug.cgi?id=722891
but it doesn't make any sense at all and causes huge memory leaks.

https://bugzilla.gnome.org/show_bug.cgi?id=744253
2015-03-15 14:19:17 +00:00