Commit graph

267 commits

Author SHA1 Message Date
Edward Hervey
1bca404191 queue2: Add details of query in debug log 2018-12-17 09:41:45 +01:00
Matthew Waters
4fc4ad87d5 query: add a new bitrate query
Allows determining from downstream what the expected bitrate of a stream
may be which is useful in queue2 for setting time based limits when
upstream does not provide timing information.

Implement bitrate query handling in queue2

https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/issues/60
2018-11-07 15:04:14 +00:00
Matthew Waters
c4ccff7861 queue2: avoid ping-pong between 0% and 100% buffering messages
If upstream is pushing buffers larger than our limits, only 1 buffer
is ever in the queue at a time.  Once that single buffer has left the
queue, a 0% buffering message would be posted followed immediately by a
100% buffering message when the next buffer was inserted into the queue
a very short time later.  As per the recommendations, This would result
in the application pausing for a short while causing the appearance of
a short stutter.

The first step of a solution involves not posting a buffering message if
there is still data waiting on the sink pad for insertion into the queue.
This successfully drops the 0% messages from being posted however a
message is still posted on each transition to 100% when the new buffer
arrives resulting in a string of 100% buffering messages.  We silence
these by storing the last posted buffering percentage and only posting a
new message when it is different from or last posted message.
2018-11-07 15:04:14 +00:00
Edward Hervey
98fabd2fa2 queue2: Reset result flow when retrying
If we ever get a GST_FLOW_EOS from downstream, we might retry
pushing new data. But if pushing that data doesn't return a
GstFlowReturn (such as pushing events), we would end up returning
the previous GstFlowReturn (i.e. EOS).

Not properly resetting it would cause cases where queue2 would
stop pushing on the first GstEvent stored (even if there is more
data contained within).
2018-10-22 13:52:30 +02:00
Dimitrios Katsaros
ac7f8b6337 queue2: use GstQueueArray
When using queue2 as a queue it was using GQueue with
individually allocated queue items, so two allocs for
each item. With GstQueueArray we can avoid those.

https://bugzilla.gnome.org/show_bug.cgi?id=796483
2018-06-04 11:10:05 +01:00
Matthew Waters
5a5a548b09 plugins: Don't force 64-bit file/seek functions variants on android
Most functions are automatically chosen from the _FILE_OFFSET_BITS
define, the remaining one (fstat) is only available on API >= 21 so
check for that
2018-03-02 18:25:21 +11:00
Tim-Philipp Müller
c2917b1ae2 plugins: use new gst_buffer_list_calculate_size() 2017-09-17 12:36:03 +01:00
Edward Hervey
6566eff20d queue2: Allow re-usability after EOS
After EOS, it is possible for a pad to be resetted by sending
either a STREAM_START or SEGMENT event

Mimic the same behaviour when receiving STREAM_START/SEGMENT events
in queue2 if we are EOS'd

https://bugzilla.gnome.org/show_bug.cgi?id=786056
2017-08-09 16:18:40 +02:00
Edward Hervey
e2f8dc0aef plugins: *queue* elements: Handle STREAM_START in EOS situation
When queue-like elements are in "EOS" situation (received GST_FLOW_EOS
from downstream or EOS was pushed), they drain buffers/events that
wouldn't be processed anyway and let through events that might
modify the EOS situation.

Previously only GST_EVENT_EOS and GST_EVENT_SEGMENT events were let
through, but we also need to allow GST_EVENT_STREAM_START to go
through since it resets the EOS state of pads since 1.6

https://bugzilla.gnome.org/show_bug.cgi?id=786034
2017-08-09 10:54:17 +02:00
Edward Hervey
4312119d16 queue2: Handle buffering levels on NOT_LINKED
When downstream returns NOT_LINKED, we return the buffering level
as being 100%.

Since the queue is no longer being consumed/used downstream, we
want applications to essentially "ignore" this queue for buffering
purposes.

If other streams are still being used, those stream buffering levels
will be used. If none are used, upstream will post an error message
on the bus indicating no streams are used.

https://bugzilla.gnome.org/show_bug.cgi?id=785799
2017-08-07 15:20:55 +02:00
Thiago Santos
045700e80e queue2: avoid return flushing if we have a not-linked
Return the correct flow return instead of returning always flushing.
This would cause queue2 to convert not-linked to flushing and making
upstream elements stop.

https://bugzilla.gnome.org/show_bug.cgi?id=776999
2017-03-11 11:19:33 -08:00
Thibault Saunier
a87b4551a6 Port gtk-doc comments to their equivalent markdown syntax
Modernizing our documentation and preparing a possible move to hotdoc.
This commits also adds missing @title metadatas to all SECTIONs
2017-01-27 16:36:38 -03:00
Sebastian Dröge
3fab00b46a queue/queue2: Protect against spurious condition variable wakeups
Make sure that we only wake up when we have to flush, or when this
specific query was handled.

https://bugzilla.gnome.org/show_bug.cgi?id=776039
2016-12-13 21:03:54 +02:00
Sebastian Dröge
33c239828b queue/queue2: Ensure that the streaming thread is unlocked after deactivating the srcpad
It might happen that the srcpad task function is never called at all, in
which case unlocking everything from there will never happen.

Make sure to unlock everything another time after the task function is
definitely stopped.

https://bugzilla.gnome.org/show_bug.cgi?id=776039
2016-12-13 21:03:54 +02:00
Jan Schmidt
a6ca8dfb42 queues: Don't return negative position queries.
When subtracting queued data sizes from upstream queries
in queue, queue2, downloadbuffer and typefind, clamp the
result to not go negative, in case upstream returned
a nonsense value that's too small (as could happen if
upstream is estimating, or just broken)
2016-11-16 00:12:22 +11:00
Edward Hervey
ace418d226 queue2: Implement gst_event_full_func handling
Same as we do for queue
2016-10-08 13:56:18 +02:00
Carlos Rafael Giani
7413064f06 queue2: Update buffering if its enabled and low/high watermarks are changed
https://bugzilla.gnome.org/show_bug.cgi?id=771210
2016-09-17 09:54:29 +01:00
Thibault Saunier
4714ef2f8e Make use of the new GST_ELEMENT_FLOW_ERROR API all around.
https://bugzilla.gnome.org/show_bug.cgi?id=770158
2016-08-27 09:33:20 -03:00
Carlos Rafael Giani
db66cb51b3 queue2: Add higher-resolution low/high-watermark properties
low/high-watermark are of type double, and given in range 0.0-1.0. This
makes it possible to set low/high watermarks with greater resolution,
which is useful with large queue2 max sizes and watermarks like 0.5%.

Also adding a test to check the fill and watermark level behavior.

https://bugzilla.gnome.org/show_bug.cgi?id=769449
2016-08-25 11:54:33 +03:00
Carlos Rafael Giani
e0f1a9e618 queue2: Distinguish between buffering percentage and buffering level
To make the code clearer, and to facilitate future improvements, introduce
a distinction between the buffering level and the buffering percentage.

Buffering level: the queue's current fill level. The low/high watermarks
are in this range.

Buffering percentage: percentage relative to the low/high watermarks
(0% = low watermark, 100% = high watermark).

To that end, get_buffering_percent() is renamed to get_buffering_level(),
and the code at the end that transforms to the buffering percentage is
factored out into a new convert_to_buffering_percent() function. Also,
the buffering level range is parameterized by adding a new constant called
MAX_BUFFERING_LEVEL.

https://bugzilla.gnome.org/show_bug.cgi?id=769449
2016-08-25 11:54:33 +03:00
Edward Hervey
5154dcfbdc queue2: Post buffering messages earlier in ringbuffer mode
In ringbuffer mode we need to make sure we post buffering messages *before*
blocking to wait for data to be drained.

Without this, we would end up in situations like this:
* pipeline is pre-rolling
* Downstream demuxer/decoder has pushed data to all sinks, and demuxer thread
  is blocking downstream (i.e. not pulling from upstream/queue2).
* Therefore pipeline has pre-rolled ...
* ... but queue2 hasn't filled up yet, therefore the application waits for
  the buffering 100% messages before setting the pipeline to PLAYING
* But queue2 can't post that message, since the 100% message will be posted
  *after* there is room available for that last buffer.

https://bugzilla.gnome.org/show_bug.cgi?id=769802
2016-08-12 16:26:40 +02:00
Edward Hervey
1ff80fbf52 queue2: Fix average input rate calculation on small input range
When dealing with small-ish input data coming into queue2, such as
adaptivedemux fragments, we would never take into account the last
<200ms of data coming in.

The problem is that usually on TCP connection the download rate
gradually increases (i.e. the rate is lower at the beginning of a
download than it is later on). Combined with small download time (less
than a second) we would end up with a computed average input rate
which was sometimes up to 30-50% off from the *actual* average input
rate for that fragment.

In order to fix this, force the average input rate calculation when
we receive an EOS so that we take into account that final window
of data.

https://bugzilla.gnome.org/show_bug.cgi?id=768649
2016-07-11 10:08:44 +02:00
Vincent Penquerc'h
b3802f7a9e queue2: fix crash deleting current region for small ring buffers
Ensure we do not attempt to destroy the current range. Doing so
causes the current one to be left dangling, and it may be dereferenced
later, leading to a crash.

This can happen with a very small queue2 ring buffer (10000 bytes)
and 4 kB buffers.

repro case:

gst-launch-1.0 fakesrc sizetype=2 sizemax=4096 ! \
queue2 ring-buffer-max-size=1000 ! fakesink sync=true

https://bugzilla.gnome.org/show_bug.cgi?id=767688
2016-06-21 10:20:20 +01:00
Sebastian Dröge
722ad08733 queue: Only unblock upstream waiting for the query once downstream is finished
... when flushing and deactivating pads. Otherwise downstream might have a
query that was already unreffed by upstream, causing crashes or other
interesting effects.

https://bugzilla.gnome.org/show_bug.cgi?id=763496
2016-05-15 11:06:08 +03:00
Tim-Philipp Müller
cf0680017e elements: use new gst_element_class_add_static_pad_template()
https://bugzilla.gnome.org/show_bug.cgi?id=762778
2016-02-28 13:10:07 +00:00
Edward Hervey
555df9d614 plugins: Check return values of gst_buffer_map()
They can fail for various reasons.

For non-fatal cases (such as the dump feature of identiy and fakesink),
we just silently skip it.

For other cases post an error message.

https://bugzilla.gnome.org/show_bug.cgi?id=728326
2016-02-23 17:30:45 +01:00
Sebastian Dröge
fd253ab1ab Revert "queue2: add overrun signal"
This reverts commit 8ae8b2723d.

It's not used anymore by anything and was considered a bad idea in general.
2016-02-16 19:11:59 +02:00
Jan Schmidt
c1f4920859 queue2: Add use-tags-bitrate property
The use-tags-bitrate property makes queue2 look at
tag events in the stream and extract a bitrate for the
stream to use when calculating a duration for buffers
that don't have one explicitly set.

This lets queue2 sensibly buffer to a time threshold
for any bytestream for which the general bitrate is known.
2016-01-20 03:13:44 +11:00
Tim-Philipp Müller
61e2f1eab1 queue2: avoid calculating fill levels multiple times
Macro expansion means we might calculate the fill level once
for the check and then possibly again for the return value.
2016-01-06 20:00:46 +00:00
Tim-Philipp Müller
8c6c0bef88 queue2: fix fill level arithmetic overflow with large values
Based on patch by: Aleksander Wabik <awabik@opera.com>

https://bugzilla.gnome.org/show_bug.cgi?id=755971
2016-01-06 20:00:46 +00:00
Reynaldo H. Verdejo Pinochet
85e7cfd811 queue2: fix some typos 2015-12-12 13:13:54 -08:00
Reynaldo H. Verdejo Pinochet
ea8bbd58df queue2: enable large file support on Android
https://bugzilla.gnome.org/show_bug.cgi?id=758980
2015-12-12 13:13:54 -08:00
Jan Schmidt
725a71ea0c queue2: Don't report 0% unless empty
When preparing a buffering message, don't report 0% if there
is any bytes left in the queue at all. We still have something
to push, so don't tell the app to start buffering - maybe
we'll get more data before actually running dry.
2015-12-02 19:32:12 +11:00
Tim-Philipp Müller
ffecef4fa0 queue2: don't print criticals when receiving custom events in ring buffer mode
Downgrade from g_warning to GST_WARNING log message.

https://bugzilla.gnome.org/show_bug.cgi?id=758276
2015-11-18 11:46:45 +00:00
Duncan Palmer
45fa81e564 queue2: Add the avg-in-rate property.
https://bugzilla.gnome.org/show_bug.cgi?id=733959
2015-11-06 12:32:16 -03:00
Philippe Normand
8ae8b2723d queue2: add overrun signal
Notifies that the queue2 is full, same as queue does

https://bugzilla.gnome.org/show_bug.cgi?id=733959
2015-11-06 12:32:10 -03:00
Edward Hervey
a9c923d585 queue/queue2: Use GST_BUFFER_DTS_OR_PTS
The input of queue/queue2 might have DTS set, in which cas we want
to take that into account (instead of the PTS) to calculate position
and queue levels.

https://bugzilla.gnome.org/show_bug.cgi?id=756507
2015-10-27 08:49:02 +01:00
Tim-Philipp Müller
c55bfecc55 Revert "queue2: Process SEEKING query"
This caused problems with oggdemux when queue2 was
operating in queue mode and the souphttpsrc upstream
is not seekable because the server doesn't support
range requests. It would then still claim seekability
and then things go wrong from there.

This reverts commit 7b0b93dafe.

https://bugzilla.gnome.org/show_bug.cgi?id=753887
2015-08-31 12:07:10 +01:00
Eunhae Choi
ed7e0e744f queue2: not update upstream size with negative value
upstream_size can be negative but queue->upstream_size is unsigned type.
to get a chance to update queue->upstream_size in gst_queue2_get_range()
it should keep the default value.

https://bugzilla.gnome.org/show_bug.cgi?id=753011
2015-08-04 07:03:24 -03:00
Jan Alexander Steffens (heftig)
7b0b93dafe queue2: Process SEEKING query
Add QUERY_SEEKING handling to queue2, so RTMP live streams become
seekable when a queue2 in download or ringbuffer mode is inserted:

rtmpsrc ! queue2 ! flvdemux

https://bugzilla.gnome.org/show_bug.cgi?id=733351
2015-03-23 10:47:41 +01:00
Sebastian Dröge
bc77a3fa0a queue2: Signal the sinkpad thread if a flow error happened
It might still be waiting for a query to be handled, or the queue to become
empty again for the next item. Also if downstream returns FLUSHING, flush the
queue like we do in queue and multiqueue.
2015-03-03 12:53:13 +01:00
Sebastian Dröge
927666642e queue2: Count the number of buffers in a buffer list for updating the current levels
instead of just assuming one buffer.
2015-02-18 11:03:08 +02:00
Thiago Santos
8d835ec400 queue2: percentage is relative to high-percent
When comparing percentage values, compare with 0-100 scale as it
has already been made relative to 0-high_percent, otherwise we mark
the queue as not buffering and report a 50% to the user. This leads to
a buffering stall as the user assumes the queue is still buffering but
it thinks it isn't.

https://bugzilla.gnome.org/show_bug.cgi?id=736969
2014-11-23 05:51:12 -03:00
Sebastian Dröge
03376a2086 queue2: update segment position on GAP events to calculate levels properly
https://bugzilla.gnome.org/show_bug.cgi?id=737498
2014-10-02 10:58:24 +03:00
Thiago Santos
ecf479e3d1 queue2: do not post buffering messages holding the lock
It might cause deadlocks to post messages while holding the queue2
lock. To avoid this a new boolean flag is set whenever a new
buffering percent is found. The message is posted after the lock
is released.

To make sure the buffering messages are posted in the right order, messages
are posted holding another lock. This prevents 2 threads trying to post
messages at the same time.

https://bugzilla.gnome.org/show_bug.cgi?id=736969
2014-09-19 20:35:29 -03:00
Tim-Philipp Müller
49c6f923de coreelements: mark properties with MUTABLE_PLAYING 2014-09-14 16:08:07 +01:00
Sebastian Dröge
ee662efdd1 queue2: Post errors if we receive EOS after downstream reported an error
There will be no further data flow that would allow us to propagate the
error upstream, causing nobody at all to post an error message.
2014-08-13 12:20:51 +03:00
Tim-Philipp Müller
6eb6d9ec38 docs: remove outdated and pointless 'Last reviewed' lines from docs
They are very confusing for people, and more often than not
also just not very accurate. Seeing 'last reviewed: 2005' in
your docs is not very confidence-inspiring. Let's just remove
those comments.
2014-04-26 21:21:51 +01:00
Vincent Penquerc'h
f3dd7937a2 queue2: fix event/preroll deadlock differently
The qlock is released between popping a buffer from the queue
and pushing it. When this buffer causes the sink to wait in
preroll, this lets a query see that the queue is empty, and
push the query then wait for it to be serviced. However, this
will not be done till after peroll, and this will thus block.
If upstream was waiting on buffering to reach 100% before
switching to PLAYING, a deadlock would ensue.

This had been fixed recently by failing queries when the
queue2 was buffering, but this happens to break some other
case (playbin on a local http server and matroska), while
this patch works for both.

See https://bugzilla.gnome.org/show_bug.cgi?id=728345
2014-04-16 15:17:04 +01:00
Tim-Philipp Müller
d5c2439f08 queue2: use g_strerror() instead of strerror()
Need UTF-8 encoding.
2014-04-07 17:49:14 +01:00