Commit graph

2137 commits

Author SHA1 Message Date
Sebastian Dröge
e953f20786 identity: Add a " " after pts: in the silent=false output 2016-02-25 14:12:00 +02:00
Edward Hervey
555df9d614 plugins: Check return values of gst_buffer_map()
They can fail for various reasons.

For non-fatal cases (such as the dump feature of identiy and fakesink),
we just silently skip it.

For other cases post an error message.

https://bugzilla.gnome.org/show_bug.cgi?id=728326
2016-02-23 17:30:45 +01:00
Sebastian Dröge
fd253ab1ab Revert "queue2: add overrun signal"
This reverts commit 8ae8b2723d.

It's not used anymore by anything and was considered a bad idea in general.
2016-02-16 19:11:59 +02:00
Alex Henrie
78ba7f9a8e typefindelement: Improve English grammar
https://bugzilla.gnome.org/show_bug.cgi?id=761273
2016-01-29 10:34:44 +01:00
Stefan Sauer
ec75b68984 tracerrecord: don't leak the spec structures
Change the gst_tracer_record_new() api to take the parameters the make the
spec structure directly. This allows us to own the top-level structure and
also collect the args so that we can take ownership of the sub-structures.

https://bugzilla.gnome.org/show_bug.cgi?id=760821
2016-01-22 10:07:26 +01:00
Tim-Philipp Müller
469475f06b multiqueue: two small fixes for when an existing pad is requested
Unlock when returning NULL from gst_single_queue_new(), and don't
crash with debug logging enabled if NULL is returned.

Spotted by Steven Hoving.
2016-01-20 10:06:14 +00:00
Stefan Sauer
f69382c5f9 tracer/gststats: fix mismatch between '.class' and tracer args
Clean up from the recent changes. The logging descriptiors did not match what we logged.
2016-01-20 09:14:12 +01:00
Jan Schmidt
c1f4920859 queue2: Add use-tags-bitrate property
The use-tags-bitrate property makes queue2 look at
tag events in the stream and extract a bitrate for the
stream to use when calculating a duration for buffers
that don't have one explicitly set.

This lets queue2 sensibly buffer to a time threshold
for any bytestream for which the general bitrate is known.
2016-01-20 03:13:44 +11:00
Tim-Philipp Müller
8641d7c911 tracers: fix thread-id casts to 64-bit ints on 32-bit systems
https://bugzilla.gnome.org/show_bug.cgi?id=760762
2016-01-18 11:28:28 +00:00
Stefan Sauer
84554af301 tracer/stats: use the right log template
When porting we used the wrong record (copy and paste).
2016-01-16 21:23:10 +01:00
Stefan Sauer
af91d23204 tracer: use the new flags to create the optional field in the format string
This spares us explicitly listing the field in the spec. and thus hide this
implementation detail.
2016-01-16 19:21:00 +01:00
Stefan Sauer
8cd76a918a tracer: add a GstTracerValueFlags and replace strings
This allows us to document the flags and makes the logs a bit smaller.
2016-01-16 18:52:32 +01:00
Tim-Philipp Müller
8b88fc487f gst.h: Don't spew warnings if GST_USE_UNSTABLE_API is not defined
Only hide GstTracer and GstTracerRecord API behind GST_USE_UNSTABLE_API,
but don't spew any warnings, otherwise everyone has to define this
to avoid compiler warnings.

This reverts parts of commit 89ee5d948d.
2016-01-16 16:00:16 +00:00
Stefan Sauer
89ee5d948d tracer: add a GFlag for the tracer scope
Port all tracers. Add the GST_USE_UNSTABLE_API flag to the internal CFLAGS so
that we don't have to specify this for gir, docs, mkenum, ...
2016-01-16 13:31:51 +01:00
Sebastian Dröge
485b15d8a9 output-selector: Make access to the active pad and last buffer thread-safe
Both can be modified from different threads at the same time.
2016-01-16 10:48:02 +01:00
Sebastian Dröge
0e132260a3 output-selector: Notify when the active-pad property is changing 2016-01-16 10:47:36 +01:00
Stefan Sauer
10b78d872b tracerrecord: add a log record class
We use this class to register tracer log entry metadata and build a log
template. With the log template we can serialize log data very efficiently.
This also simplifies the logging code, since that is now a simple varargs
function that is not exposing the implementation details.

Add docs for the new class and basic tests.

Remove the previous log handler.

Fixes #760267
2016-01-15 21:54:01 +01:00
Stefan Sauer
30741e0206 tracers: code clean ups
Drop some trailing whilespace. Make field order consistent.
2016-01-10 14:30:05 +01:00
HoonHee Lee
788938640d funnel: improve debug message
https://bugzilla.gnome.org/show_bug.cgi?id=745939
2016-01-08 19:28:43 +00:00
Stefan Sauer
2f41e7bc6a tracer: harmonize the query hooks
In post hooks always pass the return value as the last param. Pass the query
also to post hooks since it is still alive.
2016-01-07 23:03:48 +01:00
Stefan Sauer
fb249dadcb tracers/stats: add missing parameters to callback functions 2016-01-07 22:47:37 +01:00
Tim-Philipp Müller
61e2f1eab1 queue2: avoid calculating fill levels multiple times
Macro expansion means we might calculate the fill level once
for the check and then possibly again for the return value.
2016-01-06 20:00:46 +00:00
Tim-Philipp Müller
8c6c0bef88 queue2: fix fill level arithmetic overflow with large values
Based on patch by: Aleksander Wabik <awabik@opera.com>

https://bugzilla.gnome.org/show_bug.cgi?id=755971
2016-01-06 20:00:46 +00:00
Stefan Sauer
5e40639be7 tracer: make gst_tracing_register_hook_id static
We don't need to expose this as public API. Change the only plugin that was
using it.
2016-01-06 20:46:58 +01:00
Sebastian Dröge
7e761798bd tee: Check if parsing the name template with sscanf() was successful
If not, go back to the automatic pad numbering.

CID 1195129
2015-12-14 13:06:57 +01:00
Sebastian Dröge
33ded76ab8 queue/multiqueue: Don't special-case CAPS events in the event handlers
For CAPS events we will never ever have a FALSE return value here, so just
remove the dead code instead of causing future confusion.
2015-12-14 11:20:43 +01:00
Sebastian Dröge
0636fe1391 Revert "queue: Illegal memory access of sink event"
This reverts commit 78614c505a.

The code it was fixing does not have any effect anyway and will be removed in
the next commit.
2015-12-14 11:20:22 +01:00
Vineeth TM
78614c505a queue: Illegal memory access of sink event
Once event is pushed to pad, then queue should not access the event.
This is leading to invalid read valgrind errors

https://bugzilla.gnome.org/show_bug.cgi?id=759430
2015-12-14 10:41:50 +01:00
Reynaldo H. Verdejo Pinochet
2a17bad7e8 downloadbuffer: drop unneeded macros for G_OS_WIN32 2015-12-12 13:13:54 -08:00
Reynaldo H. Verdejo Pinochet
b3a704de6a fdsrc: drop unneeded macros for G_OS_WIN32 2015-12-12 13:13:54 -08:00
Reynaldo H. Verdejo Pinochet
e71344010a fdsrc: enable large file support in Android
https://bugzilla.gnome.org/show_bug.cgi?id=758980
2015-12-12 13:13:54 -08:00
Reynaldo H. Verdejo Pinochet
806b48f6ba fdsink: enable large file support in Android
https://bugzilla.gnome.org/show_bug.cgi?id=758980
2015-12-12 13:13:54 -08:00
Reynaldo H. Verdejo Pinochet
85e7cfd811 queue2: fix some typos 2015-12-12 13:13:54 -08:00
Reynaldo H. Verdejo Pinochet
ea8bbd58df queue2: enable large file support on Android
https://bugzilla.gnome.org/show_bug.cgi?id=758980
2015-12-12 13:13:54 -08:00
Reynaldo H. Verdejo Pinochet
5261283dd9 downloadbuffer: fix some typos 2015-12-12 13:13:54 -08:00
Reynaldo H. Verdejo Pinochet
63727152e0 downloadbuffer: enable large file support on Android
https://bugzilla.gnome.org/show_bug.cgi?id=758980
2015-12-12 13:13:54 -08:00
Reynaldo H. Verdejo Pinochet
2a51ae6a4c sparsefile: drop bogus reference to file descriptor
+fix typo on return value comment
2015-12-12 13:13:54 -08:00
Reynaldo H. Verdejo Pinochet
6e563cf353 sparsefile: enable large file support on Android
https://bugzilla.gnome.org/show_bug.cgi?id=758980
2015-12-12 13:13:54 -08:00
Reynaldo H. Verdejo Pinochet
a7f8ad9c9a filesrc: enable large file support in Android
https://bugzilla.gnome.org/show_bug.cgi?id=758980
2015-12-09 14:15:26 -08:00
Reynaldo H. Verdejo Pinochet
add4526314 filesink: enable large file support on Android
https://bugzilla.gnome.org/show_bug.cgi?id=758980
2015-12-09 14:15:26 -08:00
Reynaldo H. Verdejo Pinochet
c0d2c1e793 filesink: indentation fixes 2015-12-09 14:15:26 -08:00
Edward Hervey
5f1764232f multiqueue: Don't use random segment.position from upstream
segment.position is meant for internal usage only, but the various
GST_EVENT_SEGMENT creationg/parsing functions won't clear that field.

Use the appropriate segment boundary as an initial value instead
2015-12-09 17:41:49 +01:00
Edward Hervey
906a50b995 multiqueue: Fix set/get property
Blame it on the rebasing :)
2015-12-02 16:28:58 +01:00
Edward Hervey
56d4650789 pad: Implement GstPadEventFullFunction
API: GstPadEventFullFunction

Returns a GstFlowReturn, allows asynchronous elements to properly
propagate flow returns

https://bugzilla.gnome.org/show_bug.cgi?id=757821
2015-12-02 16:04:03 +01:00
Edward Hervey
4d96e5b834 multiqueue: Use signed clock values for running time calculation
This improves the accuracy of queue levels and when to push buffers
for buffers falling outside of the segment

https://bugzilla.gnome.org/show_bug.cgi?id=757193
2015-12-02 16:03:20 +01:00
Edward Hervey
c51d2b2a37 multiqueue: Add an extra cache time for unlinked streams
When synchronizing the output by time, there are some use-cases (like
allowing gapless playback downstream) where we want the unlinked streams
to stay slightly behind the linked streams.

The "unlinked-cache-time" property allows the user to specify by how
much time the unlinked streams should wait before pushing again.
2015-12-02 16:03:20 +01:00
Edward Hervey
72c31876aa multiqueue: Optimize multiqueue sizes based on interleave
Multiqueue should only be used to cope with:
* decoupling upstream and dowstream threading (i.e. having separate threads
  for elementary streams).
* Ensuring individual queues have enough space to cope with upstream interleave
  (distance in stream time between co-located samples). This is to guarantee
  that we have enough room in each individual queues to provide new data in
  each, without being blocked.
* Limit the queue sizes to that interleave distance (and an extra minimal
  buffering size). This is to ensure we don't consume too much memory.

Based on that, multiqueue now continuously calculates the input interleave
(per incoming streaming thread). Based on that, it calculates a target
interleave (currently 1.5 x real_interleave + 250ms padding).

If the target interleave is greater than the current max_size.time, it will
update it accordingly (to allow enough margin to not block).
If the target interleave goes down by more than 50%, we re-adjust it once
we know we have gone past a safe distance (2 x current max_size.time).

This mode can only be used for incoming streams that are guaranteed to be
properly timestamped.

Furthermore, we ignore sparse streams when calculating interleave and maximum
size of queues.

For the simplest of use-cases (single stream), multiqueue acts as a single
queue with a time limit of 250ms.
If there are multiple inputs, but each come from a different streaming thread,
the maximum time limit will also end up being 250ms.

On regular files (more than one input stream from the same upstream streaming
thread), it can reduce the total memory used as much as 10x, ending up with
max_size.time around 500ms.

Due to the adaptive nature, it can also cope with changing interleave (which
can happen commonly on some files at startup/pre-roll time)
2015-12-02 16:03:20 +01:00
Vincent Penquerc'h
f6069c2c72 multiqueue: use new stream-start event sparse flag to avoid overreading subtitles
This will mean a much lower delay before a subtitles track changes take
effect. Also avoids excessive memory usage in many cases.

This will also consider sparse streams as (individually) never full, so
as to avoid blocking all playback due to one sparse stream.

https://bugzilla.gnome.org/show_bug.cgi?id=600648
2015-12-02 16:03:20 +01:00
Edward Hervey
785edec270 multiqueue: Fix high_time computation
* Avoid the computation completely if we know we don't need it (not in
  sync time mode)
* Make sure we don't override highest time with GST_CLOCK_TIME_NONE on
  unlinked pads
* Ensure the high_time gets properly updated if all pads are not linked
* Fix the comparision in the loop whether the target high time is the same
  as the current time
* Split wake_up_next_non_linked method to avoid useless calculation

https://bugzilla.gnome.org/show_bug.cgi?id=757353
2015-12-02 16:03:16 +01:00
Jan Schmidt
725a71ea0c queue2: Don't report 0% unless empty
When preparing a buffering message, don't report 0% if there
is any bytes left in the queue at all. We still have something
to push, so don't tell the app to start buffering - maybe
we'll get more data before actually running dry.
2015-12-02 19:32:12 +11:00
Tim-Philipp Müller
ffecef4fa0 queue2: don't print criticals when receiving custom events in ring buffer mode
Downgrade from g_warning to GST_WARNING log message.

https://bugzilla.gnome.org/show_bug.cgi?id=758276
2015-11-18 11:46:45 +00:00
Aleix Conchillo Flaqué
2c238705a1 identity: add drop-buffer-flags property
New property drop-buffer-flags that will discard buffers that have the
given flags set.

https://bugzilla.gnome.org/show_bug.cgi?id=751182
2015-11-13 07:37:12 -08:00
Thiago Santos
9f26e5cc63 tracers: log: add missing hooks
Log all possible hooks
2015-11-12 15:39:32 -03:00
Duncan Palmer
45fa81e564 queue2: Add the avg-in-rate property.
https://bugzilla.gnome.org/show_bug.cgi?id=733959
2015-11-06 12:32:16 -03:00
Philippe Normand
8ae8b2723d queue2: add overrun signal
Notifies that the queue2 is full, same as queue does

https://bugzilla.gnome.org/show_bug.cgi?id=733959
2015-11-06 12:32:10 -03:00
Anton Bondarenko
f468eb7db0 filesink: continue element cleanup even if fclose fails
Sometimes filesink cleanup during stop may fail due to fclose error.
In this case object left partial cleanup with no file opened
but still holding old file descriptor.

It's not possible to change location property in a such state,
so next start will cause old file overwrite if 'append' does not set.

According to man page and POSIX standard about fclose behavior(extract):
------------------------------------------------------------------------
The fclose() function shall cause the stream pointed to by stream
to be flushed and the associated file to be closed.
...
Whether or not the call succeeds, the stream shall be disassociated
from the file and any buffer set by the setbuf() or setvbuf()
function shall be disassociated from the stream.
...
The fclose() function shall perform the equivalent of a close()
on the file descriptor that is associated with the stream
pointed to by stream.

After the call to fclose(), any use of stream results
in undefined behavior.
------------------------------------------------------------------------

So file is in 'closed' state no matter if fclose succeed or not.
And cleanup could be continued.

https://bugzilla.gnome.org/show_bug.cgi?id=757596
2015-11-05 10:17:41 +01:00
Thiago Santos
cf5e7c03c2 stats: log the element-new hook properly
To be able to get the time the elements were created instead of
just logging them without a time
2015-10-29 07:27:35 -03:00
Edward Hervey
a9c923d585 queue/queue2: Use GST_BUFFER_DTS_OR_PTS
The input of queue/queue2 might have DTS set, in which cas we want
to take that into account (instead of the PTS) to calculate position
and queue levels.

https://bugzilla.gnome.org/show_bug.cgi?id=756507
2015-10-27 08:49:02 +01:00
Edward Hervey
692c0dc060 multiqueue: Use buffer DTS if present, else PTS
In order to accurately determine the amount (in time) of data
travelling in queues, we should use an increasing value.

If buffers are encoded and potentially reordered, we should be
using their DTS (increasing) and not PTS (reordered)

https://bugzilla.gnome.org/show_bug.cgi?id=756507
2015-10-27 08:38:42 +01:00
Edward Hervey
7c8f76a39f multiqueue: Improve incoming SEGMENT handling
Previously this code was just blindly setting the cached flow return
of downstream to GST_FLOW_OK when we get a SEGMENT.

The problem is that this can not be done blindly. If downstream was
not linked, the corresponding sinqlequeue source pad thread might be
waiting for the next ID to be woken up upon.

By blindly setting the cached return value to GST_FLOW_OK, and if that
stream was the only one that was NOT_LINKED, then the next time we
check (from any other thread) to see if we need to wake up a source pad
thread ... we won't even try, because none of the cached flow return
are equal to GST_FLOW_NOT_LINKED.

This would result in that thread never being woken up

https://bugzilla.gnome.org/show_bug.cgi?id=756645
2015-10-19 14:31:05 +02:00
Edward Hervey
14ad763698 multiqueue: Accept STREAM_START after EOS
In the same way core now allows STREAM_START to remove the flushing
state from pads, we need to do the same thing in multiqueue
2015-10-15 17:21:40 +02:00
Sebastian Dröge
2b8413e1cd tracers: Only link against libgstprintf.la if the debugging system is enabled
It does not exist otherwise and linking will fail.
2015-10-07 12:01:16 +01:00
Stefan Sauer
d626c5b996 Revert "tracers: Only build getrusage() tracer if RUSAGE_THREAD is available"
This reverts commit 8ddbf76626.
2015-10-07 12:22:34 +02:00
Stefan Sauer
03045d428b tracers/rusage: ifdef the RUSAGE_THREAD usage
Some versions of andoid don't seem to have it.
2015-10-07 12:21:56 +02:00
Sebastian Dröge
8ddbf76626 tracers: Only build getrusage() tracer if RUSAGE_THREAD is available 2015-10-07 11:13:28 +01:00
Stefan Sauer
03821d18a0 tracers: disable the log tracer if debug logging is disabled 2015-10-06 18:46:24 +02:00
Stefan Sauer
34cf8f3252 makefile.am: Remove obsolete Android build cruft
This is not needed any longer.
2015-10-06 18:45:41 +02:00
Thiago Santos
7c5f7173c8 tracers: stats: add message structure to output
The name of the message is not enough. For example, state-change
is not enough to know the transition.

https://bugzilla.gnome.org/show_bug.cgi?id=754496
2015-10-05 19:06:41 -03:00
Thiago Santos
0913f699e2 tracer: gststats: add thread-id to log line 2015-10-05 20:59:39 +02:00
Thiago Santos
567865c81c tracer: add pad query hooks 2015-10-05 20:59:39 +02:00
Mathieu Duponchelle
a9aab3c6c7 tracer: Use GST_TIME_ARGS when printing with GST_TIME_FORMAT. 2015-10-05 20:59:39 +02:00
Stefan Sauer
e5eb05741b stats: TIMESTAMP -> PTS 2015-10-05 20:59:39 +02:00
Stefan Sauer
660049bb38 tracing: rename the global api to gst_tracing
This makes it more obvious what is the api for tracer elements and what is api
for the global state.
2015-10-05 20:59:39 +02:00
Stefan Sauer
5e2770566f tracers: eliminate var_args
Register tracer hooks as GCallback. Use macros for hook dispatch and cast the
hook functions back to the appropriate type.
2015-10-05 20:59:39 +02:00
Stefan Sauer
b10b10cf1e tracers: code cleanups
Move static variables to instance variables. Add finalize methods. Remove code
that is commented out. Cleanup locking code.
2015-10-05 20:59:39 +02:00
Stefan Sauer
4881d1c16c tracer: use GQuark or strings for the hook id
This way one can define new tracing probes without changing the core. We are
using our own quark table, as 1) we only want to initialize them if we're
tracing, 2) we want to share them with the tracers.
2015-10-05 20:59:39 +02:00
Stefan Sauer
1a3e218b89 tracer: simplify hook api
Instead of a single invoke() function and a 'mask', register to individual
hooks. This avoids one level of indirection and allows us to remove the
hook enums. The message enms are now renamed to hook enums.
2015-10-05 20:59:39 +02:00
Stefan Sauer
7b24d76260 tracer: drop the HookId hid from the invoke method
The MessageId is more detailed and anyway needed to interpret the varargs.
2015-10-05 20:59:39 +02:00
Stefan Sauer
4ab1e6e3f7 stats: fixup doc name and remove commented code 2015-10-05 20:59:39 +02:00
Stefan Sauer
4f6f8cb811 tracers: add metadata for the logged values 2015-10-05 20:59:39 +02:00
Stefan Sauer
3baaa4ba6f rusage: improve cpu load meassurements
Get the number of cpus and scale process cpu-load accordingly. Switch the
cpuload to be per-mille to get smoother graphs. Add a bit more logging and use
the _OBJECT variant.
2015-10-05 20:59:39 +02:00
Stefan Sauer
dbb1897e9b rusage: implement windowing of cpuload
Add a local help to the rusage plugin that supports windowing of values. We want
to generalize this for use in other plugins.
2015-10-05 20:59:39 +02:00
Stefan Sauer
018858980b rusage: announce the data format
Rusage will now announce what is meassures and how it is logged. Use the new format in stats.
Cleanup the the code and naming.
2015-10-05 20:59:39 +02:00
Stefan Sauer
c13ee1f2e9 stats: improve cpu load meassurements
Rename variables for clarity. Handle the initial disparity between debug time
and the time already spent in the proc and main thread.
2015-10-05 20:59:39 +02:00
Stefan Sauer
46ea9729df tracer: split into tracer and tracerutils
Keep tracer base class in tracer and move core support into the utils module.
Add a unstable-api guard to the tracer.h so that external modules would need to
acknowledge the status by setting GST_USE_UNSTABLE_API.
2015-10-05 20:59:39 +02:00
Stefan Sauer
6ea2b41364 docs: add gtk-doc blobs 2015-10-05 20:59:39 +02:00
Stefan Sauer
757d3400cd latency: take stop time when buffer is handled
Now we meassure time from 'before buffer transfer on src' to when the 'buffer is processed on sink'.
2015-10-05 20:59:39 +02:00
Stefan Sauer
7dfc0a8ead tracers: add a logging helper to remove identical copies from the tracers 2015-10-05 20:59:39 +02:00
Stefan Sauer
635a4b5c00 tracers: tweak the get_real_pad_parent()
By using the we ended up on the actual element, not the parent.
2015-10-05 20:59:39 +02:00
Stefan Sauer
111b18115a tracers: add a new latency tracer
Add a new tracer with pushes extra events to meassure src-to-sink processing latency.
2015-10-05 20:59:39 +02:00
Stefan Sauer
fc9392c10f rusage: add a new rusage tracer
The tracer hooks up to all probes and logs resource usage figures.
2015-10-05 20:59:39 +02:00
Stefan Sauer
0c26387e62 stats: improve the handling of parentage
Log new object after we did the check for parents.
2015-10-05 20:59:39 +02:00
Stefan Sauer
b8eef6ba2c stats: add a stats frontend
Parse the log and collect data from tracer messages.
2015-10-05 20:59:39 +02:00
Stefan Sauer
ac92e5674e log: add query log category 2015-10-05 20:59:39 +02:00
Stefan Sauer
ce7ce2d866 tracer: add more hooks and handle it in the plugins 2015-10-05 20:59:39 +02:00
Stefan Sauer
37721fdc9d stats: handle buffer lists 2015-10-05 20:59:39 +02:00
Stefan Sauer
75fb1f1a73 log: make the log tracer more verbose again
Define log formats per message type and print details.
2015-10-05 20:59:39 +02:00
Stefan Sauer
11d6e13d73 tracer: use a macros for the enabled check
Avoid a function call and check the variables from the macro.
2015-10-05 20:59:39 +02:00
Stefan Sauer
9d3975b5da tracer: pass the timestamp directly
Avoid the structure mashalling (and weird field naming).
2015-10-05 20:59:39 +02:00
Stefan Sauer
d663b78612 stats: add a tracer that collects pipeline statistics
This is more or less equiv to the the statistics in gst-tracelib.
2015-10-05 20:59:39 +02:00