Commit graph

1510 commits

Author SHA1 Message Date
Sebastian Dröge
9243418a23 audioaggregator: Only post QoS messages if the property is enabled
Previously one of the branches did not check for the property value. To
avoid this in the future, check inside the QoS calculation function
instead.

As a side effect this now always prints the debug messages into the logs
when samples are dropped, which is useful information even without the
QoS messages.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1224>
2021-07-12 09:42:39 +03:00
Sebastian Dröge
71e46bcf38 audioaggregator: Resync on the next buffer when dropping a buffer on discont resyncing
If a buffer is dropped during resyncing on a discont because either its
end offset is already before the current output offset of the
aggregator or because it fully overlaps with the part of the current
output buffer that was already filled, then don't just assume that the
next buffer is going to start at exactly the expected offset. It might
still require some more dropping of samples.

This caused the input to be mixed with an offset to its actual position
in the output stream, causing additional latency and wrong
synchronization between the different input streams.

Instead consider each buffer after a discont as a discont until the
aggregator actually resynced and starts mixing samples from the input
again.

Also update the start output offset of a new input buffer if samples
have to be dropped at the beginning. Otherwise it might be mixed too
early into the output and overwrite part of the output buffer that
already took samples from this input into account.

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/issues/912
which is a regression introduced by https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1180/

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1224>
2021-07-12 09:42:39 +03:00
Olivier Crête
49f6d3bf33 audio aggregator: Post QoS message when dropping audio
Post a QoS message every time some audio samples are dropped.
Also print log messages to make it easier to debug

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1209>
2021-07-08 23:01:13 -04:00
Olivier Crête
ea516aee33 audio aggregator: Count samples that are dropped or processed
Keep a count of samples that are dropped or processed as statistics

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1209>
2021-07-08 23:01:13 -04:00
Olivier Crête
e3be1b8490 audio aggregator: Add QoS property to pad
Add a property to emit a QoS message whenever any data is dropped.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1209>
2021-07-08 23:01:13 -04:00
Olivier Crête
1eff5ffef6 audio aggregator: Rename property enum to match class name
Add "CONVERT" into the property enum as we're going to add an
enum specifically for the base pad.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1209>
2021-07-08 22:34:49 +00:00
Jan Alexander Steffens (heftig)
b835356d6c audio-converter: Free config when gst_audio_converter_new fails
The config got leaked when parameter validation fails.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1140>
2021-06-25 13:55:39 +00:00
Sergei Kovalev
ba4f031864 audiobasesink: Fix of double lock release
Add missing "return;" which prevents double lock release.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1208>
2021-06-21 15:58:01 +00:00
François Laignel
973f0bf552 Check mandatory ClockTime arguments
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1205>
2021-06-16 14:52:38 +00:00
Olivier Crête
fd73fd05ca audioaggregator: Don't overwrite already written samples
On re-sync, don't forget what has already been written. Instead, just
drop any samples that overlap with parts that were already filled.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1180>
2021-05-27 16:33:00 -04:00
Seungha Yang
5ad59ce725 audiobasesrc: Fix divide by zero assertion
GstAudioRingBufferSpec can be cleared from other thread, then
rate value will be zero

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1179>
2021-05-27 07:51:48 +00:00
Marijn Suijten
9a502c15c6 audio,video-format: Make generate_raw_formats idempotent for assertions
When compiling without assertions `g_assert` and its contents disappear
resulting in no list being deserialized at all and the
`gst_{audio,video}_formats_raw` functions to return an empty collection.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1177>
2021-05-26 23:35:16 +02:00
Marijn Suijten
33167573e1 Drop @ documentation references from functions and external types
`@` references are used to reference function parameters, struct members
or enum variants _within_ the current type/function.  It cannot and
should not be used to reference to types outside that.

Since C has no notion of member functions it makes little sense to
prefix these with `@`; most of the documentation here was referencing
functions on _different_ types anyway.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1090>
2021-04-15 15:49:39 +02:00
Doug Nazar
1d5ad7d1da audio/alsa: Exit write loop if underlying device is already paused.
If the alsasink thread starts the write loop but another thread pauses
the underlying alsa device, the sink thread will endlessly loop.

snd_pcm_writei() will return 0 if the state is SND_PCM_STATE_PAUSED
and the loop will never make any progress.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1097>
2021-04-08 07:28:21 +00:00
Matthew Waters
98249a57db gst: don't use volatile to mean atomic
volatile is not sufficient to provide atomic guarantees and real atomics
should be used instead.  GCC 11 has started warning about using volatile
with atomic operations.

https://gitlab.gnome.org/GNOME/glib/-/merge_requests/1719

Discovered in https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/-/issues/868

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1073>
2021-03-19 04:20:19 +00:00
Jan Alexander Steffens (heftig)
a379e0e5f1 audioaggregator: Consider converting for equal audio formats
The converter might have a non-passthrough mix-matrix. The converter
can determine whether it should pass through, so let it, then remove it
if it's indeed a passthrough.

FIXME: Not converting when we need to but the config is invalid (e.g.
because the mix-matrix is not the right size) produces garbage. An
invalid config should cause a GST_FLOW_NOT_NEGOTIATED.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1070>
2021-03-16 13:46:56 +01:00
Jan Alexander Steffens (heftig)
43449d9fb2 audioaggregator: Clean up _convert_pad_update_converter
No functional change.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1070>
2021-03-16 13:46:55 +01:00
Guillaume Desmottes
b7c1810aa3 audioaggregator: fix input_buffer ownership
The way pad->priv->input_buffer reference was managed was pretty
spurious:
- it was overridden without unrefing it, which could potentially lead to
  leaks.
- we were unreffing it while keeping the pointer around, which could
  potentially lead to use-after-free or double-free.

As priv->input_buffer is actually no longer used outside of the
aggregate() method, remove it from pad->priv to simplify the code and
prevent the issues desribed above.

Fix a single buffer leak when shutting down the pipeline as the buffer
returned from gst_aggregator_pad_drop_buffer() was never unreffed.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1061>
2021-03-10 16:38:03 +01:00
Guillaume Desmottes
44358f1eaf audioaggregator: fix input buffer when converting
This code path is meant to convert the current buffer to the new format
on update. It was using priv->input_buffer as input which is either
priv->buffer or a converted version of it.
Use priv->buffer instead as priv->input_buffer may no longer be a valid
reference.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1061>
2021-03-10 16:34:28 +01:00
Robert Rosengren
e99a6f3142 audio: Use GST_BUFFER_PTS instead of deprecated GST_BUFFER_TIMESTAMP
GST_BUFFER_PTS already used in audio code base (e.g. gstaudiodecoder),
so migrate completely from deprecated GST_BUFFER_TIMESTAMP for better
readability, as gstcompat.h defines GST_BUFFER_TIMESTAMP directly to PTS
anyway.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1048>
2021-02-25 02:04:44 +00:00
Sebastian Dröge
f5381ba9f5 audioaggregator: Log if the sample rate of one sinkpad is not accepted
Otherwise this can silently cause not-negotiated errors without any
direct hint about what went wrong.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1049>
2021-02-24 19:53:02 +02:00
Vivia Nikolaidou
2527c8f9f8 libs: audio: Handle meta changes in gst_audio_buffer_truncate
Set timestamp and duration to GST_CLOCK_TIME_NONE unless trim==0,
because that function doesn't know the rate and therefore can't
calculate them. Set offset and offset_end to appropriate values. Make it
clear in the documentation that the caller is responsible for setting
the timestamp and duration.

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/issues/869

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1039>
2021-02-18 11:25:32 +02:00
Jan Alexander Steffens (heftig)
297a5f09b1 libs: audio: Fix gst_audio_buffer_truncate meta handling
In the non-interleaved case, it made `buffer` writable but then changed
the meta of the non-writable buffer.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1035>
2021-02-15 17:32:04 +01:00
Alejandro González
319da90d4c audioencoder: Fix gst_audio_encoder_get_audio_info return ownership GTK-Doc
GTK-Doc specifies that, by default, the caller owns returned objects, so that the caller should free them when it is done. However, in the case of this function, the returned GstAudioInfo is owned by the decoder, so this default choice is incorrect. This creates double free problems when using GStreamer Rust bindings, because they are generated using the information contained in the docs.

Fix this by correctly specifying that the caller does not own the returned object.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1032>
2021-02-13 21:25:18 +00:00
Alejandro González
2fd2540ea5 audiodecoder: Fix gst_audio_decoder_get_audio_info return ownership GTK-Doc
GTK-Doc specifies that, by default, the caller owns returned objects, so that the caller should free it when it is done. However, in the case of this function, the returned GstAudioInfo is owned by the decoder, so this default choice is incorrect. This creates double free problems when using GStreamer Rust bindings, because they are generated using the information contained in the docs.

Fix this by correctly specifying that the caller does not own the returned object.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1032>
2021-02-13 17:24:37 +00:00
Havard Graff
0f866832b1 audio: add GstAudioLevelMeta
Will be used to implement RTP extension https://tools.ietf.org/html/rfc6464

Co-authored-by: Guillaume Desmottes <guillaume.desmottes@collabora.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/706>
2021-02-04 10:25:24 +01:00
Marijn Suijten
9ab400e267 gstaudiostreamalign: Pass self as const pointer in getter functions
It was noticed in [1] that `GstAudioStreamAlign` is a simple boxed type
that is passed as const in the copy function, but not as such in the
getters. These functions turn out to be the only users of `const = true`
overrides in `gstreamer-rs`. Since there is no locking or other advanced
caching/sharing going on (as happens with miniobjects) these functions
can safely take self as const pointer.

[1]: https://gitlab.freedesktop.org/gstreamer/gstreamer-rs/-/merge_requests/683#note_783129

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1025>
2021-01-29 21:42:47 +01:00
Marijn Suijten
fa8b5b9a6d audio/audio-buffer: @buffer in audio_buffer_map is out caller-allocates
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1005>
2021-01-14 11:53:10 +00:00
Marijn Suijten
a263919f06 audio,video: Add out caller-allocates to init and from_caps
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1005>
2021-01-14 11:53:10 +00:00
Ratchanan Srirattanamet
cc8f54468e audiobasesrc: always acquire if not acquired in _setcaps
audiobasesrc's setcaps contains an optimization that makes it not re-
acquire the ringbuffer if the caps have not changed. However, it doesn't
check if it has successfully acquired it or not. It's possible to have
the caps set but not having ringbuffer acquired if the previous attempt
to acquire fails.

This commit replaces the caps existence check with whether the
ringbuffer is acquired or not. There's no need to check for caps
existence because 1.) it's unlikely to be NULL if the ringbuffer is
acquired, and 2.) _setcaps shouldn't be called with a NULL caps.

This should also let the element retry on acquiring ringbuffer after an
error by re-setting the element's state to READY and back to PLAYING.
Whether this behavior is correct is up for debate.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/512>
2020-12-04 13:57:58 +00:00
Arun Raghavan
27ce682940 audioencoder: Fix incorrect GST_LOG_OBJECT usage
GstBuffer is not a GstObject, so this causes a warning to be emitted.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/956>
2020-12-03 12:46:33 +00:00
Sanchayan Maity
5aa836848e audiodecoder: Move max_errors out of GstAudioDecoderContext
Currently max-errors gets set during init to default or via property.
However, if a decoder element calls gst_audio_decoder_reset with 'full'
argument set to TRUE, it would result in all the fields of context being
zeroed with memset. This effectively results in max-errors getting a
value of 0 overriding the default or user requested value set during
init.

This would result in calls to GST_AUDIO_DECODER_ERROR which track error
counts and allow max-errors, to be ineffective.

To fix this move max-errors out of GstAudioDecoderContext, as changes to
context should not affect this. The error_count is anyways also in
GstAudioDecoderPrivate and not in context.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/946>
2020-11-27 14:49:10 +05:30
Marijn Suijten
3ec795f613 audio: Move fill_silence into audio_format_info
With the function named gst_audio_format_fill_silence it would get
associated to the GstAudioFormat type in .gir which is incorrect and
confusing. See [1] for the discussion sparking this change.

https://gitlab.freedesktop.org/gstreamer/gstreamer-rs/-/merge_requests/630#note_694795

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/940>
2020-11-25 19:18:25 +01:00
Xavier Claessens
a28a75652e Meson: Use pkg-config generator 2020-10-23 11:19:11 -04:00
Sebastian Dröge
6af87dee17 audio/videodecoder: Initialize max_errors in instance_init()
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/882>
2020-10-20 12:46:07 +03:00
Sebastian Dröge
ed62e78c6e audio/videodecoder: Don't reset max-errors in reset()
Otherwise setting the property on the elements has no effect at all
because it's immediately reset during startup.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/881>
2020-10-20 11:52:07 +03:00
Jan Alexander Steffens (heftig)
aa89ae8beb audio: video: Fix in/outbuf confusion of transform_meta
There are three instances where in- and outbuf have been swapped. This
didn't affect the correctness of the libs *filter code, but the
videoscale implementation swapped the arguments of meta->transform_func.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/853>
2020-10-08 18:30:39 +02:00
Sebastian Dröge
1208d4e635 audioaggregator: Reset offset if the output rate is renegotiated
On next aggregation the new offset will be calculated based on the
segment position.

Without this a rate change would cause a jump forwards or backwards in
the output timeline.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/794>
2020-09-09 09:09:17 +00:00
Sebastian Dröge
391d09dc24 audio/video: Copy more metas by default in the codec base classes
For audio we copy metas that have no tags at all, or that only have the
"audio" and/or "audio-channels" tag. Audio codecs don't change the
audio aspect of the stream and in almost all cases don't change the
number of channels. They might however change the sample rate (e.g.
Opus). Subclasses that change the number of channels will have to
override ::transform_meta() accordingly.

For video we copy metas that have no tags at all, or that only have the
"video" and/or "video-size" and/or "video-orientation" tag. Video codecs
don't change the "video" aspect of the stream and in almost all cases
don't change the resolution or orientation. Subclasses that rescale or
change the orientation will have to override ::transform_meta()
accordingly.

See https://gitlab.freedesktop.org/gstreamer/gst-plugins-good/-/merge_requests/576#note_610581

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/801>
2020-08-30 22:12:22 +00:00
Sebastian Dröge
6b14080941 audioaggregator: Add support for new sample selection API
Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/issues/805

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/780>
2020-08-07 19:23:50 +03:00
Sebastian Dröge
f5a02639e1 audioaggregator: Only check downstream caps when handling CAPS events if we didn't negotiate with downstream yet
If we already negotiated with downstream there is not point in checking
if the caps are supported. We already know that this is the case.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/768>
2020-07-28 10:59:25 +03:00
Sebastian Dröge
607dc1d135 audioaggregator: Check all downstream allowed caps structures if they support the upstream rate
Otherwise it might happen that downstream prefers a different rate (i.e.
puts it into the first structure) and also supports other rates, but
audioaggregator would then fail negotiation.

Also this now correctly handles downstream returning a range of
supported rates.

Fixes https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/issues/795

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/768>
2020-07-27 18:49:48 +03:00
Silvio Lazzeretti
aa4bea913b audioutilsprivate: restore thread priority before ending
The priority of the thread that executes audioringbuffer_thread_func
is incremented on Windows by the usage of the AvSetMmThreadCharacteristics
API. This change has to be restored, as described on the documentation
of the API (https://docs.microsoft.com/en-us/windows/win32/api/avrt/nf-avrt-avsetmmthreadcharacteristicsw#remarks),
with a call to the AvRevertMmThreadCharacteristics. If this is not done,
a handle will be leaked.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/760>
2020-07-18 13:00:00 +02:00
Seungha Yang
f0a9907097 audioutilsprivate: Don't try to load avrt for UWP application
All APIs in avrt.h are desktop only.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/763>
2020-07-17 18:55:31 +09:00
Sebastian Dröge
f94c7ae3c9 audioaggregator: Fix negotiation with downstream if there is no peer yet
get_allowed_caps() will return NULL, which is not a problem in itself.
Just take the template caps for negotiation in that case instead of
erroring out.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/744>
2020-07-09 16:48:02 +00:00
Tim-Philipp Müller
6bb3e01918 meson: add update-orc-dist target
Add target to update backup orc -dist.[ch] files.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/734>
2020-07-04 14:01:56 +01:00
Havard Graff
0826fb95b7 audio: video: Optimize by using cached quark for meta tag
Avoid taking the global quark lock for every single buffer.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/295>
2020-06-27 09:23:10 +00:00
Sebastian Dröge
63933da9e8 audiodecoder: Add max-errors property
The number of consecutive decode errors that should be tolerated before
returning flow error should be up to the application, not the element.

Hence max-error should be exposed as a property.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/720>
2020-06-23 07:17:00 +00:00
Sebastian Dröge
f2af205a78 Fix up and add various "Since" markers and other related docs fixes
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/713>
2020-06-19 12:17:55 +03:00
Guillaume Desmottes
008d72d5da audio: add missing space in GST_AUDIO_FORMATS_ALL
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/694>
2020-06-10 10:43:42 +02:00