Commit graph

46 commits

Author SHA1 Message Date
Sebastian Dröge
87b72f768b livesync: Report latency query results when handling latency query too
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1739>
2024-08-22 07:33:47 +00:00
Sebastian Dröge
98b28d69ce Update for new debug log macro syntax
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1658>
2024-07-08 11:25:23 +03:00
Sebastian Dröge
bd2a039c8d livesync: Use the actual output buffer duration of gap filler buffers
Otherwise the following can happen:

  - 25fps stream
  - buffer with PTS 0ms, duration 20ms arrives, is output
  - buffer with PTS 40ms, duration 20ms arrives
  - is considered early because 20ms < 40ms
  - filler buffer with PTS 20ms and 40ms duration is output
  - buffer with PTS 40ms is output

After this change no filler would be inserted because the gap is smaller
than the duration of a filler buffer.

Also, previously the 40ms duration would be used if a filler was
previously output because in that case the cached output buffer duration
would've already been patched from 20ms to 40ms.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1647>
2024-07-02 17:15:58 +03:00
Sebastian Dröge
960529d90d livesync: Add sync property for allowing to output buffers as soon as they arrive
By default livesync will wait for each buffer on the clock. If sync is
set to false, it will output buffers immediately once they're available
and only waits on the clock for outputting gap filler buffers.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1635>
2024-06-26 16:21:42 +00:00
Sebastian Dröge
bbf131086a livesync: Synchronize on the first buffer too
Previously the first buffer would be output immediately and
synchronization would only happen from the second buffer onwards.
This would mean that the first buffer would potentially be output too
early.

Instead, if there is no known output timestamp yet but a buffer with a
timestamp, first of all take its start as the initial output timestamp
and synchronize on that buffer.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1635>
2024-06-26 16:21:42 +00:00
Sebastian Dröge
7caf6b2073 livesync: Use let-else in a few more places
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1635>
2024-06-26 16:21:41 +00:00
Sebastian Dröge
505fab2e1c livesync: Allow queueing up to latency buffers
This was already reported by the latency query, and not doing this would
require to always put a queue before livesync.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1635>
2024-06-26 16:21:41 +00:00
Sebastian Dröge
9b323a6519 Use Option::is_some_and(...) instead of Option::map_or(false, ...)
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1630>
2024-06-19 13:03:37 +00:00
Sebastian Dröge
4ad101b53b Use once_cell crate directly again
The glib crate does not depend on it anymore and also does not re-export
it anymore.

Also switch some usages of OnceCell to OnceLock from std.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1441>
2024-01-31 18:07:57 +02:00
Michael Tretter
4bb867bf52 livesync: add support for image formats
The livesync element is also useful for Motion JPEG streams. However,
Motion JPEG uses image/ caps instead of video/ caps.

The framerate is defined for image/, too.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1440>
2024-01-29 11:07:30 +00:00
Guillaume Desmottes
c616423edb livesync: properly format jitter in debug logs
Easier to read that way.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1430>
2024-01-16 13:46:34 +01:00
Sebastian Dröge
16b917abb1 Update for gst::Rank API changes 2023-11-02 14:10:59 +02:00
Jan Alexander Steffens (heftig)
e3e58ac0be livesync: Remove the stop from outgoing segments
Our buffer duplication can extend a segment indefinitely.

Fixes: https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/452
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1372>
2023-10-25 19:34:47 +02:00
Jan Alexander Steffens (heftig)
f1ba498b52 livesync: Keep existing buffer duration in some cases
Resize a repeat buffer only if caps gave us a duration to use, or we
consider its current duration unreasonable.

In particular, for audio streams we should prefer reusing the buffer
size upstream gave us, as we did before 6633cc4046.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1372>
2023-10-25 19:34:47 +02:00
Jan Alexander Steffens (heftig)
59beade079 livesync: Split fallback_duration into in_ and out_duration
Make it independent of the `latency`; this was inconsistent anyway,
where the default latency of zero got you a fallback duration of 100 ms
and something else got you half the latency.

Maintain a separate duration for the `in` and the `out` side so we
change the duration of repeat buffers after a caps change, not just
before.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1372>
2023-10-25 19:08:16 +02:00
Guillaume Desmottes
f94ecfc7a6 livesync: display jitter when waiting on clock
We already log the result of the clock wait call so may as well log the
returned jitter.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1370>
2023-10-25 14:26:19 +02:00
Guillaume Desmottes
13dae0f0d0 livesync: log new pending segments
The debug print of the event does not display details about the segment:
  Unqueueing Some(Event(Event { ptr: 0x7fa3e0002580, type: "segment", seqnum: Seqnum(479), structure: Some(GstEventSegment { segment: (GstSegment) ((GstSegment*) 0x7fa3e8001d00) }) }))

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1370>
2023-10-25 14:24:35 +02:00
Jan Alexander Steffens (heftig)
6633cc4046 livesync: Use fallback_duration for audio repeat buffers as well
Don't depend on upstream giving us sanely-sized buffers if we want to
repeat.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1369>
2023-10-25 12:55:06 +02:00
Jan Alexander Steffens (heftig)
4ac7d0415b livesync: Separate out_buffer duplicate status from GAP flag
Otherwise we might get confused by upstream GAP buffers.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1369>
2023-10-25 12:10:40 +02:00
Jan Alexander Steffens (heftig)
2f36bd5d77 livesync: Handle flags and late buffer patching after queueing
This makes the chain function almost independent of the output state. We
still do the early discard check with `buffer_is_backwards` so we don't
try to queue buffers we can't use, allowing us to fast-forward upstream
without blocking on the src task.

Don't accept `LateOverThreshold` buffers when we have `pending_caps` or
a `pending_segment`. We need to apply these first before we can sensibly
patch buffers from the new stream.

Deduplicate most of the output buffer patching code into a new
`patch_output_buffer` method.

For: https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/450
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1369>
2023-10-25 11:52:41 +02:00
Jan Alexander Steffens (heftig)
7c48a299c3 livesync: Simplify num_duplicate counting
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1369>
2023-10-25 11:52:40 +02:00
Jan Alexander Steffens (heftig)
17a2448237 livesync: Move num_in counting to the src task
This is in preparation for moving more accept/discard logic to the src
task, so we can only count `num_in` here.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1369>
2023-10-25 11:52:40 +02:00
Jan Alexander Steffens (heftig)
1740a8e363 livesync: Move a notify closer to the interesting state change
Move the `notify_all` to where we pop the buffer. We're moving within a
single state lock so no change in behavior.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1369>
2023-10-25 11:52:40 +02:00
Jan Alexander Steffens (heftig)
44f2195674 livesync: Replace an if-let with match
No change in behavior, yet. Separate commit to ease reviewing.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1369>
2023-10-25 11:52:40 +02:00
Jan Alexander Steffens (heftig)
62791bfb47 livesync: Clean up state handling
- Separate resetting state more cleanly, introducing `set_flushing`,
  `sink_reset` and `src_reset`.
- Clear the queue early when we flush, in order to unblock waits on
  query responses.
- Return an error when we fail to start, pause or stop the task.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1369>
2023-10-25 11:52:40 +02:00
Jan Alexander Steffens (heftig)
d663f708ef livesync: Log a category error when we are missing the segment
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1369>
2023-10-25 11:52:40 +02:00
Jan Alexander Steffens (heftig)
6567041a3d livesync: Improve audio duration fixups
- An entirely missing duration is now only logged at debug level instead
  of pretending the duration was zero and warning about it.
- Silently fix up a duration difference up to one sample.
- Error when we fail to calculate the duration; don't try to apply the
  `fallback_duration` to a non-video stream.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1369>
2023-10-25 11:52:40 +02:00
Jan Alexander Steffens (heftig)
0a45f776e0 livesync: Simplify start_src_task and src_loop
This should effect no change in behavior.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1369>
2023-10-25 11:03:15 +02:00
Jan Alexander Steffens (heftig)
01386b8451 livesync: Rename activatemode methods to *_activatemode
This matches the other plugins.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1369>
2023-10-25 11:03:14 +02:00
Bilal Elmoussaoui
dd2d7d9215 Use re-exported once_cell
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1268>
2023-07-06 17:50:49 +03:00
Sebastian Dröge
1119ed6620 livesync: Wait for the end timestamp of the previous buffer before looking at queue
Previously livesync was waiting for the start timestamp of the current
buffer after looking at the queue and right before pushing it
downstream. This meant that it generally looked too early at the queue
and especially that upstream had to provide the next buffer already at
the start timestamp of the previous one.

Instead, now wait before looking at the queue and wait for the end
timestamp of the previous buffer. Once the previous buffer has expired,
a new buffer really needs to be available or otherwise a filler buffer
has to be pushed downstream.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1250>
2023-06-20 13:01:39 +00:00
Jan Alexander Steffens (heftig)
52ded6e8cc livesync: Improve EOS handling
I've looked at the GstQueue code again and tried making livesync behave
better with EOS. This isn't very well tested, though. My goal was to
make this look saner but I think this should be reviewed by someone who
knows the queue code.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1251>
2023-06-20 13:18:17 +02:00
François Laignel
7ba0073052 use Pad builders for optional name definition
Also, apply auto-naming in the following cases

* When building from a non wildcard-named template, the name of the template is
  automatically assigned to the Pad. User can override with a specific name by
  calling `name()` on the `PadBuilder`.
* When building with a target and no name was provided via the above, the
  GhostPad is named after the target.

See https://gitlab.freedesktop.org/gstreamer/gstreamer-rs/-/issues/448
Auto-naming discussion: https://gitlab.freedesktop.org/gstreamer/gstreamer-rs/-/merge_requests/1255#note_1891181

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1197>
2023-05-12 12:55:31 +02:00
Vivia Nikolaidou
c6e1efa0fe livesync: Actually assume zero upstream latency when query fails
The code said "assuming zero" but left latency at None instead of
Some(0), failing to unwrap the value later.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1153>
2023-03-31 17:40:32 +03:00
Talha Khan
a12a8c566d livesync: Support variable framerate in fallback buffer duration calc
Avoids a divide by zero error

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1123>
2023-03-10 09:18:28 +00:00
Sebastian Dröge
ff2f7a8505 livesync: Correctly calculate fallback buffer duration from framerate
Numerator and denominator were switched.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1108>
2023-02-28 12:52:11 +02:00
Jan Alexander Steffens (heftig)
f55c32ed37 livesync: Document State's fields
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1083>
2023-02-09 13:07:33 +01:00
Jan Alexander Steffens (heftig)
953773a314 livesync: Improve formatting
Move some code around to make it a bit more readable. No change in
behavior.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1083>
2023-02-09 13:07:33 +01:00
Jan Alexander Steffens (heftig)
c1bfeb4c23 livesync: Fix log message capitalization
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1083>
2023-02-09 13:07:33 +01:00
Jan Alexander Steffens (heftig)
0af7151ae9 livesync: Extract LiveSync::flow_error
And add details so it behaves more like the `GST_ELEMENT_FLOW_ERROR`
macro.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1083>
2023-02-09 13:07:32 +01:00
Jan Alexander Steffens (heftig)
f03ee95bf0 livesync: Extract audio_info_from_caps
And adjust it slightly so it never panics.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1083>
2023-02-09 13:07:32 +01:00
Jan Alexander Steffens (heftig)
c971c4d1d5 livesync: Move single segment prop
Keep it with the settings, not after the stats.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1083>
2023-02-09 13:07:32 +01:00
Jan Alexander Steffens (heftig)
165b5f8c50 livesync: Fix queueing
The logic of the element requires the next buffer to be available
immediately after we are done pushing the previous, otherwise we insert
a repeat.

Making the src loop handle events and queries broke this, as upstream is
almost guaranteed not to deliver a buffer in time if we allow non-buffer
items to block upstream's push.

To fix this, replace our single-item `Option` with a `VecDeque` that we
allow to hold an unlimited number of events or queries, but only one
buffer at a time.

In addition, the code was confused about the current caps and segment.

This wasn't an issue before making the src loop handle events and
queries, as only the sinkpad cared about the current segment, using it
to buffers received, and only the srcpad cared about the current caps,
sending it just before sending the next received buffer.

Now the sinkpad cares about caps (through `update_fallback_duration`)
and the srcpad cares about the segment (when not in single-segment
mode).

Fix this by
  - making `in_caps` always hold the current caps of the sinkpad,
  - adding `pending_caps`, which is used by the srcpad to store
    caps to be sent with the next received buffer,
  - adding `in_segment`, holding the current segment of the sinkpad,
  - adding `pending_segment`, which is used by the srcpad to store
    the segment to be sent with the next received buffer,
  - adding `out_segment`, holding the current segment of the srcpad.

Maybe a fix for
https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/issues/298.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1082>
2023-02-09 12:44:47 +01:00
Jan Alexander Steffens (heftig)
33696a8aed livesync: Only resend segment if not in single-segment mode
In single-segment mode, the outgoing segment does not change when the
incoming segment changes. We only need to resend the segment if we got
flushed or deactivated.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1069>
2023-01-30 15:37:00 +00:00
Guillaume Desmottes
570eb7463a livesync: fix late-threshold property min value
The code is handling 0 as "always over threshold" but it was not
possible to set the property to 0.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/1049>
2023-01-17 10:54:05 +01:00
Jan Alexander Steffens (heftig)
42385c81be Add livesync plugin
It attempts to produce a (nearly) gapless live stream by synchronizing
its output to the running time and forwarding the next input buffer if
its start is (nearly) flush with the end of the last output buffer.

If the input buffer is missing or too far in the future, it duplicates
the last output buffer with adjusted timestamps. If it is operating on a
raw audio stream, it will fill duplicate buffers with silence.

If an input buffer arrives too late, it is thrown away. If the last
input buffer was accepted too long ago (according to `late-threshold`),
a late input buffer is accepted anyway, but immediately considered a
duplicate. Due to the silence-filling, this has no effect on audio, but
video gets a "slideshow" effect instead of freezing completely.

The "many-repeats" property will be notified when this element has
recently duplicated a lot of buffers or recovered from such a state.

Co-authored-by: Vivia Nikolaidou <vivia@ahiru.eu>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/708>
2022-12-14 18:51:36 +02:00