... unless the muxer uses the same audio pad template name as
splitmuxsink. We can't request a pad called "audio_0" on a muxer that
wants pads to be "sink_%d".
Majorly change the way that splitmuxsink collects
incoming data and sends it to the output, so that it
makes all decisions about when / where to split files
on the input side.
Use separate queues for each stream, so they can be
grown individually and kept as small as possible.
This removes raciness I observed where sometimes
some data would end up put in a different output file
over multiple runs with the same input.
Also fixes hangs with input queues getting full
and causing muxing to stall out.
Add a new signal for formatting the filename, which receives
a GstSample containing the first buffer from the reference
stream that will be muxed into that file.
Useful for creating filenames that are based on the
running time or other attributes of the buffer.
To make it work, opening of files and setting filenames is
now deferred until there is some data to write to it,
which also requires some changes to how async state changes
and gap events are handled.
splitmuxsink requests pad from element using pad template like "video_%u", "audio_%u" and "sink_%d". This is true for most of the muxers.
But splitmuxsink not able to request pad to flvmux as flvmux has "audio" and "video" as pad templates.
fix: splitmuxsink should fallback to "audio" and "video" when template not found.
https://bugzilla.gnome.org/show_bug.cgi?id=774507
Do not use last buffer TS + buffer duration because buffer duration
might be inaccurate, especially for frame rates like 30fps where a
rounding error is observed.
https://bugzilla.gnome.org/show_bug.cgi?id=773785
Commit 83e718 added a pad template to splitmux request
pads, which means that GstElement now releases the pads on
dispose, but after having removed all elements in the bin
and unlinked them. Make sure we can handle cleanup in that case
without throwing assertions.
https://bugzilla.gnome.org/show_bug.cgi?id=773784
The pacing of the overall muxing is controlled
by the video GOPs arriving, so we can only handle
1 video stream, and the request pad is named accordingly.
Ignore a request for a 2nd video pad if there's already
an active one.
On 32-bit x86: gstsplitmuxsink.c:966:31: warning: format ‘%u’ expects
argument of type ‘unsigned int’, but argument 9 has type
‘guint64 {aka long long unsigned int}’
This reverts commit fa008f271a.
async-handling in GstBin causes the pipeline to spin at 100%
CPU as the top-level pipeline tries to change that state
to PLAYING constantly. This is a workaround for a core
problem, essentially, but an improvement in this case for now.
After dropping the splitmux lock, re-check the state,
don't just fall through and sleep unconditionally,
as we may have already missed the wakeup.
https://bugzilla.gnome.org/show_bug.cgi?id=769514
Use signed clock times for running time everywhere
so that we handle negative running times without
going haywire, similar to what queue and multiqueue
do these days.
Set the async-handling property on GstBin to let it manage
async-handling instead of the local handling from the previous
commit. Works because of #174a5e in core
When switching fragments, hide the async-start/async-done
messages from the parent bin, as otherwise we sometimes (very rarely)
hang in PAUSED instead of returning / continuing to PLAYING
state.
Make sure that all data is drained out when the reference pad
goes EOS. Fixes a problem where data that arrives on other
pads after the reference pad finishes can stall forever and
never pass EOS.
https://bugzilla.gnome.org/show_bug.cgi?id=763711
Deadlock occurs when splitting files if one stream received no buffer during
the first GOP of the next file. That can happen in that scenario for example:
1) The first GOP of video is collected, it has a duration of 10s.
max_in_running_time is set to 10s.
2) Other streams catchup and we receive the first subtitle buffer at ts=0 and
has a duration of 1min.
3) We receive the 2nd subtitle buffer with a ts=1min. in_running_time is set to
1min. That buffer is blocked in handle_mq_input() because
max_in_running_time is still 10s.
4) Since all in_running_time are now > 10s, max_out_running_time is now set to
10s. That first GOP gets recorded into the file. The muxer pop buffers out
of the mq, when it tries to pop a 2nd subtitle buffer it blocks because the
GstDataQueue is empty.
5) A 2nd GOP of video is collected and has a duration of 10s as well.
max_in_running_time is now 20s. Since subtitle's in_running_time is already
1min, that GOP is already complete.
6) But let's say we overran the max file size, we thus set state to
SPLITMUX_STATE_ENDING_FILE now. As soon as a buffer with ts > 10s (end of
previous GOP) arrives in handle_mq_output(), EOS event is sent downstream
instead. But since the subtitle queue is empty, that's never going to
happen. Pipeline is now deadlocked.
To fix this situation we have to:
- Send a dummy event through the queue to wakeup output thread.
- Update out_running_time to at least max_out_running_time so it sends EOS.
- Respect time order, so we set out_running_tim=max_in_running_time because
that's bigger than previous buffer and smaller than next.
https://bugzilla.gnome.org/show_bug.cgi?id=763711
Don't increment running time from every buffer. The correct
logic to only increment when running time advances is a
little further down, so delete this left-over line.
Instead, delay it until all request pads have been released. This is
because the release_pad() vfunc requires the multiqueue and muxer to
be there in order to release their request pads as well. If those
elements are destroyed earlier, release_pad() does not work, no
pads are released and some resources are leaked.
https://bugzilla.gnome.org/show_bug.cgi?id=753622
mux_start_time refers to the running_time of the buffer
that goes first in the output file. Normally this time is
0, so this variable is initialized to 0 during the state
change to PAUSED.
However, when dealing with dynamic pipelines and starting
a recording while the pipeline has already run for a while,
the running_time of the first buffer is > 0 and this causes
a problem with detecting the end of the first file(s) when
splitting by duration, because the code will later compare
the threshold_time with (last buffer running_time - mux_start_time)
and will get it wrong until mux_start_time advances enough
to make this difference < threshold_time, creating empty files
in the meantime.
https://bugzilla.gnome.org/show_bug.cgi?id=753624
If the GOP is completed, pads have to start gathering for the
next one but it is possible that the the state might go to
COLLECTING_GOP_START and back to WAITING_GOP_COMPLETE before the
thread has a chance to wake up and proceed, leaving it trapped in
the check_completed_gop loop and deadlocking the other threads
waiting for it to advance.
To solve it, this patch also checks that tha input running time
hasn't changed to prevent this scenario.
In certain applications, splitting into files named after a base
location template and an incremental sequence number is not enough.
This signal gives more fine-grained control to the application to
decide how to name the files.
https://bugzilla.gnome.org/show_bug.cgi?id=750106
Sometimes, extra async-start/done from the internal sink
while the element is still starting up can cause splitmuxsink
to stall in PAUSED state when it has been set to PLAYING
by the app. Drop the child's async-start/done messages while
switching, so they don't cause state changes at the
splitmuxsink level.
https://bugzilla.gnome.org/show_bug.cgi?id=750747
When deciding whether it's time to switch to a new file, take into
account data that's been released for pushing, but hasn't yet
been pushed - because downstream is slow or the threads haven't been
scheduled.
Fixes a race in the unit test and probably in practice - sometimes
failing to switch when it should for an extra GOP or two.
Also fix a problem in splitmuxsrc where playback sometimes
stalls at startup if types are found too quickly.
https://bugzilla.gnome.org/show_bug.cgi?id=750747
because _release_pad tries to release it from ctx->sinkpad, which is
multiqueue's sink pad, and currently fails because the probe is not
installed there
Implement 2 new elements - splitmuxsink and splitmuxsrc.
splitmuxsink is a bin which wraps a muxer and takes 1 video stream,
plus audio/subtitle streams, and starts a new file
whenever necessary to avoid overrunning a threshold of either bytes
or time. New files are started at a keyframe, and corresponding audio
and subtitle streams are split at packet boundaries to match
video GOP timestamps.
splitmuxsrc is a corresponding source element which handles
the splitmux:// URL and plays back all component files,
reconstructing the original elementary streams as it goes.
