Same changes as done for wasapisink in cbe2fc40a. Turns out this is
sometimes also needed for capture. Reported by Mathieu_Du.
Also improve logging in that case for easier debugging.
Sometimes the minimum period advertised by a card results in an
unaligned buffer size error during initialization in exclusive mode.
In that case, we can fetch the actual buffer size in frames and
calculate the period from that.
We can't do this pre-emptively because we can't call GetBufferSize
till Initialize has been called at least once.
https://bugzilla.gnome.org/show_bug.cgi?id=793289
This reduces the chances of startup glitches, and also reduces the
chances that we'll get garbled output due to driver bugs.
Recommended by the WASAPI documentation.
https://bugzilla.gnome.org/show_bug.cgi?id=793289
So far, we have been completely discarding the values of latency-time
and buffer-time and trying to always open the device in the lowest
latency mode possible. However, sometimes this is a bad idea:
1. When we want to save power/CPU and don't want low latency
2. When the lowest latency setting causes glitches
3. Other audio-driver bugs
Now we will try to follow the user-set values of latency-time and
buffer-time in shared mode, and only latency-time in exclusive mode (we
have no control over the hardware buffer size, and there is no use in
setting GstAudioRingBuffer size to something larger).
The elements will still try to open the devices in the lowest latency
mode possible if you set the "low-latency" property to "true".
https://bugzilla.gnome.org/show_bug.cgi?id=793289
This requires using allocated strings, but it's the best option. For
instance, a call could fail because CoInitialize() wasn't called, or
because some other thing in the stack failed.
https://bugzilla.gnome.org/show_bug.cgi?id=793289
This is particularly important when running in exclusive mode because
any delays will immediately cause glitching.
The MinGW version in Cerbero is too old, so we can only enable this when
building with MSVC or when people build GStreamer for MSYS2 or other
MinGW-based distributions.
To force-enable this code when building with MinGW, build with
CFLAGS="-DGST_FORCE_WIN_AVRT -lavrt".
https://bugzilla.gnome.org/show_bug.cgi?id=793289
This provides much lower latency compared to opening in shared mode,
but it also means that the device cannot be opened by any other
application. The advantage is that the achievable latency is much
lower.
In shared mode, WASAPI's engine period is 10ms, and so that is the
lowest latency achievable.
In exclusive mode, the limit is the device period itself, which in my
testing with USB DACs, on-board PCI sound-cards, and HDMI cards is
between 2ms and 3.33ms.
We set our audioringbuffer limits to match the device, so the
achievable sink latency is 6-9ms. Further improvements can be made if
needed.
https://bugzilla.gnome.org/show_bug.cgi?id=793289
We will use ->device for storing a pointer to the IMMDevice structure
which is needed for fetching the caps supported by devices in
exclusive mode.
https://bugzilla.gnome.org/show_bug.cgi?id=793289
This will set the actual-latency-time and actual-buffer-time of the sink
and source.
We completely ignore the latency-time/buffer-time values set
on the element because WASAPI is happiest when it is reading/writing at
the default period. Improving this will likely require the use of the
IAudioClient3 interfaces which are not available in MinGW yet.
https://bugzilla.gnome.org/show_bug.cgi?id=792897
Currently only does probing and does not handle messages from
endpoints/devices. In the future we want to do proper monitoring which
is well-supported in WASAPI.
https://bugzilla.gnome.org/show_bug.cgi?id=792897
We need to parse the WAVEFORMATEXTENSIBLE structure, figure out what
positions the channels have (if they are positional), and reorder them
as necessary.
https://bugzilla.gnome.org/show_bug.cgi?id=792897
There is no fixed limitation for the number of devices on the
decklink API side according to BlackMagic. Many PC motherboards
are able support 6 decklink cards each with up to 8 inputs so
a limit of 16 might well be too low.
https://bugzilla.gnome.org/show_bug.cgi?id=777239
Both the source and the sink elements were broken in a number of ways:
* prepare() was assuming that the format was always S16LE 2ch 44.1KHz.
We now probe the preferred format with GetMixFormat().
* Device initialization was done with the wrong buffer size
(buffer_time is in microseconds, not nanoseconds).
* sink_write() and src_read() were just plain wrong and would never
write or read anything useful.
* Some functions in prepare() were always returning FALSE which meant
trying to use the elements would *always* fail.
* get_caps() and delay() were not implemented at all.
TODO: support for >2 channels
TODO: pro-audio low-latency
TODO: SPDIF and other encoded passthroughs
Three new properties are now implemented: role, mute, and device.
* 'role' designates the stream role of the initialized device, see:
https://msdn.microsoft.com/en-us/library/windows/desktop/dd370842(v=vs.85).aspx
* 'device' is a system-wide GUIDesque string for a specific device.
* 'mute' is a sink property and simply mutes it.
On my Windows 8.1 system, the lowest latency that works is:
wasapisrc buffer-time=20000
wasapisink buffer-time=10000
aka, 20ms and 10ms respectively. These values are close to the lowest
possible with the IAudioClient interface. Further improvements require
porting to IAudioClient2 or IAudioClient3.
https://docs.microsoft.com/en-us/windows-hardware/drivers/audio/low-latency-audio
Sometimes we might get an audio packet without a corresponding video
frame. In these cases, the stream and hardware reference timestamps
would be missing, because they're called on the video frame. Instead of
potentially breaking stuff downstream that might depend on these, we now
extrapolate them.
https://bugzilla.gnome.org/show_bug.cgi?id=792042
When we receive a video or audio buffer, we calculate the next stream
time based on the current stream time + buffer duration. If the next
buffer's stream time is after that, we issue a warning.
This happens because the stream time incoming from Decklink should be
really constant and without gaps. If there is a gap, it means that
something went wrong, e.g. the internal buffer pool is empty (too many
buffers queued up downstream).
https://bugzilla.gnome.org/show_bug.cgi?id=781776
Sometimes we might get an audio packet without a corresponding video
frame. In these cases, the stream and hardware reference timestamps
would be missing, because they're called on the video frame. Instead of
potentially breaking stuff downstream that might depend on these, we now
extrapolate them.
https://bugzilla.gnome.org/show_bug.cgi?id=792042
The correct behaviour of anything stuck in the ->render() function
between ->unlock() and ->unlock_stop() is to call
gst_base_sink_wait_preroll() and only return an error if this returns an
error, otherwise, it must continue where it left off!
https://bugzilla.gnome.org/show_bug.cgi?id=774950
Not only if the video sink is set to PLAYING so far. Also give more
useful debug output about why we don't start, and don't start if already
started.
Also refactor the function to early-return instead of having a huge
if-else block over the whole function.
https://bugzilla.gnome.org/show_bug.cgi?id=790114
The Decklink and GstAudioBaseSink APIs don't fit very well together,
which causes various problems due to inaccuracies in the clock
calculations and the actual ringbuffer and GStreamer's copy getting of
sync.
Problems are audio drop-outs and A/V sync getting wrong after
pausing/seeking.
https://bugzilla.gnome.org/show_bug.cgi?id=790114
