This requires using allocated strings, but it's the best option. For
instance, a call could fail because CoInitialize() wasn't called, or
because some other thing in the stack failed.
https://bugzilla.gnome.org/show_bug.cgi?id=793289
This is particularly important when running in exclusive mode because
any delays will immediately cause glitching.
The MinGW version in Cerbero is too old, so we can only enable this when
building with MSVC or when people build GStreamer for MSYS2 or other
MinGW-based distributions.
To force-enable this code when building with MinGW, build with
CFLAGS="-DGST_FORCE_WIN_AVRT -lavrt".
https://bugzilla.gnome.org/show_bug.cgi?id=793289
This provides much lower latency compared to opening in shared mode,
but it also means that the device cannot be opened by any other
application. The advantage is that the achievable latency is much
lower.
In shared mode, WASAPI's engine period is 10ms, and so that is the
lowest latency achievable.
In exclusive mode, the limit is the device period itself, which in my
testing with USB DACs, on-board PCI sound-cards, and HDMI cards is
between 2ms and 3.33ms.
We set our audioringbuffer limits to match the device, so the
achievable sink latency is 6-9ms. Further improvements can be made if
needed.
https://bugzilla.gnome.org/show_bug.cgi?id=793289
We will use ->device for storing a pointer to the IMMDevice structure
which is needed for fetching the caps supported by devices in
exclusive mode.
https://bugzilla.gnome.org/show_bug.cgi?id=793289
This will set the actual-latency-time and actual-buffer-time of the sink
and source.
We completely ignore the latency-time/buffer-time values set
on the element because WASAPI is happiest when it is reading/writing at
the default period. Improving this will likely require the use of the
IAudioClient3 interfaces which are not available in MinGW yet.
https://bugzilla.gnome.org/show_bug.cgi?id=792897
Currently only does probing and does not handle messages from
endpoints/devices. In the future we want to do proper monitoring which
is well-supported in WASAPI.
https://bugzilla.gnome.org/show_bug.cgi?id=792897
We need to parse the WAVEFORMATEXTENSIBLE structure, figure out what
positions the channels have (if they are positional), and reorder them
as necessary.
https://bugzilla.gnome.org/show_bug.cgi?id=792897
There is no fixed limitation for the number of devices on the
decklink API side according to BlackMagic. Many PC motherboards
are able support 6 decklink cards each with up to 8 inputs so
a limit of 16 might well be too low.
https://bugzilla.gnome.org/show_bug.cgi?id=777239
Both the source and the sink elements were broken in a number of ways:
* prepare() was assuming that the format was always S16LE 2ch 44.1KHz.
We now probe the preferred format with GetMixFormat().
* Device initialization was done with the wrong buffer size
(buffer_time is in microseconds, not nanoseconds).
* sink_write() and src_read() were just plain wrong and would never
write or read anything useful.
* Some functions in prepare() were always returning FALSE which meant
trying to use the elements would *always* fail.
* get_caps() and delay() were not implemented at all.
TODO: support for >2 channels
TODO: pro-audio low-latency
TODO: SPDIF and other encoded passthroughs
Three new properties are now implemented: role, mute, and device.
* 'role' designates the stream role of the initialized device, see:
https://msdn.microsoft.com/en-us/library/windows/desktop/dd370842(v=vs.85).aspx
* 'device' is a system-wide GUIDesque string for a specific device.
* 'mute' is a sink property and simply mutes it.
On my Windows 8.1 system, the lowest latency that works is:
wasapisrc buffer-time=20000
wasapisink buffer-time=10000
aka, 20ms and 10ms respectively. These values are close to the lowest
possible with the IAudioClient interface. Further improvements require
porting to IAudioClient2 or IAudioClient3.
https://docs.microsoft.com/en-us/windows-hardware/drivers/audio/low-latency-audio
Sometimes we might get an audio packet without a corresponding video
frame. In these cases, the stream and hardware reference timestamps
would be missing, because they're called on the video frame. Instead of
potentially breaking stuff downstream that might depend on these, we now
extrapolate them.
https://bugzilla.gnome.org/show_bug.cgi?id=792042
When we receive a video or audio buffer, we calculate the next stream
time based on the current stream time + buffer duration. If the next
buffer's stream time is after that, we issue a warning.
This happens because the stream time incoming from Decklink should be
really constant and without gaps. If there is a gap, it means that
something went wrong, e.g. the internal buffer pool is empty (too many
buffers queued up downstream).
https://bugzilla.gnome.org/show_bug.cgi?id=781776
Sometimes we might get an audio packet without a corresponding video
frame. In these cases, the stream and hardware reference timestamps
would be missing, because they're called on the video frame. Instead of
potentially breaking stuff downstream that might depend on these, we now
extrapolate them.
https://bugzilla.gnome.org/show_bug.cgi?id=792042
The correct behaviour of anything stuck in the ->render() function
between ->unlock() and ->unlock_stop() is to call
gst_base_sink_wait_preroll() and only return an error if this returns an
error, otherwise, it must continue where it left off!
https://bugzilla.gnome.org/show_bug.cgi?id=774950
Not only if the video sink is set to PLAYING so far. Also give more
useful debug output about why we don't start, and don't start if already
started.
Also refactor the function to early-return instead of having a huge
if-else block over the whole function.
https://bugzilla.gnome.org/show_bug.cgi?id=790114
The Decklink and GstAudioBaseSink APIs don't fit very well together,
which causes various problems due to inaccuracies in the clock
calculations and the actual ringbuffer and GStreamer's copy getting of
sync.
Problems are audio drop-outs and A/V sync getting wrong after
pausing/seeking.
https://bugzilla.gnome.org/show_bug.cgi?id=790114
When we cannot scale, we need to enforce the pixel aspect ratio.
This was partly implemented in the previous patch. Doing this
simplify some of the code.
https://bugzilla.gnome.org/show_bug.cgi?id=784599
1. Similar to 880f3d8, don't consider not getting an output buffer as
an error during flushing. I've seen the following sometimes when
encoding:
W GStreamer+amcvideoenc: java.lang.IllegalStateException
W GStreamer+amcvideoenc: at android.media.MediaCodec.getBuffer(Native Method)
W GStreamer+amcvideoenc: at android.media.MediaCodec.getOutputBuffer(MediaCodec.java:2886)
2. For amcvideodec/enc, call _find_nearest_frame (which grabs a fresh
reference on a GstVideoCodecFrame) after we have an output buffer,
so as to not leak the reference, in case getting an output buffer
fails.
Otherwise, if we get an error grabbing the output buffer, we leak
the reference to the frame. This can cause issues with a
v4l2bufferpool feeding the encoder not being able to clean itself
up properly due to buffers still being marked as in-use.
https://bugzilla.gnome.org/show_bug.cgi?id=791258
This is to be used with gst_video_overlay_set_render_rectangle()
so the application can calculate a rectangle that fits inside
the display. The property changes are notify in a way that you
can watch either notify::display-width or notify::display-height
and both will be up-to-data when this is called back. Before the
element is started, the size will be 0x0.
https://bugzilla.gnome.org/show_bug.cgi?id=784599
Implement videooverlay interface in kmssink, divided into two cases:
when driver supports scale, then we do refresh in show_frame(); if
not, send a reconfigure event to upstream and re-negotiate, using the
new size.
https://bugzilla.gnome.org/show_bug.cgi?id=784599
If the driver requires more data, just unref the frame at the moment
then retreive/finish the frame after encoding is finished.
This also fixes a memory leak.
https://bugzilla.gnome.org/show_bug.cgi?id=790312
Fixes outputted frame sequence when performing a seek
i.e. when seeking backwards, the first frame after the seek was a frame
from the future. This would result in GstVideoDecoder essentially
marking all the timestamps as essentially bogus and the base class would
attempt to compensate. A visible indication of this was 'decreasing timestamp'
warning after a seek.
https://bugzilla.gnome.org/show_bug.cgi?id=790478
The "fields" flag is ignored because currently GStreamer doesn't support
having only top or only bottom fields inside a frame. The "drop frame"
flag is ignored because some occurrences have been spotted where it
wasn't set while it should have been. In practice, when we have 29.97 or
59.94 FPS, it's always drop-frame.
https://bugzilla.gnome.org/show_bug.cgi?id=790112
When we receive a video or audio buffer, we calculate the next stream
time based on the current stream time + buffer duration. If the next
buffer's stream time is after that, we issue a warning.
This happens because the stream time incoming from Decklink should be
really constant and without gaps. If there is a gap, it means that
something went wrong, e.g. the internal buffer pool is empty (too many
buffers queued up downstream).
https://bugzilla.gnome.org/show_bug.cgi?id=781776
If we drop many frames at once, printing one message per video frame and
one per audio packet would cause a lot of disk IO. Just print a total at
the end.
https://bugzilla.gnome.org/show_bug.cgi?id=788780
Now that we are doing lazy allocation, we may endup calling _stop()
before the allocator was created. As a side effect, we need to nul-check
the pointer before calling it's method (_clear_cache()).
https://bugzilla.gnome.org/show_bug.cgi?id=787593
DRM_RDWR was not defined until libdrm 2.4.68. However,
in configure.ac we only require libdrm >= 2.4.55.
Seems silly to to bump minimum libdrm version for a simple
define. Thus, define DRM_RDWR if it's not defined.
This fixes compilation error introduced in:
commit 922031b0f9
Author: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Date: Tue Sep 12 12:07:13 2017 -0400
kms: Export DMABuf from Dumb buffer when possible
https://bugzilla.gnome.org/show_bug.cgi?id=787593
Signed-off-by: U. Artie Eoff <ullysses.a.eoff@intel.com>
When we guess the strides, we need to also update the GstVideoInfo.size
otherwise the memory size will be set to something smaller then needed.
This was causing crash with the DMABuf exportation, since we would not
mmap() a large enough buffer.
https://bugzilla.gnome.org/show_bug.cgi?id=787593
The buffer itself is 128 bytes into the allocated memory area, to be
able to store the size and other metadata before it. Freeing the buffer
directly will make malloc moderately unhappy.
If bo allocation failed we destroy the buffer and return GST_FLOW_ERROR,
but the @buffer pointer was still pointing to the address of the
destroyed buffer. gst_kms_sink_copy_to_dumb_buffer() was then trying to
unref it when bailing out causing a crash.
Leave @buffer untouched if allocation failed to fix the crash.
Also remove the check on *buffer being not NULL as gst_buffer_new()
will abort if it failed.
https://bugzilla.gnome.org/show_bug.cgi?id=787442
Implement videooverlay interface in kmssink, divided into two cases:
when driver supports scale, then we do refresh in show_frame(); if
not, send a reconfigure event to upstream and re-negotiate, using the
new size.
https://bugzilla.gnome.org/show_bug.cgi?id=784599
We used to to handle the driver pitch only for single plan video format.
Add support for multi planes format by re-using the extrapolate function
from the v4l2 element.
Also use this pitch to calculate the proper offsets.
Prevent DRM drivers to pick a slow path if the pitches/offsets don't
match the ones it reported.
https://bugzilla.gnome.org/show_bug.cgi?id=785029
No semantic change, just renamed the 'tmp' variable to a more meaningful
name and to use the same structure as in gst_kms_allocator_bo_alloc().
Needed as I'm going to move the gst_memory_init() call after the
allocation of the DUMB buffer.
https://bugzilla.gnome.org/show_bug.cgi?id=785029
HRESULT is unsigned long on Windows, but the Decklink headers define
it to 'int' on Linux. Confusingly, the defines that talk about the
possible return values for it use long constants. The easy fix would
be to change the linux/LinuxCOM.h header, but that's copied from the
decklink SDK.
Change the logging to always upcast to unsigned long while printing
HRESULT for consistency across platforms.
gstdecklinkvideosrc.cpp:425:7: warning: format '%x' expects argument of type 'unsigned int', but argument 8 has type 'HRESULT {aka long int}' [-Wformat]
[and so on]
gstdecklinkaudiosink.cpp:155:19: error: conflicting type attributes specified for 'virtual HRESULT GStreamerAudioOutputCallback::QueryInterface(const IID&, void**)'
In file included from /var/lib/jenkins/workspace/cerbero-cross-mingw32/workdir/mingw/w32/bin/../lib/gcc/i686-w64-mingw32/4.7.3/../../../../i686-w64-mingw32/include/objbase.h:153:0,
from /var/lib/jenkins/workspace/cerbero-cross-mingw32/workdir/mingw/w32/bin/../lib/gcc/i686-w64-mingw32/4.7.3/../../../../i686-w64-mingw32/include/ole2.h:16,
from /var/lib/jenkins/workspace/cerbero-cross-mingw32/workdir/mingw/w32/bin/../lib/gcc/i686-w64-mingw32/4.7.3/../../../../i686-w64-mingw32/include/windows.h:94,
from /var/lib/jenkins/workspace/cerbero-cross-mingw32/workdir/mingw/w32/bin/../lib/gcc/i686-w64-mingw32/4.7.3/../../../../i686-w64-mingw32/include/rpc.h:16,
from win/DeckLinkAPI.h:27,
from gstdecklink.h:35,
from gstdecklinkaudiosink.h:27,
from gstdecklinkaudiosink.cpp:25:
/var/lib/jenkins/workspace/cerbero-cross-mingw32/workdir/mingw/w32/bin/../lib/gcc/i686-w64-mingw32/4.7.3/../../../../i686-w64-mingw32/include/unknwn.h:67:25: error: overriding 'virtual HRESULT IUnknown::QueryInterface(const IID&, void**)'
(and many more)
https://ci.gstreamer.net/job/cerbero-cross-mingw32/6407/console
The default memory allocator of the decklink library allocates
a fixed pool of buffers, and the number of buffers is unknown.
This makes it impossible do useful queuing downstream. The new
memory allocator can create an unlimited number of buffers,
giving all queuing features one would expect from a live source.
https://bugzilla.gnome.org/show_bug.cgi?id=782556
In this patch we keep track of the cached kmsmem in a way
that we can clear the cache during the drain process. This
release the framebuffer before waiting for the next vblank,
hence add support for DRM driver (like Intel one) that release
the associated DMABuf reference asynchronously.
https://bugzilla.gnome.org/show_bug.cgi?id=782774
kmssink keeps a reference on the last rendered buffer. If this buffer
refers to an upstream buffer, it should be should be released on DRAIN
and ALLOCATION queries so all upstream buffers can be returned to the
pool if needed. As the buffer may be used for scanout, we copy this
buffer into a dumb buffer prior to let it go.
Based on patch from Guillaume Desmottes <guillaume.desmottes@collabora.com>
https://bugzilla.gnome.org/show_bug.cgi?id=782774
This otherwise breaks DMABuf reclaiming. This is not visible from
userspace, but inside the kernel, the DRM driver will hold a ref to the
DMABuf object. With a V4L2 driver allocating those DMABuf, it then
prevent changing the resolution and re-allocation new buffers.
https://bugzilla.gnome.org/show_bug.cgi?id=782774
Milliseconds was wrong and made use of this timeout quite
confusing. The code uses the value as microsenconds so
any meaningful number was off by orders of magnitude.
Set the pts and dts on the frame that we receive from the msdk.
Also fix the inverted logic in setting sync points, previously we
were marking all frames as sync points except IDRs.
https://bugzilla.gnome.org/show_bug.cgi?id=782801
When extracting an aux buffer from an MJPG carrier, at
*least* put the original timestamp on it, even if we
fail to apply any other timestamp (which we always do
at the moment, because the timestamp calculating code
was never finished). Apply a DTS using the camera
supplied delay value as well, assuming that there's
no re-ordering going on (there isn't in the C920,
which is really the only extant camera doing this
stuff) and a warning if that turns out not to be true.
This is basically a frame counter provided by the driver and it's
advancing at the speed of the HDMI/SDI input. Having this available on
each buffer allows to know what constant-framerate-based timestamp each
frame is corresponding to and can be used e.g. to write out files
accordingly without having the local pipeline clock timestamps used.
https://bugzilla.gnome.org/show_bug.cgi?id=779213
The main advantage is that our sleeps can be interrupted in case of
an src_reset(). Earlier, we would need to wait for a read to complete
before we could do a reset, which could take a long time.
https://bugzilla.gnome.org/show_bug.cgi?id=781249
The audio packet times can be completely unrelated to the video stream
time, depending on the card. While this looks like a bug in the driver,
just always using the video stream time (which is correct) works as a
workaround for now.
Earlier, the plugin was ignoring those settings and blindly setting
buffer-time to 2 seconds and latency-time to 200ms, which forced all
pipelines to have a minimum latency of 200ms + sink latency.
The values of segsize and segtotal were also not derived correctly.
Now we obey these values, and you can get close to the previous
behaviour by setting buffer-time and latency-time manually. Note that
they are set in microseconds.
As a consequence, when we haven't received enough data from the
device, we now sleep for a time proportional to the data remaining.
However, Directsound is a deprecated API so it maintains its own
software ringbuffer which updates at arbitrary intervals. Hence we
might have to wait a full segsize to get the last 10% of data. To
avoid tight loops, we clamp our sleep floor at 10ms.
In my testing, this keeps the wakeups not-too-high (proportional to
the latency-time set on the source). Further improvements should be
made by fixing the WASAPI audio source plugin instead of this.
Directsound is deprecated and as the comments explain, it is
impossible to get low latency, decent quality, or good performance
from it.
Based on a patch by Sebastian Dröge <sebastian@centricular.com>
https://bugzilla.gnome.org/show_bug.cgi?id=781249
This reverts commit 845832263b.
The commit broke cross-mingw CI:
https://ci.gstreamer.net/job/GStreamer-master/8659/console
It seems that cross-mingw on Autotools and native-mingw on Meson
disagree about the size of HRESULT. Revert for now till I can
investigate the Meson side of things some more.
MinGW does not provide comsupp.lib, so there's no implementation of
_com_util::ConvertBSTRToString. Use a fallback implementation that
uses wcstombs() instead.
On MinGW we also truncate the name to 100 chars which should be fine.
The QTKit framework had been deprecated for long in favour of AVFundation
framework and we already have avfvideosrc that provides the same
functionality.
https://bugzilla.gnome.org/show_bug.cgi?id=782078
MediaCodec gives us a presentation timestamp of 0 if it does not know
anything, but GStreamer gives us GST_CLOCK_TIME_NONE. Don't mix up these
two.
https://bugzilla.gnome.org/show_bug.cgi?id=780190
This is basically a frame counter provided by the driver and it's
advancing at the speed of the HDMI/SDI input. Having this available on
each buffer allows to know what constant-framerate-based timestamp each
frame is corresponding to and can be used e.g. to write out files
accordingly without having the local pipeline clock timestamps used.
https://bugzilla.gnome.org/show_bug.cgi?id=779213
This reverts commit 6d256d9908.
It was configuring the period/buffer size in a way that often causes
drop-outs or complete underruns. Needs further investigation.
"meson encountered an error in file
sys/decklink/meson.build, line 33, column 2:
Invalid use of addition: must be str, not list"
Also remove nonsensical linker flags on windows.
https://bugzilla.gnome.org/show_bug.cgi?id=781156
segsize should be based on latency-time, and must be a multiple of the
frame size. segtotal should be based on buffer-time and segsize.
This prevents errors caused by outputting buffers that are not a
multiple of the frame size, and actually makes the buffer-time and
latency-time properties do what they're supposed to do.
gstkmssink.c: In function ‘gst_kms_sink_get_input_buffer’:
gstkmssink.c:1102:29: error: ‘mems[0]’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
kmsmem = (GstKMSMemory *) get_cached_kmsmem (mems[0]);
^~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors
Avfvideosrc represents an iphone camera or, on mac, a screencapture session.
The old API allowed you to select an input device by device index only. The new
API adds the ability to select the position (front or back facing) and
device-type (wide angle, telephoto, etc.). Furthermore, you can now specify
the orientation (portrait, landscape, etc.) of the videostream.
https://bugzilla.gnome.org/show_bug.cgi?id=778333
All code interacting with Objective-C objects should now use Automated
Reference Counting rather than manual memory management or Garbage
Collection. Because ARC prohibits C-structs from containing
references to Objective-C objects, all such fields are now typed
'gpointer'. Setting and gettings Objective-C fields on such a
struct now uses explicit __bridge_* calls to tell ARC about
object lifetimes.
https://bugzilla.gnome.org/show_bug.cgi?id=777847
It was previously possible for videotexturecache to be finalized before all of
its textures. Finalizing outstanding textures in this circumstance leads
to a crash. This patch ensure resources are freed in the proper order.
https://bugzilla.gnome.org/show_bug.cgi?id=779247
This seems to happen sometimes on some hardware, and is not really
critical as long as the scheduling of the normal frames works fine.
Only post a warning message for this case.
Overriding the pad query function completely overrides all the default
query handling implemented in basesrc, including caps etc. The correct
thing to do is just override the basesrc query vfunc and then chain up
for the queries we don't handle.
The cached texture was treated as user_data passed to GstGLBaseMemory
and freed with a GDestroyNotify function. However, this data must
be treated specially: it must be destroyed in the GL thread.
https://bugzilla.gnome.org/show_bug.cgi?id=778434
Enforce exactly the same raw video format on both sides, include a
videoconvert and queue before the video sink and make the shm area a
little bit bigger so that things don't get stuck.
and error out here already otherwise. We currently don't support
reconfiguration here and it can't happen really either unless the auto
mode is selected.
15:18:47 gstdecklinkaudiosrc.cpp:745:45: error: cannot initialize a parameter of type 'int64_t *' (aka 'long long *') with an rvalue of type 'gint64 *' (aka 'long *')
15:18:47 (BMDDeckLinkMaximumAudioChannels, &self->channels_found);
15:18:47 ^~~~~~~~~~~~~~~~~~~~~
15:18:47 ./linux/DeckLinkAPI.h:970:87: note: passing argument to parameter 'value' here
15:18:47 virtual HRESULT GetInt (/* in */ BMDDeckLinkAttributeID cfgID, /* out */ int64_t *value) = 0;
15:18:47 ^
gstdecklink.cpp:821:11: warning: variable 'dtc' is used uninitialized whenever 'if' condition is false [-Wsometimes-uninitialized]
if (m_input->videosrc) {
^~~~~~~~~~~~~~~~~
gstdecklink.cpp:837:41: note: uninitialized use occurs here
stream_time, stream_duration, dtc, no_signal);
^~~
gstdecklink.cpp:821:7: note: remove the 'if' if its condition is always true
if (m_input->videosrc) {
^~~~~~~~~~~~~~~~~~~~~~~
gstdecklink.cpp:810:29: note: initialize the variable 'dtc' to silence this warning
IDeckLinkTimecode *dtc;
^
= NULL
In some places a GST_FLOW_FLUSHING result was return as a FALSE
gboolean and then returned from a parent function as
GST_FLOW_ERROR. This prevented seeking from working.
https://bugzilla.gnome.org/show_bug.cgi?id=776360
gstamcvideodec.c: In function 'gst_amc_video_dec_src_query':
gstamcvideodec.c:2412:55: error: 'self' undeclared (first use in this function)
if (gst_gl_handle_context_query ((GstElement *) self, query,
This logic did not belong to the channel configuration
parser (only used by dvbbasebin) but to dvbsrc, which
is the element directly using this value and honoring
the "adapter" property.
Allows previously non-working cases like this to work:
GST_DVB_ADAPTER=1 gst-launch-1.0 dvbsrc delsys=11 modulation=7 frequency=689000000 ! fakesink
If they were not ported after 4+ years it seems unlikely that anybody is
ever going to need them again. They're still in the GIT history if
needed.
https://bugzilla.gnome.org/show_bug.cgi?id=774530
