There is no fixed limitation for the number of devices on the
decklink API side according to BlackMagic. Many PC motherboards
are able support 6 decklink cards each with up to 8 inputs so
a limit of 16 might well be too low.
https://bugzilla.gnome.org/show_bug.cgi?id=777239
Both the source and the sink elements were broken in a number of ways:
* prepare() was assuming that the format was always S16LE 2ch 44.1KHz.
We now probe the preferred format with GetMixFormat().
* Device initialization was done with the wrong buffer size
(buffer_time is in microseconds, not nanoseconds).
* sink_write() and src_read() were just plain wrong and would never
write or read anything useful.
* Some functions in prepare() were always returning FALSE which meant
trying to use the elements would *always* fail.
* get_caps() and delay() were not implemented at all.
TODO: support for >2 channels
TODO: pro-audio low-latency
TODO: SPDIF and other encoded passthroughs
Three new properties are now implemented: role, mute, and device.
* 'role' designates the stream role of the initialized device, see:
https://msdn.microsoft.com/en-us/library/windows/desktop/dd370842(v=vs.85).aspx
* 'device' is a system-wide GUIDesque string for a specific device.
* 'mute' is a sink property and simply mutes it.
On my Windows 8.1 system, the lowest latency that works is:
wasapisrc buffer-time=20000
wasapisink buffer-time=10000
aka, 20ms and 10ms respectively. These values are close to the lowest
possible with the IAudioClient interface. Further improvements require
porting to IAudioClient2 or IAudioClient3.
https://docs.microsoft.com/en-us/windows-hardware/drivers/audio/low-latency-audio
Sometimes we might get an audio packet without a corresponding video
frame. In these cases, the stream and hardware reference timestamps
would be missing, because they're called on the video frame. Instead of
potentially breaking stuff downstream that might depend on these, we now
extrapolate them.
https://bugzilla.gnome.org/show_bug.cgi?id=792042
When we receive a video or audio buffer, we calculate the next stream
time based on the current stream time + buffer duration. If the next
buffer's stream time is after that, we issue a warning.
This happens because the stream time incoming from Decklink should be
really constant and without gaps. If there is a gap, it means that
something went wrong, e.g. the internal buffer pool is empty (too many
buffers queued up downstream).
https://bugzilla.gnome.org/show_bug.cgi?id=781776
Sometimes we might get an audio packet without a corresponding video
frame. In these cases, the stream and hardware reference timestamps
would be missing, because they're called on the video frame. Instead of
potentially breaking stuff downstream that might depend on these, we now
extrapolate them.
https://bugzilla.gnome.org/show_bug.cgi?id=792042
The correct behaviour of anything stuck in the ->render() function
between ->unlock() and ->unlock_stop() is to call
gst_base_sink_wait_preroll() and only return an error if this returns an
error, otherwise, it must continue where it left off!
https://bugzilla.gnome.org/show_bug.cgi?id=774950
Not only if the video sink is set to PLAYING so far. Also give more
useful debug output about why we don't start, and don't start if already
started.
Also refactor the function to early-return instead of having a huge
if-else block over the whole function.
https://bugzilla.gnome.org/show_bug.cgi?id=790114
The Decklink and GstAudioBaseSink APIs don't fit very well together,
which causes various problems due to inaccuracies in the clock
calculations and the actual ringbuffer and GStreamer's copy getting of
sync.
Problems are audio drop-outs and A/V sync getting wrong after
pausing/seeking.
https://bugzilla.gnome.org/show_bug.cgi?id=790114
When we cannot scale, we need to enforce the pixel aspect ratio.
This was partly implemented in the previous patch. Doing this
simplify some of the code.
https://bugzilla.gnome.org/show_bug.cgi?id=784599
1. Similar to 880f3d8, don't consider not getting an output buffer as
an error during flushing. I've seen the following sometimes when
encoding:
W GStreamer+amcvideoenc: java.lang.IllegalStateException
W GStreamer+amcvideoenc: at android.media.MediaCodec.getBuffer(Native Method)
W GStreamer+amcvideoenc: at android.media.MediaCodec.getOutputBuffer(MediaCodec.java:2886)
2. For amcvideodec/enc, call _find_nearest_frame (which grabs a fresh
reference on a GstVideoCodecFrame) after we have an output buffer,
so as to not leak the reference, in case getting an output buffer
fails.
Otherwise, if we get an error grabbing the output buffer, we leak
the reference to the frame. This can cause issues with a
v4l2bufferpool feeding the encoder not being able to clean itself
up properly due to buffers still being marked as in-use.
https://bugzilla.gnome.org/show_bug.cgi?id=791258
This is to be used with gst_video_overlay_set_render_rectangle()
so the application can calculate a rectangle that fits inside
the display. The property changes are notify in a way that you
can watch either notify::display-width or notify::display-height
and both will be up-to-data when this is called back. Before the
element is started, the size will be 0x0.
https://bugzilla.gnome.org/show_bug.cgi?id=784599
Implement videooverlay interface in kmssink, divided into two cases:
when driver supports scale, then we do refresh in show_frame(); if
not, send a reconfigure event to upstream and re-negotiate, using the
new size.
https://bugzilla.gnome.org/show_bug.cgi?id=784599
If the driver requires more data, just unref the frame at the moment
then retreive/finish the frame after encoding is finished.
This also fixes a memory leak.
https://bugzilla.gnome.org/show_bug.cgi?id=790312
Fixes outputted frame sequence when performing a seek
i.e. when seeking backwards, the first frame after the seek was a frame
from the future. This would result in GstVideoDecoder essentially
marking all the timestamps as essentially bogus and the base class would
attempt to compensate. A visible indication of this was 'decreasing timestamp'
warning after a seek.
https://bugzilla.gnome.org/show_bug.cgi?id=790478
The "fields" flag is ignored because currently GStreamer doesn't support
having only top or only bottom fields inside a frame. The "drop frame"
flag is ignored because some occurrences have been spotted where it
wasn't set while it should have been. In practice, when we have 29.97 or
59.94 FPS, it's always drop-frame.
https://bugzilla.gnome.org/show_bug.cgi?id=790112
When we receive a video or audio buffer, we calculate the next stream
time based on the current stream time + buffer duration. If the next
buffer's stream time is after that, we issue a warning.
This happens because the stream time incoming from Decklink should be
really constant and without gaps. If there is a gap, it means that
something went wrong, e.g. the internal buffer pool is empty (too many
buffers queued up downstream).
https://bugzilla.gnome.org/show_bug.cgi?id=781776
If we drop many frames at once, printing one message per video frame and
one per audio packet would cause a lot of disk IO. Just print a total at
the end.
https://bugzilla.gnome.org/show_bug.cgi?id=788780
Now that we are doing lazy allocation, we may endup calling _stop()
before the allocator was created. As a side effect, we need to nul-check
the pointer before calling it's method (_clear_cache()).
https://bugzilla.gnome.org/show_bug.cgi?id=787593
DRM_RDWR was not defined until libdrm 2.4.68. However,
in configure.ac we only require libdrm >= 2.4.55.
Seems silly to to bump minimum libdrm version for a simple
define. Thus, define DRM_RDWR if it's not defined.
This fixes compilation error introduced in:
commit 922031b0f9
Author: Nicolas Dufresne <nicolas.dufresne@collabora.com>
Date: Tue Sep 12 12:07:13 2017 -0400
kms: Export DMABuf from Dumb buffer when possible
https://bugzilla.gnome.org/show_bug.cgi?id=787593
Signed-off-by: U. Artie Eoff <ullysses.a.eoff@intel.com>
When we guess the strides, we need to also update the GstVideoInfo.size
otherwise the memory size will be set to something smaller then needed.
This was causing crash with the DMABuf exportation, since we would not
mmap() a large enough buffer.
https://bugzilla.gnome.org/show_bug.cgi?id=787593
The buffer itself is 128 bytes into the allocated memory area, to be
able to store the size and other metadata before it. Freeing the buffer
directly will make malloc moderately unhappy.
If bo allocation failed we destroy the buffer and return GST_FLOW_ERROR,
but the @buffer pointer was still pointing to the address of the
destroyed buffer. gst_kms_sink_copy_to_dumb_buffer() was then trying to
unref it when bailing out causing a crash.
Leave @buffer untouched if allocation failed to fix the crash.
Also remove the check on *buffer being not NULL as gst_buffer_new()
will abort if it failed.
https://bugzilla.gnome.org/show_bug.cgi?id=787442
Implement videooverlay interface in kmssink, divided into two cases:
when driver supports scale, then we do refresh in show_frame(); if
not, send a reconfigure event to upstream and re-negotiate, using the
new size.
https://bugzilla.gnome.org/show_bug.cgi?id=784599
We used to to handle the driver pitch only for single plan video format.
Add support for multi planes format by re-using the extrapolate function
from the v4l2 element.
Also use this pitch to calculate the proper offsets.
Prevent DRM drivers to pick a slow path if the pitches/offsets don't
match the ones it reported.
https://bugzilla.gnome.org/show_bug.cgi?id=785029
No semantic change, just renamed the 'tmp' variable to a more meaningful
name and to use the same structure as in gst_kms_allocator_bo_alloc().
Needed as I'm going to move the gst_memory_init() call after the
allocation of the DUMB buffer.
https://bugzilla.gnome.org/show_bug.cgi?id=785029
HRESULT is unsigned long on Windows, but the Decklink headers define
it to 'int' on Linux. Confusingly, the defines that talk about the
possible return values for it use long constants. The easy fix would
be to change the linux/LinuxCOM.h header, but that's copied from the
decklink SDK.
Change the logging to always upcast to unsigned long while printing
HRESULT for consistency across platforms.
gstdecklinkvideosrc.cpp:425:7: warning: format '%x' expects argument of type 'unsigned int', but argument 8 has type 'HRESULT {aka long int}' [-Wformat]
[and so on]
gstdecklinkaudiosink.cpp:155:19: error: conflicting type attributes specified for 'virtual HRESULT GStreamerAudioOutputCallback::QueryInterface(const IID&, void**)'
In file included from /var/lib/jenkins/workspace/cerbero-cross-mingw32/workdir/mingw/w32/bin/../lib/gcc/i686-w64-mingw32/4.7.3/../../../../i686-w64-mingw32/include/objbase.h:153:0,
from /var/lib/jenkins/workspace/cerbero-cross-mingw32/workdir/mingw/w32/bin/../lib/gcc/i686-w64-mingw32/4.7.3/../../../../i686-w64-mingw32/include/ole2.h:16,
from /var/lib/jenkins/workspace/cerbero-cross-mingw32/workdir/mingw/w32/bin/../lib/gcc/i686-w64-mingw32/4.7.3/../../../../i686-w64-mingw32/include/windows.h:94,
from /var/lib/jenkins/workspace/cerbero-cross-mingw32/workdir/mingw/w32/bin/../lib/gcc/i686-w64-mingw32/4.7.3/../../../../i686-w64-mingw32/include/rpc.h:16,
from win/DeckLinkAPI.h:27,
from gstdecklink.h:35,
from gstdecklinkaudiosink.h:27,
from gstdecklinkaudiosink.cpp:25:
/var/lib/jenkins/workspace/cerbero-cross-mingw32/workdir/mingw/w32/bin/../lib/gcc/i686-w64-mingw32/4.7.3/../../../../i686-w64-mingw32/include/unknwn.h:67:25: error: overriding 'virtual HRESULT IUnknown::QueryInterface(const IID&, void**)'
(and many more)
https://ci.gstreamer.net/job/cerbero-cross-mingw32/6407/console
The default memory allocator of the decklink library allocates
a fixed pool of buffers, and the number of buffers is unknown.
This makes it impossible do useful queuing downstream. The new
memory allocator can create an unlimited number of buffers,
giving all queuing features one would expect from a live source.
https://bugzilla.gnome.org/show_bug.cgi?id=782556
In this patch we keep track of the cached kmsmem in a way
that we can clear the cache during the drain process. This
release the framebuffer before waiting for the next vblank,
hence add support for DRM driver (like Intel one) that release
the associated DMABuf reference asynchronously.
https://bugzilla.gnome.org/show_bug.cgi?id=782774
kmssink keeps a reference on the last rendered buffer. If this buffer
refers to an upstream buffer, it should be should be released on DRAIN
and ALLOCATION queries so all upstream buffers can be returned to the
pool if needed. As the buffer may be used for scanout, we copy this
buffer into a dumb buffer prior to let it go.
Based on patch from Guillaume Desmottes <guillaume.desmottes@collabora.com>
https://bugzilla.gnome.org/show_bug.cgi?id=782774
This otherwise breaks DMABuf reclaiming. This is not visible from
userspace, but inside the kernel, the DRM driver will hold a ref to the
DMABuf object. With a V4L2 driver allocating those DMABuf, it then
prevent changing the resolution and re-allocation new buffers.
https://bugzilla.gnome.org/show_bug.cgi?id=782774
Milliseconds was wrong and made use of this timeout quite
confusing. The code uses the value as microsenconds so
any meaningful number was off by orders of magnitude.
Set the pts and dts on the frame that we receive from the msdk.
Also fix the inverted logic in setting sync points, previously we
were marking all frames as sync points except IDRs.
https://bugzilla.gnome.org/show_bug.cgi?id=782801
When extracting an aux buffer from an MJPG carrier, at
*least* put the original timestamp on it, even if we
fail to apply any other timestamp (which we always do
at the moment, because the timestamp calculating code
was never finished). Apply a DTS using the camera
supplied delay value as well, assuming that there's
no re-ordering going on (there isn't in the C920,
which is really the only extant camera doing this
stuff) and a warning if that turns out not to be true.
This is basically a frame counter provided by the driver and it's
advancing at the speed of the HDMI/SDI input. Having this available on
each buffer allows to know what constant-framerate-based timestamp each
frame is corresponding to and can be used e.g. to write out files
accordingly without having the local pipeline clock timestamps used.
https://bugzilla.gnome.org/show_bug.cgi?id=779213
The main advantage is that our sleeps can be interrupted in case of
an src_reset(). Earlier, we would need to wait for a read to complete
before we could do a reset, which could take a long time.
https://bugzilla.gnome.org/show_bug.cgi?id=781249
The audio packet times can be completely unrelated to the video stream
time, depending on the card. While this looks like a bug in the driver,
just always using the video stream time (which is correct) works as a
workaround for now.
Earlier, the plugin was ignoring those settings and blindly setting
buffer-time to 2 seconds and latency-time to 200ms, which forced all
pipelines to have a minimum latency of 200ms + sink latency.
The values of segsize and segtotal were also not derived correctly.
Now we obey these values, and you can get close to the previous
behaviour by setting buffer-time and latency-time manually. Note that
they are set in microseconds.
As a consequence, when we haven't received enough data from the
device, we now sleep for a time proportional to the data remaining.
However, Directsound is a deprecated API so it maintains its own
software ringbuffer which updates at arbitrary intervals. Hence we
might have to wait a full segsize to get the last 10% of data. To
avoid tight loops, we clamp our sleep floor at 10ms.
In my testing, this keeps the wakeups not-too-high (proportional to
the latency-time set on the source). Further improvements should be
made by fixing the WASAPI audio source plugin instead of this.
Directsound is deprecated and as the comments explain, it is
impossible to get low latency, decent quality, or good performance
from it.
Based on a patch by Sebastian Dröge <sebastian@centricular.com>
https://bugzilla.gnome.org/show_bug.cgi?id=781249
This reverts commit 845832263b.
The commit broke cross-mingw CI:
https://ci.gstreamer.net/job/GStreamer-master/8659/console
It seems that cross-mingw on Autotools and native-mingw on Meson
disagree about the size of HRESULT. Revert for now till I can
investigate the Meson side of things some more.
MinGW does not provide comsupp.lib, so there's no implementation of
_com_util::ConvertBSTRToString. Use a fallback implementation that
uses wcstombs() instead.
On MinGW we also truncate the name to 100 chars which should be fine.
The QTKit framework had been deprecated for long in favour of AVFundation
framework and we already have avfvideosrc that provides the same
functionality.
https://bugzilla.gnome.org/show_bug.cgi?id=782078
MediaCodec gives us a presentation timestamp of 0 if it does not know
anything, but GStreamer gives us GST_CLOCK_TIME_NONE. Don't mix up these
two.
https://bugzilla.gnome.org/show_bug.cgi?id=780190
This is basically a frame counter provided by the driver and it's
advancing at the speed of the HDMI/SDI input. Having this available on
each buffer allows to know what constant-framerate-based timestamp each
frame is corresponding to and can be used e.g. to write out files
accordingly without having the local pipeline clock timestamps used.
https://bugzilla.gnome.org/show_bug.cgi?id=779213
This reverts commit 6d256d9908.
It was configuring the period/buffer size in a way that often causes
drop-outs or complete underruns. Needs further investigation.
"meson encountered an error in file
sys/decklink/meson.build, line 33, column 2:
Invalid use of addition: must be str, not list"
Also remove nonsensical linker flags on windows.
https://bugzilla.gnome.org/show_bug.cgi?id=781156
segsize should be based on latency-time, and must be a multiple of the
frame size. segtotal should be based on buffer-time and segsize.
This prevents errors caused by outputting buffers that are not a
multiple of the frame size, and actually makes the buffer-time and
latency-time properties do what they're supposed to do.
gstkmssink.c: In function ‘gst_kms_sink_get_input_buffer’:
gstkmssink.c:1102:29: error: ‘mems[0]’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
kmsmem = (GstKMSMemory *) get_cached_kmsmem (mems[0]);
^~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors
Avfvideosrc represents an iphone camera or, on mac, a screencapture session.
The old API allowed you to select an input device by device index only. The new
API adds the ability to select the position (front or back facing) and
device-type (wide angle, telephoto, etc.). Furthermore, you can now specify
the orientation (portrait, landscape, etc.) of the videostream.
https://bugzilla.gnome.org/show_bug.cgi?id=778333
All code interacting with Objective-C objects should now use Automated
Reference Counting rather than manual memory management or Garbage
Collection. Because ARC prohibits C-structs from containing
references to Objective-C objects, all such fields are now typed
'gpointer'. Setting and gettings Objective-C fields on such a
struct now uses explicit __bridge_* calls to tell ARC about
object lifetimes.
https://bugzilla.gnome.org/show_bug.cgi?id=777847
It was previously possible for videotexturecache to be finalized before all of
its textures. Finalizing outstanding textures in this circumstance leads
to a crash. This patch ensure resources are freed in the proper order.
https://bugzilla.gnome.org/show_bug.cgi?id=779247
This seems to happen sometimes on some hardware, and is not really
critical as long as the scheduling of the normal frames works fine.
Only post a warning message for this case.
Overriding the pad query function completely overrides all the default
query handling implemented in basesrc, including caps etc. The correct
thing to do is just override the basesrc query vfunc and then chain up
for the queries we don't handle.
The cached texture was treated as user_data passed to GstGLBaseMemory
and freed with a GDestroyNotify function. However, this data must
be treated specially: it must be destroyed in the GL thread.
https://bugzilla.gnome.org/show_bug.cgi?id=778434
Enforce exactly the same raw video format on both sides, include a
videoconvert and queue before the video sink and make the shm area a
little bit bigger so that things don't get stuck.
and error out here already otherwise. We currently don't support
reconfiguration here and it can't happen really either unless the auto
mode is selected.
15:18:47 gstdecklinkaudiosrc.cpp:745:45: error: cannot initialize a parameter of type 'int64_t *' (aka 'long long *') with an rvalue of type 'gint64 *' (aka 'long *')
15:18:47 (BMDDeckLinkMaximumAudioChannels, &self->channels_found);
15:18:47 ^~~~~~~~~~~~~~~~~~~~~
15:18:47 ./linux/DeckLinkAPI.h:970:87: note: passing argument to parameter 'value' here
15:18:47 virtual HRESULT GetInt (/* in */ BMDDeckLinkAttributeID cfgID, /* out */ int64_t *value) = 0;
15:18:47 ^
gstdecklink.cpp:821:11: warning: variable 'dtc' is used uninitialized whenever 'if' condition is false [-Wsometimes-uninitialized]
if (m_input->videosrc) {
^~~~~~~~~~~~~~~~~
gstdecklink.cpp:837:41: note: uninitialized use occurs here
stream_time, stream_duration, dtc, no_signal);
^~~
gstdecklink.cpp:821:7: note: remove the 'if' if its condition is always true
if (m_input->videosrc) {
^~~~~~~~~~~~~~~~~~~~~~~
gstdecklink.cpp:810:29: note: initialize the variable 'dtc' to silence this warning
IDeckLinkTimecode *dtc;
^
= NULL
In some places a GST_FLOW_FLUSHING result was return as a FALSE
gboolean and then returned from a parent function as
GST_FLOW_ERROR. This prevented seeking from working.
https://bugzilla.gnome.org/show_bug.cgi?id=776360
gstamcvideodec.c: In function 'gst_amc_video_dec_src_query':
gstamcvideodec.c:2412:55: error: 'self' undeclared (first use in this function)
if (gst_gl_handle_context_query ((GstElement *) self, query,
This logic did not belong to the channel configuration
parser (only used by dvbbasebin) but to dvbsrc, which
is the element directly using this value and honoring
the "adapter" property.
Allows previously non-working cases like this to work:
GST_DVB_ADAPTER=1 gst-launch-1.0 dvbsrc delsys=11 modulation=7 frequency=689000000 ! fakesink
If they were not ported after 4+ years it seems unlikely that anybody is
ever going to need them again. They're still in the GIT history if
needed.
https://bugzilla.gnome.org/show_bug.cgi?id=774530
Configure the display mode when setting the negotiated caps instead of
during showing the first frame.
A framebuffer is required to set the mode. Allocate a buffer object
according to the negotiated caps and use it to set the mode. This buffer
object cannot be freed until another page flip happened on the crtc
(i.e., until the first frame is rendered).
https://bugzilla.gnome.org/show_bug.cgi?id=773473
Signed-off-by: Víctor Manuel Jáquez Leal <vjaquez@igalia.com>
The force-modesetting parameter forces the kmssink to ignore already
configured display modes, to configure the display mode itself and use
the base plane for output.
https://bugzilla.gnome.org/show_bug.cgi?id=773473
If the input buffers have a different size than the display, the frames
would have to be scaled or positioned on the display. The kmssink cannot
decide which behaviour would be appropriate for which use case.
In order to avoid scaling or positioning of the input stream, allow only
the supported connector resolutions in the sink caps.
https://bugzilla.gnome.org/show_bug.cgi?id=773473
Signed-off-by: Víctor Manuel Jáquez Leal <vjaquez@igalia.com>
Displays usually support multiple modes. Therefore, the kmssink should
not only support the preferred mode, but any mode that is supported by
the display.
https://bugzilla.gnome.org/show_bug.cgi?id=773473
The kmssink assumed that the mode was already set by another application
and used an overlay plane for displaying the frames.
Use the preferred mode of the monitor and render to the base plane if
the crtc does not have a valid mode.
https://bugzilla.gnome.org/show_bug.cgi?id=773473
Signed-off-by: Víctor Manuel Jáquez Leal <vjaquez@igalia.com>
gstdecklink.cpp: In member function ‘virtual HRESULT GStreamerDecklinkInputCallback::VideoInputFrameArrived(IDeckLinkVideoInputFrame*, IDeckLinkAudioInputPacket*)’:
gstdecklink.cpp:766:34: error: ‘base_time’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
capture_time -= base_time;
^
First of all, all the HD and UHD modes should be top-field-first, as
also returned by the Decklink mode iterator API.
Then we should include the caps field "field-order" in the caps of the
source (not the sink due to negotiation problems with optional fields).
And finally we should set the TFF flag on interlaced buffers that are
top-field-first.
On some hardware the first few frames are bogus and not very useful.
Their timestamps are off, they have no timecodes, or there are spurious
black frames / no-signal frames. After a few frames this stabilizes
though.
https://bugzilla.gnome.org/show_bug.cgi?id=774850
Based on this we calculate the actual capture time, which should get us
rid of any capturing jitter by averaging it out.
Also add a output-stream-time property which forces the elements to
output the stream time directly instead of doing any conversion to the
pipeline clock. Use with care.
https://bugzilla.gnome.org/show_bug.cgi?id=774850
The hardware timestamps have no relation to when frames were produced,
only when frames arrived somewhere in the hardware. Especially there is
no guarantee that audio and video will have the same hardware timestamps
although they belong together, and even more important: the rate with
which the hardware timestamps increase is completely unrelated to the
rate with which the frames are captured!
As such we can as well use the pipeline clock directly and stop doing
complicated calculations. Also as a side effect this allows now running
without any pipeline clock, by directly making use of the stream times
as reported by the driver.
https://bugzilla.gnome.org/show_bug.cgi?id=774850
libkms should not be used, because it imposes limitations on the DRM
API, especially regarding bpp and stride. Instead the DRM IOCTL should
be used directly.
Switch from libkms to the IOCTL interface. Set bpp and height for
framebuffer allocation to properly handle planar video formats.
https://bugzilla.gnome.org/show_bug.cgi?id=773473
Signed-off-by: Víctor Jáquez <vjaquez@igalia.com>
When a frame is found to not have an associated input source (cable
unplugged, wrong mode selected), an element warning will be issued. When
the next frame in the stream is found to have an input source selected
(e.g. cable replugged), an element info will be issued.
https://bugzilla.gnome.org/show_bug.cgi?id=774629
Fixes:
Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '*** +[NSString stringWithUTF8String:]: NULL cString
in the state change test.
The default get_times() function of the base sink is just fine.
Remove the custom get_times() function, because the default function
already reads the timestamps from the buffers.
Signed-off-by: Michael Tretter <m.tretter@pengutronix.de>
https://bugzilla.gnome.org/show_bug.cgi?id=773473
Unfortunately this does not go through the normal state change
machinery, so we don't get notified about this in change_state().
However we need to stop scheduled playback, so that once PLAYING is
reached again we can start scheduled playback with the correct time.
Without this, flushing seeks in PLAYING will not work correctly:
decklinkvideosink will wait before showing the new frames for the amount
of time the pipeline was in PLAYING before.
Drawing is done via the GDI drawing functions. The cursor is
converted to a monochrome version before drawing. This is because
the GDI drawing functions seem to have undefined behavior with
cursor images including an alpha channel.
I could not find any other reliable way to draw these alpha
channel cursors without producing unwanted artifacts. These type
of cursors were introduced with Window Vista when run with it's
Aero theme.
Also adjust the cursor coordinates when capturing non-primary
screens via the "monitor" option.
https://bugzilla.gnome.org/show_bug.cgi?id=760172
* Rephrase tune error to be delsys-neutral
* Refer to the actual check in the 'missing sanity check' warnings
* Use "Delivery system" instead of 'delsys'. The
latter is OK as a shorthand in the code but not
even a real word
Currently dx9screencapsrc prints a verbose warning in case the screen
index is out of range for the current number of detected monitors. This
value is then dropped.
However there is no initial indication (beside the console print) if it
worked or not. This may result in capturing an unwanted screen as it
would capture the last set index that was not rejected.
This patch sets the index regardless. Instead, the element throws an
error when it tries to run or getting caps for an invalid index.
https://bugzilla.gnome.org/show_bug.cgi?id=771817
In most display sink, the logic is to use as much as possible
of the given window. In this case, the window is the screen,
hence it's logical to scale up.
https://bugzilla.gnome.org/show_bug.cgi?id=767422
The source region was scaled for display before being passed
to drmModeSetPlane, which resulted in a portion of the video
being cropped. While when crop meta was present, the rectangle
was not centered since we where using unscaled width/height.
https://bugzilla.gnome.org/show_bug.cgi?id=767422
Some kms drivers demands specific pitches over the ones calculated by
GstVideoInfo. For example, intel driver demands strides round up 64.
This patch queries the driver for the prefered pitch and overwrites it
in the pool's GstVideoInfo structure.
https://bugzilla.gnome.org/show_bug.cgi?id=768446
While gint64 and int64_t are always the same, clang does not agree with that.
/Applications/Xcode.app/Contents/Developer/usr/bin/make -C decklink
CXX libgstdecklink_la-gstdecklinkaudiosink.lo
gstdecklinkaudiosink.cpp:675:79: error: cannot initialize a parameter of type 'int64_t *' (aka 'long long *') with an rvalue of type 'gint64 *' (aka 'long *')
ret = buf->output->attributes->GetInt (BMDDeckLinkMaximumAudioChannels, &max_channels);
^~~~~~~~~~~~~
./linux/DeckLinkAPI.h:692:87: note: passing argument to parameter 'value' here
virtual HRESULT GetInt (/* in */ BMDDeckLinkAttributeID cfgID, /* out */ int64_t *value) = 0;
^
Scale down to milliseconds, otherwise at least some hardware has problems
scheduling the frames (or schedules them too slow) and we run out of available
frames.
https://bugzilla.gnome.org/show_bug.cgi?id=770282
This commit introduces IOSGLMemory which is a GLMemory that falls back to
GstAppleCoreVideoMemory for CPU access. This is a temporary solution until
IOSurface gets exposed as a public framework on iOS and so we can use
IOSurfaceMemory on both MacOS and iOS.
https://bugzilla.gnome.org/show_bug.cgi?id=769210
Add systemstream=false to caps, otherwise the decoder
may be picked for MPEG-PS files. Also parsed=true,
as video toolbox expects entire frame in
VTDecompressionSessionDecodeFrame.
https://bugzilla.gnome.org/show_bug.cgi?id=770049
https://github.com/mesonbuild/meson
With contributions from:
Tim-Philipp Müller <tim@centricular.com>
Matej Knopp <matej.knopp@gmail.com>
Jussi Pakkanen <jpakkane@gmail.com> (original port)
Highlights of the features provided are:
* Faster builds on Linux (~40-50% faster)
* The ability to build with MSVC on Windows
* Generate Visual Studio project files
* Generate XCode project files
* Much faster builds on Windows (on-par with Linux)
* Seriously fast configure and building on embedded
... and many more. For more details see:
http://blog.nirbheek.in/2016/05/gstreamer-and-meson-new-hope.htmlhttp://blog.nirbheek.in/2016/07/building-and-developing-gstreamer-using.html
Building with Meson should work on both Linux and Windows, but may
need a few more tweaks on other operating systems.
_stdint.h is generated by Autotools and we don't really need it. All
supported platforms now ship with stdint.h. The only stickler was MSVC,
and since Visual Studio 2015 it also ships stdint.h now.
Uncompressed RGB frames can be (usually are) bottom-up
layout in DirectShow, and the code to flip them wasn't
properly ported from 0.10. Fix it.
Fix post-processing of RGB buffers. We need a writable
buffer, but the requests pool is holding an extra ref.
This could use more fixing to use a buffer pool
On the ODroid C1+ the H265 and H264 have the same name but are listed as two
different codecs. We have to handle them as the same one that supports both,
as otherwise we will register the same GType name twice which fails and we
then only have H265 support and not H264 support.
ahssrc is a new plugin that enables Gstreamer to read from the
android.hardware.Sensor Android sensors. These sensors are treated as
buffers and can be passed through and manipulated by the pipeline.
https://bugzilla.gnome.org/show_bug.cgi?id=768110
The calculation of the offset table was done base on a plane size
estimation. This does not always work. Instead, use memory offset the
same we as it's implement in GstVideoMeta map functions.
Without setting the DRM_CLIENT_CAP_UNIVERSAL_PLANES capability bit, only
overlay planes are made available for compatibility with legacy clients.
But if a CRTC doesn't have an overlay plane associated, then kmssink is
not able to find a plane for the CRTC and the pipeline will fail, i.e:
ERROR kmssink gstkmssink.c:482:gst_kms_sink_start:<kmssink0> Could not find a plane for crtc
If no overlay planes were found for a given CRTC, fallback to universal
planes so DRM will also return primary planes that can be used instead.
https://bugzilla.gnome.org/show_bug.cgi?id=768183
Signed-off-by: Javier Martinez Canillas <javier@osg.samsung.com>
Without setting the DRM_CLIENT_CAP_UNIVERSAL_PLANES capability bit, only
overlay planes are made available for compatibility with legacy clients.
But if a CRTC doesn't have an overlay plane associated, then kmssink is
not able to find a plane for the CRTC and the pipeline will fail, i.e:
ERROR kmssink gstkmssink.c:482:gst_kms_sink_start:<kmssink0> Could not find a plane for crtc
This patch adds a plane-id property to the kmssink element so a specific
plane can be used in case that a CRTC has only a primary plane associated.
https://bugzilla.gnome.org/show_bug.cgi?id=768183
Rather than assuming something. e.g. zerocopy on iOS with GLES3 requires
the use of Luminance/Luminance Alpha formats and does not work with
Red/RG textures.
Some names were incorrect. Authoritative source for
the dvbv5 format taken from v4l-utils' lib/libdvbv5/dvb-v5.c
Aditionally, add the missing setter mapping for the
modulation param.
This change makes ATSC work.
https://bugzilla.gnome.org/show_bug.cgi?id=764957
The hardware decoder can become (temporarily) unavailable across
VTDecompressionSessionCreate/Destroy calls. During negotiation if the currently
configured caps are still accepted by downstream we keep using them so we don't
have to destroy and recreate the decoding session.
This indirectly fixes https://bugzilla.gnome.org/show_bug.cgi?id=767429, by
making vtdec stick to GLMemory.
strcasecmp is not defined on MSVC, so just use the glib wrapper. Also pretend to
be Windows XP explicitly since the API we use was deprecated and removed
(ifdef-ed) from the SDK after this version of Windows. This will be especially
relevant once we stop supporting Windows XP soon:
https://bugzilla.gnome.org/show_bug.cgi?id=756866
The URI must already be escaped by the caller, we don't support passing around
invalid (unescaped) URIs via the GstURIHandler interface.
Also it will escape too much of the URI in this case, e.g.
ipod-library://item/item.m4a?id=3143338395173862951
becomes
ipod-library://item/item.m4a%3Fid%3D3143338395173862951
https://bugzilla.gnome.org/show_bug.cgi?id=767492
Move calling gst_vtdec_push_frames_if_needed from ::set_format to ::negotiate so
that we always drain even when renegotiation is triggered by downstream.
vtdec specifies sysmem; GLMemory as template caps. When negotiating, we used to
call gst_pad_peer_query_caps (..., filter) with our template caps as filter. The
query does gst_caps_intersect (filter, peercaps) internally which gives
precedence to the order of the filter caps. While we want to output sysmem by
default, when negotiating with glimagesink which returns GLMemory; sysmem; we
do want to do GL, so we now query using a NULL filter and intersect the result
with our template caps giving precedence to downstream's caps.
tl;dr: make sure we end up negotiating GLMemory with glimagesink
If for some reason the avdtpsink element can't go READY then the
gsta2dpsink can't either and so should release the ressources it
allocates when trying to do so.
Fix a leak with the generic/states test.
https://bugzilla.gnome.org/show_bug.cgi?id=767161
Similar to vtdec_hw, this commit adds a vtenc_h264_hw element that fails
caps negotiation unless a hardware encoder could actually be acquired.
This is useful in situations where a fallback to a software encoder
other than the vtenc_h264 software encoder is desired (e.g. to x264enc).
https://bugzilla.gnome.org/show_bug.cgi?id=767104
When renegotiating mid stream - for example with variable bitrate
streams - and therefore destroying and recreating VTSessions, the
hw decoder might become temporarily unavailable.
To deal with this and avoid erroring out on bitrate changes,
vtdec_hw now falls back to using the software decoder if the hw
one was available at some point but isn't anymore. At
renegotiation/bitrate change time, it will still retry to open
the hardware one.
::negotiate can be called several times before the CAPS event is sent downstream
so use the currently configured output state caps instead of the pad current
caps when deciding whether to recreate the VTSession or not.
This leads to creating/destroying less VTSessions which makes renegotiation more
reliable especially when using hw decoding.
There's no need for an end-of-list marker in the filter
PIDs array if full, as the absolute maximum number of
elements (MAX_FILTERS) is known.
CID #1362441
This bug was found via cppcheck static analysis.
If android.hardware.Camera.getParameters returns NULL, then object will
be NULL, and we won't allocate params. This means that the GST_DEBUG
statement referencing params->object will be invalid. Fix this by
exiting early if android.hardware.Camera.getParameters returns NULL.
https://bugzilla.gnome.org/show_bug.cgi?id=766638
There is no way to tell one over the other when parameters
seem valid for DVB-T and DVB-T2 and the adapter supports
both. Reason to go with the former here is that, from
experience, most DVB-T2 channels out there seem to use
parameters that are not valid for DVB-T, like QAM_256
https://bugzilla.gnome.org/show_bug.cgi?id=765731
DVB-T/T2 have the same number of fields so we were
wrongly assuming DVB-T for DVB-T2 broadcasts. Not
setting the delivery system here allows for dvbsrc
to make an informed guess based on the channel
parameters.
When there's no explicit delivery system information
for a channel in the channel configuration file and
the user hasn't selected one via setting the delsys
property, we *guessed* it by selecting the last
supported delsys reported by the driver. This change
provides the basis for smarter delsys auto detection
and implements a rule for DVB-T2. Rules for other
delivery systems can be added in _guess_delsys() in
a similar way.
Additionally: Store list of adapter-supported
delivery systems instead of querying the driver each
time this information is needed.
Related to:
https://bugzilla.gnome.org/show_bug.cgi?id=765731
The device name and descriptions returned are in the locale encoding, not
UTF8. Our device name property is in UTF8 though, so we need to convert.
https://bugzilla.gnome.org/show_bug.cgi?id=756948
The only mandatory frontend information for our use case
is its status. Make sure we output what we know instead
of choking at the first error getting SNR, BER or any of
the other informational parameters.
Some cameras (IDS) have broken DirectShow drivers which incorrectly fill some
fields in the VIDEOINFOHEADER structure; comparison between suggested and
supported media types in CBaseRenderer should ignore deprecated and/or not
essential fields; additionaly explicitely setting the mediatype for the capture
pin before trying to connect it works around another IDS driver bug, and
should have been already done anyway.
https://bugzilla.gnome.org/show_bug.cgi?id=765428
Add include path so that the cmake-generated project
is able to find gstconfig.h
Add /SAFESEH:NO to MSVC linker options so it can link with
gstreamer libraries on Windows.
https://bugzilla.gnome.org//show_bug.cgi?id=765426
This patch requests for drmModePageFlip() for the used CRTC, if the kernel
module suppports async page flip. If it does not, the element requests for a
vblank event. A GstPoll waits for the event to happen.
https://bugzilla.gnome.org/show_bug.cgi?id=761059
This patch will enable the import of dmabufs into a KMS buffer using
the PRIME kernel interface.
If the driver does not support prime import, the method is skipped.
It has been tested with a Freescale I.MX6 board.
https://bugzilla.gnome.org/show_bug.cgi?id=761059
This is simple video sink that use libdrm/libkms API to render frames.
The element uses planes to render through drmModeSetPlane().
It has been tested in an Exynos4412 board and in a Freescale I.MX6 board.
https://bugzilla.gnome.org/show_bug.cgi?id=761059
Some presets are not always supported on all devices and will cause an error if
used. Specifically, the LOSSLESS presets are known to not work everywhere.
We have no idea which timestamps they are supposed to have so the only thing
we can do at this point is to drop them. Packets without timestamps happen if
audio was captured but no corresponding video, which shouldn't happen under
normal circumstances.
https://bugzilla.gnome.org/show_bug.cgi?id=747633
And creating one is causing assertions. Also get rid of the other CONSTRUCT
property as it's a) unneeded for default initialization and b) you're not
supposed to use constructor properties when creating element instances and the
GStreamer API doesn't provide direct ways for doing so.
https://bugzilla.gnome.org/show_bug.cgi?id=764339
In many cases, we use g_slice_new0 and then immediately overwrite the
allocated memory. This is inefficient. Since we're going to immediately
overwrite it, we might as well use plain g_slice_new.
https://bugzilla.gnome.org/show_bug.cgi?id=763998
Currently, we use AHC*_CALL macros to call many of the Camera functions.
However, we already have helper classes to call the Camera functions, so
eliminate the macros.
As a nice side-benefit, we also get improved error handling and
reporting when something goes wrong calling these functions, because a
GError gets populated, and we log a GST_ERROR when something fails. This
was harder to do using macros, as all error handling was hidden from the
caller.
https://bugzilla.gnome.org/show_bug.cgi?id=763065
In the androidmedia plugin_init, we initialize various resources on the
Android device. If anything fails during this series of initializations,
we need to deinitialize any initializations that already occurred.
However, we don't do so if we fail to register the ahcsrc element. Fix
this.
https://bugzilla.gnome.org/show_bug.cgi?id=763065
The error message is specific to only one of the failure cases and is
misleading in the others. Correct it to be more generic and cover all
the failure cases.
https://bugzilla.gnome.org/show_bug.cgi?id=763065
Don't wait until later, we want to know here if the codec can be opened or not
for the requested format. This was removed (accidentially?) by
119e09eac3
Without this decodebin has no way to switch to a different decoder if this one
does not work.
https://bugzilla.gnome.org/show_bug.cgi?id=762613
Leave kCVOpenGLESTextureCacheMaximumTextureAgeKey to the default (1s). We used
to set it to 0 and flush manually, but apparently (looking at the GLES profiler)
0 means "disable the cache entirely".
CPU waits are more expensive and are only required if the CPU is ever going to
access the data. GPU waits perform inter-context synchronisation and are cheaper
as they don't require CPU intervention.
When we are not waiting, we need to pass -1 to signal that we just want to check
that the frame was/n't rendered. Avoids waiting for frames that will never be
rendered.
https://bugzilla.gnome.org/show_bug.cgi?id=761014
When not rendering the video frame, e.g. when freeing an unreleased sync frame,
we will not receive a frame listener callback.
Reduces the amount of 'on_frame_available miss detected' messages when dropping
frames.
https://bugzilla.gnome.org/show_bug.cgi?id=761014
The frame available callback can be called after deconfiguring the amc codec.
Guard against this by setting the back pointer to NULL on close() and ignoring
any NULL data pointer.
https://bugzilla.gnome.org/show_bug.cgi?id=761014
Rework the GL context code. Now both avfvideosrc and vtdec can create an
internal GL context for pushing textures. Both elements will still try to
use/switch to a local context where available (including after RECONFIGURE
events).
Actually set the configured framerate. Before we only used to set the first
matching framerate range. On iOS where the camera reports ranges [2, 60], we
used to configure the camera to output anything between 2 and 60fps.
Instead of just ignoring that error and then calling JNI functions with NULL,
which will kill the virtual machine.
The error handling here needs some further improvements though, errors in more
places are just ignored.
Happens when doing zerocopy rendering, or when passing a wrong index to it.
Handle this properly for zerocopy rendering, fail properly for the other
cases.
https://bugzilla.gnome.org/show_bug.cgi?id=760961
Currently it was wrongly reporting min/max as being the shortest and
longest possible frame duration. This is not how latency works in
GStreamer.
Fix by reporting min latency as being the longest possible duration of
one frame. As we don't know how many buffers the stack can accumulate, we
simply assume that max latency is the same (the usual default behaviour).
_data_queue_item_free() calls gst_buffer_unref(), which
calls gst_ahc_src_buffer_free_func(), which calls
g_mutex_lock() on self->mutex and there you go... deadlock!
This commit is a part of portng android hardware camera from 0.10 implementation.
To preserve history and get diff clearly, the interesting files are moved to
deployment directory and the remaining files are removed.
Moved the java wrapper API into its own files and made use of the
gst-dvm macros. Also renamed the API to have the proper naming
convention and coding style in order to match the one in androidcamera.
This is a work in progress! "android/media/MediaCodecList" is still missing
and the actual elements have not been ported to use the new function names.
The on_preview callback gets called with NULL if the buffer in the queue is
too small, so we need to handle the case where the array is NULL. Also
there is a bug in the android source which makes it drop one of the buffers
so if we had 5 buffers, and we renegotiate to a higher resolution, then we'd
only get 4 calls to on_preview_frame with NULL, with one being dropped.
This means we can't reallocate the buffers in the if (data == NULL) case
because we might end up with 0 buffers in the end.
Implement a new memory type wrapping CVPixelBuffer.
There are two immediate advantages:
a) Make the GstMemory itself retain the CVPixelBuffer. Previously,
the containing GstBuffer was solely responsible for the lifetime of
the backing CVPixelBuffer.
With this change, we remove the GST_MEMORY_FLAG_NO_SHARE so that
GstMemory objects be referenced by multiple GstBuffers (doing away
with the need to copy.)
b) Delay locking CVPixelBuffer into CPU memory until it's actually
mapped -- possibly never.
The CVPixelBuffer object is shared among references, shares and
(in planar formats) planes, so a wrapper GstAppleCoreVideoPixelBuffer
structure was introduced to manage locking.
https://bugzilla.gnome.org/show_bug.cgi?id=747216
When doing GLMemory avfvideosrc negotiates UYVY. This change allows avfvideosrc
! tee name=t ! ... ! glimagesink t. ! ... ! gldownload ! vtenc_h264 ! ...
to do GLMemory and 0-copy with the encoder (with the CV meta).
Change texture format from BGRA to NV12. This allows a pipeline like avfvideosrc
! tee name=t ! ... ! glimagesink t. ! ... ! gldownload ! vtenc_h264 ! ... to
negotiate GLMemory. This makes the glimagesink branch much faster (obviously)
and triggers the 0-copy path between avfvideosrc and vtenc (using the CV meta).
Combined this results in a huge perf improvement on iOS (25-30% of CPU time in a
pipeline like the one above).
Note that this doesn't introduce a new shader conversion in the sink, since BGRA
textures had to be copied/converted from format=BGRA,texture-target=RECTANGLE to
format=RGBA,texture-target=2D anyway.
Fixate to the highest possible resolution and fps. Otherwise by default we end
up fixating at 2fps and the lowest supported resolution, which is hardly what
someone who bought an overpriced smartphone wants.
We need a static lock to protect various NVENC methods in _set_format(). Without
this the CPU use increases dramatically on initialisation of the element when
there are multiple elements being initialised at the same time.
https://bugzilla.gnome.org/show_bug.cgi?id=759742
When the mode of decklinkvideosink is set to "auto", the sink claims to
support the full set of caps that it can support for all modes. Then, every
time new caps are set, the sink will automatically find the correct mode for
these caps and set it.
Caveat: We have no way to know whether a specific mode will actually work for
your hardware. Therefore, if you try sending 4K video to a 1080 screen, it
will silently fail, we have no way to know that in advance. Manually setting
that mode at least gave the user a way to double-check what they are doing.
https://bugzilla.gnome.org/show_bug.cgi?id=759600
Otherwise qtkitvideosrc fails to build on OSX 10.10.4
because QTKit has been deprecated since OS X 10.9.
Also set -mmacosx-version-min=10.8 in front to allow
the user or cerbero to override the version.
https://bugzilla.gnome.org/show_bug.cgi?id=745564
Add gst_gl_memory_allocator_get_default to get the default allocator based on
the opengl version. Allows us to stop hardcoding the PBO allocator which isn't
supported on gles2.
Fixes GL upload on iOS9 among other things.
- Create GstGLVideoAllocationParams which is a GstGLAllocationParams subclass.
- Make it possible to allocate glmemory objects directly if no frills are
needed.
Prefer GLMemory over sysmem. Also now when pushing GLMemory we push the
original formats (UYVY in OSX, BGRA in iOS) and leave it to downstream to
convert.
It was added back in the day to make texture sharing work by default with
glimagesink inside playbin. These days glimagesink accepts (and converts) YUV
internally so it's no longer needed.
Switch to using IOSurface instead of CVOpenGLTextureCache on OSX. The latter can't be
used anymore to do YUV => RGB with opengl3 on El Capitan as GL_YCBCR_422_APPLE
has been removed from the opengl3 driver. Also switch to NV12 from UYVY, which
was the only YUV format supported by CVOpenGLTextureCache.
First of a few commits to stop using CVOpenGLTextureCache on OSX and use
IOSurfaces directly instead. CVOpenGLTextureCache hasn't been updated for OpenGL
3 which is why texture sharing is currently disabled on OSX.
rename gst-launch --> gst-launch-1.0
replace old elements with new elements(ffmpegcolorspace -> videoconvert, ffenc_** -> avenc_**)
fix caps in examples
https://bugzilla.gnome.org/show_bug.cgi?id=759432
It will fail and cause the sink to crash. Instead wait until the window is
visible again before checking if the swapchain really has to be recreated.
https://bugzilla.gnome.org/show_bug.cgi?id=741608
The video decoders tried calling gst_buffer_add_*meta() on non-writable
buffer resulting in warnings of this kind:
gstamcvideodec.c:921 (_gl_sync_render_unlocked): WARNING: amcvideodec
Failed to create the transformation meta for the gl_sync 0xabc03848
buffer 0xabb01b40 (0)
https://bugzilla.gnome.org/show_bug.cgi?id=758694
Some devices only ever keep one buffer available in the GL queue resulting in
multiple calls to release_output_buffer only causing one frame to be rendered.
If there is a queue after amcvideodec (even playsink's small one), then
multiple buffers are pushed but only a small fraction of them are actually
rendered on time. The rest will either render some number of frames ahead of
where they are meant to be or timeout waiting for a frame that's already been
rendered.
Solved by moving the release_output_buffer into the sync_meta the is pushed
downstream. When downstream renders, the custom sync implementation attempts
to release the current buffer (if not already released) and render. Once the
frame has been rendered to the screen, the next frame is released and is
hopefully available by the time the next frame is to be rendered.
This fixes a perceived frame jitter in the output.
Year 12: I still don't understand how negotiation works.
Apparently gst_pad_query_caps doesn't do what I thought it did. To get the
actual caps that can flow through vtdec:src we must call gst_pad_peer_query_caps
with the template caps as filter.
Fixes negotiation with stuff that doesn't understand GLMemory (hello videoscale).
This provides a performance and power usage improvement by removing
the texture copy from an OES texture to 2D texture.
The flow is as follows
1. Generate the output buffer with the required sync meta with the incrementing
push counter and OES GL memory
1.1 release_output_buffer (buf, render=true) and push downstream
2. Downstream waits for on the sync meta (timed wait) or drops the frame (no wait)
2.1 Timed wait for the frame number to reach the number of frame callbacks fired
2.2 Unconditionally update the image when the wait completes (success or fail).
Sets the affine transformation matrix meta on the buffer.
3. Downstream renders as usual.
At *some* point through this the on_frame_callback may or may not fire. If it
does fire, we can finish waiting early and render. Otherwise we have to
wait for a timeout to occur which may cause more buffers to be pused into the
internal GL queue which siginificantly decreases the chances of the
on_frame_callback to fire again. This is because the frame callback only occurs
when the internal GL queue changes state from empty to non-empty.
Because there is no way to reliably correlate between the number of buffers
pushed and the number of frame callbacks received, there are a number of
workarounds in place.
1. We self-increment the ready counter when it falls behind the push counter
2. Time based waits as the frame callback may not be fired for a certain frame.
3. It is assumed that the device can render at speed or performs some QoS of
the interal GL queue (which may not match the GStreamer QoS).
It holds that we call SurfaceTexture::updateTexImage for each buffer pushed
downstream however there's no guarentee that updateTexImage will result in
the exact next frame (it could skip or duplicate) so synchronization is not
guaranteed to be accurate although it seems to be close enough to be unable
to discern visually. This has not changed from before this patch. The current
requirement for synchronization is that updateTexImage is called at the point in
time when the buffers is to be rendered.
https://bugzilla.gnome.org/show_bug.cgi?id=757285
Rework negotiation implementing GstVideoDecoder::negotiate. Make it possible to
switch texture sharing on and off at runtime. Useful to (eventually) turn
texture sharing on in pipelines where glimagesink is linked only after
decoding has already started (for example OWR).
Improve decode error handling by avoiding calling into GstVideoDecoder from the
VT decode callback. This removes contention on the GST_VIDEO_DECODER_STREAM_LOCK
which used to make the decode callback slow enough for VT to start dropping lots
of frames once the first frame was dropped.
Otherwise, gst_vtenc_negotiate_profile_and_level will double-release as
it checks for profile_level != NULL. This caused crashes when the
vtenc instance is stopped and then restarted.
https://bugzilla.gnome.org/show_bug.cgi?id=757935
Use gst_gl_sized_gl_format_from_gl_format_type to get the format passed to
CVOpenGLESTextureCacheCreateTextureFromImage. Before this change extracting the
second texture from the pixel buffer was failing on ios 9.1.
No need to use G_GINT64_FORMAT for potentially negative values of
GstClockTimeDiff. Since 1.6 these can be handled with GST_STIME_ARGS.
Plus it creates more readable values in the logs.
https://bugzilla.gnome.org/show_bug.cgi?id=757480
Solved with a simple shader templating mechanism and string replacements
of the necessary sampler types/texture accesses and texture coordinate
mangling for rectangular and external-oes textures.
Add the various tokens/strings for the differnet texture types (2D, rect, oes)
Changes the GLmemory api to include the GstGLTextureTarget in all relevant
functions.
Update the relevant caps/templates for 2D only textures.
Otherwise we're going to return times starting at 0 again after shutting down
an element for a specific input/output and then using it again later.
https://bugzilla.gnome.org/show_bug.cgi?id=755426
GstVideoDecoder has its own logic for detecting when to reconfigure
which ultimately calls decide_allocation and results in a new
texture cache that has not been configured from our reconfigure check.
https://bugzilla.gnome.org/show_bug.cgi?id=755156
Fixes playback to GL memory on iOS, where the colours are messed
up by passing Luminance/LuminanceAlpha textures where
color convert expects R/RG textures.
https://bugzilla.gnome.org/show_bug.cgi?id=754504
We were converting all times to our internal running times, that is the time
the sink itself spent in PLAYING already. But forgot to do that for the
running time calculated from the buffer timestamps. As such, all buffers were
scheduled much later if the pipeline's running time did not start at 0.
This happens for example if a base time is explicitly set on the pipeline.
https://bugzilla.gnome.org/show_bug.cgi?id=754528
Casting to UINT from HMIXER generates the following warning with
64bit Windows target MinGW:
gstdirectsoundsrc.c: In function 'gst_directsound_src_mixer_find':
gstdirectsoundsrc.c:733:30: error: cast from pointer to integer of different size [-Werror=pointer-to-int-cast]
mmres = mixerGetDevCaps ((UINT) dsoundsrc->mixer,
^
cc1: all warnings being treated as errors
We can use portable GPOINTER_TO_UINT() macro for this propose.
https://bugzilla.gnome.org/show_bug.cgi?id=754756
Instead of checking for the gstreamer-video-1.0 package is installed,
just assume it is since we already check for the -base dependency.
With this replace the GST_VIDEO_* variables in makefiles and directly
link with libgstvideo.
https://bugzilla.gnome.org/show_bug.cgi?id=753820
Also implement framerate handling correctly by borrowing the code from
ximagesrc. GstBaseSrc::get_times() can't be used for that, we have to
implement proper waiting ourselves.
The block that is dispatched async to the main thread assumed the
wrapping GstAvSampleVideoSink to be alive. However, at the time of
the block execution the GstObject instance that is deferenced to access
the CA layer might already be freed, which caused occasional crashes.
Instead, we now only pass the CoreAnimation layer that needs to be
released to the block. We use __block to make sure the block is not
increasing the refcount of the CA layer again on its own.
https://bugzilla.gnome.org/show_bug.cgi?id=753081
