Don't create many short lived locks/conds in gst_gl_window_send_message. This is
a micro optimization to save a bunch of pthread_* calls which are expensive on
OSX/iOS and possibly other platforms.
while calling eglCreateImage without a GL context current in the executing
thread works on the RPi, some other implementations will return errors.
Marshall the eglCreateImage to the GL thread to appease these implementations.
There are numerous slight differences required between Desktop GL and GLES3 for
multiple render targets.
1. gl_FragData doesn't exist at all and one is required to use
'layout (location = ?) out ...' instead.
2. gl_FragColor doesn't exist, same as 1
3. texture2D() has been deprecated
Fortunately most of these have been taken care of with GL3 and the shader
mangling already exists so just expand the conditions they are used in. The
gl_FragData issue requires a new mangle pass though. We also use this new
pass on desktop GL for consistency.
By default, reading GL_RED or GL_RG us unsupported by glReadPixels unless
exposed through GL_COLOR_READ_IMPLEMENTATION_FORMAT/TYPE. This allows
downloading multiple-planar video frames where possible.
There are some cases where it's needed for binding in/out variables in shaders.
e.g. glsl 150 (gl 3.2) doesn't support the 'layout (location = ?)' specifiers in
the shader source so we have to bind them ourselves.
Without the GST_GL_API_GLES2 bit set, we will not even attempt to look
for the function pointers in the core library and will fallback to
glFlush/glFinish.
If the user uploads their own texture without setting the unpack length, then
then the result will have the appearance of stride mismanagement due to
an incorrect row length.
If we are given caps with extra features (like the overlay composition
features), we can only deal with that when we are in passthrough mode.
Previously we were bailing entirely and not allowing passthrough filter elements
with things like textoverlay.
Fixes the following pipeline (assuming glfilter supports passthrough):
gl ! textoverlay ! glfilter ! ... ! glimagesinkelement
https://bugzilla.gnome.org/show_bug.cgi?id=763756
When transforming, xplode it out into the necessary caps features both
with and without the passthough features.
Fixes negotiation in the following class of pipelines:
gl ! textoverlay ! glupload ! glimagesinkelement
https://bugzilla.gnome.org/show_bug.cgi?id=763756
GL 1.4 (with GL_ARB_shader_objects) doesn't have glIsProgram or glIsShader
equivalents. As they are simply assertions, skip them when there isn't a
valid function pointer.
With e38af23044 returning the correct contexts,
gst_gl_display_add_context() was susceptible to causing infinte loops when
adding the same GstGLContext more than once. Fix and add a test for
gst_gl_display_add_context().
Fixes glvideomixer gst-validate tests.
Execute GL calls without marshalling them to the context thread. In the cocoa
and eagl backends calling gst_gl_context_activate is cheap and therefore calling
it on the current thread and serializing GL calls with a per-context lock is
more efficient (faster and has less overhead) than marshalling everything to the
context thread.
This optimization cuts a large overhead in g_poll (continuously waking up the
context thread) and in g_mutex_*/g_cond_* (waiting for results from the context
thread).
When requesting a glcontext (regardless of thread), the result was correct.
However, when requesting current glcontext on a specific thread, it could
come up with a glcontext active on another thread.
https://bugzilla.gnome.org/show_bug.cgi?id=763168
Maxsize is initialized once and should never change. Allocating data
should have no impact on the selected max size for this memory. This
causing memory map failure as the maxsize would become smaller then
size. This happened when using direct rendering in avviddec on GL that
does not support PBO transfer.
https://bugzilla.gnome.org/show_bug.cgi?id=763045
Intended for use with wrapped contexts that are created shared with gst's
gl contexts in order to manage the internal sharegroup state correctly.
e.g. with caopengllayer (which is used in glimagesink and caopengllayersink
on OS X), we create a CGL context from the gst context and the sharing state
was not being correctly set on either GL context and gst_gl_context_is_shared()
was always returning FALSE.
With 11fb4fff80 only flushing with multiple
shared contexts, the required flush was not occuring causing screen
corruption or stuttering.
Note: this didn't affect GST_GL_API=opengl pipelines
https://bugzilla.gnome.org/show_bug.cgi?id=762620
Usually gl debug is initialized in gst_gl_context_create_thread.
But this function is not used when using the GstGLContextGPUProcess
from ChromiumGStreamerBackend.
Received signal 11 SEGV_MAPERR 000000000000
gst_debug_category_get_threshold
gst_gl_insert_debug_marker
gst_gl_base_filter_gl_start
CC libgstgl_x11_la-gstglcontext_glx.lo
In file included from gstglcontext_glx.c:39:0:
../utils/opengl_versions.h:52:43: error: ‘gles2_versions’ defined but not used [-Werror=unused-const-variable]
static const struct { int major, minor; } gles2_versions[] = {
^~~~~~~~~~~~~~
Invoke the callback right away when called on the context thread. Removes
overhead when nesting libgstgl calls (for example when working with the sync
meta).
CPU waits are more expensive and are only required if the CPU is ever going to
access the data. GPU waits perform inter-context synchronisation and are cheaper
as they don't require CPU intervention.
All uses of query->context in gstglquery.c assume it exists. We can assume
this as well before unrefing it. Furthermore, gst_object_unref() will just
silently return if it ever were to not exist.
Improves the responsiveness of the pipeline when resources are close/above the
limitations of the hardware.
Any subclass that wishes not to enable qos can do so themselves.
https://bugzilla.gnome.org/show_bug.cgi?id=761519
This reverts commit 96b9666d59.
This reverts commit d11385d167.
This breaks the texture sharing with the applemedia elements as
CVOpenGLESTextureCache seems to have an arbitrary restriction on GLES2 only.
We may need them to transform into a different set of formats.
Fixes YUV->YUV with two glcolorconverts, e.g:
format=I420 ! glcolorconvert ! glcolorconvert ! format=NV12
1.0 / width does not offset by one pixel in rectangular textures (which use
unnormalized coordinates).
Provide the actual pixel offset as a uniform to the shader.
1. Correctly describe what we can caps we can transform to/from.
i.e. no YUV->YUV or GRAY->YUV or YUV->GRAY (except for passthrough).
2. Prefer similar formats and ignore incompatible formats on fixation.
Useful when gst_gl_window.c::gst_gl_window_new is not used.
This is the case when using a custom GstGLWindow.
(ex: GstGLWindowGPUProcess from Chromium)
The messages are stored by gst_gl_async_debug_store_log_msg() and output later
by a corresponding store(), output() or an unset()/free().
Some wrapper macros are provided to avoid callers explicitly using __FILE__,
GST_FUNCTION and __LINE__
This makes a pipeline like:
... ! video/x-raw(memory:GLMemory),format=UYVY ! glcolorconvert !
video/x-raw(memory:GLMemory),format={UYVY, NV12} ! ...
passthrough instead of converting UYVY => NV12. The conversion would happen
before this change since the element (and basetransform) transform the src caps
to format={NV12, UYVY} (since NV12 comes first in the glcolorconvert:src
template) and then the default caps fixate func would fixate to NV12. Blah.
Also there's no need to intersect against the template caps in ::transform_caps
since basetransform does that right after calling the vfunc.
The optimistic download_transfer was not setting the required flag to not
perform glReadPixels on subsequent map (READ). resulting in glReadPixels
happening twice.
Some operations are unnecessary when running with only a single GL
context.
e.g. glFlush when setting a fence object as the flush happens on wait.
API: gst_gl_context_is_shared
1. Various elements/base classes only perform a subset check on accept-caps
2. Some GL elements have texture-target in their pad template
3. When checking subsets, only the caps to check are allowed to contain extra
fields. If the 'template' caps have extra fields, the subset fails.
Thus without texture-target on the caps, various accept-caps implementations
were failing.
Also, add some convenience functions for setting and retrieving
texture targets to/from GValue.
https://bugzilla.gnome.org/show_bug.cgi?id=759860
GL 2.1 only supports pbo upload.
The wrapped data pointer was only being set on the pbo memory and on the
glmemory so when a download was requested (in GL 2.1), glmemory was
allocating a new data pointer and thus not returning the wrapped data.
Exposing the navigation thread's main context, GSourceFuncs and structs called
key_event and mouse_event is exposing a bit too much of the internals. Let's
just go with two functions to asynchronously send navigation events on the
window with the same API as the synchronous ones.
This upload method detect and optimize uploads of DMABuf memory. This is
done by creating and caching EGLImages wrapper around DMABuf. The
EGLImages are then binded to a texture which get converter using
standard shader.
Example pipeline:
GST_GL_PLATFORM=egl \
gst-launch-1.0 v4l2src device=/dev/video1 io-mode=4 ! \
video/x-raw,format=NV12 ! glimagesink
https://bugzilla.gnome.org/show_bug.cgi?id=743345
Maps GstVideoFormats to suitable DRM fourccs which work with
glcolorconvert, using gst_gl_memory_alloc(). We require mostly
only 4 formats to be supported by the driver. We require DRM
equivalent to RGB16, RGBA, R8 and RG88. This way it's compatible with
DesktopGL, since GL_TEXTURE_2D is used and limit driver requirements.
With this we can virtually support all formats the glcolorconvert
supports.
https://bugzilla.gnome.org/show_bug.cgi?id=743345
Add gst_gl_memory_allocator_get_default to get the default allocator based on
the opengl version. Allows us to stop hardcoding the PBO allocator which isn't
supported on gles2.
Fixes GL upload on iOS9 among other things.
e.g when wrapping a data pointer we don't want to map/unmap off the end of
pointer with the alignment bytes.
Instead track that information separately as maxsize is used for mapping by
GstMemory and thus represents a size without any alignment padding bytes.
Requires the usage of GstGLVideoAllocationParams however any user can set their
own parameters along with an allocator which will be used to allocate the
correct memory type.
- Create GstGLVideoAllocationParams which is a GstGLAllocationParams subclass.
- Make it possible to allocate glmemory objects directly if no frills are
needed.
This is made possible by a subclassable GstGLAllocationParams that holds
the allocation parameters
Every allocation would now go through gst_gl_base_memory_alloc with the
allocation parameters now being specified in a single struct to allow
extension by different allocators.
The imported memory has already been allocated, passing allocation
parameters with alignment confuses the memory which endup with a
size different from maxsize and lead to overrun when the memory
is being copied.
GCC automatically disable redundance warnings for system headers. As
soon as we start using a non-system installed mesa, we would start
having issues. The test for both wasn't setting any flags, so it would
work but then fail at runtime.
This is being fixed by disabling in the code (where needed only) that
GCC warning. The test is also fixed to avoid the false positive we had.
The base class is useful for having multiple backing memory types other
than the default. e.g. IOSurface, EGLImage, dmabuf?
The PBO transfer logic is now inside GstGLMemoryPBO which uses GstGLBuffer
to manage the PBO memory.
This also moves the format utility functions into their own file.
Heavily based on GstGLBaseBuffer that is a subclass of GstGLBaseMemory.
Provides GPU and CPU accessible GL buffer objects by GL handle or by
sysmem data pointer.
It handles the following
- GstAllocationParams -> gst_memory_init transformation
- Makes sure that map/unmap/create/destroy happen on the GL thread with
a GL context current.
- Holds a possible sysmem accessible data pointer with alignment.
- Holds the need upload/download transfer state
Rectangle textures don't use normalized coordinates so subsampling needs to be
factored in explicitly.
Fixes YUV => RGB conversion for rectangle textures.
To use GLMemory and EGLImage allocators, one need to know the
libgstgl API. This is only expected if the associated caps features
have been negotiated. Generic element that otherwise receive those
allocators may fail, resulting in broken pieline. We don't want to
force all generic element to check if the allocator is a custom
allocator or a normal allocator (which implement the _alloc method).
https://bugzilla.gnome.org/show_bug.cgi?id=758877
gstglsyncmeta.c -fPIC -DPIC -o .libs/libgstgl_1.0_la-gstglsyncmeta.o
gstglsyncmeta.c: In function 'gst_buffer_add_gl_sync_meta':
gstglsyncmeta.c:131:1: error: control reaches end of non-void function [-Werror=return-type]
}
^
there could be other ways/requirements for synchronising two GPU command
streams (whether GL or platform specific).
e.g. glfencesync/eglwaitnative/cond/etc
given a NULL-terminated string, s.
s[i] = '\0';
i++;
does not guarentee that s[i] is NULL terminated and thus string operations
could read off the end of the array.
https://bugzilla.gnome.org/show_bug.cgi?id=758039
gst_gl_shader_detach_unlocked already removes the list entry so attempting to
use the element to iterate to the next stage could read invalid data.
Based on patch by Vineeth TM <vineeth.tm@samsung.com>
https://bugzilla.gnome.org/show_bug.cgi?id=758039
Some drivers don't provide the compatibility definition and we need to provide
our own 'out vec4' variable to put the results of the fragment shader into.
https://bugzilla.gnome.org/show_bug.cgi?id=757938
Rectangular textures are unavailable in unextended
GLES2 #version 100 shaders.
Fixes
texture-target=rectangle ! glcolorconvert ! texture-target=2D
There's a couple of differences between GL3 and GLES2/GL
- varying -> in or out depending on the stage (vertex/fragment)
- attribute -> in
- filtered texture access is a single function, texture()
Make gst_gl_context_gen_shader/_compile_shader assume GST_GLSL_PROFILE_ES |
GST_GLSL_PROFILE_COMPATIBILITY as the profile. Without this, the shader compiler
doesn't inject the #version tag resulting in a compilation error on Mountain
Lion.
This is a workaround for old code using gst_gl_context_gen_shader. New code
should use the gst_glsl_stage_* API directly which allows the caller to
explicitly specify version/profile.
The ret variable may be uninitialized and so its contents were undefined and
the results were erratic (failing with glvideomixer, succeeding in other cases)
P.S. No idea why gcc/clang et al never picked up on this like they normally do
(probably due to some optimisation pass figuring out it's only set once...)
They are already defined in the forward decleration header and defining them
more than once will give an error with OSX's clang about typedef redefinition
being a C11 feature.
This was chosen over relying solely on the caps as glupload needs to propose an
allocation and set the texture target based on the output caps. Setting the
caps in the config is currently pointless as they are overwritten in a lot of
element's decide_allocation functions.
This provides a mechanism for the buffer pool to be configured for a certain
texture target when none has been configured.
Solved with a simple shader templating mechanism and string replacements
of the necessary sampler types/texture accesses and texture coordinate
mangling for rectangular and external-oes textures.
Add the various tokens/strings for the differnet texture types (2D, rect, oes)
Changes the GLmemory api to include the GstGLTextureTarget in all relevant
functions.
Update the relevant caps/templates for 2D only textures.
the USING_GLES2 includes all GLES3 contexts as well which does support
drawing to multiple buffers. Instead make or decision solely based on
whether glDrawBuffers is available or not.
e.g:
gstglcontext_egl.c:613:7: error: implicit declaration of function 'strcmp'
[-Werror=implicit-function-declaration]
if (strcmp (G_MODULE_SUFFIX, "so") == 0)
While technically, i is always 0 and *vertex_sources[i++] is equivalant
to (*vertex_sources)[i++]. Be future-proof in the case of code
moves/changes/etc.
CID 1327406
A GstGLShader is now simply a collection of stages that are
compiled and linked together into a program. The uniform/attribute
interface has remained the same.
1. So we get tracking inside GstElement properly when e.g. adding to a bin
2. Removes redundant code. Now only one place where
GstContext->GstGLDisplay/GstGLContext transformation occurs
3. Fixes a memory leak in the process
4. Make the retrieval of debug categories thread safe
These markers are visible in tools that record the GL function calls
such as apitrace, et al.
Makes it easier to match up GL draw commands with specific elements.
- glimagesink needs to be able to resize the viewport on aspect ratio
changes resulting from either caps changes or 3d output mode changes.
- Performing a glViewport outside the GstGLWindow::resize callback
will not have the winsys' stack of viewports required to correctly
place the output frame.
Provide a function to request a resize on the next draw event from the
winsys.
Also track size changes inside the base GstGLWindow class rather
than in each subclass.
https://bugzilla.gnome.org/show_bug.cgi?id=755111
when allocating memory. Fixes crashes with avdec_h265 in the AVX2
code path which requires 32-byte stride alignment, but the
GstAllocationParams only specified a 16-byte alignment.
https://bugzilla.gnome.org/show_bug.cgi?id=754120
HAVE_IOS is only defined for the build of this module so
attempting to use gstgl in iOS would result in incorrect GL
includes.
Use GST_GL_HAVE_PLATFORM_EAGL instead for choosing the iOS GL
header.
During allocation query, when this element is not passthrough, it must
relay the overlay compostion meta and it's parameters. Fortunatly, base
transform can do this for us.
https://bugzilla.gnome.org/show_bug.cgi?id=753850
GL_SHADING_LANGUAGE_VERSION was introduced since ES 2.0, but in some
android emulator doesn't support this feature. To prevent confusion for
developer, the error message need to be more clear.
https://bugzilla.gnome.org/show_bug.cgi?id=753905
Removes the redundant GL object creation/deletion on every
decide_allocation call which is being called for every caps change.
Thus reduces the required GL state changes on reconfigure events
which are being sent by glimagesink/xvimagesink
As we only expose the mapped portion of the frame into the GL
memory object (and not the original padding) we need to
re-calculate the size and offset.
Adding this private header to the list of sources. We don't want to make
this header public, but we need it in the list of sources otherwise it
won't be included in the tarball. This fixes make distcheck.
This regression was introduced by commit 1a6fe3db
Depending on the bytes order we will get BGRA (little) and ARGB (big)
from the composition overlay buffer while our GL code expects RGBA. Add
a fragment shader that do this conversion.
https://bugzilla.gnome.org/show_bug.cgi?id=752842
We can't know if the GstGLUpload type is initialized at this point already,
and thus our debug category might not be initialized yet... and cause an
assertion here.
As we don't print debug output for any of the other transform functions, let's
defer this problem for now.
The coordinate are relative to the texture dimension and not
the window dimension now. There is no need to pass the window
dimension or to update the overlay if the dimension changes.
https://bugzilla.gnome.org/show_bug.cgi?id=745107
They require to get_proc_address some functions through the
platform specific {glX,egl}GetProcAddress rather than the default
GL library symbol lookup.
nvidia drivers return the exact version in glGstString (GL_VERSION)
we request on creation so start with the highest known version and
work our way down.
The previous approach of traversing the other_context weak ref tree was
1. Less performant
2. Incorrect for context destruction removing a link in the tree
Example of 2:
c1 = context_create (NULL)
c2 = context_create (c1)
c3 = context_create (c2)
context_can_share (c1, c3) == TRUE
context_destroy (c2)
unref (c2)
context_can_share (c1, c3) returns FALSE when it should be TRUE!
This does not remove the restriction that context sharedness can only
be tracked between GstGLContext's.
This will deadlock if the main thread is the one who creates the GstGLContext.
All things we call from the main thread should be possible from any thread.
https://bugzilla.gnome.org/show_bug.cgi?id=751101
Add API for a helper object that can convert between different
stereoscopic video representations, and later do filtering
of multiple view streams.
https://bugzilla.gnome.org/show_bug.cgi?id=611157
Provides generic handling of GL buffer objects accessible using
the GL bind points (GL_ARRAY_BUFFER, GL_PIXEL_*_BUFFER).
Implementation based off the current GstGLMemory.
5697b6b89b causes us to possibly listen
on a toolkit provided Display connection. We thus could eat their
precious winsys events. Only listen if we need to
(!foreign_display or videooverlay).
Add a vfunc that is called by glfilter before it sets
caps features and intersects with the peer caps, and
move removing the size from caps into its default
implementation. Allows sub-classes to do more
sophisticated management of the size fields in case they
don't support arbitrary resizing or have distinct
preferences.
Previously when compiling GstGL with both GL and GLES2,
GL_RGBA8 was picked from GL/gl.h. But a clash may happen at
runtime when one is selecting GLES2.
gst_gl_internal_format_rgba allows to check at runtime
if it should use GL_RGBA or GL_RGBA8.
Simple implementation split from GstGLWindowWayland
Can now have multiple glimagesink elements all displaying output
linked via GL or otherwise (barring GL platform limitations).
The intel driver is racy and can crash setting up the two glimagesink contexts.
e.g.
videotestsrc ! tee name=t ! queue ! glupload ! glimagesinkelement
t. ! queue ! gleffects_blur ! glimagesinkelement
videotestsrc ! glupload ! glfiltercube ! tee name=t ! queue ! glimagesinkelement
t. ! queue ! gleffects_blur ! glimagesinkelement
Otherwise we could end up being mistaken for the diference between a
gl3 and a gl2 context resulting in a failure getting the list of
extensions from the wrapped context due to the difference between
glGetString and glGetStringi for the GL_EXTENSIONS token.
https://bugzilla.gnome.org/show_bug.cgi?id=749728
When called from gst_gl_window_win32_close(), internal window
could not exist, and if it does it's going to be destroyed just
after that anyway. Also it causes window_proc() to be called
and crash because it gets a NULL context.
When called from gst_gl_window_win32_set_window_handle() we are
going to set another parent anyway, and it's probably better to
reparent directly instead of passing by a NULL parent which could
cause the internal window to popup briefly.
https://bugzilla.gnome.org/show_bug.cgi?id=749601
gst_gl_context_finalize() is calling gst_gl_window_win32_quit()
which was posting a message. But then window_proc takes window's
context and get a NULL.
Now that we've got a GMainLoop we can do like other backends and
simply call g_main_loop_quit().
This also remove duplicated code to release the parent window and
potential crash there because parent_proc could be NULL if we never
created the internal window. That could happen for example if setting
state to READY then setting a window_handle, and go back to NULL state.
https://bugzilla.gnome.org/show_bug.cgi?id=749601
gst_gl_window_win32_send_message_async() could be called before the
internal window is created so we cannot use PostMessage there.
x11 and wayland backends both create a custom GSource for this,
so there is no reason to not do that for win32.
https://bugzilla.gnome.org/show_bug.cgi?id=749601
Otherwise it could stay client side without being submitted to the GL
server resulting in another context waiting on a Fence that will never
become signalled causing a deadlock.
Make the passthrough check contingent on only the fields we
can modify being unchanged, and pre-compute it when caps
change instead of checking on each buffer. Makes the passthrough
more lenient if consumers are lax about making input and output
caps complete.
Show the DispmanX window only if there's no shared external GL context
set up. When a window is required by the context a transparent
DispmanX element is created and later on made visible by the ::show
method.
https://bugzilla.gnome.org/show_bug.cgi?id=746632
In some upload implementations the out buffer has more than one references,
turning the buffer not writable, so it won't be possible to modify its
meta-data.
This patch moves the meta-data copy before increasing the reference of the out
buffer.
https://bugzilla.gnome.org/show_bug.cgi?id=746173
Chaining a downstream pool would lead to two owner of the same
pool. In dynamic pipeline, if one owner is removed from the pipeline
the pool will be stopped, and the rest of the pipeline will fail
since the pool will now be flushing. Also fix proposed pool caching,
filter->pool was never set, never unrefed.
https://bugzilla.gnome.org/show_bug.cgi?id=745705
Otherwise the pipeline stalls when running
more than one glimagesink with gst-launch.
Also only register the custom nsapp loop
when setting up the nsapp from gstgl.
We also need to recalculate the offset, since otherwise the frame
mapping will be forward two lines in the U and V planes (I420) due
to gst_video_info_align() round up the Y plane to a even number of
lines.
https://bugzilla.gnome.org/show_bug.cgi?id=745054
Make sure we support offset and video alignment when downloading too.
This is currently not used (plane_start is always 0), but it makes
the code correct if we want to use that later.
Provide the right size to GL when uploading. Using maxsize is wrong
since we offset the data point with the memory offset and video
alignement offset.
https://bugzilla.gnome.org/show_bug.cgi?id=744246
When the memory is partial copy, the texture size and videoinfo no
longer make sense. As we cannot guess what the application wants, we
safely copy into a sysmem memory.
https://bugzilla.gnome.org/show_bug.cgi?id=744246
This implements support for GstAllocationParams and memory alignments.
The parameters where simply ignored which could lead to crash on
certain platform when used with libav and no luck.
https://bugzilla.gnome.org/show_bug.cgi?id=744246
When trying to render buffers with meta:GLTextureUpload the glimagesink crashes
with a segmentation fault.
This patch workarounds this crash setting to NULL the method implementation
after free.
https://bugzilla.gnome.org/show_bug.cgi?id=745206
When setting a new window handle, we need to ensure all implementations
will detect the change.
For that we deactivate the context before setting the window handle, then
reactivate the context
https://bugzilla.gnome.org/show_bug.cgi?id=745090
When (re)activating the context, the backing window handle might have changed.
If that happened, destroy the previous surface and create a new one
https://bugzilla.gnome.org/show_bug.cgi?id=745090
This fix a very slow rendering rate regression that only
happens when using gst-launch, i.e. in the case where
the main thread does not run any NSApp loop.
Git bisect reported it has been introduced by the commit
e10d2417e2:
"move to CGL and CAOpenGLLayer for rendering".
Then the commit 7d46357627:
"gstglwindow_cocoa: fix slow render rate" attempted to fix
the slow rendering rate problem when using gst-launch.
At least for me it does not work. I tried several
combinations, for example to flush CA transactions in the
custom app loop, as mentioned in the doc, but the only solution
that fixes the slow rendering is by reducing the loop latency.
From what I tested, no need to put less than 60ms, even if the
framerate has an interval much lower (16.6ms for 60 fps).
Anytime else, we have no idea how to match up map and unmaps.
We also don't know exactly how the calling code is using us.
Also fixes the case where we're trying to transfer while someone else
is accessing our data pointer or texture resulting in mismatched video
frames.
https://bugzilla.gnome.org/show_bug.cgi?id=744839
Where possible, use the _OBJECT variants in order to track better from
which object the debug statement is coming from
Define (and use) GST_CAT_DEFAULT where applicable
Use GST_PTR_FORMAT where applicable
In gst_gl_filter_fixate_caps () it can goto done without freeing the memory of
the tmp GstStructure. This makes it go out of scope and leak.
CID #1265765
Allows finer grain decisions about formats and features at each
stage of the pipeline.
Also provide propose_allocation for glupload besed on the supported
methods.
In gst_gl_window_cocoa_draw we used to just call setNeedsDisplay:YES. That was
creating an implicit CA transaction which was getting committed at the next
runloop iteration. Since we don't know how often the main runloop is running,
and when we run it implicitly (from gst_gl_window_cocoa_nsapp_iteration) we only
do so every 200ms, use an explicit CA transaction instead and commit it
immediately. CA transactions nest and debounce automatically so this will never
result in extra work.
Make GstGLMemory hold the texture target (tex_target) the texture it represents
(tex_id) is bound to. Modify gst_gl_memory_wrapped_texture and
gst_gl_download_perform_with_data to take the texture target as an argument.
This change is needed to support wrapping textures created outside libgstgl,
which might be bound to a target other than GL_TEXTURE_2D. For example on OSX
textures coming from VideoToolbox have target GL_TEXTURE_RECTANGLE.
With this change we still keep (and sometimes imply) GL_TEXTURE_2D as the
target of textures created with libgstgl.
API: modify GstGLMemory
API: modify gst_gl_memory_wrapped_texture
API: gst_gl_download_perform_with_data
Don't call glClear && glClearColor at each draw since we're going to draw the
whole viewport anyway. Gets rid of a glFlush triggered by glClear on OSX.
In order to use pbo's efficiently, the transfer operation has to
be separated from the use of the downloaded data which requires some
rearchitecturing around glcolorconvert/gldownload and elements
Depending on the platform, it was only ever implemented to 1) set a
default surface size, 2) resize based on the video frame or 3) nothing.
Instead, provide a set_preferred_size () that elements/applications
can use to request a certain size which may be ignored for
videooverlay/other cases.
Removes the use of NSOpenGL* variety and functions. Any Cocoa
specific functions that took/returned a NSOpenGL* object now
take/return the CGL equivalents.
The hack causes deadlocks and other interesting problems and it really
can only be fixed properly inside GLib. We will include a patch for
GLib in our builds for now that handles this, and hopefully at some
point GLib will also merge a proper solution.
A proper solution would first require to refactor the polling in
GMainContext to only provide a single fd, e.g. via epoll/kqueue
or a thread like the one added by our patch. Then this single
fd could be retrieved from the GMainContext and directly integrated
into a NSRunLoop.
https://bugzilla.gnome.org/show_bug.cgi?id=741450https://bugzilla.gnome.org/show_bug.cgi?id=704374
Soon after setting two variables to 1, the code checks if their values are
different from each other. This would never be true. Removing this.
CID 1226443
Actually we should always recalculate buffer size since our buffer size
even when not-padded is smaller for many sub-sampled formats. This is
because we don't add padding between the planes.
https://bugzilla.gnome.org/show_bug.cgi?id=740900
Problem was that if buffer was mapped READWRITE (state of buffers from
libav right now), mapping it READ/GL will not upload. This is because the
flag is only set when the buffer is unmapped. We can fix this by setting
the flags in map. This result in already mapped buffer that get mapped
to be read in GL will be uploaded. The problem is that if the write
mapper makes modification afterward, the modification will never get
uploaded.
https://bugzilla.gnome.org/show_bug.cgi?id=740900
A context can create a GLsync object that can be waited on in order
to ensure that GL resources created in one context are able to be
used in another shared context without any chance of reading invalid
data.
This meta would be placed on buffers that are known to cross from
one context to another. The receiving element would then wait
on the sync object to ensure that the data to be used is complete.
Floating point numbers are written differently in different
locales, e.g. in many countries 1/2 = 0,5 instead of 0.5, and
strtod will not be able to parse "0.5" correctly in such a
locale.
Otherwise interesting things will happen in Cocoa applications, like
infinite event loops that block the NSApplication loop forever.
This was only needed for GNUStep and thus can safely be removed now.
Until gcc and GNUStep properly support Objective-C blocks and other
"new" features of Objective-C we can't properly support them without
making the code much more ugly.
https://bugzilla.gnome.org/show_bug.cgi?id=739152
It's architecture dependent and should not be placed into the include
directory as the assumption is that all those headers are architecture
independent.
https://bugzilla.gnome.org/show_bug.cgi?id=739767
Otherwise when resizing the window you will also get messages like:
class NSConcreteMapTable autoreleased with no pool in place - just leaking
class NSConcreteValue autoreleased with no pool in place - just leaking
class NSConcreteValue autoreleased with no pool in place - just leaking
class __NSCFDictionary autoreleased with no pool in place - just leaking
Need to set the ':' as the reshape method now takes one parameter.
For the story, the GstGLNSView was previously inheriting from
NSOpenGLView which has a reshape function without any parameter.
Now the GstGLNSView inherits from NSView and we re-use the reshape
function manually.
Use the reshape function after being defined. The other way
would have been to declare the reshape function in the header.
gstglwindow_cocoa.m: In function '-[GstGLNSView drawRect:]':
gstglwindow_cocoa.m:555: warning: 'GstGLNSView' may not respond to '-reshape'
gstglwindow_cocoa.m:555: warning: (Messages without a matching method signature
gstglwindow_cocoa.m:555: warning: will be assumed to return 'id' and accept
gstglwindow_cocoa.m:555: warning: '...' as arguments.)
GTK-Doc uses a special syntax for code documentation. A multiline comment that
starts with an additional '*' marks a documentation block that will be processed
by the GTK-Doc tools. So GTK-Doc is confused if a comment block starts with that
additional '*' but isn't meant to be processed. Removing this additional '*'.
https://bugzilla.gnome.org/show_bug.cgi?id=739444
gstglwindow_cocoa.m: In function '-[GstGLNSView drawRect:]':
gstglwindow_cocoa.m:555: warning: 'GstGLNSView' may not respond to '-reshape'
gstglwindow_cocoa.m:555: warning: (Messages without a matching method signature
gstglwindow_cocoa.m:555: warning: will be assumed to return 'id' and accept
gstglwindow_cocoa.m:555: warning: '...' as arguments.)
Using NSApp directly seems to confuse something, as the compiler
was expecting an id<NSFileManagerDelegate>. Switched to using
[NSApplication sharedApplication], and specified the delegate
protocol on the window class as well.
https://bugzilla.gnome.org/show_bug.cgi?id=738740
- The shader was outputing the wrong values compared with raw
videotestsrc.
- deal with the texture edge properly.
- properly sample the 2x1 rectangle for the u and v values
- don't double sample the y value
The visible rect and bounds might be the same as before, but Cocoa
might've changed our viewport without us nothing. This happens if
you hide the view and show it again.
This is only for non-Cocoa apps but previously caused a 2 second
waiting during startup for Cocoa apps. This is unacceptable.
Instead we now check a bit more extensive if something actually
runs on the GLib default main context, and if not don't even
bother waiting for something to happen from there.
Otherwise we could pass on a RGBA formatted buffer and downstream would
misinterpret that as some other video format.
Fixes pipelines of the form
gleffects ! tee ! xvimagesink
Allows callers to properly reference count the buffers used for
rendering.
Fixes a redraw race in glimagesink where the previous buffer
(the one used for redraw operations) is freed as soon as the next
buffer is uploaded.
1. glimagesink uploads in _prepare() to texture n
1.1 glupload holds buffer n
2. glimagesink _render()s texture n
3. glimagesink uploads texture n+1
3.1 glupload free previous buffer which deletes texture n
3.2 glupload holds buffer n+1
4. glwindow resize/expose
5. glimagesink redraws with texture n
The race is that the buffer n (the one used for redrawing) is freed as soon as
the buffer n+1 arrives. There could be any amount of time and number of
redraws between this event and when buffer n+1 is actually rendered and thus
replaces buffer n as the redraw source.
https://bugzilla.gnome.org/show_bug.cgi?id=736740
This thread dispatches navigation events. It is needed to avoid deadlocks
between window backend threads that emit navigation events (e.g. X11/GMainLoop
thread) and consumers of navigation events such as glimagesink, see
https://bugzilla.gnome.org/show_bug.cgi?id=733661
GstGlWindow_x11 thread is changed to invoke the navigation thread for navigation
dispatching, instead of emiting the event itself. Othe backends beside X11 do
not dispatch navigation events yet, but should use this thread when dispatching
these events in the future.
The navigation thread is currently part of GstGLWindow and not implemented in
separate subclasses / backends. This will be needed in the future.
gst_gl_window_x11_get_surface_dimensions is also changed to use a cached value
of the window's width, height. These values are now retrieved in the X11
thread, function gst_gl_window_x11_handle_event. This change is needed because
otherwise the XGetWindowAttributes gets called from the navigation thread,
leading to xlib aborting due to multithreaded access (if XInitThreads is not
called before, as is the case for gst-launch)
EGL_CONTEXT_FLAGS_KHR and EGL_CONTEXT_OPENGL_DEBUG_BIT_KHR
don't exist in the Android NDK. Wrap their usage in an #ifdef
EGL_KHR_create_context to fix the build there.
The text for EGL_KHR_create_context added the possiblity for ES
contexts to ask for a debug context however that has not been
fully realized by all implementations. Fallback to a non-debug
context when the implementation errors.
If window is resized, GstStructure pointer values have to be rescaled to
original geometry. A get_surface_dimensions GLWindow class method is added for
this purpose and used in the navigation send_event function.
https://bugzilla.gnome.org/show_bug.cgi?id=703486
Certain elements expect that there be a certain number of lines
that they can write into. e.g. for odd heights, I420, YV12, NV12,
NV21 (and others) Y lines are expected to have exactly twice the
number of U/UV lines.
https://bugzilla.gnome.org/show_bug.cgi?id=733717
