From a multimedia perspective GLES >= 2 has the big advantage of
supporting external textures (`OES_EGL_image_external` /
`OES_EGL_image_external_essl3`), allowing various YUV formats to be
imported directly by drivers.
It appears unlikely by now that the extension will ever be ported to
GL with Vulkan becoming more popular, leaving GL without an "official"
way to import YUV formats.
Further more, for Gst internal purposes it's likely that GLES2 works
equally well if not better on most drivers these days, especially on
embedded devices.
Thus switch the default for EGL context creation to GLES2. This won't
affect apps that create their own context, but `gst-launch-1.0` etc.,
which are often used for testing so people don't have to pass
`GST_GL_API=gles2`.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5509>
In rare cases - notably on macOS, because of multiple GL contexts - the lack of a sync point was causing overlays
to disappear for a frame after being redrawn, or sometimes not appear at all. This change makes sure that the display
in one GL context will be correctly synchronised with the other GL context where the overlay texture was uploaded.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5610>
In _gl_memory_upload_propose_allocation(), when output target is "external-oes",
then we should not provide GL allocator and pool in the allocation query.
This is because the "external-oes" kind memory can never be mapped directly
and the upstream element may misuse it.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5468>
The propose and decide allocation vfuncs are called directly from
basetransform and need to use the locked accessor function for
retrieving a reliable reference to the GstGLContext (if available)
Fixes spurious crashes on shutdown during pad reconfiguration
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5437>
The number of planes is a meta we carry around in the GstVideoMeta with
DMA_DRM format. In cannot be decuded correctly from knowledge of the
base format. Notably, some compression modifier may introduce an extra
plane to store the compression parameters.
So use n_planes from GstVideoMeta and pass this explicitly when
importing to EGLImage.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5461>
The DMAbuf accept function was ensuring the in_dma_info values was valid if
the in_caps have change. But the check was bogus since the in_caps was being
modified without a pointer change. As a side effect, on the second accept
call, the drm_fourcc was reset to 0, which cause the uploader to fallback.
Fix this by ensuring we always have a valid dma_frm info directly in the
set_caps() function. Also remove the bogus caps changed check and remove any
modification to the info structure and always do that inner checks.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5461>
DRM Modifiers are not generically transferrable from a format like NV12 to
their indirect shading format (R8 / RG88). So the helper to this do needs
to be removed from our API.
To make things worse, we support indirect formats that aren't DRM format in
the first place. Notably NV12_16L32 (aka MM21) is not (yet) a DRM format. Yet,
each plane can be indirectly imported using R8/RG88 and a detiling shader.
This patch also removes this constraint restoring zero-copy playback on
Mediatek SoC.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5461>
This was causing a memory leak in cases like `gltestsrc ! gltransformation scale-x=0.5 ! glimagesink`.
Parent meta was being added in assumption that those buffers are different, which was not the case here,
creating a reference loop and never freeing the buffer.
Co-authored-by: Matthew Waters <matthew@centricular.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5445>
The GST_VIDEO_FORMAT_Y410, GST_VIDEO_FORMAT_Y412_LE and GST_VIDEO_FORMAT_Y412_BE
formats in fact are packed formats, which have just 1 plane. But we have special
setting for them rather than using get_single_planar_format_gl_swizzle_order().
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5314>
As we don't have any mapping from YUV formats + modifiers to an equivalent
emulated format (e.g. NV12 + modifier -> R8+modifier/RG88+modifier), do no
allow these formats to be used with the indirect DMABuf uploader.
Fixes#2942
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5270>
Fixes a potential GPU stall if an immediately freed texture/buffer is
attempted to be reused immediately by the CPU, e.g. when uploading.
Problematic scenario is this:
1. element does GPU processing reading from texture
2. frees the buffer back to the pool
3. pool acquire returns the just released buffer
4. GPU processing then has to wait for the previous GPU operation to
complete causing a stall
If there was a reliable way to know whether a buffer had been finished
with across all GPU drivers, we would use it. However as that does not
exist, this workaround is to keep the released buffer unusable until the
next released buffer.
This is the same approach as is used in the qml (Qt5) elements.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5144>
When this flag is enabled, the transform_caps() simply set passthrough
to generate the raw caps. This is not correct, because the sink and
src have different format/drm-format fields.
We already add system memory conversion for DMABuf manner, so no more
need for this flag.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3524>
The current _nvmm_upload_transform_caps() only simply apply
"memory:NVMM" to all input caps to transform the output caps.
This is not precise and may cause problem. For example, if the
input caps include:
video/x-raw(memory:DMABuf), width=(int)1920, height=(int)1080, \
interlace-mode=(string)progressive, multiview-mode=(string)mono, \
framerate=(fraction)30/1, drm-format=(string)NV12:0x0100000000000002
it will be changed as
video/x-raw(memory:GLMemory), width=(int)1920, height=(int)1080, \
interlace-mode=(string)progressive, multiview-mode=(string)mono, \
framerate=(fraction)30/1, drm-format=(string)NV12:0x0100000000000002
For GLMemory kind caps, no drm-format should appear.
So we should let it only transforms which it can recognize.
And in _nvmm_upload_accept(), we should only accept the "memory:NVMM"
feature in input caps.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3524>
The current _directviv_upload_transform_caps() only simply apply
"memory:GLMemory" to all input caps to transform the output caps.
This is not precise and may cause problem. For example, if the
input caps include:
video/x-raw(memory:DMABuf), width=(int)1920, height=(int)1080, \
interlace-mode=(string)progressive, multiview-mode=(string)mono, \
framerate=(fraction)30/1, drm-format=(string)NV12:0x0100000000000002
it will be changed as
video/x-raw(memory:GLMemory), width=(int)1920, height=(int)1080, \
interlace-mode=(string)progressive, multiview-mode=(string)mono, \
framerate=(fraction)30/1, drm-format=(string)NV12:0x0100000000000002
For GLMemory kind caps, no drm-format should appear.
So we should let it only transforms which it can recognize.
And in _directviv_upload_accept(), we should only accept the system
memory as input caps.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3524>
The current _gl_memory_upload_transform_caps() only simply apply
"memory:GLMemory" to all input caps to transform the output caps.
This is not precise and may cause problem. For example, if the
input caps include:
video/x-raw(memory:DMABuf), width=(int)1920, height=(int)1080, \
interlace-mode=(string)progressive, multiview-mode=(string)mono, \
framerate=(fraction)30/1, drm-format=(string)NV12:0x0100000000000002
it will be changed as
video/x-raw(memory:GLMemory), width=(int)1920, height=(int)1080, \
interlace-mode=(string)progressive, multiview-mode=(string)mono, \
framerate=(fraction)30/1, drm-format=(string)NV12:0x0100000000000002
For GLMemory kind caps, no drm-format should appear.
So we should let it only transforms which it can recognize.
And in _gl_memory_upload_propose_allocation(), we should only allocate
the allocator and buffer pool for the caps with "memory:GLMemory"
feature.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3524>
The current _upload_meta_upload_transform_caps() only simply apply
"memory:GLMemory" to all input caps to transform the output caps.
This is not precise and may cause problem. For example, if the
input caps include:
video/x-raw(memory:DMABuf), width=(int)1920, height=(int)1080, \
interlace-mode=(string)progressive, multiview-mode=(string)mono, \
framerate=(fraction)30/1, drm-format=(string)NV12:0x0100000000000002
it will be changed as
video/x-raw(memory:GLMemory), width=(int)1920, height=(int)1080, \
interlace-mode=(string)progressive, multiview-mode=(string)mono, \
framerate=(fraction)30/1, drm-format=(string)NV12:0x0100000000000002
For GLMemory kind caps, no drm-format should appear.
So we should let it only transforms which it can recognize.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3524>
The current _raw_data_upload_transform_caps() only simply apply
"memory:GLMemory" to all input caps to transform the output caps.
This is not precise and may cause problem. For example, if the
input caps include:
video/x-raw(memory:DMABuf), width=(int)1920, height=(int)1080, \
interlace-mode=(string)progressive, multiview-mode=(string)mono, \
framerate=(fraction)30/1, drm-format=(string)NV12:0x0100000000000002
it will be changed as
video/x-raw(memory:GLMemory), width=(int)1920, height=(int)1080, \
interlace-mode=(string)progressive, multiview-mode=(string)mono, \
framerate=(fraction)30/1, drm-format=(string)NV12:0x0100000000000002
For GLMemory kind caps, no drm-format should appear.
So we should let it only transforms which it can recognize.
We also should recognize the system memory caps in _accept() early, if
the input is not system memory, we just return early.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3524>
Most of the time, the RGB kind formats are OpenGL native supported
format which has only one plane. They can be imported at one shot
using no matter DIRECT or INDIRECT mode.
While YUV kind formats which have multi planes have two ways to import.
They can be DIRECT imported, which requires GL_OES_EGL_image_external
extension. The output format should be RGBA and TARGET should be set
as OES after imported. The other way, they can be INDIRECT imported,
which makes each plane as a texture. In this mode, the imported textures
have different fourcc from the original format. For example, the NV12
format can be imported as a R8 texture for the first plane and RG88
texture for the second plane. The output TARGET should be sets as 2D
in this mode.
When converting sink caps to src caps, we first filter the feature of
"video/x-raw(memory:DMABuf)" and system memory. Then Based on the
external_only flag (INDIRECT mode does not care while DIRECT mode cares),
we transform the drm-format into the gst video format.
When converting src caps into sink caps, we first filter the correct
TARGET(INDIRECT mode contains 2D only while DIRECT mode contains 2D,
OES or both of them) gstructure. Then Based on the include_external flag
(INDIRECT mode always true while DIRECT mode depends on TARGET), we
transform the gst video format into drm-format.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3524>
Shader compilation was failing on macOS:
gstglslstage.c:519:_compile_shader:<glslstage1> fragment shader compilation failed:
ERROR: 0:10: 'input_swizzle' : syntax error: Array size must appear after variable name
Co-authored-by: Matthew Waters <matthew@centricular.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/5123>
glfilter will unref input buffer after _transform() call immidiately,
but gpu may still reading input buffer for rendering because gl
api is executed async. Need hold reference for input buffer by
adding parent meta to output buffer.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4801>
This patch adds gst_egl_image_from_dmabuf_direct_target_with_dma_drm() and
add gst_egl_image_from_dmabuf_with_dma_drm() functions
New function gst_egl_image_from_dmabuf_direct_target_with_dma_drm(), where
gst_egl_image_from_dmabuf_direct_target() is a specialization of the first.
And gst_egl_image_from_dmabuf() is a specialization of new function
gst_egl_image_from_dmabuf_with_dma_drm()
Co-authored-by: Victor Jaquez <vjaquez@igalia.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4680>
It internally uses gst_gl_context_egl_get_dma_formats() instead of fetching
modifiers by itself.
Thus gst_egl_image_check_dmabuf_direct() is a decorator of this new function.
Co-authored-by: He Junyan <junyan.he@intel.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4680>
By calling the internal function gst_gl_context_egl_fetch_dma_formats() the an
array of structures holding a DMA fourcc format and its modifiers (another array of
structure holing modifier and if it's external only) will be stored.
Users would call gst_gl_context_egl_get_format_modifiers() to get the array of
modifiers of a specific DMA fourcc format.
Co-authored-by: He Junyan <junyan.he@intel.com>
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4680>
Subclasses may want to override the pad template with different formats
or with a different pad subclass.
The original beahviour is still available by calling
gst_gl_mixer_class_add_rgba_pad_templates() in _class_init() of the
subclass.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4608>
Upon creating a window, glimagesink and osxvideosink now set the policy to
NSApplicationActivationPolicyRegular, which lets us show an icon in the Dock
for convenience and appear in the top menu bar like other apps.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4573>
This is no longer needed since the introduction of `gst_macos_main()` in 1.22.
Before that existed, we had a patch for GLib in Cerbero, which did work but made it
impossible to update GLib at all. The code being removed was a fail-safe in case of
running without said patch being applied. It's no longer needed, since for macOS
we just wrap our GStreamer with an NSApplication using `gst_macos_main()`.
Warnings will be displayed if no NSApp/NSRunLoop is found wherever needed,
pointing the user towards using the new API.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4366>
The proxy and queue are created in the gst_gl_window_wayland_egl_open()
function and will be recreated on open. This leaks both objects, the
wayland client documentation mentions that they should be destroyed
using the appropriate destroy functions.
Found during valgrind memory leak testing, these blocks were marked as
definitely lost.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/4354>
Do not store cached EGL images in GstMemory QData. Instead, use a
per-DmabufUpload GHashTable to store cache entries with a weak
reference to the GstMemory.
This allows two glupload elements on separate tee branches to have
their own EGL image cache. For this pipeline:
gst-launch-1.0 v4l2src ! tee name=t \
t. ! queue ! glupload ! fakesink
t. ! queue ! glupload ! fakesink
this gets rid of the occasional critical error message:
GStreamer-CRITICAL **: 08:26:33.194: gst_mini_object_unref: assertion 'GST_MINI_OBJECT_REFCOUNT_VALUE (mini_object) > 0' failed
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3880>
We create a new context in `gst_gl_context_create_thread()` and then
activate it on the current thread. Thereafter we assume that the
current thread continues to be the active thread for that context and
call `gst_gl_context_fill_info()` which asserts that the current
thread is the active thread.
However, if at the same time a different thread calls
`send_message_async()`, it will call into
`gst_gl_window_cocoa_send_message_async()` which will schedule the
message to be invoked using GCD. That anonymous function will also
call `gst_gl_context_activate()`, which creates a race, which can lead
to:
```
gst_gl_context_fill_info: assertion 'context->priv->active_thread == g_thread_self ()' failed
```
Fix it by using `gst_gl_context_thread_add()` to invoke `fill_info()`
on the context.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3732>
In current tile representation, only tiles with power of two
width and height in bytes are supported. This limitation
prevents adding more complex tiles formats.
In this patch, we deprecate tile_ws and tile_hs from GstVideoFormatInfo and
replace if with an array of GstVideoTileInfo. Each plane tiles are then
described with their pixels width/height, line stride and total size.
The helper gst_video_format_info_get_tile_sizes() that depends on the
deprecated API is also being removed. This can simply be removed as it wasn't
in any stable release yet.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3424>
When a tile format is padded and imported as DMABuf, the stride
contains the information about the actual width and height in
number of tiles. This information is needed by the detiling shader
in order accuratly calculate the location of pixels. To fix that,
we also copy the offset and strides into the otuput format and
the converter will ensure that the shader is recompiled whenever
the stride changes.
This fixes video corruptions observed when decoding on MT8195
with videos that aren't not aligned to 64bytes in width.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3365>
duplicate symbol '__invoke_on_main' in:
/Library/Frameworks/GStreamer.framework/Versions/1.0/lib/libgstvulkan-1.0.a(cocoa_gstvkwindow_cocoa.m.o)
/Library/Frameworks/GStreamer.framework/Versions/1.0/lib/libgstgl-1.0.a(cocoa_gstglwindow_cocoa.m.o)
ld: 1 duplicate symbol for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Also make the same change in iOS for consistency.
Continuation of https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/issues/1132
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3242>
