The issue can be reproduced on a computer with a Radeon graphics card
when trying to force GStreamer Editing Services to use GL for video
mixing in GESSmartMixer, instead of the GstCompositor that smart mixer
would normally use. This change causes the resulting video stream to
have "video/x-raw(memory:GLMemory) ... texture-target: 2D" caps (instead
of "video/x-raw ..." caps). At the PlaySink stage of the pipeline, a
GstGLImageSinkBin is plugged, with a GstGLColorBalance on it. For some
reason that is still to be debugged (and out of the scope of this
patch), gst_gl_filter_set_caps() is never called on that color balance
element, leaving filter->in_texture_target set to its default
GST_GL_TEXTURE_TARGET_NONE value. The incomplete _create_shader() logic
does the rest and silently generates a shader code that doesn't build.
This is the command I use to reproduce the issue (I'm not sure if I
would be able to isolate the issue in a simple pipeline, though):
GST_PLUGIN_FEATURE_RANK=vaapih265enc:NONE,vaapih264enc:NONE,vaapisink:NONE,vaapidecodebin:NONE,vaapipostproc:NONE,vaapih265dec:NONE,vaapivc1dec:NONE,vaapih264dec:NONE,vaapimpeg2dec:NONE,vaapijpegdec:NONE,glvideomixer:260
ges-launch-1.0 +clip /tmp/video.mp4
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1159>
The current manner in transform_caps() for src direction is not very correct. For example,
when the src caps is:
video/x-raw(memory:DMABuf); video/x-raw; video/x-raw(memory:GLMemory)
this function returns:
video/x-raw(memory:DMABuf); video/x-raw; video/x-raw(memory:GLMemory)
as the sink caps. This is not correct, because DMABuf feature is not even in the sink pad's
caps template. The correct answer should be:
video/x-raw(memory:GLMemory); video/x-raw
only.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1058>
These parameters are incorrectly regarded as mutable in G-IR making them
"incompatible" with languages that are explicit about mutability like
Rust. In order to clean up the code and expected API there, update the
signatures here, right at the source (instead of overriding them in
Gir.toml and hoping for the best).
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/1005>
The sink_query just uses context, other_context and display to query info.
But all these objects can be changed or distroyed in state_change() func
and other places.
This patch is not very perfect. The condition race still exists in other
places in this element. All the functions directly access these objects
without protection. Most of them are executed when the data is pushing and
draw context/window have already been established, so they should not have
problems. But the sink_query and propose_allocation functions are the query
-like functions and executed in query context, which can be called in any
state of the element. So it can cause some crash issues because of destroyed
context object.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/922>
The operations for the inside GstGLUploadElement->upload have race
condition. The _transform_caps() will creates this object if it does
not exist, while the _stop() and change_state() can destroy this object.
The _transform_caps() is called by the gst_base_transform_query(),
so it does not hold the stream lock. It may use the upload while the
_stop() and change_state() has already destroy that object, and then
crash.
Fix: #645
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/913>
If there are two elements and threads attempting to query each other for
an OpenGL context. The locking may result in a deadlock.
We need to unlock each element's context_lock when querying another
element for the OpenGL context in order to allow any other element to
take the lock when the other element is querying for an OpenGL context.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-base/-/merge_requests/642>
Downstream may provide extra things when we ask it for caps that we may
need to remove.
Fixes the following pipeline warning:
gltestsrc ! glvideomixerelement ! glimagesinkelement
** (gst-launch-1.0:908581): WARNING **: 13:53:28.518: glvideomixer0: update_src_caps returned caps which are not a real subset of the filter caps
* A copy-paste error was getting the information from the wrong
query
* The 'allocation_meta' GstStructure was being leaked
* No check was done on whether the query existed (to try to set the
resulting allocation meta on)
CID: 1439872
CID: 1439873
CID: 1439874
CID: 1439875
CID: 1439876
CID: 1439877
When DMABuf was tried, we would renegotiate back and fourth between
DMABuf and system memory if the export failed. This would happen for
every single frame.
This patch introduces try_dmabuf_exports boolean, which is unset when
an export failed. This boolean is then put back to TRUE when upstream
pushes new caps, or downstream pushes a reconfigure event.
This introduces an enum in order to cleanup how we select the
transfer mode. It also fixes the case where we callback to PBO but
we didn't execute the PBO. That was not causing any issue, just that
the processing latency would be delayed to the next element, which
can be confusing.
This is possibly not strictly needed when pixels are being downloaded to
CPU memory, but would cause issue when exporting DMABuf, as the data may
not be yet ready when the DMABuf reaches the consumer.
Fixes macos werror build
../ext/gl/caopengllayersink.m:336:23: error: '__bridge_retained' casts have no effect when not using ARC [-Werror,-Warc-bridge-casts-disallowed-in-nonarc]
ca_sink->layer = (__bridge_retained gpointer)layer;
^~~~~~~~~~~~~~~~~~
By passing NULL to `g_signal_new` instead of a marshaller, GLib will
actually internally optimize the signal (if the marshaller is available
in GLib itself) by also setting the valist marshaller. This makes the
signal emission a bit more performant than the regular marshalling,
which still needs to box into `GValue` and call libffi in case of a
generic marshaller.
Note that for custom marshallers, one would use
`g_signal_set_va_marshaller()` with the valist marshaller instead.
The gltestsrc element was refactored to inherit from this base class which
handles the GL context. The sub-class only needs to implement the gl_start,
gl_stop and fill_gl_memory vfuncs, along with properly advertizing the GL APIs
it supports through the supported_gl_api GstGLBaseSrc class attribute.
