The matrices were converting the wrong values with non-diagonal-only matrices.
e.g. a typical yflip matrix in [-1,1]^3 such as
1 0 0 0
0 -1 0 0
0 0 1 0
0 0 0 1
Would have actually required a matrix like this in [0,1]^3
1 0 0 0
0 -1 0 0
0 0 1 0
0 -2 0 1
Which is
1. not consistent with our multiplication convention and would require
transposing matrices or changing our multiplication order (from what is
generally used on opengl matrix guides/tutorials).
2. Produces incorrect values when input with actual vertices accounting for
the difference in multiplication order. e.g. some vertices multiplied by
the yflip matrix using vertex * yflip(== transpose(yflip) * vertex):
vertex: -> result: expected:
vec4(1,0,1,1) -> vec4(1,-2,1,1) vec4(1,1,1,1)
vec4(1,1,1,1) -> vec4(1,-3,1,1) vec4(1,0,1,1)
With the updated values, we now get the expected values.
Includes a test for this behaviour and the example above
A granule is a 64bit signed integer, shifting by 63 or more is
undefined and most likely an indication that the stream is
corrupted or invalid.
Detected by oss-fuzz
glcolorbalance is in the default GL path inside glimagesink,
so has been causing an possibly-unnecessary extra texture copy
on Android for a while now. If we're just doing passthrough,
we can support OES directly. If not, they'll be transformed
to 2D textures and colourbalanced.
This moves all the conversion related code to a single place, allows
less code-duplication inside compositor and makes the glmixer code less
awkward. It's also the same pattern as used by GstAudioAggregator.
The aggregated_frame is now called prepared_frame and passed to the
prepare_frame and cleanup_frame virtual methods directly. For the
currently queued buffer there is a method on the video aggregator pad
now.
It performs similarly to the existing alpha element however performs
calculations in floating point rather than with small (guint8) integers
so some differences are to be expected.
https://bugzilla.gnome.org/show_bug.cgi?id=794070
Previously we assumed that the texture ID is going to be valid even
after unmapping the frame, as it was immediately unmapped before even
being used. Now we only unmap once we're done with the texture.
When doing a 3D/multiview transformation and rescaling to
match the output window size, the resulting PAR may
not match the input any more and needs recalculating,
or else the GstSample reported to client-draw has the
wrong PAR.
If new headers arrive after we are initialized, we need to make
sure that they are indeed valid.
A vorbis bitstream always begins with three header packets and must
be in order.
Also some streams have unframed (invalid?) headers that might
confuse and disrupt the decoding process.
Therefore if ever we see new headers, we accumulate them and once
we get a non-header packet we check them to make sure that:
* We have at least 3 headers
* They are the expected ones (identification, comments and setup)
* They are in order
* Any other "header" is ignored
If those conditions are met, we reset and reconfigure the decoder
https://bugzilla.gnome.org/show_bug.cgi?id=784530
Even if the input is monoscopic, the app might want to display
it in a different layout, to do side-by-side for VR for example,
so if the app changes the output-multiview-mode always use that.
We can either receive an element that is floating or not and need to
accomodate that in the signal return values. Do so by removing the
floating flag.
https://bugzilla.gnome.org/show_bug.cgi?id=792597
We call the base class first as this will remove the pad from
the aggregator, thus stopping misc callbacks from being called,
one of which (process_textures) will recreate the vertex_buffer
if it is destroyed
https://bugzilla.gnome.org/show_bug.cgi?id=760873
Alsasrc introduced delay_lock in commit 519f85a43e73efb8f3fb2c7be45226e
because alsa-lib is not thread safe for the same handle.
Alsasrc uses the same threading pattern, it should be locked too.
https://bugzilla.gnome.org/show_bug.cgi?id=746015
Tests and documentation will follow separately.
The mixer elements in the opengl plugin need to stay
in -bad for now since they use GstVideoAggregator.
https://bugzilla.gnome.org/show_bug.cgi?id=754094
When compiling with clang, an enum conversion error is triggered
since GstVideoFrameFlags are not GstVideoFlags.
This patch sets GST_VIDEO_FRAME_FLAG_NONE to the added video meta.
https://bugzilla.gnome.org/show_bug.cgi?id=791251
This patch adds code to gldownload to export the image as a
dmabuf if requested. The element now exposes memory:DMABuf as
a cap feature, and if it is selected, the element exports the
texture to an EGL image and then a dmabuf. It also implements a
fallback to system memory download in case the exportation failed.
https://bugzilla.gnome.org/show_bug.cgi?id=776927
when using internal window, window resize should work
when pause state, but expose only do redisplay when
window_id is valid. So expose should do redisplay all
the time.
https://bugzilla.gnome.org/show_bug.cgi?id=787394
Except for gst/gl/gstglfuncs.h
It is up to the client app to include these headers.
It is coherent with the fact that gstreamer-gl.pc does not
require any egl.pc/gles.pc. I.e. it is the responsability
of the app to search these headers within its build setup.
For example gstreamer-vaapi includes explicitly EGL/egl.h
and search for it in its configure.ac.
For example with this patch, if an app includes the headers
gst/gl/egl/gstglcontext_egl.h
gst/gl/egl/gstgldisplay_egl.h
gst/gl/egl/gstglmemoryegl.h
it will *no longer* automatically include EGL/egl.h and GLES2/gl2.h.
Which is good because the app might want to use the gstgl api only
without the need to bother about gl headers.
Also added a test: cd tests/check && make libs/gstglheaders.check
https://bugzilla.gnome.org/show_bug.cgi?id=784779
Add a function to install the default RGBA pad templates,
but don't make them required so that there can be
GstGLFilter sub-classes with different input/output
caps if they want. Remove the hard-coded RGBA restriction in
the set_caps_features call, as it will be taken care
of by intersecting with the pad templates.
Update all the sub-classes to match
All code interacting with Objective-C objects should now use Automated
Reference Counting rather than manual memory management or Garbage
Collection. Because ARC prohibits C-structs from containing
references to Objective-C objects, all such fields are now typed
'gpointer'. Setting and gettings Objective-C fields on such a
struct now uses explicit __bridge_* calls to tell ARC about
object lifetimes.
https://bugzilla.gnome.org/show_bug.cgi?id=777847
The same symbol also exists in libgstgl, although marked as private and
internal. This has no effect when doing static linking and there's a
symbol conflict.
gst_caps_intersect () may return an increased reference of one of the
input caps.
Fixes critical in the simple-launch-lines test:
Unexpected critical/warning: gst_caps_set_features: assertion 'IS_WRITABLE (caps)' failed
middle textures in gleffects do not need to use GstGLMemoryPBO as they
aren't transfering data to/from the GPU. This will cost too much DMA
memory and cause performance issue. Change the allocator to use non-PBO
GstGLMemory.
https://bugzilla.gnome.org/show_bug.cgi?id=776072
glvideomixer does not support it currently and it needs special support
for handling this correctly, and is rather non-trivial to implement for
all formats.
MSVC warns about this because it's a C++ compiler, and this actually
results in useful things such as the incorrect 'gboolean' return value
for functions that return GstFlowReturn, so let's do explicit
conversions to reduce the noise and increase its efficacy.
With MSVC, this gives the following warning:
warning C4305: 'function': truncation from 'double' to 'gfloat'
Apparently, MSVC does not figure out what type to use for constants
based on the assignment. This warning is very spammy, so let's try to
fix it.
The headers we include already define boolean on Windows with MSVC, and
it leads to a typedef redefinition error with jpeglib.h which tries to
redefine it in jmorecfg.h
When only linking the element, the upload object will be created from
_transform_caps() but will never be unreffed as the only case is in _stop().
Add an unref if non-NULL to a new finalize handler for this case.
Fixes an assertion when moving from passthrough to non-passthrough
Without an explicit reconfigure, glfiter won't have created the GL
resources such as the FBO, GL bufferpool, etc and basetransform will
allocate sysmem buffers instead.
Previously it was created in the init function and destroyed in ::stop, which
lead to segfaults when reusing the element.
Now the upload object is created in ::transform_caps if it is NULL, which is the
earliest we need it. The other vfuncs already bail out if the upload object is
NULL, which means that negotiation wasn't done.
Now when used with video/x-raw as input, the GLMemoryUpload method checks for
->tex_target in input GLMemory(es) and sets the output texture-target
accordingly.
Fixes video corruption with a pipeline like avfvideosrc ! video/x-raw !
glimagesink where on macos avfvideosrc pushes RECTANGLE textures but glupload
was configuring texture-target=2D as output.
The videoaggregator negotiation sequence changed some time
back and broke glstereomix. Instead of doing nego incorrectly
in the find_best_format() vfunc, do it directly in the
update_caps() method.
It implements now this interface with its video-direction
property. Values are changed to GstVideoOrientationMethod but they have
the same value than the originals.
https://bugzilla.gnome.org/show_bug.cgi?id=768687
It now returns the correct values for both orthographic and perspective
projections and takes into account the aspect ratio of the video, handles
the Y-flipping in GL and by us and uses some more helpers from graphene.
There's no need for the jump to an extra thread in most cases, especially
when relying solely on a shader to render. We can use the provided
render_to_target() functions to simplify filter writing.
Facilities are given to create fbo's and attach GL memory (renderbuffers
or textures). It also keeps track of the renderable size for use with
effective use with glViewport().
Rather than assuming something. e.g. zerocopy on iOS with GLES3 requires
the use of Luminance/Luminance Alpha formats and does not work with
Red/RG textures.
These two shader will calculate the vector length and use it as denominator.
But length could be zero which will cause undefine behaviour. Add protection for
this condition
https://bugzilla.gnome.org/show_bug.cgi?id=767635
_get_gl_context() can be called concurrently from either propose_allocation() or
decide_allocation(). If it so happens that this happens at the same time,
the check for whether we already had a GL context was outside the lock. Inside
the lock and loop, the first thing that happens is that we unref the current GL
context (if valid) as if there was a conflict adding it to the display. If the
timing was unlucky, subsequent use of the GL context would be referencing an
already unreffed GL context object resulting in a critical:
g_object_ref: assertion 'object->ref_count > 0' failed
https://bugzilla.gnome.org/show_bug.cgi?id=766703
The gltestsrc element uses two shaders: color_shader and snow_shader.
Those are alternatively assigned to the SrcShader->shader pointer and
their reference was transferred to it. Only the SrcShader->shader was
unreffed (in _src_shader_deinit()) so only one shader was properly
freed, the other one was leaked.
Fixed this by giving an extra ref to SrcShader->shader and unreffing the
2 shaders in _src_smpte_free().
https://bugzilla.gnome.org/show_bug.cgi?id=766661
We can avoid a render pass if downstream supports the affine transformation meta
and increase the performance of some pipelines involving gltransformation.
Implemented by checking for the affine transformation in the allocation query
from downstream and combining our matrix with that of upstream's (or creating
our own).
Provide a function to get the affine matrix in the meta in terms of NDC
coordinates and use as a standard opengl matrix.
Also advertise support for the affine transformation meta in the allocation
query.
We were always failing the allocation query as a flag was never being set to
signal a successful negotiation. Fix by setting the required flag on a
successful caps event from upstream.
1.Porting the exist deinterlace shader and OpenGL callback
to be compatible with OpenGL ES.
2.Add a our blur vertical shader to gldeinterlace.
3.Add a property named “method” to let user choose which
deinterlace function to use. Default to choose blur vertical
method for better performance.
[Matthew Waters]: fix name of greedyh in method property (was greedhy) and port
to git master.
https://bugzilla.gnome.org/show_bug.cgi?id=764873
When application change pipeline state NULL->READY and then READY->NULL,
glimagesink will not clear glsink->window_id. After that, when application
change state NULL->READY, the new_window_id is equal to window_id, glimagesink
will not set window handle. It will use the internal window but not the window
create by application.
https://bugzilla.gnome.org/show_bug.cgi?id=765241
We weren't using the result of find_best_format at all.
Also, move the find_best_format usage to the default update_caps() to make
sure that it is also overridable.
https://bugzilla.gnome.org/show_bug.cgi?id=764363
