When doing a 3D/multiview transformation and rescaling to
match the output window size, the resulting PAR may
not match the input any more and needs recalculating,
or else the GstSample reported to client-draw has the
wrong PAR.
If new headers arrive after we are initialized, we need to make
sure that they are indeed valid.
A vorbis bitstream always begins with three header packets and must
be in order.
Also some streams have unframed (invalid?) headers that might
confuse and disrupt the decoding process.
Therefore if ever we see new headers, we accumulate them and once
we get a non-header packet we check them to make sure that:
* We have at least 3 headers
* They are the expected ones (identification, comments and setup)
* They are in order
* Any other "header" is ignored
If those conditions are met, we reset and reconfigure the decoder
https://bugzilla.gnome.org/show_bug.cgi?id=784530
Even if the input is monoscopic, the app might want to display
it in a different layout, to do side-by-side for VR for example,
so if the app changes the output-multiview-mode always use that.
We can either receive an element that is floating or not and need to
accomodate that in the signal return values. Do so by removing the
floating flag.
https://bugzilla.gnome.org/show_bug.cgi?id=792597
We call the base class first as this will remove the pad from
the aggregator, thus stopping misc callbacks from being called,
one of which (process_textures) will recreate the vertex_buffer
if it is destroyed
https://bugzilla.gnome.org/show_bug.cgi?id=760873
Alsasrc introduced delay_lock in commit 519f85a43e73efb8f3fb2c7be45226e
because alsa-lib is not thread safe for the same handle.
Alsasrc uses the same threading pattern, it should be locked too.
https://bugzilla.gnome.org/show_bug.cgi?id=746015
Tests and documentation will follow separately.
The mixer elements in the opengl plugin need to stay
in -bad for now since they use GstVideoAggregator.
https://bugzilla.gnome.org/show_bug.cgi?id=754094
When compiling with clang, an enum conversion error is triggered
since GstVideoFrameFlags are not GstVideoFlags.
This patch sets GST_VIDEO_FRAME_FLAG_NONE to the added video meta.
https://bugzilla.gnome.org/show_bug.cgi?id=791251
This patch adds code to gldownload to export the image as a
dmabuf if requested. The element now exposes memory:DMABuf as
a cap feature, and if it is selected, the element exports the
texture to an EGL image and then a dmabuf. It also implements a
fallback to system memory download in case the exportation failed.
https://bugzilla.gnome.org/show_bug.cgi?id=776927
when using internal window, window resize should work
when pause state, but expose only do redisplay when
window_id is valid. So expose should do redisplay all
the time.
https://bugzilla.gnome.org/show_bug.cgi?id=787394
Except for gst/gl/gstglfuncs.h
It is up to the client app to include these headers.
It is coherent with the fact that gstreamer-gl.pc does not
require any egl.pc/gles.pc. I.e. it is the responsability
of the app to search these headers within its build setup.
For example gstreamer-vaapi includes explicitly EGL/egl.h
and search for it in its configure.ac.
For example with this patch, if an app includes the headers
gst/gl/egl/gstglcontext_egl.h
gst/gl/egl/gstgldisplay_egl.h
gst/gl/egl/gstglmemoryegl.h
it will *no longer* automatically include EGL/egl.h and GLES2/gl2.h.
Which is good because the app might want to use the gstgl api only
without the need to bother about gl headers.
Also added a test: cd tests/check && make libs/gstglheaders.check
https://bugzilla.gnome.org/show_bug.cgi?id=784779
Add a function to install the default RGBA pad templates,
but don't make them required so that there can be
GstGLFilter sub-classes with different input/output
caps if they want. Remove the hard-coded RGBA restriction in
the set_caps_features call, as it will be taken care
of by intersecting with the pad templates.
Update all the sub-classes to match
All code interacting with Objective-C objects should now use Automated
Reference Counting rather than manual memory management or Garbage
Collection. Because ARC prohibits C-structs from containing
references to Objective-C objects, all such fields are now typed
'gpointer'. Setting and gettings Objective-C fields on such a
struct now uses explicit __bridge_* calls to tell ARC about
object lifetimes.
https://bugzilla.gnome.org/show_bug.cgi?id=777847
The same symbol also exists in libgstgl, although marked as private and
internal. This has no effect when doing static linking and there's a
symbol conflict.
gst_caps_intersect () may return an increased reference of one of the
input caps.
Fixes critical in the simple-launch-lines test:
Unexpected critical/warning: gst_caps_set_features: assertion 'IS_WRITABLE (caps)' failed
middle textures in gleffects do not need to use GstGLMemoryPBO as they
aren't transfering data to/from the GPU. This will cost too much DMA
memory and cause performance issue. Change the allocator to use non-PBO
GstGLMemory.
https://bugzilla.gnome.org/show_bug.cgi?id=776072
glvideomixer does not support it currently and it needs special support
for handling this correctly, and is rather non-trivial to implement for
all formats.
MSVC warns about this because it's a C++ compiler, and this actually
results in useful things such as the incorrect 'gboolean' return value
for functions that return GstFlowReturn, so let's do explicit
conversions to reduce the noise and increase its efficacy.
With MSVC, this gives the following warning:
warning C4305: 'function': truncation from 'double' to 'gfloat'
Apparently, MSVC does not figure out what type to use for constants
based on the assignment. This warning is very spammy, so let's try to
fix it.
The headers we include already define boolean on Windows with MSVC, and
it leads to a typedef redefinition error with jpeglib.h which tries to
redefine it in jmorecfg.h
When only linking the element, the upload object will be created from
_transform_caps() but will never be unreffed as the only case is in _stop().
Add an unref if non-NULL to a new finalize handler for this case.
Fixes an assertion when moving from passthrough to non-passthrough
Without an explicit reconfigure, glfiter won't have created the GL
resources such as the FBO, GL bufferpool, etc and basetransform will
allocate sysmem buffers instead.
Previously it was created in the init function and destroyed in ::stop, which
lead to segfaults when reusing the element.
Now the upload object is created in ::transform_caps if it is NULL, which is the
earliest we need it. The other vfuncs already bail out if the upload object is
NULL, which means that negotiation wasn't done.
Now when used with video/x-raw as input, the GLMemoryUpload method checks for
->tex_target in input GLMemory(es) and sets the output texture-target
accordingly.
Fixes video corruption with a pipeline like avfvideosrc ! video/x-raw !
glimagesink where on macos avfvideosrc pushes RECTANGLE textures but glupload
was configuring texture-target=2D as output.
The videoaggregator negotiation sequence changed some time
back and broke glstereomix. Instead of doing nego incorrectly
in the find_best_format() vfunc, do it directly in the
update_caps() method.
It implements now this interface with its video-direction
property. Values are changed to GstVideoOrientationMethod but they have
the same value than the originals.
https://bugzilla.gnome.org/show_bug.cgi?id=768687
It now returns the correct values for both orthographic and perspective
projections and takes into account the aspect ratio of the video, handles
the Y-flipping in GL and by us and uses some more helpers from graphene.
There's no need for the jump to an extra thread in most cases, especially
when relying solely on a shader to render. We can use the provided
render_to_target() functions to simplify filter writing.
Facilities are given to create fbo's and attach GL memory (renderbuffers
or textures). It also keeps track of the renderable size for use with
effective use with glViewport().
Rather than assuming something. e.g. zerocopy on iOS with GLES3 requires
the use of Luminance/Luminance Alpha formats and does not work with
Red/RG textures.
These two shader will calculate the vector length and use it as denominator.
But length could be zero which will cause undefine behaviour. Add protection for
this condition
https://bugzilla.gnome.org/show_bug.cgi?id=767635
_get_gl_context() can be called concurrently from either propose_allocation() or
decide_allocation(). If it so happens that this happens at the same time,
the check for whether we already had a GL context was outside the lock. Inside
the lock and loop, the first thing that happens is that we unref the current GL
context (if valid) as if there was a conflict adding it to the display. If the
timing was unlucky, subsequent use of the GL context would be referencing an
already unreffed GL context object resulting in a critical:
g_object_ref: assertion 'object->ref_count > 0' failed
https://bugzilla.gnome.org/show_bug.cgi?id=766703
The gltestsrc element uses two shaders: color_shader and snow_shader.
Those are alternatively assigned to the SrcShader->shader pointer and
their reference was transferred to it. Only the SrcShader->shader was
unreffed (in _src_shader_deinit()) so only one shader was properly
freed, the other one was leaked.
Fixed this by giving an extra ref to SrcShader->shader and unreffing the
2 shaders in _src_smpte_free().
https://bugzilla.gnome.org/show_bug.cgi?id=766661
We can avoid a render pass if downstream supports the affine transformation meta
and increase the performance of some pipelines involving gltransformation.
Implemented by checking for the affine transformation in the allocation query
from downstream and combining our matrix with that of upstream's (or creating
our own).
Provide a function to get the affine matrix in the meta in terms of NDC
coordinates and use as a standard opengl matrix.
Also advertise support for the affine transformation meta in the allocation
query.
We were always failing the allocation query as a flag was never being set to
signal a successful negotiation. Fix by setting the required flag on a
successful caps event from upstream.
1.Porting the exist deinterlace shader and OpenGL callback
to be compatible with OpenGL ES.
2.Add a our blur vertical shader to gldeinterlace.
3.Add a property named “method” to let user choose which
deinterlace function to use. Default to choose blur vertical
method for better performance.
[Matthew Waters]: fix name of greedyh in method property (was greedhy) and port
to git master.
https://bugzilla.gnome.org/show_bug.cgi?id=764873
When application change pipeline state NULL->READY and then READY->NULL,
glimagesink will not clear glsink->window_id. After that, when application
change state NULL->READY, the new_window_id is equal to window_id, glimagesink
will not set window handle. It will use the internal window but not the window
create by application.
https://bugzilla.gnome.org/show_bug.cgi?id=765241
We weren't using the result of find_best_format at all.
Also, move the find_best_format usage to the default update_caps() to make
sure that it is also overridable.
https://bugzilla.gnome.org/show_bug.cgi?id=764363
This makes gltestsrc work everywhere \o/
- workaround RPi returning invalid values for positive coords in the
checker shader
- reduce the number of iterations in the mandelbrot shader for gles2
https://bugzilla.gnome.org/show_bug.cgi?id=751540
We want to iterate over all the pads, not just the first one. Fix by returning
TRUE in the GstAggregatorPadForeachFunc.
Removes a GST_IS_GL_CONTEXT() assertion on shutdown with >2 inputs
using gst-launch.
On some embedded systems, sqrt() is not supported in the shader,
use the actual value of sqrt(2) instead.
Signed-off-by: Haihua Hu <b55597@freescale.com>
Bugzilla: https://bugzilla.gnome.org/show_bug.cgi?id=761271
Allows the subclass to completely override the chosen src caps.
This is needed as videoaggregator generally has no idea exactly
what operation is being performed.
- Adds a fixate_caps vfunc for fixation
- Merges gst_video_aggregator_update_converters() into
gst_videoaggregator_update_src_caps() as we need some of its info
for proper caps handling.
- Pass the downstream caps to the update_caps vfunc
https://bugzilla.gnome.org/show_bug.cgi?id=756207
1. Otherwise rotating the video will clip and show black bars due to
gltransformation's implementation.
2. The other option of make gltransformation aspect-agnostic produces
incorrect output with perspective transformations.
In file included from effects/gstgleffectrgbtocurve.c:25:0:
effects/gstgleffectscurves.h:174:32: error: 'xray_curve' defined but not used
static const GstGLEffectsCurve xray_curve = {
...
Reverses the transformation applied through the properties and forwards the
event.
The process for finding the coordinates on the video are as follows:
1. Convert the given pointer_x and pointer_y to model space at the near and far planes
2. Get the equation of the video plane
3. Find where the ray in 1 intersects the plane
4. Profit!
It performs the exact same operation as videobalance but with opengl shaders and
was tested with glvideomixer by comparing frames from videobalance and
glcolorbalance.
Allows more blending options than just A over B
e.g. frame comparisons are now possible.
glvideomixer name=m
sink_0::zorder=0
sink_1::zorder=1
sink_1::blend-equation-rgb={subtract,reverse-subtract}
sink_1::blend-function-src-rgb=src-color
sink_1::blend-function-dst-rgb=dst-color
! glimagesinkelement
videotestsrc pattern=checkers-4 ! m.sink_0
videotestsrc pattern=checkers-8 ! m.sink_1
All the gleffects shaders can be run against a gles2 or a legacy opengl glsl
compiler but weren't being advertised as such.
Fixes gleffects under desktop opengl < 3.2.
The property location has been changed in favor of vertex/fragment
string properties; the doc had not been updated and was still referring
to the previous property; also, now the #version header has become mandatory
https://bugzilla.gnome.org/show_bug.cgi?id=759902
Adds more meaningful error than
"Failed to convert multiview video buffer", which is always used
when prepare_next_buffer() fails in gst_glimage_sink_prepare().
https://bugzilla.gnome.org/show_bug.cgi?id=743345
Add gst_gl_memory_allocator_get_default to get the default allocator based on
the opengl version. Allows us to stop hardcoding the PBO allocator which isn't
supported on gles2.
Fixes GL upload on iOS9 among other things.
Performing any GL function marshalling off the GL thread with glimagesink's
render lock is prone to deadlocks between the GL thread and the non-GL thread.
What can happen is this:
1. non-GL thread attempts to function marshal to the GL thread.
2. while 1 is happening, the winsys gives an event (say resize)
3. This calls back into glimagesink which taks the render lock.
4. As the GL function marshalling is attempting to run on the GL
and already has glimagesink's render lock locked. This deadlocks
as the threads are waiting for each other.
rename gst-launch --> gst-launch-1.0
replace old elements with new elements(ffmpegcolorspace -> videoconvert, ffenc_** -> avenc_**)
fix caps in examples
https://bugzilla.gnome.org/show_bug.cgi?id=759432
The base class is useful for having multiple backing memory types other
than the default. e.g. IOSurface, EGLImage, dmabuf?
The PBO transfer logic is now inside GstGLMemoryPBO which uses GstGLBuffer
to manage the PBO memory.
This also moves the format utility functions into their own file.
Solved with a simple shader templating mechanism and string replacements
of the necessary sampler types/texture accesses and texture coordinate
mangling for rectangular and external-oes textures.
Add the various tokens/strings for the differnet texture types (2D, rect, oes)
Changes the GLmemory api to include the GstGLTextureTarget in all relevant
functions.
Update the relevant caps/templates for 2D only textures.
Currently float and int are supported by default. vec2, vec3, vec4
and mat4 are supported if graphene is used. Of course if one wants
to set custom uniforms they can also be set using the create-shader
signal.
One may not have an GstGLContext available or current in the thread where one
would need to update the shader. Support this by signalling create-shader
whenever the one-shot 'update-shader' is set to TRUE.
A GstGLShader is now simply a collection of stages that are
compiled and linked together into a program. The uniform/attribute
interface has remained the same.
This allow properly copying selected meta, like the composition
overlay. Note that output buffer need to be readable, but GlUpload
keeps a ref. For now, simply drop GlUpload ref after perform,
leaving that ref has no purpose. The method shall be removed
in the future.
https://bugzilla.gnome.org/show_bug.cgi?id=754047
Use base class default method instead of only copying flags and
timestamp. This way, selected meta's like compostion overlay will
be passed downstream as expected.
https://bugzilla.gnome.org/show_bug.cgi?id=754047
During allocation query, when this element is not passthrough, it must
relay the overlay compostion meta and it's parameters. Fortunatly, base
transform can do this for us.
https://bugzilla.gnome.org/show_bug.cgi?id=753850
The current code was ignoring the par/dar aspect when transforming
from window coordinates to stream coordinates resulting in incorrect
coordinates being sent upstream in the navigation events.
https://bugzilla.gnome.org/show_bug.cgi?id=744763
Add a pivot vector for setting the origin of rotations and scales.
With the pivot point the rotation and scale operations can have
different origins. This adds the ability to rotate around different points.
Currently the default (0, 0) pivot point is possible,
a rotation around the center, and zooming into and out of the center.
With an pivot point this is optional.
I defined the following image coordinates for the pivot point:
(-1,1) ------------------------- (1,1)
| |
| |
| |
| (0,0) |
| |
| |
| |
(-1,-1) ------------------------- (1,-1)
Example:
Rotate the video at the bottom left corner
gst-launch-1.0 videotestsrc \
! gltransformation \
scale-x=0.5 \
scale-y=0.5 \
rotation-z=25.0 \
pivot-x=-1.0 \
pivot-y=-1.0 \
! glimagesink
The pivot-z option defines the pivot point in 3D space.
This only affects rotation, since we have no Z data to scale.
With this option a video can be rotated around a point in 3D space.
Example:
Rotate around point behind the video:
gst-launch-1.0 videotestsrc \
! gltransformation \
rotation-x=10.0 \
pivot-z=-4.0 \
! glimagesink
The ref_object and object parameters were the wrong way around.
For the typical use case where an application is setting a
GstControlBinding on the returned ghost pad:
1. our control binding would be removed when the new one was set
2. sync_values calls were not being forwarded from the internal
pad to the ghost pad.
If an application attempts to perform other control binding
operations (get_* family of functions) on the internal pad, they
will also be forwarded to the ghost pad where a possible
GstControlBinding will provide the necessary values.
This is used to proxy GstControlBinding to the pad on the
parent object. This avoid having to sync the values in the proxy pad,
this is too early if you have a queue between the pad and the actual
aggregation operation.
https://bugzilla.gnome.org/show_bug.cgi?id=734060
When the sink does not know the window size (e.g not created yet)
it will not add any param to the the composition meta. This is no
reason not to forward this meta API. Fixes issue where it could not
attach until we resize the window.
https://bugzilla.gnome.org/show_bug.cgi?id=745107
The coordinate are relative to the texture dimension and not
the window dimension now. There is no need to pass the window
dimension or to update the overlay if the dimension changes.
https://bugzilla.gnome.org/show_bug.cgi?id=745107
Adds an GST_VIDEO_OVERLAY_COMPOSITION_META_API_TYPE query to glupload
and glimagesink. Detects the query from the downstream elements, so
it is executed only when downstream supports the overlay API.
This makes pipelines with textoverlay ! glupload ! gldownload ! xvimagesink possible.
Uses allocation meta struct for passing the window size upstream.
https://bugzilla.gnome.org/show_bug.cgi?id=745107
I notice that if you stop the pipeline during a renegotiation
the upload may be NULL while an allocation query is being run.
In that scenario, returning FALSE to the allocation query is the
best thing.
