There are some cases where it's needed for binding in/out variables in shaders.
e.g. glsl 150 (gl 3.2) doesn't support the 'layout (location = ?)' specifiers in
the shader source so we have to bind them ourselves.
Happens e.g. if a RECONFIGURE event is sent from downstream while we're
switching pads at this very moment. The old pad is gone and the stream has a
new pad.
https://bugzilla.gnome.org/show_bug.cgi?id=764404
Previously, while allocating the pad number for a new pad, aggregator was
maintaining an interesting relationship between the pad count and the pad
number.
If you requested a sink pad called "sink_6", padcount (which is badly named and
actually means number-of-pads-minus-one) would be set to 6. Which means that if
you then requested a sink pad called "sink_0", it would be assigned the name
"sink_6" again, which fails the non-uniqueness test inside gstelement.c.
This can be fixed by instead setting padcount to be 7 in that case, but this
breaks manual management of pad names by the application since it then becomes
impossible to request a pad called "sink_2". Instead, we fix this by always
directly using the requested name as the sink pad name. Uniqueness of the pad
name is tested separately inside gstreamer core. If no name is requested, we use
the next available pad number.
Note that this is important since the sinkpad numbering in aggregator is not
meaningless. Videoaggregator uses it to decide the Z-order of video frames.
Without the GST_GL_API_GLES2 bit set, we will not even attempt to look
for the function pointers in the core library and will fallback to
glFlush/glFinish.
If the user uploads their own texture without setting the unpack length, then
then the result will have the appearance of stride mismanagement due to
an incorrect row length.
If we are given caps with extra features (like the overlay composition
features), we can only deal with that when we are in passthrough mode.
Previously we were bailing entirely and not allowing passthrough filter elements
with things like textoverlay.
Fixes the following pipeline (assuming glfilter supports passthrough):
gl ! textoverlay ! glfilter ! ... ! glimagesinkelement
https://bugzilla.gnome.org/show_bug.cgi?id=763756
When transforming, xplode it out into the necessary caps features both
with and without the passthough features.
Fixes negotiation in the following class of pipelines:
gl ! textoverlay ! glupload ! glimagesinkelement
https://bugzilla.gnome.org/show_bug.cgi?id=763756
GL 1.4 (with GL_ARB_shader_objects) doesn't have glIsProgram or glIsShader
equivalents. As they are simply assertions, skip them when there isn't a
valid function pointer.
This is a regression from since mpegvideoparser was switched to
use the codecparsing library.
The problem is that the high bit of the profile_and_level is used
to specify non-hierarchical profiles and levels. Unfortunately we
were discarding that information.
Expose that escape bit, and use it in the element
https://bugzilla.gnome.org/show_bug.cgi?id=763220
With e38af23044 returning the correct contexts,
gst_gl_display_add_context() was susceptible to causing infinte loops when
adding the same GstGLContext more than once. Fix and add a test for
gst_gl_display_add_context().
Fixes glvideomixer gst-validate tests.
Execute GL calls without marshalling them to the context thread. In the cocoa
and eagl backends calling gst_gl_context_activate is cheap and therefore calling
it on the current thread and serializing GL calls with a per-context lock is
more efficient (faster and has less overhead) than marshalling everything to the
context thread.
This optimization cuts a large overhead in g_poll (continuously waking up the
context thread) and in g_mutex_*/g_cond_* (waiting for results from the context
thread).
When requesting a glcontext (regardless of thread), the result was correct.
However, when requesting current glcontext on a specific thread, it could
come up with a glcontext active on another thread.
https://bugzilla.gnome.org/show_bug.cgi?id=763168
Maxsize is initialized once and should never change. Allocating data
should have no impact on the selected max size for this memory. This
causing memory map failure as the maxsize would become smaller then
size. This happened when using direct rendering in avviddec on GL that
does not support PBO transfer.
https://bugzilla.gnome.org/show_bug.cgi?id=763045
When the start_type is GST_SEEK_TYPE_NONE for a forward seek
(or stop_type for a reverse) is not set on a snap seeking operation,
the element should use the current position and then snap as requested.
Also fixes uninitialized variable complaint by clang about
'ts' variable.
Intended for use with wrapped contexts that are created shared with gst's
gl contexts in order to manage the internal sharegroup state correctly.
e.g. with caopengllayer (which is used in glimagesink and caopengllayersink
on OS X), we create a CGL context from the gst context and the sharing state
was not being correctly set on either GL context and gst_gl_context_is_shared()
was always returning FALSE.
With 11fb4fff80 only flushing with multiple
shared contexts, the required flush was not occuring causing screen
corruption or stuttering.
Note: this didn't affect GST_GL_API=opengl pipelines
https://bugzilla.gnome.org/show_bug.cgi?id=762620
When caps are already negotiated it should be possible to
select formats other than the one that was negotiated. If downstream
allows alpha video caps and it has already negotiated to a non-alpha
format, caps queries should still return the alpha caps as a possible
format as caps renegotiation can happen.
Includes tests (for compositor) to check that caps queries done after
a caps has been negotiated returns complete results
https://bugzilla.gnome.org/show_bug.cgi?id=757610
Expose the expose() and set_render_rectangle() methods. These are useful for
proper functioning of the video overlay in various situations and toolkits.
H.265 7.4.7.1 says:
> When slice_deblocking_filter_disabled_flag is not present, it is
> inferred to be equal to pps_deblocking_filter_disabled_flag.
https://bugzilla.gnome.org/show_bug.cgi?id=762351
Usually gl debug is initialized in gst_gl_context_create_thread.
But this function is not used when using the GstGLContextGPUProcess
from ChromiumGStreamerBackend.
Received signal 11 SEGV_MAPERR 000000000000
gst_debug_category_get_threshold
gst_gl_insert_debug_marker
gst_gl_base_filter_gl_start
CC libgstgl_x11_la-gstglcontext_glx.lo
In file included from gstglcontext_glx.c:39:0:
../utils/opengl_versions.h:52:43: error: ‘gles2_versions’ defined but not used [-Werror=unused-const-variable]
static const struct { int major, minor; } gles2_versions[] = {
^~~~~~~~~~~~~~
Invoke the callback right away when called on the context thread. Removes
overhead when nesting libgstgl calls (for example when working with the sync
meta).
CPU waits are more expensive and are only required if the CPU is ever going to
access the data. GPU waits perform inter-context synchronisation and are cheaper
as they don't require CPU intervention.
Adaptive demuxers need to start downloading from specific positions
(fragments) for every stream, this means that all streams can snap-seek
to a different position when requested. Snap seeking in this case will
be done in 2 steps:
1) do the snap seeking on the pad that received the seek event and
get the final position
2) use this position to do a regular seek on the other streams to
make sure they all start from the same position
More arguments were added to the stream_seek function, allowing better control
of how seeking is done. Knowing the flags and the playback direction allows
subclasses to handle snap-seeking.
And also adds a new return parameter to inform of the final
selected seeking position that is used to align the other streams.
https://bugzilla.gnome.org/show_bug.cgi?id=759158
All uses of query->context in gstglquery.c assume it exists. We can assume
this as well before unrefing it. Furthermore, gst_object_unref() will just
silently return if it ever were to not exist.
Improves the responsiveness of the pipeline when resources are close/above the
limitations of the hardware.
Any subclass that wishes not to enable qos can do so themselves.
https://bugzilla.gnome.org/show_bug.cgi?id=761519
This reverts commit 96b9666d59.
This reverts commit d11385d167.
This breaks the texture sharing with the applemedia elements as
CVOpenGLESTextureCache seems to have an arbitrary restriction on GLES2 only.
We may need them to transform into a different set of formats.
Fixes YUV->YUV with two glcolorconverts, e.g:
format=I420 ! glcolorconvert ! glcolorconvert ! format=NV12
1.0 / width does not offset by one pixel in rectangular textures (which use
unnormalized coordinates).
Provide the actual pixel offset as a uniform to the shader.
1. Correctly describe what we can caps we can transform to/from.
i.e. no YUV->YUV or GRAY->YUV or YUV->GRAY (except for passthrough).
2. Prefer similar formats and ignore incompatible formats on fixation.
Useful when gst_gl_window.c::gst_gl_window_new is not used.
This is the case when using a custom GstGLWindow.
(ex: GstGLWindowGPUProcess from Chromium)
Allows the subclass to completely override the chosen src caps.
This is needed as videoaggregator generally has no idea exactly
what operation is being performed.
- Adds a fixate_caps vfunc for fixation
- Merges gst_video_aggregator_update_converters() into
gst_videoaggregator_update_src_caps() as we need some of its info
for proper caps handling.
- Pass the downstream caps to the update_caps vfunc
https://bugzilla.gnome.org/show_bug.cgi?id=756207
The function gst_adaptive_demux_stream_update_source() function creates
a new GstPad called internal_pad. This pad is not freed when releasing
the stream.
The solution is to set GST_PAD_FLAG_NEED_PARENT so that the chain
functions do not get called when the pad has no parent and then
remove the parent in the gst_adaptive_demux_stream_free() function. This
causes the refcount of the pad to be set to zero.
https://bugzilla.gnome.org/show_bug.cgi?id=760982
It's useful enough already to be used in other elements for audio aggregation,
let's give people the opportunity to use it and give it some API testing.
https://bugzilla.gnome.org/show_bug.cgi?id=760733
Handling the ghostpad and its internal pad was causing more issues
than helping because of their coupled activation/deactivation
actions.
As we have to install custom chain,event and query functions it is
better to use a floating sink pad internally in the demuxer and just
use those pad functions to push through a standard pad in the demuxer
https://bugzilla.gnome.org/show_bug.cgi?id=757951
The messages are stored by gst_gl_async_debug_store_log_msg() and output later
by a corresponding store(), output() or an unset()/free().
Some wrapper macros are provided to avoid callers explicitly using __FILE__,
GST_FUNCTION and __LINE__
This makes a pipeline like:
... ! video/x-raw(memory:GLMemory),format=UYVY ! glcolorconvert !
video/x-raw(memory:GLMemory),format={UYVY, NV12} ! ...
passthrough instead of converting UYVY => NV12. The conversion would happen
before this change since the element (and basetransform) transform the src caps
to format={NV12, UYVY} (since NV12 comes first in the glcolorconvert:src
template) and then the default caps fixate func would fixate to NV12. Blah.
Also there's no need to intersect against the template caps in ::transform_caps
since basetransform does that right after calling the vfunc.
The optimistic download_transfer was not setting the required flag to not
perform glReadPixels on subsequent map (READ). resulting in glReadPixels
happening twice.
Fixed adaptivedemux seeking without flushing that just wants
to update stop position. This required protecting the segment
variables with a new mutex so that the seeking thread and the
download threads could safely manipulate the segment and
events related to it.
This contention is only locked/unlocked when starting a new
download, when the first fragment of a segment is received and
when seeking so, hopefully, it won't damage performance.
Some operations are unnecessary when running with only a single GL
context.
e.g. glFlush when setting a fence object as the flush happens on wait.
API: gst_gl_context_is_shared
Avoids downloading and pushing a full segment just to get 1 nanosecond
of data. This happens frequently when seeking is done with flags
that adjust to boundaries or when the start is aligned with segment
starts. The later is common when segment durations is a multiple of
a second.
For reverse, set position to segment.stop when starting and also
don't set the position to fragment end timestamp when it finishes,
just leave it at the fragment start.
1. Various elements/base classes only perform a subset check on accept-caps
2. Some GL elements have texture-target in their pad template
3. When checking subsets, only the caps to check are allowed to contain extra
fields. If the 'template' caps have extra fields, the subset fails.
Thus without texture-target on the caps, various accept-caps implementations
were failing.
Also, add some convenience functions for setting and retrieving
texture targets to/from GValue.
https://bugzilla.gnome.org/show_bug.cgi?id=759860
In very few cases the simple version was actually needed and having the
parameters hidden by a _full() version caused application that actually needed
it to not use it.
GL 2.1 only supports pbo upload.
The wrapped data pointer was only being set on the pbo memory and on the
glmemory so when a download was requested (in GL 2.1), glmemory was
allocating a new data pointer and thus not returning the wrapped data.
Exposing the navigation thread's main context, GSourceFuncs and structs called
key_event and mouse_event is exposing a bit too much of the internals. Let's
just go with two functions to asynchronously send navigation events on the
window with the same API as the synchronous ones.
This upload method detect and optimize uploads of DMABuf memory. This is
done by creating and caching EGLImages wrapper around DMABuf. The
EGLImages are then binded to a texture which get converter using
standard shader.
Example pipeline:
GST_GL_PLATFORM=egl \
gst-launch-1.0 v4l2src device=/dev/video1 io-mode=4 ! \
video/x-raw,format=NV12 ! glimagesink
https://bugzilla.gnome.org/show_bug.cgi?id=743345
Maps GstVideoFormats to suitable DRM fourccs which work with
glcolorconvert, using gst_gl_memory_alloc(). We require mostly
only 4 formats to be supported by the driver. We require DRM
equivalent to RGB16, RGBA, R8 and RG88. This way it's compatible with
DesktopGL, since GL_TEXTURE_2D is used and limit driver requirements.
With this we can virtually support all formats the glcolorconvert
supports.
https://bugzilla.gnome.org/show_bug.cgi?id=743345
Add gst_gl_memory_allocator_get_default to get the default allocator based on
the opengl version. Allows us to stop hardcoding the PBO allocator which isn't
supported on gles2.
Fixes GL upload on iOS9 among other things.
e.g when wrapping a data pointer we don't want to map/unmap off the end of
pointer with the alignment bytes.
Instead track that information separately as maxsize is used for mapping by
GstMemory and thus represents a size without any alignment padding bytes.
Requires the usage of GstGLVideoAllocationParams however any user can set their
own parameters along with an allocator which will be used to allocate the
correct memory type.
- Create GstGLVideoAllocationParams which is a GstGLAllocationParams subclass.
- Make it possible to allocate glmemory objects directly if no frills are
needed.
This is made possible by a subclassable GstGLAllocationParams that holds
the allocation parameters
Every allocation would now go through gst_gl_base_memory_alloc with the
allocation parameters now being specified in a single struct to allow
extension by different allocators.
The imported memory has already been allocated, passing allocation
parameters with alignment confuses the memory which endup with a
size different from maxsize and lead to overrun when the memory
is being copied.
GCC automatically disable redundance warnings for system headers. As
soon as we start using a non-system installed mesa, we would start
having issues. The test for both wasn't setting any flags, so it would
work but then fail at runtime.
This is being fixed by disabling in the code (where needed only) that
GCC warning. The test is also fixed to avoid the false positive we had.
The base class is useful for having multiple backing memory types other
than the default. e.g. IOSurface, EGLImage, dmabuf?
The PBO transfer logic is now inside GstGLMemoryPBO which uses GstGLBuffer
to manage the PBO memory.
This also moves the format utility functions into their own file.
Heavily based on GstGLBaseBuffer that is a subclass of GstGLBaseMemory.
Provides GPU and CPU accessible GL buffer objects by GL handle or by
sysmem data pointer.
It handles the following
- GstAllocationParams -> gst_memory_init transformation
- Makes sure that map/unmap/create/destroy happen on the GL thread with
a GL context current.
- Holds a possible sysmem accessible data pointer with alignment.
- Holds the need upload/download transfer state
Rectangle textures don't use normalized coordinates so subsampling needs to be
factored in explicitly.
Fixes YUV => RGB conversion for rectangle textures.
To use GLMemory and EGLImage allocators, one need to know the
libgstgl API. This is only expected if the associated caps features
have been negotiated. Generic element that otherwise receive those
allocators may fail, resulting in broken pieline. We don't want to
force all generic element to check if the allocator is a custom
allocator or a normal allocator (which implement the _alloc method).
https://bugzilla.gnome.org/show_bug.cgi?id=758877
gstglsyncmeta.c -fPIC -DPIC -o .libs/libgstgl_1.0_la-gstglsyncmeta.o
gstglsyncmeta.c: In function 'gst_buffer_add_gl_sync_meta':
gstglsyncmeta.c:131:1: error: control reaches end of non-void function [-Werror=return-type]
}
^
there could be other ways/requirements for synchronising two GPU command
streams (whether GL or platform specific).
e.g. glfencesync/eglwaitnative/cond/etc
This no longer does anything, and it was marked as CONSTRUCT_ONLY
which means someone would really have to go out of their way to
be able to set this, which would only be done in very custom
scenarios, if ever, and those will likely target a specific
version of GStreamer then, so probably not much point keeping
it deprecated for a while before removing it.
given a NULL-terminated string, s.
s[i] = '\0';
i++;
does not guarentee that s[i] is NULL terminated and thus string operations
could read off the end of the array.
https://bugzilla.gnome.org/show_bug.cgi?id=758039
gst_gl_shader_detach_unlocked already removes the list entry so attempting to
use the element to iterate to the next stage could read invalid data.
Based on patch by Vineeth TM <vineeth.tm@samsung.com>
https://bugzilla.gnome.org/show_bug.cgi?id=758039
Some drivers don't provide the compatibility definition and we need to provide
our own 'out vec4' variable to put the results of the fragment shader into.
https://bugzilla.gnome.org/show_bug.cgi?id=757938
Rectangular textures are unavailable in unextended
GLES2 #version 100 shaders.
Fixes
texture-target=rectangle ! glcolorconvert ! texture-target=2D
There's a couple of differences between GL3 and GLES2/GL
- varying -> in or out depending on the stage (vertex/fragment)
- attribute -> in
- filtered texture access is a single function, texture()
Bitrate estimation is now handled through a queue2 element added after
the source element used to download fragments.
Original hlsdemux patch by Duncan Palmer <dpalmer@digisoft.tv>
https://bugzilla.gnome.org/show_bug.cgi?id=733959
This code will never be called as max>=min in all cases. If the upstream
latency query returned min>max, the function already returned and all
values that are added to those have max>= min.
Make gst_gl_context_gen_shader/_compile_shader assume GST_GLSL_PROFILE_ES |
GST_GLSL_PROFILE_COMPATIBILITY as the profile. Without this, the shader compiler
doesn't inject the #version tag resulting in a compilation error on Mountain
Lion.
This is a workaround for old code using gst_gl_context_gen_shader. New code
should use the gst_glsl_stage_* API directly which allows the caller to
explicitly specify version/profile.
The ret variable may be uninitialized and so its contents were undefined and
the results were erratic (failing with glvideomixer, succeeding in other cases)
P.S. No idea why gcc/clang et al never picked up on this like they normally do
(probably due to some optimisation pass figuring out it's only set once...)
They are already defined in the forward decleration header and defining them
more than once will give an error with OSX's clang about typedef redefinition
being a C11 feature.
This was chosen over relying solely on the caps as glupload needs to propose an
allocation and set the texture target based on the output caps. Setting the
caps in the config is currently pointless as they are overwritten in a lot of
element's decide_allocation functions.
This provides a mechanism for the buffer pool to be configured for a certain
texture target when none has been configured.
Solved with a simple shader templating mechanism and string replacements
of the necessary sampler types/texture accesses and texture coordinate
mangling for rectangular and external-oes textures.
Add the various tokens/strings for the differnet texture types (2D, rect, oes)
Changes the GLmemory api to include the GstGLTextureTarget in all relevant
functions.
Update the relevant caps/templates for 2D only textures.
The gst_adaptive_demux_stream_free function is trying to stop the stream's
download task. For this, it signals the task. But it fails to also set the
stream->download_finished = TRUE, so the task will go back to sleep and
only exit when the download is finished.
https://bugzilla.gnome.org/show_bug.cgi?id=755121
Initialize to 0 these parse structures before filling them: GstH264SEIMessage,
GstH264NalUnit, GstH264PPS, GstH264SPS and GstH264SliceHdr.
When calling the functions which fill those structures, they may fail, leaving
unitialized those structures. This situation may lead to future problems, such
as a segmentation fault when freeing, for example.
This patch initializes to zero these structures, before filling them.
https://bugzilla.gnome.org/show_bug.cgi?id=755161
Initialize to 0 these parse structures before filling them: GstH265SEIMessage,
GstH265NalUnit, GstH265VPS, GstH265PPS, GstH265SPS and GstH265SliceHdr.
When calling the functions which fill those structures, they may fail, leaving
unitialized those structures. This situation may lead to future problems, such
as a segmentation fault when freeing, for example.
This patch initializes to zero these structures, before filling them.
https://bugzilla.gnome.org/show_bug.cgi?id=755161
the USING_GLES2 includes all GLES3 contexts as well which does support
drawing to multiple buffers. Instead make or decision solely based on
whether glDrawBuffers is available or not.
Not all aggregator subclasses will have a single pad template called sink_%u
and might do something special depending on what the application requests.
https://bugzilla.gnome.org/show_bug.cgi?id=757018
e.g:
gstglcontext_egl.c:613:7: error: implicit declaration of function 'strcmp'
[-Werror=implicit-function-declaration]
if (strcmp (G_MODULE_SUFFIX, "so") == 0)
While technically, i is always 0 and *vertex_sources[i++] is equivalant
to (*vertex_sources)[i++]. Be future-proof in the case of code
moves/changes/etc.
CID 1327406
A GstGLShader is now simply a collection of stages that are
compiled and linked together into a program. The uniform/attribute
interface has remained the same.
1. So we get tracking inside GstElement properly when e.g. adding to a bin
2. Removes redundant code. Now only one place where
GstContext->GstGLDisplay/GstGLContext transformation occurs
3. Fixes a memory leak in the process
4. Make the retrieval of debug categories thread safe
These markers are visible in tools that record the GL function calls
such as apitrace, et al.
Makes it easier to match up GL draw commands with specific elements.
Otherwise they will receive a QOS event that has earliest_time=0 (because we
can't have negative timestamps), and consider their buffer as too late
https://bugzilla.gnome.org/show_bug.cgi?id=754356
- glimagesink needs to be able to resize the viewport on aspect ratio
changes resulting from either caps changes or 3d output mode changes.
- Performing a glViewport outside the GstGLWindow::resize callback
will not have the winsys' stack of viewports required to correctly
place the output frame.
Provide a function to request a resize on the next draw event from the
winsys.
Also track size changes inside the base GstGLWindow class rather
than in each subclass.
https://bugzilla.gnome.org/show_bug.cgi?id=755111
dashdemux seeks each live stream to its current fragment in the beginning, but
the base class does not know about this. Update the demuxer segment with this
seek so we generate the correct SEGMENT event and can actually play the
stream.
This needs some refactoring at some point.
https://bugzilla.gnome.org/show_bug.cgi?id=755047
when allocating memory. Fixes crashes with avdec_h265 in the AVX2
code path which requires 32-byte stride alignment, but the
GstAllocationParams only specified a 16-byte alignment.
https://bugzilla.gnome.org/show_bug.cgi?id=754120
We have to queue buffers based on their running time, not based on
the segment position.
Also return running time from GstAggregator::get_next_time() instead of
a segment position, as required by the API.
Also only update the segment position after we pushed a buffer, otherwise
we're going to push down a segment event with the next position already.
https://bugzilla.gnome.org/show_bug.cgi?id=753196
Each period will start again with pts 0 + period presentation offset, which is
also going to be the presentation time inside the container stream if any.
However all periods together should form a continuous timeline, with regard to
stream time and running time.
For making this possible we keep track of the "user requested segment", i.e.
the seek events, inside the demuxer without adjusting anything and taking this
demuxer segment only as orientation for modified segments per stream.
This per stream segments will have their segment.start at pts that would be
produced for this stream in this period, and the segment.base/time adjusted so
that this pts maps to the running and stream time this period should have in
the context of all other periods.
https://bugzilla.gnome.org/show_bug.cgi?id=754222
Only accept alpha if downstream has alpha as well. It could
theoretically accept alpha unconditionally if blending is
properly implemented for handle it but at the moment this
is a missing feature.
Improves the caps query by also comparing with the template
caps to filter by what the subclass supports.
https://bugzilla.gnome.org/show_bug.cgi?id=754465
If short_term_ref_pic_set_sps_flag is FALSE, the ShortTermRefPicSet
structure is supposed to derive from slice header. Which means,
CurrRpsIdx is equal to num_short_term_ref_pic_sets. But the number
of refpicsets communicated via sps header is only num_short_term_ref_pic_sets - 1.
And we are using slice_header structure to reference the last entry, which is
ShortTermRefPicSet[num_short_term_ref_pic_sets].
https://bugzilla.gnome.org/show_bug.cgi?id=754834
HAVE_IOS is only defined for the build of this module so
attempting to use gstgl in iOS would result in incorrect GL
includes.
Use GST_GL_HAVE_PLATFORM_EAGL instead for choosing the iOS GL
header.
