gstreamer-rs/docs/gstreamer-gl/docs.md

GLBaseFilter handles the nitty gritty details of retrieving an OpenGL context. It also provided some wrappers around gst_base::BaseTransform's start, stop and set_caps virtual methods that ensure an OpenGL context is available and current in the calling thread.

Implements

GLBaseFilterExt, gst::ObjectExt, glib::object::ObjectExt

Trait containing all GLBaseFilter methods.

Implementors

GLBaseFilter

GstGLBuffer is a gst::Memory subclass providing support for the mapping of GL buffers.

Data is uploaded or downloaded from the GPU as is necessary.

Initializes the GL Buffer allocator. It is safe to call this function multiple times. This must be called before any other GLBuffer operation.

GLColorConvert is an object that converts between color spaces and/or formats using OpenGL Shaders.

A GLColorConvert can be created with GLColorConvert::new, the configuration negotiated with GLColorConvert::transform_caps and the conversion performed with GLColorConvert::perform.

The glcolorconvertelement provides a GStreamer element that uses GLColorConvert to convert between video formats and color spaces.

Implements

gst::ObjectExt, glib::object::ObjectExt

context

a GLContext

Returns

a new GLColorConvert object

Provides an implementation of gst_base::BaseTransformClass::fixate_caps()

context

a GLContext to use for transforming caps

direction

a gst::PadDirection

caps

the gst::Caps of direction

other

the gst::Caps to fixate

Returns

the fixated gst::Caps

Provides an implementation of gst_base::BaseTransformClass::transform_caps()

context

a GLContext to use for transforming caps

direction

a gst::PadDirection

caps

the gst::Caps to transform

filter

a set of filter gst::Caps

Returns

the converted gst::Caps

Provides an implementation of GstBaseTransfromClass::decide_allocation()

query

a completed ALLOCATION gst::Query

Returns

whether the allocation parameters were successfully chosen

Converts the data contained by inbuf using the formats specified by the gst::Caps passed to GLColorConvert::set_caps

inbuf

the GLMemory filled gst::Buffer to convert

Returns

a converted gst::Buffer or None

Initializes self with the information required for conversion.

in_caps

input gst::Caps

out_caps

output gst::Caps

GLContext wraps an OpenGL context object in a uniform API. As a result of the limitation on OpenGL context, this object is not thread safe unless specified and must only be activated in a single thread.

Implements

GLContextExt, gst::ObjectExt, glib::object::ObjectExt

Trait containing all GLContext methods.

Implementors

GLContext

Create a new GLContext with the specified display

display

a GLDisplay

Returns

a new GLContext

Wraps an existing OpenGL context into a GLContext.

Note: The caller is responsible for ensuring that the OpenGL context represented by handle stays alive while the returned GLContext is active.

display

a GLDisplay

handle

the OpenGL context to wrap

context_type

a GLPlatform specifying the type of context in handle

available_apis

a GLAPI containing the available OpenGL apis in handle

Returns

a GLContext wrapping handle

A default implementation of the various GetProcAddress functions that looks for name in the OpenGL shared libraries or in the current process.

See also: GLContext::get_proc_address

gl_api

a GLAPI

name

then function to get the address of

Returns

an address pointing to name or None

See also GLContextExt::activate.

Returns

the GLContext active in the current thread or None

If an error occurs, major and minor are not modified and GLAPI::None is returned.

platform

the GLPlatform to retrieve the API for

major

the major version

minor

the minor version

Returns

The version supported by the OpenGL context current in the calling thread or GLAPI::None

context_type

a GLPlatform specifying the type of context to retrieve

Returns

The OpenGL context handle current in the calling thread or None

Attempts to use the context_type specific GetProcAddress implementations to retreive name.

See also GLContext::get_proc_address.

context_type

a GLPlatform

gl_api

a GLAPI

name

the name of the function to retrieve

Returns

a function pointer for name, or None

(De)activate the OpenGL context represented by this self.

In OpenGL terms, calls eglMakeCurrent or similar with this context and the currently set window. See GLContextExt::set_window for details.

activate

true to activate, false to deactivate

Returns

Whether the activation succeeded

Note: This will always fail for two wrapped GLContext's

other_context

another GLContext

Returns

whether self and other_context are able to share OpenGL resources.

Check for an OpenGL feature being supported.

Note: Most features require that the context be created before it is possible to determine their existence and so will fail if that is not the case.

feature

a platform specific feature

Returns

Whether feature is supported by self

fbo_target

the GL value of the framebuffer target, GL_FRAMEBUFFER, GL_READ_FRAMEBUFFER, GL_DRAW_FRAMEBUFFER

Returns

whether whether the current framebuffer is complete

api

api type required

maj

major version required

min

minor version required

Returns

whether OpenGL context implements the required api and specified version.

Unbind the current framebuffer

Clear's the currently set shader from the GL state machine.

Note: must be called in the GL thread.

Creates an OpenGL context with the specified other_context as a context to share shareable OpenGL objects with. See the OpenGL specification for what is shared between OpenGL contexts.

If an error occurs, and error is not None, then error will contain details of the error and false will be returned.

Should only be called once.

other_context

a GLContext to share OpenGL objects with

Returns

whether the context could successfully be created

Destroys an OpenGL context.

Should only be called after GLContextExt::create has been successfully called for this context.

Fills self's info (version, extensions, vtable, etc) from the GL context in the current thread. Typically used with wrapped contexts to allow wrapped contexts to be used as regular GLContext's.

Returns

the GLDisplay associated with this self

Get the currently enabled OpenGL api.

The currently available API may be limited by the GLDisplay in use and/or the GLWindow chosen.

Returns

the available OpenGL api

Gets the backing OpenGL context used by self.

Returns

The platform specific backing OpenGL context

Gets the OpenGL platform that used by self.

Returns

The platform specific backing OpenGL context

Get the version of the OpenGL platform (GLX, EGL, etc) used. Only valid after a call to gst_gl_context_create_context.

major

return for the major version

minor

return for the minor version

Returns the OpenGL version implemented by self. See GLContextExt::get_gl_api for retreiving the OpenGL api implemented by self.

maj

resulting major version

min

resulting minor version

Get a function pointer to a specified opengl function, name. If the the specific function does not exist, NULL is returned instead.

Platform specfic functions (names starting 'egl', 'glX', 'wgl', etc) can also be retrieved using this method.

Note: This function may return valid function pointers that may not be valid to call in self. The caller is responsible for ensuring that the returned function is a valid function to call in self by either checking the OpenGL API and version or for an appropriate OpenGL extension.

Note: On success, you need to cast the returned function pointer to the correct type to be able to call it correctly. On 32-bit Windows, this will include the GSTGLAPI identifier to use the correct calling convention. e.g.

void (GSTGLAPI *PFN_glGetIntegerv) (GLenum name, GLint * ret)

name

an opengl function name

Returns

a function pointer or None

Returns

The glib::Thread, self is current in or NULL

Returns

the currently set window

Returns

Whether the GLContext has been shared with another GLContext

Will internally set self as shared with share

share

another GLContext

Set's the current window on self to window. The window can only be changed before GLContextExt::create has been called and the window is not already running.

window

a GLWindow

Returns

Whether the window was successfully updated

version

a GLSLVersion

profile

a GLSLProfile

Returns

Whether self supports the combination of version with profile

Swap the front and back buffers on the window attached to self. This will display the frame on the next refresh cycle.

Execute func in the OpenGL thread of self with data

MT-safe

func

a GstGLContextThreadFunc

data

user data to call func with

OpenGL context errors.

Failed for an unspecified reason

The configuration requested is not correct

The OpenGL API requested is not correct

The OpenGL libraries are too old

glXCreateContext (or similar) failed

A resource is not available

GLDisplay represents a connection to the underlying windowing system. Elements are required to make use of gst::Context to share and propogate a GLDisplay.

There are a number of environment variables that influence the choice of platform and window system specific functionality.

• GST_GL_WINDOW influences the window system to use. Common values are 'x11', 'wayland', 'win32' or 'cocoa'.
• GST_GL_PLATFORM influences the OpenGL platform to use. Common values are 'egl', 'glx', 'wgl' or 'cgl'.
• GST_GL_API influences the OpenGL API requested by the OpenGL platform. Common values are 'opengl', 'opengl3' and 'gles2'.

Certain window systems require a special function to be called to initialize threading support. As this GStreamer GL library does not preclude concurrent access to the windowing system, it is strongly advised that applications ensure that threading support has been initialized before any other toolkit/library functionality is accessed. Failure to do so could result in sudden application abortion during execution. The most notably example of such a function is X11's XInitThreads().

Implements

GLDisplayExt, gst::ObjectExt, glib::object::ObjectExt

Trait containing all GLDisplay methods.

Implementors

GLDisplayEGL, GLDisplay

Returns

a new GLDisplay

context

a GLContext

Returns

whether context was successfully added. false may be returned if there already exists another context for context's active thread.

Must be called with the object lock held.

It requires the display's object lock to be held.

other_context

other GLContext to share resources with.

p_context

resulting GLContext

Returns

whether a new context could be created.

It requires the display's object lock to be held.

Returns

a new GLWindow for self or None.

limit the use of OpenGL to the requested gl_api. This is intended to allow application and elements to request a specific set of OpenGL API's based on what they support. See GLContextExt::get_gl_api for the retreiving the API supported by a GLContext.

gl_api

a GLAPI to filter with

Execute compare_func over the list of windows stored by self. The first argment to compare_func is the GLWindow being checked and the second argument is data.

data

some data to pass to compare_func

compare_func

a comparison function to run

Returns

The first GLWindow that causes a match from compare_func

see GLDisplayExt::filter_gl_api for what the returned value represents

Returns

the GLAPI configured for self

thread

a glib::Thread

Returns

the GLContext current on thread or None

Must be called with the object lock held.

Returns

the native handle for the display

Returns

the GLDisplayType of self

window

a GLWindow to remove

Returns

if window could be removed from self

Overrides the GLContext creation mechanism. It can be called in any thread and it is emitted with display's object lock held.

context

other context to share resources with.

Returns

the new context.

the contents of a GLDisplayEGL are private and should only be accessed through the provided API

Implements

GLDisplayExt, gst::ObjectExt, glib::object::ObjectExt

Create a new GLDisplayEGL using the default EGL_DEFAULT_DISPLAY.

Returns

a new GLDisplayEGL or None

Attempts to create a new EGLDisplay from display. If type_ is GLDisplayType::Any, then display must be 0. type_ must not be GLDisplayType::None.

type_

a GLDisplayType

display

pointer to a display (or 0)

Returns

A EGLDisplay or EGL_NO_DISPLAY

Single component replicated across R, G, and B textures components

Single component stored in the A texture component

Combination of GLFormat::Luminance and GLFormat::Alpha

Single component stored in the R texture component

Single 8-bit component stored in the R texture component

Two components stored in the R and G texture components

Two 8-bit components stored in the R and G texture components

Three components stored in the R, G, and B texture components

Three 8-bit components stored in the R, G, and B texture components

Three components of bit depth 5, 6 and 5 stored in the R, G, and B texture components respectively.

Four components stored in the R, G, B, and A texture components respectively.

Four 8-bit components stored in the R, G, B, and A texture components respectively.

A single 16-bit component for depth information.

A 24-bit component for depth information and a 8-bit component for stencil informat.

A GLFramebuffer represents and holds an OpenGL framebuffer object with it's associated attachments.

A GLFramebuffer can be created with GLFramebuffer::new or GLFramebuffer::new_with_default_depth and bound with GLFramebufferExt::bind. Other resources can be bound with GLFramebufferExt::attach

Note: OpenGL framebuffers are not shareable resources so cannot be used between multiple OpenGL contexts.

Implements

GLFramebufferExt, gst::ObjectExt, glib::object::ObjectExt

Trait containing all GLFramebuffer methods.

Implementors

GLFramebuffer

context

a GLContext

Returns

a new GLFramebuffer

context

a GLContext

width

width for the depth buffer

height

for the depth buffer

Returns

a new GLFramebuffer with a depth buffer of width and height

attach mem to attachment_point

attachment_point

the OpenGL attachment point to bind mem to

mem

the memory object to bind to attachment_point

Bind self into the current thread

Perform the steps necessary to have the output of a glDraw* command in func update the contents of mem.

mem

the GLMemory to draw to

func

the function to run

user_data

data to pass to func

Returns

the result of executing func

Retreive the effective dimensions from the current attachments attached to self.

width

output width

height

output height

Returns

the OpenGL id for self

Opaque GLOverlayCompositor object

Implements

gst::ObjectExt, glib::object::ObjectExt

A GLQuery represents and holds an OpenGL query object. Various types of queries can be run or counters retrieved.

Record the result of a counter

End counting the query

Frees a GLQuery

context

a GLContext

query_type

the GLQueryType

Returns

the result of the query

Start counting the query

Free any dynamically allocated resources

Performs a GST_QUERY_CONTEXT query of type "gst.gl.local_context" on all GstPads in element of direction for the local OpenGL context used by GStreamer elements.

element

a gst::Element to query from

direction

the gst::PadDirection to query

context_ptr

location containing the current and/or resulting GLContext

Returns

whether context_ptr contains a GLContext

Free with GLQuery::free

context

a GLContext

query_type

the GLQueryType to create

Returns

a new GLQuery

no query

query the time elapsed

query the current time

Compilation stage that caused an error

Compilation error occured

Link error occured

General program error occured

GLSLStage holds and represents a single OpenGL shader stage.

Implements

gst::ObjectExt, glib::object::ObjectExt

context

a GLContext

type_

the GL enum shader stage type

Returns

a new GLSLStage of the specified type_

context

a GLContext

type_

the GL enum shader stage type

version

the GLSLVersion

profile

the GLSLProfile

str

a shader string

Returns

a new GLSLStage of the specified type_

context

a GLContext

type_

the GL enum shader stage type

version

the GLSLVersion

profile

the GLSLProfile

n_strings

the number of strings in str

str

an array of strings concatted together to produce a shader

Returns

a new GLSLStage of the specified type_

Returns

whether the compilation suceeded

Returns

The GL handle for this shader stage

Returns

The GLSL profile for the current shader stage

Returns

The GL shader type for this shader stage

Returns

The GLSL version for the current shader stage

Replaces the current shader string with str.

version

a GLSLVersion

profile

a GLSLProfile

n_strings

number of strings in str

str

a GLSL shader string

GLSL version list

no version

version 100 (only valid for ES)

version 110 (only valid for compatibility desktop GL)

version 120 (only valid for compatibility desktop GL)

version 130 (only valid for compatibility desktop GL)

version 140 (only valid for compatibility desktop GL)

version 150 (valid for compatibility/core desktop GL)

version 300 (only valid for ES)

version 310 (only valid for ES)

version 320 (only valid for ES)

version 330 (valid for compatibility/core desktop GL)

version 400 (valid for compatibility/core desktop GL)

version 410 (valid for compatibility/core desktop GL)

version 420 (valid for compatibility/core desktop GL)

version 430 (valid for compatibility/core desktop GL)

version 440 (valid for compatibility/core desktop GL)

version 450 (valid for compatibility/core desktop GL)

Implements

gst::ObjectExt, glib::object::ObjectExt

Note: must be called in the GL thread

context

a GLContext

Returns

a new empty shader

Note: must be called in the GL thread

context

a GLContext

Returns

a default shader or None on failure

Each stage will attempt to be compiled and attached to shader. Then the shader will be linked. On error, None will be returned and error will contain the details of the error.

Note: must be called in the GL thread

context

a GLContext

error

a glib::Error

Returns

a new shader with the specified stages.

Each stage will attempt to be compiled and attached to shader. On error, None will be returned and error will contain the details of the error.

Note: must be called in the GL thread

context

a GLContext

error

a glib::Error

Returns

a new shader with the specified stages.

Attaches stage to self. stage must have been successfully compiled with GLSLStage::compile.

Note: must be called in the GL thread

stage

a GLSLStage to attach

Returns

whether stage could be attached to self

Attaches stage to self. stage must have been successfully compiled with GLSLStage::compile.

Note: must be called in the GL thread

stage

a GLSLStage to attach

Returns

whether stage could be attached to self

Bind attribute name to the specified location index using glBindAttributeLocation().

index

attribute index to set

name

name of the attribute

Bind attribute name to the specified location index using glBindFragDataLocation().

index

attribute index to set

name

name of the attribute

Compiles stage and attaches it to self.

Note: must be called in the GL thread

stage

a GLSLStage to attach

Returns

whether stage could be compiled and attached to self

Detaches stage from self. stage must have been successfully attached to self with GLShader::attach or GLShader::attach_unlocked.

Note: must be called in the GL thread

stage

a GLSLStage to attach

Detaches stage from self. stage must have been successfully attached to self with GLShader::attach or GLShader::attach_unlocked.

Note: must be called in the GL thread

stage

a GLSLStage to attach

name

name of the attribute

Returns

the attribute index for name in self or -1 on failure

Returns

the GL program handle for this shader

Note: must be called in the GL thread

Returns

whether self has been successfully linked

Links the current list of GLSLStage's in self.

Note: must be called in the GL thread

Returns

whether self could be linked together.

Releases the shader and stages.

Note: must be called in the GL thread

Releases the shader and stages.

Note: must be called in the GL thread

Perform glUniform1f() for name on self

name

name of the uniform

value

value to set

Perform glUniform1fv() for name on self

name

name of the uniform

count

number of values to set

value

values to set

Perform glUniform1i() for name on self

name

name of the uniform

value

value to set

Perform glUniform1iv() for name on self

name

name of the uniform

count

number of values to set

value

values to set

Perform glUniform2f() for name on self

name

name of the uniform

v0

first value to set

v1

second value to set

Perform glUniform2fv() for name on self

name

name of the uniform

count

number of values to set

value

values to set

Perform glUniform2i() for name on self

name

name of the uniform

v0

first value to set

v1

second value to set

Perform glUniform2iv() for name on self

name

name of the uniform

count

number of values to set

value

values to set

Perform glUniform3f() for name on self

name

name of the uniform

v0

first value to set

v1

second value to set

v2

third value to set

Perform glUniform3fv() for name on self

name

name of the uniform

count

number of values to set

value

values to set

Perform glUniform3i() for name on self

name

name of the uniform

v0

first value to set

v1

second value to set

v2

third value to set

Perform glUniform3iv() for name on self

name

name of the uniform

count

number of values to set

value

values to set

Perform glUniform4f() for name on self

name

name of the uniform

v0

first value to set

v1

second value to set

v2

third value to set

v3

fourth value to set

Perform glUniform4fv() for name on self

name

name of the uniform

count

number of values to set

value

values to set

Perform glUniform4i() for name on self

name

name of the uniform

v0

first value to set

v1

second value to set

v2

third value to set

v3

fourth value to set

Perform glUniform4iv() for name on self

name

name of the uniform

count

number of values to set

value

values to set

Perform glUniformMatrix2fv() for name on self

name

name of the uniform

count

number of 2x2 matrices to set

transpose

transpose the matrix

value

matrix to set

Perform glUniformMatrix2x3fv() for name on self

name

name of the uniform

count

number of 2x3 matrices to set

transpose

transpose the matrix

value

values to set

Perform glUniformMatrix2x4fv() for name on self

name

name of the uniform

count

number of 2x4 matrices to set

transpose

transpose the matrix

value

values to set

Perform glUniformMatrix3fv() for name on self

name

name of the uniform

count

number of 3x3 matrices to set

transpose

transpose the matrix

value

values to set

Perform glUniformMatrix3x2fv() for name on self

name

name of the uniform

count

number of 3x2 matrices to set

transpose

transpose the matrix

value

values to set

Perform glUniformMatrix3x4fv() for name on self

name

name of the uniform

count

number of 3x4 matrices to set

transpose

transpose the matrix

value

values to set

Perform glUniformMatrix4fv() for name on self

name

name of the uniform

count

number of 4x4 matrices to set

transpose

transpose the matrix

value

values to set

Perform glUniformMatrix4x2fv() for name on self

name

name of the uniform

count

number of 4x2 matrices to set

transpose

transpose the matrix

value

values to set

Perform glUniformMatrix4x3fv() for name on self

name

name of the uniform

count

number of 4x3 matrices to set

transpose

transpose the matrix

value

values to set

Mark's self as being used for the next GL draw command.

Note: must be called in the GL thread and self must have been linked.

Output anaglyph type to generate when downmixing to mono

no texture target

2D texture target

rectangle texture target

external oes texture target

GLUpload is an object that uploads data from system memory into GL textures.

A GLUpload can be created with GLUpload::new

Implements

gst::ObjectExt, glib::object::ObjectExt

context

a GLContext

Returns

a new GLUpload object

in_caps

the input gst::Caps

out_caps

the output gst::Caps

Uploads buffer using the transformation specified by GLUpload::set_caps creating a new gst::Buffer in outbuf_ptr.

buffer

input gst::Buffer

outbuf_ptr

resulting gst::Buffer

Returns

whether the upload was successful

Adds the required allocation parameters to support uploading.

decide_query

a gst::Query from a decide allocation

query

the proposed allocation query

Initializes self with the information required for upload.

in_caps

input gst::Caps

out_caps

output gst::Caps

Returns

whether in_caps and out_caps could be set on self

No further processing required

An unspecified error occured

The configuration is unsupported.

This element requires a reconfiguration.

Convert stereoscopic/multiview video using fragment shaders.

Implements

gst::ObjectExt, glib::object::ObjectExt

Returns

a new GLViewConvert

Provides an implementation of gst_base::BaseTransformClass::fixate_caps()

direction

a gst::PadDirection

caps

the gst::Caps of direction

othercaps

the gst::Caps to fixate

Returns

the fixated gst::Caps

Retrieve the processed output buffer placing the output in outbuf_ptr.

outbuf_ptr

a gst::Buffer

Returns

a gst::FlowReturn

Converts the data contained by inbuf using the formats specified by the gst::Caps passed to GLViewConvert::set_caps

inbuf

the GLMemory filled gst::Buffer to convert

Returns

a converted gst::Buffer or None

Reset self to the default state. Further operation will require setting the caps with GLViewConvert::set_caps.

Initializes self with the information required for conversion.

in_caps

input gst::Caps

out_caps

output gst::Caps

Set context on self

context

the GLContext to set

Submit input to be processed by self

is_discont

true if we have a discontinuity

input

a gst::Buffer

Returns

a gst::FlowReturn

Provides an implementation of gst_base::BaseTransformClass::transform_caps()

direction

a gst::PadDirection

caps

the gst::Caps to transform

filter

a set of filter gst::Caps

Returns

the converted gst::Caps

GstGLWindow represents a window that elements can render into. A window can either be a user visible window (onscreen) or hidden (offscreen).

Implements

GLWindowExt, gst::ObjectExt, glib::object::ObjectExt

Trait containing all GLWindow methods.

Implementors

GLWindow

display

a GLDisplay

Returns

a new GLWindow using display's connection

Redraw the window contents. Implementations should invoke the draw callback.

Returns

the GLContext associated with this self

Returns

the windowing system display handle for this self

width

resulting surface width

height

resulting surface height

Returns

the window handle we are currently rendering into

Tell a self that it should handle events from the window system. These events are forwarded upstream as navigation events. In some window systems events are not propagated in the window hierarchy if a client is listening for them. This method allows you to disable events handling completely from the self.

handle_events

a gboolean indicating if events should be handled or not.

Quit the runloop's execution.

Start the execution of the runloop.

Invoke callback with data on the window thread. callback is guarenteed to have executed when this function returns.

callback

function to invoke

data

data to invoke callback with

Invoke callback with data on the window thread. The callback may not have been executed when this function returns.

callback

function to invoke

data

data to invoke callback with

destroy

called when data is not needed anymore

Sets the callback called when the window is about to close.

callback

function to invoke

data

data to invoke callback with

destroy_notify

called when data is not needed any more

Sets the draw callback called everytime GLWindowExt::draw is called

callback

function to invoke

data

data to invoke callback with

destroy_notify

called when data is not needed any more

Set the preferred width and height of the window. Implementations are free to ignore this information.

width

new preferred width

height

new preferred height

Tell a self that it should render into a specific region of the window according to the gst_video::VideoOverlay interface.

x

x position

y

y position

width

width

height

height

Returns

whether the specified region could be set

Sets the resize callback called everytime a resize of the window occurs.

callback

function to invoke

data

data to invoke callback with

destroy_notify

called when data is not needed any more

Sets the window that this self should render into. Some implementations require this to be called with a valid handle before drawing can commence.

handle

handle to the window

Present the window to the screen.

Will be emitted when a key event is received by the GstGLwindow.

id

the name of the event

key

the id of the key pressed

Will be emitted when a mouse event is received by the GstGLwindow.

id

the name of the event

button

the id of the button

x

the x coordinate of the mouse event

y

the y coordinate of the mouse event

failed for a unspecified reason

the implementation is too old

no such resource was found