GStreamer bindings for Rust
Find a file
Marijn Suijten f5bafe5a07 examples/glupload: Provide new shared GLContext via pad probe
On certain GL drivers on Windows (e.g. those for Intel Arc) we see that
creation of a shared context based on our wrapped WGL context (from
`glutin`) fails with `ERROR_BUSY`:

    0:00:00.509113900 29460 000001E07A782CC0 DEBUG              glcontext gstglcontext.c:1227:gst_gl_context_create_thread:<glcontextwgl0> Creating thread
    0:00:00.509352100 29460 000001E07A782CC0 FIXME              glcontext gstglcontext.c:2041:gst_gl_wrapped_context_get_config:<glwrappedcontext0> wrapped context could not retrieve config. The application may be missing a call to gst_gl_context_fill_info() or the specific platform implemention is not implemented for retrieving the config from a wrapped OpenGL context.
    0:00:00.543157300 29460 000001E07A782CC0 INFO               glcontext gstglcontext_wgl.c:402:gst_gl_context_wgl_choose_format:<glcontextwgl0> chosen config gst-gl-context-config, platform=(GstGLPlatform)GST_GL_PLATFORM_WGL, red-size=(int)8, blue-size=(int)8, green-size=(int)8, alpha-size=(int)8, depth-size=(int)24, stencil-size=(int)8, native-visual-id=(uint)5, surface-type=(GstGLConfigSurfaceType)GST_GL_CONFIG_SURFACE_TYPE_WINDOW;
    0:00:00.543770000 29460 000001E07A782CC0 INFO               glcontext gstglcontext.c:1322:gst_gl_context_create_thread:<glcontextwgl0> Attempting to create opengl context. user chosen api(s) (any), compiled api support (opengl opengl3) display api (any)
    0:00:00.553201100 29460 000001E07A782CC0 DEBUG              glcontext gstglcontext_wgl.c:186:gst_gl_context_wgl_create_context: gl context created: 65537
    0:00:00.613589500 29460 000001E07A782CC0 DEBUG              glcontext gstglcontext_wgl.c:214:gst_gl_context_wgl_create_context:<glcontextwgl0> Available WGL extensions WGL_EXT_depth_float WGL_ARB_buffer_region WGL_ARB_extensions_string WGL_ARB_make_current_read WGL_ARB_pixel_format WGL_ARB_pbuffer WGL_EXT_extensions_string WGL_EXT_swap_control WGL_ARB_multisample WGL_ARB_pixel_format_float WGL_ARB_framebuffer_sRGB WGL_ARB_create_context WGL_ARB_create_context_profile WGL_EXT_pixel_format_packed_float WGL_EXT_create_context_es_profile WGL_EXT_create_context_es2_profile WGL_NV_DX_interop WGL_NV_DX_interop2 WGL_ARB_robustness_application_isolation WGL_ARB_robustness_share_group_isolation WGL_ARB_create_context_robustness WGL_ARB_context_flush_control
    0:00:00.614036500 29460 000001E07A782CC0 DEBUG              glcontext gstglcontext_wgl.c:235:gst_gl_context_wgl_create_context:<glcontextwgl0> trying to create a GL 4.5 context
    0:00:00.631649700 29460 000001E07A782CC0 DEBUG              glcontext gstglcontext_wgl.c:235:gst_gl_context_wgl_create_context:<glcontextwgl0> trying to create a GL 4.4 context
    0:00:00.650484600 29460 000001E07A782CC0 DEBUG              glcontext gstglcontext_wgl.c:235:gst_gl_context_wgl_create_context:<glcontextwgl0> trying to create a GL 4.3 context
    0:00:00.669332300 29460 000001E07A782CC0 DEBUG              glcontext gstglcontext_wgl.c:235:gst_gl_context_wgl_create_context:<glcontextwgl0> trying to create a GL 4.2 context
    0:00:00.687633700 29460 000001E07A782CC0 DEBUG              glcontext gstglcontext_wgl.c:235:gst_gl_context_wgl_create_context:<glcontextwgl0> trying to create a GL 4.1 context
    0:00:00.706444500 29460 000001E07A782CC0 DEBUG              glcontext gstglcontext_wgl.c:235:gst_gl_context_wgl_create_context:<glcontextwgl0> trying to create a GL 4.0 context
    0:00:00.725763400 29460 000001E07A782CC0 DEBUG              glcontext gstglcontext_wgl.c:235:gst_gl_context_wgl_create_context:<glcontextwgl0> trying to create a GL 3.3 context
    0:00:00.745413200 29460 000001E07A782CC0 DEBUG              glcontext gstglcontext_wgl.c:235:gst_gl_context_wgl_create_context:<glcontextwgl0> trying to create a GL 3.2 context
    0:00:00.781065300 29460 000001E07A782CC0 WARN               glcontext gstglcontext.c:1326:gst_gl_context_create_thread:<glcontextwgl0> Failed to create context
    0:00:00.781358600 29460 000001E076862100 INFO               glcontext gstglcontext.c:1079:gst_gl_context_create:<glcontextwgl0> gl thread created
    0:00:00.781584200 29460 000001E076862100 DEBUG              glcontext gstglcontext.c:746:gst_gl_context_finalize:<glcontextwgl0> End of finalize
    0:00:00.781787700 29460 000001E076862100 WARN            glbasefilter gstglbasefilter.c:608:gst_gl_base_filter_find_gl_context_unlocked:<gluploadelement0> error: failed to share contexts through wglShareLists 0xaa
    0:00:00.782145700 29460 000001E076862100 INFO               glcontext gstglcontext.c:349:gst_gl_context_new: creating a context for display <gldisplay0>, user choice:(null)
    0:00:00.782381300 29460 000001E076862100 DEBUG              glcontext gstglcontext.c:383:gst_gl_context_new:<glcontextwgl1> Done creating context for display <gldisplay0> (user_choice:(null))
    0:00:00.782632900 29460 000001E076862100 DEBUG              glcontext gstglcontext.c:1058:gst_gl_context_create:<glcontextwgl1>  other_context:<glwrappedcontext0>
    0:00:00.782921300 29460 000001E076862100 INFO                glwindow gstglwindow.c:295:gst_gl_window_new: creating a window, user choice:(null)
    0:00:00.783246300 29460 000001E076862100 DEBUG              glcontext gstglcontext.c:956:gst_gl_context_set_window:<glcontextwgl1> window:<glwindowwin32-1>
    0:00:00.783815200 29460 000001E07A782D00 DEBUG              glcontext gstglcontext.c:1227:gst_gl_context_create_thread:<glcontextwgl1> Creating thread
    0:00:00.784130000 29460 000001E07A782D00 FIXME              glcontext gstglcontext.c:2041:gst_gl_wrapped_context_get_config:<glwrappedcontext0> wrapped context could not retrieve config. The application may be missing a call to gst_gl_context_fill_info() or the specific platform implemention is not implemented for retrieving the config from a wrapped OpenGL context.
    thread 'main' panicked at examples\src\bin\..\glupload.rs:755:44:
    called `Result::unwrap()` on an `Err` value: Received error from /GstPipeline:pipeline0/GstGLSinkBin:glsinkbin0/GstGLUploadElement:gluploadelement0: failed to share contexts through wglShareLists 0xaa (debug: Some("../gst-libs/gst/gl/gstglbasefilter.c(608): gst_gl_base_filter_find_gl_context_unlocked (): /GstPipeline:pipeline0/GstGLSinkBin:glsinkbin0/GstGLUploadElement:gluploadelement0"))

This happens because the context is made "current" inside
the `winit` loop before asynchronous GL initialization inside
GStreamer creates a shared context for contexts replied to
`NeedContext("gst.gl.app_context")`.  `wglShareLists()` requires neither
context to be current on separate threads.  As we have a `Surface` on
the `glutin` side, we're not uncurrenting it anymore, and instead want
to perform GStreamer initialization up-front without a thread, or on a
separate `GLContext`.

One way to prevent GStreamer from creating yet another context on top
of our wrapped context asynchronously, is by creating a new shared GL
context based on our wrapped (WGL) context ourselves.

By adding that context to `GLDisplay` (that we provide in the
relevant `gst.gl.GLDisplay` context query) instead of via the
`gst.gl.app_context` `NeedContext` query, the underlying initialization
code no longer attempts to create a shared context because it
will use the one from `GLDisplay` directly (specifically in e.g.
`gst_gl_base_filter_find_gl_context_unlocked()`).

The preferred way however is to reply this new shared context to
a `Query` for the `gst.gl.local_context` context, which arbitrary
applications can achieve by inserting a pad probe on one of the pads
in the pipeline. GL elements and functions like the one mentioned above
call into `gst_gl_query_local_gl_context()` which performs this query in
both directions, before reading contexts from `GLDisplay` or ultimately
sending a `NeedContext` message outlined above.

With this patch the initialization flow becomes clear from the logs,
where the above lines that will call `wglShareLists()` are now called
directly when `gl_context.create(wrapped_context)` is called in our
Rust code.

In theory, since `GLContext` tracks whether it is current and on
which thread, the upsteam source should emit an error of sorts when
`wglShareLists()` is called on a thread where this `other_context` in
`gst_gl_context_create(_thread)` was not yet current, to not make such
mistakes go unnoticed as most drivers (i.e. AMD's) seem to not make an
issue out of it.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer-rs/-/merge_requests/1486>
2024-10-30 10:15:19 +01:00
.config ci: Use cargo nextest as the test runner and export junit reports 2024-10-03 00:17:50 +03:00
.gitlab/issue_templates gitlab: add issue template 2023-06-15 11:37:18 +00:00
ci Update minimum supported Rust version to 1.80 2024-10-20 17:48:44 +00:00
examples examples/glupload: Provide new shared GLContext via pad probe 2024-10-30 10:15:19 +01:00
gir@a172dda2a7 Update gtk-rs dependencies from master to main branch 2024-10-22 11:31:19 +00:00
gir-files@f7af0e7113 Update gtk-rs dependencies from master to main branch 2024-10-22 11:31:19 +00:00
gst-gir-files@05cc12709e Update GStreamer gir files 2024-10-20 17:48:45 +00:00
gstreamer Regenerate with latest gir 2024-10-22 11:31:19 +00:00
gstreamer-allocators Regenerate with latest gir 2024-10-22 11:31:19 +00:00
gstreamer-analytics Regenerate with latest gir 2024-10-22 11:31:19 +00:00
gstreamer-app Regenerate with latest gir 2024-10-22 11:31:19 +00:00
gstreamer-audio Regenerate with latest gir 2024-10-22 11:31:19 +00:00
gstreamer-base Regenerate with latest gir 2024-10-22 11:31:19 +00:00
gstreamer-check Regenerate with latest gir 2024-10-22 11:31:19 +00:00
gstreamer-controller Regenerate with latest gir 2024-10-22 11:31:19 +00:00
gstreamer-editing-services Regenerate with latest gir 2024-10-22 11:31:19 +00:00
gstreamer-gl Regenerate with latest gir 2024-10-22 11:31:19 +00:00
gstreamer-mpegts Regenerate with latest gir 2024-10-22 11:31:19 +00:00
gstreamer-net Regenerate with latest gir 2024-10-22 11:31:19 +00:00
gstreamer-pbutils Regenerate with latest gir 2024-10-22 11:31:19 +00:00
gstreamer-play play: Improve API of PlayMessage 2024-10-28 11:29:37 +00:00
gstreamer-player Regenerate with latest gir 2024-10-22 11:31:19 +00:00
gstreamer-rtp Regenerate with latest gir 2024-10-22 11:31:19 +00:00
gstreamer-rtsp Regenerate with latest gir 2024-10-22 11:31:19 +00:00
gstreamer-rtsp-server Regenerate with latest gir 2024-10-22 11:31:19 +00:00
gstreamer-sdp Regenerate with latest gir 2024-10-22 11:31:19 +00:00
gstreamer-tag Regenerate with latest gir 2024-10-22 11:31:19 +00:00
gstreamer-utils Remove once_cell dependency 2024-10-21 17:53:15 +00:00
gstreamer-validate validate: tests: Handle the fact that tests can run in parallel in a same process 2024-10-22 14:47:26 +00:00
gstreamer-video Regenerate with latest gir 2024-10-22 11:31:19 +00:00
gstreamer-webrtc Regenerate with latest gir 2024-10-22 11:31:19 +00:00
public Add more redirects for old documentation links 2021-05-29 18:56:03 +03:00
tutorials Add workaround for linking against macOS SDK's relocatable dylibs 2024-09-10 16:48:29 +00:00
.gitignore Add .helix to .gitignore 2024-10-04 12:12:10 +00:00
.gitlab-ci.yml examples: Update to derive_more 1.0 2024-10-20 17:48:45 +00:00
.gitmodules Update gir submodule 2023-07-06 14:08:32 +03:00
Cargo.lock Update gtk-rs dependencies from master to main branch 2024-10-22 11:31:19 +00:00
Cargo.toml Update gtk-rs dependencies from master to main branch 2024-10-22 11:31:19 +00:00
COPYRIGHT Add a COPYRIGHT file that describes the general copyright and license situation 2020-12-16 12:42:33 +02:00
deny.toml deny: Remove toml_edit override 2024-08-27 20:31:50 +03:00
DEVELOPMENT.md gir-files/fix: Autoamtically delete duplicate ONEFIELDs in GstVideo.gir 2020-11-25 22:44:18 +01:00
generator.py generator.py: Replace with link to gir/generator.py 2021-04-24 09:39:04 +02:00
LICENSE-APACHE Add README.md 2017-08-01 19:27:56 +03:00
LICENSE-MIT Add README.md 2017-08-01 19:27:56 +03:00
README.md Add some more information about setup on Windows/macOS to the README.md 2022-10-14 21:26:12 +05:30
rustfmt.toml rustfmt.toml: Add file 2022-04-06 07:46:29 +00:00
typos.toml typos.toml: Ignore anc and sys/tests 2024-04-05 11:59:08 +03:00

gstreamer-rs crates.io pipeline status

GStreamer bindings for Rust. Documentation can be found here.

These bindings are providing a safe API that can be used to interface with GStreamer, e.g. for writing GStreamer-based applications and GStreamer plugins.

The bindings are mostly autogenerated with gir based on the GObject-Introspection API metadata provided by the GStreamer project.

Table of Contents

  1. Installation
    1. Linux/BSDs
    2. macOS
    3. Windows
  2. Getting Started
  3. License
  4. Contribution

Installation

To build the GStreamer bindings or anything depending on them, you need to have at least GStreamer 1.14 and gst-plugins-base 1.14 installed. In addition, some of the examples/tutorials require various GStreamer plugins to be available, which can be found in gst-plugins-base, gst-plugins-good, gst-plugins-bad, gst-plugins-ugly and/or gst-libav.

Linux/BSDs

You need to install the above mentioned packages with your distributions package manager, or build them from source.

On Debian/Ubuntu they can be installed with

$ apt-get install libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev \
      gstreamer1.0-plugins-base gstreamer1.0-plugins-good \
      gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly \
      gstreamer1.0-libav libgstrtspserver-1.0-dev libges-1.0-dev

The minimum required version of the above libraries is >= 1.14. If you build the gstreamer-player sub-crate, or any of the examples that depend on gstreamer-player, you must ensure that in addition to the above packages, libgstreamer-plugins-bad1.0-dev is installed. See the Cargo.toml files for the full details,

$ apt-get install libgstreamer-plugins-bad1.0-dev

Package names on other distributions should be similar. Please submit a pull request with instructions for yours.

macOS

You can install GStreamer and the plugins via Homebrew or by installing the binaries provided by the GStreamer project.

We recommend using the official GStreamer binaries over Homebrew, especially as GStreamer in Homebrew is currently broken.

GStreamer Binaries

You need to download the two .pkg files from the GStreamer website and install them, e.g. gstreamer-1.0-1.20.4-universal.pkg and gstreamer-1.0-devel-1.20.4-universal.pkg.

After installation, you also need to set the PATH environment variable as follows

$ export PATH="/Library/Frameworks/GStreamer.framework/Versions/1.0/bin${PATH:+:$PATH}"

Also note that the pkg-config from GStreamer should be the first one in the PATH as other versions have all kinds of quirks that will cause problems.

Homebrew

Homebrew only installs various plugins if explicitly enabled, so some extra --with-* flags may be required.

$ brew install gstreamer gst-plugins-base gst-plugins-good \
      gst-plugins-bad gst-plugins-ugly gst-libav gst-rtsp-server \
      gst-editing-services --with-orc --with-libogg --with-opus \
      --with-pango --with-theora --with-libvorbis --with-libvpx \
      --enable-gtk3

Make sure the version of these libraries is >= 1.14.

Windows

You can install GStreamer and the plugins via MSYS2 with pacman or by installing the binaries provided by the GStreamer project.

We recommend using the official GStreamer binaries over MSYS2.

GStreamer Binaries

You need to download the two .msi files for your platform from the GStreamer website and install them, e.g. gstreamer-1.0-x86_64-1.20.4.msi and gstreamer-1.0-devel-x86_64-1.20.4.msi. Make sure to select the version that matches your Rust toolchain, i.e. MinGW or MSVC.

After installation set the ``PATH` environment variable as follows:

# For a UNIX-style shell:
$ export PATH="c:/gstreamer/1.0/msvc_x86_64/bin${PATH:+:$PATH}"

# For cmd.exe:
$ set PATH=C:\gstreamer\1.0\msvc_x86_64\bin;%PATH%

Make sure to update the path to where you have actually installed GStreamer and for the corresponding toolchain.

Also note that the pkg-config.exe from GStreamer should be the first one in the PATH as other versions have all kinds of quirks that will cause problems.

MSYS2 / pacman

$ pacman -S glib2-devel pkg-config \
      mingw-w64-x86_64-gstreamer mingw-w64-x86_64-gst-plugins-base \
      mingw-w64-x86_64-gst-plugins-good mingw-w64-x86_64-gst-plugins-bad \
      mingw-w64-x86_64-gst-plugins-ugly mingw-w64-x86_64-gst-libav \
      mingw-w64-x86_64-gst-rtsp-server

Make sure the version of these libraries is >= 1.14.

Note that the version of pkg-config included in MSYS2 is known to have problems compiling GStreamer, so you may need to install another version. One option would be pkg-config-lite.

Getting Started

The API reference can be found here, however it is only the Rust API reference and does not explain any of the concepts.

For getting started with GStreamer development, the best would be to follow the documentation on the GStreamer website, especially the Application Development Manual. While being C-centric, it explains all the fundamental concepts of GStreamer and the code examples should be relatively easily translatable to Rust. The API is basically the same, function/struct names are the same and everything is only more convenient (hopefully) and safer.

In addition there are tutorials on the GStreamer website. Many of them were ported to Rust already and the code can be found in the tutorials directory.

Some further examples for various aspects of GStreamer and how to use it from Rust can be found in the examples directory.

Various GStreamer plugins written in Rust can be found in the gst-plugins-rs repository.

LICENSE

gstreamer-rs and all crates contained in here are licensed under either of

at your option.

GStreamer itself is licensed under the Lesser General Public License version 2.1 or (at your option) any later version: https://www.gnu.org/licenses/lgpl-2.1.html

Contribution

Any kinds of contributions are welcome as a pull request.

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in gstreamer-rs by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.