mirror of
https://gitlab.freedesktop.org/gstreamer/gstreamer.git
synced 2024-09-08 13:18:52 +00:00
GStreamer multimedia framework
Add a way to support drawing on application's texture instead of usual window handle. To make use of this new feature, application should follow below step. 1) Enable this feature by using "draw-on-shared-texture" property 2) Watch "begin-draw" signal 3) On "begin-draw" signal handler, application can request drawing by using "draw" signal action. Note that "draw" signal action should be happen before "begin-draw" signal handler is returned NOTE 1) For texture sharing, creating a texture with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend if possible because we cannot ensure sync a texture which was created with D3D11_RESOURCE_MISC_SHARED and it would cause glitch with ID3D11VideoProcessor use case. NOTE 2) Direct9Ex doesn't support texture sharing which was created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words, D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop. NOTE 3) Because of missing synchronization around ID3D11VideoProcessor, If shared texture was created with D3D11_RESOURCE_MISC_SHARED, d3d11videosink might use fallback texture to convert DXVA texture to normal Direct3D texture. Then converted texture will be copied to user-provided shared texture. * Why not use generic appsink approach? In order for application to be able to store video data which was produced by GStreamer in application's own texture, there would be two possible approaches, one is copying our texture into application's own texture, and the other is drawing on application's own texture directly. The former (appsink way) cannot be a zero-copy by nature. In order to support zero-copy processing, we need to draw on application's own texture directly. For example, assume that application wants RGBA texture. Then we can imagine following case. "d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink" ^ |_ allocate new Direct3D texture for RGBA format In above case, d3d11convert will allocate new texture(s) for RGBA format and then application will copy again the our RGBA texutre into application's own texture. One texture allocation plus per frame GPU copy will hanppen in that case therefore. Moreover, in order for application to be able to access our texture, we need to allocate texture with additional flags for application's Direct3D11 device to be able to read texture data. That would be another implementation burden on our side But with this MR, we can configure pipeline in this way "d3d11h264dec ! d3d11videosink". In that way, we can save at least one texture allocation and per frame texutre copy since d3d11videosink will convert incoming texture into application's texture format directly without copy. * What if we expose texture without conversion and application does conversion by itself? As mentioned above, for application to be able to access our texture from application's Direct3D11 device, we need to allocate texture in a special form. But in some case, that might not be possible. Also, if a texture belongs to decoder DPB, exposing such texture to application is unsafe and usual Direct3D11 shader cannot handle such texture. To convert format, ID3D11VideoProcessor API needs to be used but that would be a implementation burden for application. Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873> |
||
---|---|---|
data | ||
docs | ||
ext | ||
gst | ||
gst-libs | ||
hooks | ||
po | ||
scripts | ||
sys | ||
tests | ||
tools | ||
.gitignore | ||
.gitlab-ci.yml | ||
.indentignore | ||
AUTHORS | ||
ChangeLog | ||
COPYING | ||
gst-plugins-bad.doap | ||
MAINTAINERS | ||
meson.build | ||
meson_options.txt | ||
NEWS | ||
README | ||
README.static-linking | ||
RELEASE | ||
REQUIREMENTS |
GStreamer 1.18.x stable series WHAT IT IS ---------- This is GStreamer, a framework for streaming media. WHERE TO START -------------- We have a website at https://gstreamer.freedesktop.org Our documentation, including tutorials, API reference and FAQ can be found at https://gstreamer.freedesktop.org/documentation/ You can subscribe to our mailing lists: https://lists.freedesktop.org/mailman/listinfo/gstreamer-announce https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel We track bugs, feature requests and merge requests (patches) in GitLab at https://gitlab.freedesktop.org/gstreamer/ You can join us on IRC - #gstreamer on irc.freenode.org GStreamer 1.0 series -------------------- Starring GSTREAMER The core around which all other modules revolve. Base functionality and libraries, some essential elements, documentation, and testing. BASE A well-groomed and well-maintained collection of GStreamer plug-ins and elements, spanning the range of possible types of elements one would want to write for GStreamer. And introducing, for the first time ever, on the development screen ... THE GOOD --- "Such ingratitude. After all the times I've saved your life." A collection of plug-ins you'd want to have right next to you on the battlefield. Shooting sharp and making no mistakes, these plug-ins have it all: good looks, good code, and good licensing. Documented and dressed up in tests. If you're looking for a role model to base your own plug-in on, here it is. If you find a plot hole or a badly lip-synced line of code in them, let us know - it is a matter of honour for us to ensure Blondie doesn't look like he's been walking 100 miles through the desert without water. THE UGLY --- "When you have to shoot, shoot. Don't talk." There are times when the world needs a color between black and white. Quality code to match the good's, but two-timing, backstabbing and ready to sell your freedom down the river. These plug-ins might have a patent noose around their neck, or a lock-up license, or any other problem that makes you think twice about shipping them. We don't call them ugly because we like them less. Does a mother love her son less because he's not as pretty as the other ones ? No - she commends him on his great personality. These plug-ins are the life of the party. And we'll still step in and set them straight if you report any unacceptable behaviour - because there are two kinds of people in the world, my friend: those with a rope around their neck and the people who do the cutting. THE BAD --- "That an accusation?" No perfectly groomed moustache or any amount of fine clothing is going to cover up the truth - these plug-ins are Bad with a capital B. They look fine on the outside, and might even appear to get the job done, but at the end of the day they're a black sheep. Without a golden-haired angel to watch over them, they'll probably land in an unmarked grave at the final showdown. Don't bug us about their quality - exercise your Free Software rights, patch up the offender and send us the patch on the fastest steed you can steal from the Confederates. Because you see, in this world, there's two kinds of people, my friend: those with loaded guns and those who dig. You dig. The Lowdown ----------- --- "I've never seen so many plug-ins wasted so badly." GStreamer Plug-ins has grown so big that it's hard to separate the wheat from the chaff. Also, distributors have brought up issues about the legal status of some of the plug-ins we ship. To remedy this, we've divided the previous set of available plug-ins into four modules: - gst-plugins-base: a small and fixed set of plug-ins, covering a wide range of possible types of elements; these are continuously kept up-to-date with any core changes during the development series. - We believe distributors can safely ship these plug-ins. - People writing elements should base their code on these elements. - These elements come with examples, documentation, and regression tests. - gst-plugins-good: a set of plug-ins that we consider to have good quality code, correct functionality, our preferred license (LGPL for the plug-in code, LGPL or LGPL-compatible for the supporting library). - We believe distributors can safely ship these plug-ins. - People writing elements should base their code on these elements. - gst-plugins-ugly: a set of plug-ins that have good quality and correct functionality, but distributing them might pose problems. The license on either the plug-ins or the supporting libraries might not be how we'd like. The code might be widely known to present patent problems. - Distributors should check if they want/can ship these plug-ins. - People writing elements should base their code on these elements. - gst-plugins-bad: a set of plug-ins that aren't up to par compared to the rest. They might be close to being good quality, but they're missing something - be it a good code review, some documentation, a set of tests, a real live maintainer, or some actual wide use. If the blanks are filled in they might be upgraded to become part of either gst-plugins-good or gst-plugins-ugly, depending on the other factors. - If the plug-ins break, you can't complain - instead, you can fix the problem and send us a patch, or bribe someone into fixing them for you. - New contributors can start here for things to work on. PLATFORMS --------- - Linux is of course fully supported - FreeBSD is reported to work; other BSDs should work too; same for Solaris - MacOS works, binary 1.x packages can be built using the cerbero build tool - Windows works; binary 1.x packages can be built using the cerbero build tool - MSys/MinGW builds - Microsoft Visual Studio builds are also available and supported - Android works, binary 1.x packages can be built using the cerbero build tool - iOS works INSTALLING FROM PACKAGES ------------------------ You should always prefer installing from packages first. GStreamer is well-maintained for a number of distributions, including Fedora, Debian, Ubuntu, Mandrake, Arch Linux, Gentoo, ... Only in cases where you: - want to hack on GStreamer - want to verify that a bug has been fixed - do not have a sane distribution should you choose to build from source tarballs or git. Find more information about the various packages at https://gstreamer.freedesktop.org/download/ COMPILING FROM SOURCE TARBALLS ------------------------------ - again, make sure that you really need to install from source! If GStreamer is one of your first projects ever that you build from source, consider taking on an easier project. - you need a recent version of Meson installed, see http://mesonbuild.com/Getting-meson.html and https://gitlab.freedesktop.org/gstreamer/gst-build/blob/master/README.md - run meson build ninja -C build to build GStreamer. - if you want to install it (not required, but what you usually want to do), run ninja -C build install - try out a simple test: gst-launch-1.0 -v fakesrc num_buffers=5 ! fakesink (If you didn't install GStreamer, run `./build/tools/gst-launch-1.0`) If it outputs a bunch of messages from fakesrc and fakesink, everything is ok. If it did not work, keep in mind that you might need to adjust the PATH and/or LD_LIBRARY_PATH environment variables to make the system find GStreamer in the prefix where you installed (by default that is /usr/local). - After this, you're ready to install gst-plugins, which will provide the functionality you're probably looking for by now, so go on and read that README. COMPILING FROM GIT ------------------ You can build an uninstalled GStreamer from git for development or testing purposes without affecting your system installation. Get started with: git clone https://gitlab.freedesktop.org/gstreamer/gst-build meson build ninja -C build ninja -C build uninstalled For more information, see the `gst-build` module and its documentation: https://gitlab.freedesktop.org/gstreamer/gst-build/blob/master/README.md PLUG-IN DEPENDENCIES AND LICENSES --------------------------------- GStreamer is developed under the terms of the LGPL (see COPYING file for details). Some of our plug-ins however rely on libraries which are available under other licenses. This means that if you are distributing an application which has a non-GPL compatible license (for instance a closed-source application) with GStreamer, you have to make sure not to distribute GPL-linked plug-ins. When using GPL-linked plug-ins, GStreamer is for all practical reasons under the GPL itself. HISTORY ------- The fundamental design comes from the video pipeline at Oregon Graduate Institute, as well as some ideas from DirectMedia. It's based on plug-ins that will provide the various codec and other functionality. The interface hopefully is generic enough for various companies (ahem, Apple) to release binary codecs for Linux, until such time as they get a clue and release the source.