mirror of
https://gitlab.freedesktop.org/gstreamer/gstreamer.git
synced 2024-11-28 04:31:06 +00:00
GStreamer multimedia framework
60e223f4fd
Add a way to support drawing on application's texture instead of
usual window handle.
To make use of this new feature, application should follow below step.
1) Enable this feature by using "draw-on-shared-texture" property
2) Watch "begin-draw" signal
3) On "begin-draw" signal handler, application can request drawing
by using "draw" signal action. Note that "draw" signal action
should be happen before "begin-draw" signal handler is returned
NOTE 1) For texture sharing, creating a texture with
D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX flag is strongly recommend
if possible because we cannot ensure sync a texture
which was created with D3D11_RESOURCE_MISC_SHARED
and it would cause glitch with ID3D11VideoProcessor use case.
NOTE 2) Direct9Ex doesn't support texture sharing which was
created with D3D11_RESOURCE_MISC_SHARED_KEYEDMUTEX. In other words,
D3D11_RESOURCE_MISC_SHARED is the only option for Direct3D11/Direct9Ex interop.
NOTE 3) Because of missing synchronization around ID3D11VideoProcessor,
If shared texture was created with D3D11_RESOURCE_MISC_SHARED,
d3d11videosink might use fallback texture to convert DXVA texture
to normal Direct3D texture. Then converted texture will be
copied to user-provided shared texture.
* Why not use generic appsink approach?
In order for application to be able to store video data
which was produced by GStreamer in application's own texture,
there would be two possible approaches,
one is copying our texture into application's own texture,
and the other is drawing on application's own texture directly.
The former (appsink way) cannot be a zero-copy by nature.
In order to support zero-copy processing, we need to draw on
application's own texture directly.
For example, assume that application wants RGBA texture.
Then we can imagine following case.
"d3d11h264dec ! d3d11convert ! video/x-raw(memory:D3D11Memory),format=RGBA ! appsink"
^
|_ allocate new Direct3D texture for RGBA format
In above case, d3d11convert will allocate new texture(s) for RGBA format
and then application will copy again the our RGBA texutre into
application's own texture. One texture allocation plus per frame GPU copy will hanppen
in that case therefore.
Moreover, in order for application to be able to access
our texture, we need to allocate texture with additional flags for
application's Direct3D11 device to be able to read texture data.
That would be another implementation burden on our side
But with this MR, we can configure pipeline in this way
"d3d11h264dec ! d3d11videosink".
In that way, we can save at least one texture allocation and
per frame texutre copy since d3d11videosink will convert incoming texture
into application's texture format directly without copy.
* What if we expose texture without conversion and application does
conversion by itself?
As mentioned above, for application to be able to access our texture
from application's Direct3D11 device, we need to allocate texture
in a special form. But in some case, that might not be possible.
Also, if a texture belongs to decoder DPB, exposing such texture
to application is unsafe and usual Direct3D11 shader cannot handle
such texture. To convert format, ID3D11VideoProcessor API needs to
be used but that would be a implementation burden for application.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1873>
|
||
|---|---|---|
| data | ||
| docs | ||
| ext | ||
| gst | ||
| gst-libs | ||
| hooks | ||
| po | ||
| scripts | ||
| sys | ||
| tests | ||
| tools | ||
| .gitignore | ||
| .gitlab-ci.yml | ||
| .indentignore | ||
| AUTHORS | ||
| ChangeLog | ||
| COPYING | ||
| gst-plugins-bad.doap | ||
| MAINTAINERS | ||
| meson.build | ||
| meson_options.txt | ||
| NEWS | ||
| README | ||
| README.static-linking | ||
| RELEASE | ||
| REQUIREMENTS | ||
GStreamer 1.18.x stable series
WHAT IT IS
----------
This is GStreamer, a framework for streaming media.
WHERE TO START
--------------
We have a website at
https://gstreamer.freedesktop.org
Our documentation, including tutorials, API reference and FAQ can be found at
https://gstreamer.freedesktop.org/documentation/
You can subscribe to our mailing lists:
https://lists.freedesktop.org/mailman/listinfo/gstreamer-announce
https://lists.freedesktop.org/mailman/listinfo/gstreamer-devel
We track bugs, feature requests and merge requests (patches) in GitLab at
https://gitlab.freedesktop.org/gstreamer/
You can join us on IRC - #gstreamer on irc.freenode.org
GStreamer 1.0 series
--------------------
Starring
GSTREAMER
The core around which all other modules revolve. Base functionality and
libraries, some essential elements, documentation, and testing.
BASE
A well-groomed and well-maintained collection of GStreamer plug-ins and
elements, spanning the range of possible types of elements one would want
to write for GStreamer.
And introducing, for the first time ever, on the development screen ...
THE GOOD
--- "Such ingratitude. After all the times I've saved your life."
A collection of plug-ins you'd want to have right next to you on the
battlefield. Shooting sharp and making no mistakes, these plug-ins have it
all: good looks, good code, and good licensing. Documented and dressed up
in tests. If you're looking for a role model to base your own plug-in on,
here it is.
If you find a plot hole or a badly lip-synced line of code in them,
let us know - it is a matter of honour for us to ensure Blondie doesn't look
like he's been walking 100 miles through the desert without water.
THE UGLY
--- "When you have to shoot, shoot. Don't talk."
There are times when the world needs a color between black and white.
Quality code to match the good's, but two-timing, backstabbing and ready to
sell your freedom down the river. These plug-ins might have a patent noose
around their neck, or a lock-up license, or any other problem that makes you
think twice about shipping them.
We don't call them ugly because we like them less. Does a mother love her
son less because he's not as pretty as the other ones ? No - she commends
him on his great personality. These plug-ins are the life of the party.
And we'll still step in and set them straight if you report any unacceptable
behaviour - because there are two kinds of people in the world, my friend:
those with a rope around their neck and the people who do the cutting.
THE BAD
--- "That an accusation?"
No perfectly groomed moustache or any amount of fine clothing is going to
cover up the truth - these plug-ins are Bad with a capital B.
They look fine on the outside, and might even appear to get the job done, but
at the end of the day they're a black sheep. Without a golden-haired angel
to watch over them, they'll probably land in an unmarked grave at the final
showdown.
Don't bug us about their quality - exercise your Free Software rights,
patch up the offender and send us the patch on the fastest steed you can
steal from the Confederates. Because you see, in this world, there's two
kinds of people, my friend: those with loaded guns and those who dig.
You dig.
The Lowdown
-----------
--- "I've never seen so many plug-ins wasted so badly."
GStreamer Plug-ins has grown so big that it's hard to separate the wheat from
the chaff. Also, distributors have brought up issues about the legal status
of some of the plug-ins we ship. To remedy this, we've divided the previous
set of available plug-ins into four modules:
- gst-plugins-base: a small and fixed set of plug-ins, covering a wide range
of possible types of elements; these are continuously kept up-to-date
with any core changes during the development series.
- We believe distributors can safely ship these plug-ins.
- People writing elements should base their code on these elements.
- These elements come with examples, documentation, and regression tests.
- gst-plugins-good: a set of plug-ins that we consider to have good quality
code, correct functionality, our preferred license (LGPL for the plug-in
code, LGPL or LGPL-compatible for the supporting library).
- We believe distributors can safely ship these plug-ins.
- People writing elements should base their code on these elements.
- gst-plugins-ugly: a set of plug-ins that have good quality and correct
functionality, but distributing them might pose problems. The license
on either the plug-ins or the supporting libraries might not be how we'd
like. The code might be widely known to present patent problems.
- Distributors should check if they want/can ship these plug-ins.
- People writing elements should base their code on these elements.
- gst-plugins-bad: a set of plug-ins that aren't up to par compared to the
rest. They might be close to being good quality, but they're missing
something - be it a good code review, some documentation, a set of tests,
a real live maintainer, or some actual wide use.
If the blanks are filled in they might be upgraded to become part of
either gst-plugins-good or gst-plugins-ugly, depending on the other factors.
- If the plug-ins break, you can't complain - instead, you can fix the
problem and send us a patch, or bribe someone into fixing them for you.
- New contributors can start here for things to work on.
PLATFORMS
---------
- Linux is of course fully supported
- FreeBSD is reported to work; other BSDs should work too; same for Solaris
- MacOS works, binary 1.x packages can be built using the cerbero build tool
- Windows works; binary 1.x packages can be built using the cerbero build tool
- MSys/MinGW builds
- Microsoft Visual Studio builds are also available and supported
- Android works, binary 1.x packages can be built using the cerbero build tool
- iOS works
INSTALLING FROM PACKAGES
------------------------
You should always prefer installing from packages first. GStreamer is
well-maintained for a number of distributions, including Fedora, Debian,
Ubuntu, Mandrake, Arch Linux, Gentoo, ...
Only in cases where you:
- want to hack on GStreamer
- want to verify that a bug has been fixed
- do not have a sane distribution
should you choose to build from source tarballs or git.
Find more information about the various packages at
https://gstreamer.freedesktop.org/download/
COMPILING FROM SOURCE TARBALLS
------------------------------
- again, make sure that you really need to install from source!
If GStreamer is one of your first projects ever that you build from source,
consider taking on an easier project.
- you need a recent version of Meson installed, see
http://mesonbuild.com/Getting-meson.html
and
https://gitlab.freedesktop.org/gstreamer/gst-build/blob/master/README.md
- run
meson build
ninja -C build
to build GStreamer.
- if you want to install it (not required, but what you usually want to do), run
ninja -C build install
- try out a simple test:
gst-launch-1.0 -v fakesrc num_buffers=5 ! fakesink
(If you didn't install GStreamer, run `./build/tools/gst-launch-1.0`)
If it outputs a bunch of messages from fakesrc and fakesink, everything is
ok.
If it did not work, keep in mind that you might need to adjust the
PATH and/or LD_LIBRARY_PATH environment variables to make the system
find GStreamer in the prefix where you installed (by default that is /usr/local).
- After this, you're ready to install gst-plugins, which will provide the
functionality you're probably looking for by now, so go on and read
that README.
COMPILING FROM GIT
------------------
You can build an uninstalled GStreamer from git for development or testing
purposes without affecting your system installation.
Get started with:
git clone https://gitlab.freedesktop.org/gstreamer/gst-build
meson build
ninja -C build
ninja -C build uninstalled
For more information, see the `gst-build` module and its documentation:
https://gitlab.freedesktop.org/gstreamer/gst-build/blob/master/README.md
PLUG-IN DEPENDENCIES AND LICENSES
---------------------------------
GStreamer is developed under the terms of the LGPL (see COPYING file for
details). Some of our plug-ins however rely on libraries which are available
under other licenses. This means that if you are distributing an application
which has a non-GPL compatible license (for instance a closed-source
application) with GStreamer, you have to make sure not to distribute GPL-linked
plug-ins.
When using GPL-linked plug-ins, GStreamer is for all practical reasons
under the GPL itself.
HISTORY
-------
The fundamental design comes from the video pipeline at Oregon Graduate
Institute, as well as some ideas from DirectMedia. It's based on plug-ins that
will provide the various codec and other functionality. The interface
hopefully is generic enough for various companies (ahem, Apple) to release
binary codecs for Linux, until such time as they get a clue and release the
source.