gstreamer/subprojects/gst-docs/markdown/tutorials/basic/platform-specific-elements.md

191 lines
7.3 KiB
Markdown
Raw Normal View History

# Basic tutorial 16: Platform-specific elements
2016-05-16 14:30:34 +00:00
2016-06-16 00:00:24 +00:00
## Goal
2016-05-16 14:30:34 +00:00
Even though GStreamer is a multiplatform framework, not all the elements
2016-06-16 00:00:24 +00:00
are available on all platforms. For example, the video sinks
2016-05-16 14:30:34 +00:00
depend heavily on the underlying windowing system, and a different one
needs to be selected depending on the platform. You normally do not need
2016-06-16 00:00:24 +00:00
to worry about this when using elements like `playbin` or
2016-05-16 14:30:34 +00:00
`autovideosink`, but, for those cases when you need to use one of the
sinks that are only available on specific platforms, this tutorial hints
you some of their peculiarities.
2016-06-16 00:00:24 +00:00
## Cross Platform
### `glimagesink`
This video sink is based on
2016-06-16 01:01:51 +00:00
[OpenGL](http://en.wikipedia.org/wiki/OpenGL) or [OpenGL ES](https://en.wikipedia.org/wiki/OpenGL_ES). It supports rescaling
2016-06-16 00:00:24 +00:00
and filtering of the scaled image to alleviate aliasing. It implements
the VideoOverlay interface, so the video window can be re-parented
(embedded inside other windows). This is the video sink recommended on
most platforms except for Windows (On Windows, `d3d11videosink` is recommended).
In particular, on Android and iOS, it is the only
2016-06-16 00:00:24 +00:00
available video sink. It can be decomposed into
`glupload ! glcolorconvert ! glimagesinkelement` to insert further OpenGL
hardware accelerated processing into the pipeline.
## Linux
2016-05-16 14:30:34 +00:00
### `ximagesink`
2016-06-16 00:00:24 +00:00
A standard RGB only X-based video sink. It implements the VideoOverlay
interface, so the video window can be re-parented (embedded inside
other windows). It does not support scaling or color formats other
than RGB; it has to be performed by different means (using the
`videoscale` element, for example).
2016-05-16 14:30:34 +00:00
### `xvimagesink`
An X-based video sink, using the [X Video
Extension](http://en.wikipedia.org/wiki/X_video_extension) (Xv). It
2016-06-16 00:00:24 +00:00
implements the VideoOverlay interface, so the video window can be
2016-05-16 14:30:34 +00:00
re-parented (embedded inside other windows). It can perform scaling
efficiently, on the GPU. It is only available if the hardware and
corresponding drivers support the Xv extension.
### `alsasink`
This audio sink outputs to the sound card via
[ALSA](http://www.alsa-project.org/) (Advanced Linux Sound
Architecture). This sink is available on almost every Linux platform. It
is often seen as a “low level” interface to the sound card, and can be
2016-06-16 00:00:24 +00:00
complicated to configure (See the comment on
[](tutorials/playback/digital-audio-pass-through.md)).
2016-05-16 14:30:34 +00:00
### `pulsesink`
This sink plays audio to a [PulseAudio](http://www.pulseaudio.org/)
server. It is a higher level abstraction of the sound card than ALSA,
and is therefore easier to use and offers more advanced features. It has
been known to be unstable on some older Linux distributions, though.
2016-06-16 00:00:24 +00:00
## Mac OS X
2016-05-16 14:30:34 +00:00
### `osxvideosink`
2016-06-16 00:00:24 +00:00
This is the video sink available to GStreamer on Mac OS X. It is also
possible to draw using `glimagesink` using OpenGL.
2016-05-16 14:30:34 +00:00
### `osxaudiosink`
This is the only audio sink available to GStreamer on Mac OS X.
2016-06-16 00:00:24 +00:00
## Windows
2016-05-16 14:30:34 +00:00
### `d3d11videosink`
2016-05-16 14:30:34 +00:00
This video sink is based on [Direct3D11](https://en.wikipedia.org/wiki/Direct3D#Direct3D_11)
and is the recommended element on Windows.
It supports VideoOverlay interface and rescaling/colorspace conversion
in [zero-copy](https://en.wikipedia.org/wiki/Zero-copy) manner. This element
is the most performant and featureful video sink element on Windows.
### `d3dvideosink`
This video sink is based on
[Direct3D9](https://en.wikipedia.org/wiki/Direct3D#Direct3D_9).
It supports rescaling and filtering of the scaled image to alleviate aliasing.
It implements the VideoOverlay interface, so the video window can be re-parented (embedded inside other windows).
This element is not recommended for applications targetting Windows 8 or more recent.
2016-05-16 14:30:34 +00:00
### `dshowvideosink (deprecated)`
2016-05-16 14:30:34 +00:00
This video sink is based on [Direct
2016-06-16 00:00:24 +00:00
Show](http://en.wikipedia.org/wiki/Direct_Show). It can use different
2016-05-16 14:30:34 +00:00
rendering back-ends, like
[EVR](http://en.wikipedia.org/wiki/Enhanced_Video_Renderer),
[VMR9](http://en.wikipedia.org/wiki/Direct_Show#Video_rendering_filters)
or
[VMR7](http://en.wikipedia.org/wiki/Direct_Show#Video_rendering_filters),
EVR only being available on Windows Vista or more recent. It supports
2016-06-16 00:00:24 +00:00
rescaling and filtering of the scaled image to alleviate aliasing. It
implements the VideoOverlay interface, so the video window can be
re-parented (embedded inside other windows). This element is not recommended
in most cases.
2016-05-16 14:30:34 +00:00
### `wasapisink` and `wasapi2sink`
2016-05-16 14:30:34 +00:00
Those elements are the default audio sink elements on Windows, based on
[WASAPI](https://docs.microsoft.com/en-us/windows/win32/coreaudio/wasapi),
which is available on Vista or more recent. Note that `wasapi2sink` is
a replacement of `wasapisink` and `wasapi2sink` is default for Windows 8 or
more recent. Otherwise `wasapisink` will be default audio sink element.
2016-05-16 14:30:34 +00:00
### `directsoundsink (deprecated)`
2016-05-16 14:30:34 +00:00
This audio sink element is based on
[DirectSound](http://en.wikipedia.org/wiki/DirectSound), which is available in
2016-05-16 14:30:34 +00:00
all Windows versions.
### `dshowdecwrapper`
2016-06-16 00:00:24 +00:00
[Direct Show](http://en.wikipedia.org/wiki/Direct_Show) is a multimedia
2016-05-16 14:30:34 +00:00
framework similar to GStreamer. They are different enough, though, so
that their pipelines cannot be interconnected. However, through this
element, GStreamer can benefit from the decoding elements present in
2016-06-16 00:00:24 +00:00
Direct Show. `dshowdecwrapper` wraps multiple Direct Show decoders so
they can be embedded in a GStreamer pipeline. Use the `gst-inspect-1.0` tool
(see [](tutorials/basic/gstreamer-tools.md)) to see the
2016-05-16 14:30:34 +00:00
available decoders.
2016-06-16 00:00:24 +00:00
## Android
2016-05-16 14:30:34 +00:00
### `openslessink`
This is the only audio sink available to GStreamer on Android. It is
2016-06-16 00:00:24 +00:00
based on [OpenSL ES](http://en.wikipedia.org/wiki/OpenSL_ES).
### `openslessrc`
This is the only audio source available to GStreamer on Android. It is
based on [OpenSL ES](http://en.wikipedia.org/wiki/OpenSL_ES).
2016-05-16 14:30:34 +00:00
### `androidmedia`
[android.media.MediaCodec](http://developer.android.com/reference/android/media/MediaCodec.html)
is an Android specific API to access the codecs that are available on
the device, including hardware codecs. It is available since API level
2016-06-16 00:00:24 +00:00
16 (JellyBean) and GStreamer can use it via the androidmedia plugin
for audio and video decoding. On Android, attaching the hardware
decoder to the `glimagesink` element can produce a high performance
zero-copy decodebin pipeline.
2016-05-16 14:30:34 +00:00
2016-06-16 00:00:24 +00:00
### `ahcsrc`
2016-05-16 14:30:34 +00:00
2016-06-16 00:00:24 +00:00
This video source can capture from the cameras on Android devices, it is part
of the androidmedia plugin and uses the [android.hardware.Camera API](https://developer.android.com/reference/android/hardware/Camera.html).
2016-05-16 14:30:34 +00:00
2016-06-16 00:00:24 +00:00
## iOS
2016-05-16 14:30:34 +00:00
### `osxaudiosink`
This is the only audio sink available to GStreamer on iOS.
### `iosassetsrc`
Source element to read iOS assets, this is, documents stored in the
Library (like photos, music and videos). It can be instantiated
2016-06-16 00:00:24 +00:00
automatically by `playbin` when URIs use the
`assets-library://` scheme.
2016-05-16 14:30:34 +00:00
### `iosavassetsrc`
Source element to read and decode iOS audiovisual assets, this is,
documents stored in the Library (like photos, music and videos). It can
2016-06-16 00:00:24 +00:00
be instantiated automatically by `playbin` when URIs use the
`ipod-library://` scheme. Decoding is performed by the system, so
2016-05-16 14:30:34 +00:00
dedicated hardware will be used if available.
2016-06-16 00:00:24 +00:00
## Conclusion
2016-05-16 14:30:34 +00:00
This tutorial has shown a few specific details about some GStreamer
elements which are not available on all platforms. You do not have to
2016-06-16 00:00:24 +00:00
worry about them when using multiplatform elements like `playbin` or
2016-05-16 14:30:34 +00:00
`autovideosink`, but it is good to know their personal quirks if
instancing them manually.
2016-06-16 00:00:24 +00:00
It has been a pleasure having you here, and see you soon!