gstreamer/sdk-basic-tutorial-platform-specific-elements.md

180 lines
6.7 KiB
Markdown
Raw Normal View History

# Basic tutorial 16: Platform-specific elements
2016-05-16 14:30:34 +00:00
2016-06-16 00:00:24 +00:00
## Goal
2016-05-16 14:30:34 +00:00
Even though GStreamer is a multiplatform framework, not all the elements
2016-06-16 00:00:24 +00:00
are available on all platforms. For example, the video sinks
2016-05-16 14:30:34 +00:00
depend heavily on the underlying windowing system, and a different one
needs to be selected depending on the platform. You normally do not need
2016-06-16 00:00:24 +00:00
to worry about this when using elements like `playbin` or
2016-05-16 14:30:34 +00:00
`autovideosink`, but, for those cases when you need to use one of the
sinks that are only available on specific platforms, this tutorial hints
you some of their peculiarities.
2016-06-16 00:00:24 +00:00
## Cross Platform
### `glimagesink`
This video sink is based on
2016-06-16 01:01:51 +00:00
[OpenGL](http://en.wikipedia.org/wiki/OpenGL) or [OpenGL ES](https://en.wikipedia.org/wiki/OpenGL_ES). It supports rescaling
2016-06-16 00:00:24 +00:00
and filtering of the scaled image to alleviate aliasing. It implements
the VideoOverlay interface, so the video window can be re-parented
(embedded inside other windows). This is the video sink recommended on
most platforms. In particular, on Android and iOS, it is the only
available video sink. It can be decomposed into
`glupload ! glcolorconvert ! glimagesinkelement` to insert further OpenGL
hardware accelerated processing into the pipeline.
## Linux
2016-05-16 14:30:34 +00:00
### `ximagesink`
2016-06-16 00:00:24 +00:00
A standard RGB only X-based video sink. It implements the VideoOverlay
interface, so the video window can be re-parented (embedded inside
other windows). It does not support scaling or color formats other
than RGB; it has to be performed by different means (using the
`videoscale` element, for example).
2016-05-16 14:30:34 +00:00
### `xvimagesink`
An X-based video sink, using the [X Video
Extension](http://en.wikipedia.org/wiki/X_video_extension) (Xv). It
2016-06-16 00:00:24 +00:00
implements the VideoOverlay interface, so the video window can be
2016-05-16 14:30:34 +00:00
re-parented (embedded inside other windows). It can perform scaling
efficiently, on the GPU. It is only available if the hardware and
corresponding drivers support the Xv extension.
### `alsasink`
This audio sink outputs to the sound card via
[ALSA](http://www.alsa-project.org/) (Advanced Linux Sound
Architecture). This sink is available on almost every Linux platform. It
is often seen as a “low level” interface to the sound card, and can be
2016-06-16 00:00:24 +00:00
complicated to configure (See the comment on
[](sdk-playback-tutorial-digital-audio-pass-through.md)).
2016-05-16 14:30:34 +00:00
### `pulsesink`
This sink plays audio to a [PulseAudio](http://www.pulseaudio.org/)
server. It is a higher level abstraction of the sound card than ALSA,
and is therefore easier to use and offers more advanced features. It has
been known to be unstable on some older Linux distributions, though.
2016-06-16 00:00:24 +00:00
## Mac OS X
2016-05-16 14:30:34 +00:00
### `osxvideosink`
2016-06-16 00:00:24 +00:00
This is the video sink available to GStreamer on Mac OS X. It is also
possible to draw using `glimagesink` using OpenGL.
2016-05-16 14:30:34 +00:00
### `osxaudiosink`
This is the only audio sink available to GStreamer on Mac OS X.
2016-06-16 00:00:24 +00:00
## Windows
2016-05-16 14:30:34 +00:00
### `directdrawsink`
This is the oldest of the Windows video sinks, based on [Direct
Draw](http://en.wikipedia.org/wiki/DirectDraw). It requires DirectX 7,
so it is available on almost every current Windows platform. It supports
rescaling and filtering of the scaled image to alleviate aliasing.
### `dshowvideosink`
This video sink is based on [Direct
2016-06-16 00:00:24 +00:00
Show](http://en.wikipedia.org/wiki/Direct_Show). It can use different
2016-05-16 14:30:34 +00:00
rendering back-ends, like
[EVR](http://en.wikipedia.org/wiki/Enhanced_Video_Renderer),
[VMR9](http://en.wikipedia.org/wiki/Direct_Show#Video_rendering_filters)
or
[VMR7](http://en.wikipedia.org/wiki/Direct_Show#Video_rendering_filters),
EVR only being available on Windows Vista or more recent. It supports
2016-06-16 00:00:24 +00:00
rescaling and filtering of the scaled image to alleviate aliasing. It
implements the VideoOverlay interface, so the video window can be
2016-05-16 14:30:34 +00:00
re-parented (embedded inside other windows).
### `d3dvideosink`
This video sink is based on
[Direct3D](http://en.wikipedia.org/wiki/Direct3D) and its the most
recent Windows video sink. It supports rescaling and filtering of the
2016-06-16 00:00:24 +00:00
scaled image to alleviate aliasing. It implements the VideoOverlay
2016-05-16 14:30:34 +00:00
interface, so the video window can be re-parented (embedded inside other
windows).
### `directsoundsink`
This is the default audio sink for Windows, based on [Direct
Sound](http://en.wikipedia.org/wiki/DirectSound), which is available in
all Windows versions.
### `dshowdecwrapper`
2016-06-16 00:00:24 +00:00
[Direct Show](http://en.wikipedia.org/wiki/Direct_Show) is a multimedia
2016-05-16 14:30:34 +00:00
framework similar to GStreamer. They are different enough, though, so
that their pipelines cannot be interconnected. However, through this
element, GStreamer can benefit from the decoding elements present in
2016-06-16 00:00:24 +00:00
Direct Show. `dshowdecwrapper` wraps multiple Direct Show decoders so
they can be embedded in a GStreamer pipeline. Use the `gst-inspect-1.0` tool
2016-06-17 19:49:34 +00:00
(see [](sdk-basic-tutorial-gstreamer-tools.md)) to see the
2016-05-16 14:30:34 +00:00
available decoders.
2016-06-16 00:00:24 +00:00
## Android
2016-05-16 14:30:34 +00:00
### `openslessink`
This is the only audio sink available to GStreamer on Android. It is
2016-06-16 00:00:24 +00:00
based on [OpenSL ES](http://en.wikipedia.org/wiki/OpenSL_ES).
### `openslessrc`
This is the only audio source available to GStreamer on Android. It is
based on [OpenSL ES](http://en.wikipedia.org/wiki/OpenSL_ES).
2016-05-16 14:30:34 +00:00
### `androidmedia`
[android.media.MediaCodec](http://developer.android.com/reference/android/media/MediaCodec.html)
is an Android specific API to access the codecs that are available on
the device, including hardware codecs. It is available since API level
2016-06-16 00:00:24 +00:00
16 (JellyBean) and GStreamer can use it via the androidmedia plugin
for audio and video decoding. On Android, attaching the hardware
decoder to the `glimagesink` element can produce a high performance
zero-copy decodebin pipeline.
2016-05-16 14:30:34 +00:00
2016-06-16 00:00:24 +00:00
### `ahcsrc`
2016-05-16 14:30:34 +00:00
2016-06-16 00:00:24 +00:00
This video source can capture from the cameras on Android devices, it is part
of the androidmedia plugin and uses the [android.hardware.Camera API](https://developer.android.com/reference/android/hardware/Camera.html).
2016-05-16 14:30:34 +00:00
2016-06-16 00:00:24 +00:00
## iOS
2016-05-16 14:30:34 +00:00
### `osxaudiosink`
This is the only audio sink available to GStreamer on iOS.
### `iosassetsrc`
Source element to read iOS assets, this is, documents stored in the
Library (like photos, music and videos). It can be instantiated
2016-06-16 00:00:24 +00:00
automatically by `playbin` when URIs use the
`assets-library://` scheme.
2016-05-16 14:30:34 +00:00
### `iosavassetsrc`
Source element to read and decode iOS audiovisual assets, this is,
documents stored in the Library (like photos, music and videos). It can
2016-06-16 00:00:24 +00:00
be instantiated automatically by `playbin` when URIs use the
`ipod-library://` scheme. Decoding is performed by the system, so
2016-05-16 14:30:34 +00:00
dedicated hardware will be used if available.
2016-06-16 00:00:24 +00:00
## Conclusion
2016-05-16 14:30:34 +00:00
This tutorial has shown a few specific details about some GStreamer
elements which are not available on all platforms. You do not have to
2016-06-16 00:00:24 +00:00
worry about them when using multiplatform elements like `playbin` or
2016-05-16 14:30:34 +00:00
`autovideosink`, but it is good to know their personal quirks if
instancing them manually.
2016-06-16 00:00:24 +00:00
It has been a pleasure having you here, and see you soon!