2016-05-27 02:21:04 +00:00
|
|
|
|
# Basic tutorial 16: Platform-specific elements
|
2016-05-16 14:30:34 +00:00
|
|
|
|
|
2016-06-16 00:00:24 +00:00
|
|
|
|
## Goal
|
2016-05-16 14:30:34 +00:00
|
|
|
|
|
|
|
|
|
Even though GStreamer is a multiplatform framework, not all the elements
|
2016-06-16 00:00:24 +00:00
|
|
|
|
are available on all platforms. For example, the video sinks
|
2016-05-16 14:30:34 +00:00
|
|
|
|
depend heavily on the underlying windowing system, and a different one
|
|
|
|
|
needs to be selected depending on the platform. You normally do not need
|
2016-06-16 00:00:24 +00:00
|
|
|
|
to worry about this when using elements like `playbin` or
|
2016-05-16 14:30:34 +00:00
|
|
|
|
`autovideosink`, but, for those cases when you need to use one of the
|
|
|
|
|
sinks that are only available on specific platforms, this tutorial hints
|
|
|
|
|
you some of their peculiarities.
|
|
|
|
|
|
2016-06-16 00:00:24 +00:00
|
|
|
|
## Cross Platform
|
|
|
|
|
|
|
|
|
|
### `glimagesink`
|
|
|
|
|
|
|
|
|
|
This video sink is based on
|
2016-06-16 01:01:51 +00:00
|
|
|
|
[OpenGL](http://en.wikipedia.org/wiki/OpenGL) or [OpenGL ES](https://en.wikipedia.org/wiki/OpenGL_ES). It supports rescaling
|
2016-06-16 00:00:24 +00:00
|
|
|
|
and filtering of the scaled image to alleviate aliasing. It implements
|
|
|
|
|
the VideoOverlay interface, so the video window can be re-parented
|
|
|
|
|
(embedded inside other windows). This is the video sink recommended on
|
|
|
|
|
most platforms. In particular, on Android and iOS, it is the only
|
|
|
|
|
available video sink. It can be decomposed into
|
|
|
|
|
`glupload ! glcolorconvert ! glimagesinkelement` to insert further OpenGL
|
|
|
|
|
hardware accelerated processing into the pipeline.
|
|
|
|
|
|
|
|
|
|
## Linux
|
2016-05-16 14:30:34 +00:00
|
|
|
|
|
|
|
|
|
### `ximagesink`
|
|
|
|
|
|
2016-06-16 00:00:24 +00:00
|
|
|
|
A standard RGB only X-based video sink. It implements the VideoOverlay
|
|
|
|
|
interface, so the video window can be re-parented (embedded inside
|
|
|
|
|
other windows). It does not support scaling or color formats other
|
|
|
|
|
than RGB; it has to be performed by different means (using the
|
|
|
|
|
`videoscale` element, for example).
|
2016-05-16 14:30:34 +00:00
|
|
|
|
|
|
|
|
|
### `xvimagesink`
|
|
|
|
|
|
|
|
|
|
An X-based video sink, using the [X Video
|
|
|
|
|
Extension](http://en.wikipedia.org/wiki/X_video_extension) (Xv). It
|
2016-06-16 00:00:24 +00:00
|
|
|
|
implements the VideoOverlay interface, so the video window can be
|
2016-05-16 14:30:34 +00:00
|
|
|
|
re-parented (embedded inside other windows). It can perform scaling
|
|
|
|
|
efficiently, on the GPU. It is only available if the hardware and
|
|
|
|
|
corresponding drivers support the Xv extension.
|
|
|
|
|
|
|
|
|
|
### `alsasink`
|
|
|
|
|
|
|
|
|
|
This audio sink outputs to the sound card via
|
|
|
|
|
[ALSA](http://www.alsa-project.org/) (Advanced Linux Sound
|
|
|
|
|
Architecture). This sink is available on almost every Linux platform. It
|
|
|
|
|
is often seen as a “low level” interface to the sound card, and can be
|
2016-06-16 00:00:24 +00:00
|
|
|
|
complicated to configure (See the comment on
|
2016-10-21 13:23:28 +00:00
|
|
|
|
[](tutorial-playback-digital-audio-pass-through.md)).
|
2016-05-16 14:30:34 +00:00
|
|
|
|
|
|
|
|
|
### `pulsesink`
|
|
|
|
|
|
|
|
|
|
This sink plays audio to a [PulseAudio](http://www.pulseaudio.org/)
|
|
|
|
|
server. It is a higher level abstraction of the sound card than ALSA,
|
|
|
|
|
and is therefore easier to use and offers more advanced features. It has
|
|
|
|
|
been known to be unstable on some older Linux distributions, though.
|
|
|
|
|
|
2016-06-16 00:00:24 +00:00
|
|
|
|
## Mac OS X
|
2016-05-16 14:30:34 +00:00
|
|
|
|
|
|
|
|
|
### `osxvideosink`
|
|
|
|
|
|
2016-06-16 00:00:24 +00:00
|
|
|
|
This is the video sink available to GStreamer on Mac OS X. It is also
|
|
|
|
|
possible to draw using `glimagesink` using OpenGL.
|
2016-05-16 14:30:34 +00:00
|
|
|
|
|
|
|
|
|
### `osxaudiosink`
|
|
|
|
|
|
|
|
|
|
This is the only audio sink available to GStreamer on Mac OS X.
|
|
|
|
|
|
2016-06-16 00:00:24 +00:00
|
|
|
|
## Windows
|
2016-05-16 14:30:34 +00:00
|
|
|
|
|
|
|
|
|
### `directdrawsink`
|
|
|
|
|
|
|
|
|
|
This is the oldest of the Windows video sinks, based on [Direct
|
|
|
|
|
Draw](http://en.wikipedia.org/wiki/DirectDraw). It requires DirectX 7,
|
|
|
|
|
so it is available on almost every current Windows platform. It supports
|
|
|
|
|
rescaling and filtering of the scaled image to alleviate aliasing.
|
|
|
|
|
|
|
|
|
|
### `dshowvideosink`
|
|
|
|
|
|
|
|
|
|
This video sink is based on [Direct
|
2016-06-16 00:00:24 +00:00
|
|
|
|
Show](http://en.wikipedia.org/wiki/Direct_Show). It can use different
|
2016-05-16 14:30:34 +00:00
|
|
|
|
rendering back-ends, like
|
|
|
|
|
[EVR](http://en.wikipedia.org/wiki/Enhanced_Video_Renderer),
|
|
|
|
|
[VMR9](http://en.wikipedia.org/wiki/Direct_Show#Video_rendering_filters)
|
|
|
|
|
or
|
|
|
|
|
[VMR7](http://en.wikipedia.org/wiki/Direct_Show#Video_rendering_filters),
|
|
|
|
|
EVR only being available on Windows Vista or more recent. It supports
|
2016-06-16 00:00:24 +00:00
|
|
|
|
rescaling and filtering of the scaled image to alleviate aliasing. It
|
|
|
|
|
implements the VideoOverlay interface, so the video window can be
|
2016-05-16 14:30:34 +00:00
|
|
|
|
re-parented (embedded inside other windows).
|
|
|
|
|
|
|
|
|
|
### `d3dvideosink`
|
|
|
|
|
|
|
|
|
|
This video sink is based on
|
|
|
|
|
[Direct3D](http://en.wikipedia.org/wiki/Direct3D) and it’s the most
|
|
|
|
|
recent Windows video sink. It supports rescaling and filtering of the
|
2016-06-16 00:00:24 +00:00
|
|
|
|
scaled image to alleviate aliasing. It implements the VideoOverlay
|
2016-05-16 14:30:34 +00:00
|
|
|
|
interface, so the video window can be re-parented (embedded inside other
|
|
|
|
|
windows).
|
|
|
|
|
|
|
|
|
|
### `directsoundsink`
|
|
|
|
|
|
|
|
|
|
This is the default audio sink for Windows, based on [Direct
|
|
|
|
|
Sound](http://en.wikipedia.org/wiki/DirectSound), which is available in
|
|
|
|
|
all Windows versions.
|
|
|
|
|
|
|
|
|
|
### `dshowdecwrapper`
|
|
|
|
|
|
2016-06-16 00:00:24 +00:00
|
|
|
|
[Direct Show](http://en.wikipedia.org/wiki/Direct_Show) is a multimedia
|
2016-05-16 14:30:34 +00:00
|
|
|
|
framework similar to GStreamer. They are different enough, though, so
|
|
|
|
|
that their pipelines cannot be interconnected. However, through this
|
|
|
|
|
element, GStreamer can benefit from the decoding elements present in
|
2016-06-16 00:00:24 +00:00
|
|
|
|
Direct Show. `dshowdecwrapper` wraps multiple Direct Show decoders so
|
|
|
|
|
they can be embedded in a GStreamer pipeline. Use the `gst-inspect-1.0` tool
|
2016-10-21 13:23:28 +00:00
|
|
|
|
(see [](tutorial-basic-gstreamer-tools.md)) to see the
|
2016-05-16 14:30:34 +00:00
|
|
|
|
available decoders.
|
|
|
|
|
|
2016-06-16 00:00:24 +00:00
|
|
|
|
## Android
|
2016-05-16 14:30:34 +00:00
|
|
|
|
|
|
|
|
|
### `openslessink`
|
|
|
|
|
|
|
|
|
|
This is the only audio sink available to GStreamer on Android. It is
|
2016-06-16 00:00:24 +00:00
|
|
|
|
based on [OpenSL ES](http://en.wikipedia.org/wiki/OpenSL_ES).
|
|
|
|
|
|
|
|
|
|
### `openslessrc`
|
|
|
|
|
|
|
|
|
|
This is the only audio source available to GStreamer on Android. It is
|
|
|
|
|
based on [OpenSL ES](http://en.wikipedia.org/wiki/OpenSL_ES).
|
2016-05-16 14:30:34 +00:00
|
|
|
|
|
|
|
|
|
### `androidmedia`
|
|
|
|
|
|
|
|
|
|
[android.media.MediaCodec](http://developer.android.com/reference/android/media/MediaCodec.html)
|
|
|
|
|
is an Android specific API to access the codecs that are available on
|
|
|
|
|
the device, including hardware codecs. It is available since API level
|
2016-06-16 00:00:24 +00:00
|
|
|
|
16 (JellyBean) and GStreamer can use it via the androidmedia plugin
|
|
|
|
|
for audio and video decoding. On Android, attaching the hardware
|
|
|
|
|
decoder to the `glimagesink` element can produce a high performance
|
|
|
|
|
zero-copy decodebin pipeline.
|
2016-05-16 14:30:34 +00:00
|
|
|
|
|
2016-06-16 00:00:24 +00:00
|
|
|
|
### `ahcsrc`
|
2016-05-16 14:30:34 +00:00
|
|
|
|
|
2016-06-16 00:00:24 +00:00
|
|
|
|
This video source can capture from the cameras on Android devices, it is part
|
|
|
|
|
of the androidmedia plugin and uses the [android.hardware.Camera API](https://developer.android.com/reference/android/hardware/Camera.html).
|
2016-05-16 14:30:34 +00:00
|
|
|
|
|
2016-06-16 00:00:24 +00:00
|
|
|
|
## iOS
|
2016-05-16 14:30:34 +00:00
|
|
|
|
|
|
|
|
|
### `osxaudiosink`
|
|
|
|
|
|
|
|
|
|
This is the only audio sink available to GStreamer on iOS.
|
|
|
|
|
|
|
|
|
|
### `iosassetsrc`
|
|
|
|
|
|
|
|
|
|
Source element to read iOS assets, this is, documents stored in the
|
|
|
|
|
Library (like photos, music and videos). It can be instantiated
|
2016-06-16 00:00:24 +00:00
|
|
|
|
automatically by `playbin` when URIs use the
|
|
|
|
|
`assets-library://` scheme.
|
2016-05-16 14:30:34 +00:00
|
|
|
|
|
|
|
|
|
### `iosavassetsrc`
|
|
|
|
|
|
|
|
|
|
Source element to read and decode iOS audiovisual assets, this is,
|
|
|
|
|
documents stored in the Library (like photos, music and videos). It can
|
2016-06-16 00:00:24 +00:00
|
|
|
|
be instantiated automatically by `playbin` when URIs use the
|
|
|
|
|
`ipod-library://` scheme. Decoding is performed by the system, so
|
2016-05-16 14:30:34 +00:00
|
|
|
|
dedicated hardware will be used if available.
|
|
|
|
|
|
2016-06-16 00:00:24 +00:00
|
|
|
|
## Conclusion
|
2016-05-16 14:30:34 +00:00
|
|
|
|
|
|
|
|
|
This tutorial has shown a few specific details about some GStreamer
|
|
|
|
|
elements which are not available on all platforms. You do not have to
|
2016-06-16 00:00:24 +00:00
|
|
|
|
worry about them when using multiplatform elements like `playbin` or
|
2016-05-16 14:30:34 +00:00
|
|
|
|
`autovideosink`, but it is good to know their personal quirks if
|
|
|
|
|
instancing them manually.
|
|
|
|
|
|
2016-06-16 00:00:24 +00:00
|
|
|
|
It has been a pleasure having you here, and see you soon!
|