gstreamer/subprojects/gst-docs/markdown/tutorials/playback/audio-visualization.md

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

260 lines
8.5 KiB
Markdown
Raw Normal View History

# Playback tutorial 6: Audio visualization
2016-05-16 14:30:34 +00:00
{{ ALERT_PY.md }}
{{ ALERT_JS.md }}
2016-06-16 01:26:59 +00:00
## Goal
2016-05-16 14:30:34 +00:00
GStreamer comes with a set of elements that turn audio into video. They
can be used for scientific visualization or to spice up your music
player, for example. This tutorial shows:
- How to enable audio visualization
- How to select the visualization element
2016-06-16 01:26:59 +00:00
## Introduction
2016-05-16 14:30:34 +00:00
2016-06-16 01:26:59 +00:00
Enabling audio visualization in `playbin` is actually very easy. Just
set the appropriate `playbin` flag and, when an audio-only stream is
2016-05-16 14:30:34 +00:00
found, it will instantiate the necessary elements to create and display
the visualization.
If you want to specify the actual element that you want to use to
generate the visualization, you instantiate it yourself and then tell
2016-06-16 01:26:59 +00:00
`playbin` about it through the `vis-plugin` property.
2016-05-16 14:30:34 +00:00
This tutorial searches the GStreamer registry for all the elements of
2016-06-16 01:26:59 +00:00
the Visualization class, tries to select `goom` (or another one if it is
2016-05-27 18:19:02 +00:00
not available) and passes it to `playbin`.
2016-05-16 14:30:34 +00:00
2016-06-16 01:26:59 +00:00
## A fancy music player
2016-05-16 14:30:34 +00:00
2016-06-16 01:26:59 +00:00
Copy this code into a text file named `playback-tutorial-6.c`.
2016-05-16 14:30:34 +00:00
**playback-tutorial-6.c**
2016-06-06 00:58:09 +00:00
``` c
2016-05-16 14:30:34 +00:00
#include <gst/gst.h>
2016-05-27 18:19:02 +00:00
/* playbin flags */
2016-05-16 14:30:34 +00:00
typedef enum {
GST_PLAY_FLAG_VIS = (1 << 3) /* Enable rendering of visualizations when there is no video stream. */
} GstPlayFlags;
2016-05-16 14:30:34 +00:00
/* Return TRUE if this is a Visualization element */
static gboolean filter_vis_features (GstPluginFeature *feature, gpointer data) {
GstElementFactory *factory;
2016-05-16 14:30:34 +00:00
if (!GST_IS_ELEMENT_FACTORY (feature))
return FALSE;
factory = GST_ELEMENT_FACTORY (feature);
if (!g_strrstr (gst_element_factory_get_klass (factory), "Visualization"))
return FALSE;
2016-05-16 14:30:34 +00:00
return TRUE;
}
2016-05-16 14:30:34 +00:00
int main(int argc, char *argv[]) {
GstElement *pipeline, *vis_plugin;
GstBus *bus;
GstMessage *msg;
GList *list, *walk;
GstElementFactory *selected_factory = NULL;
guint flags;
2016-05-16 14:30:34 +00:00
/* Initialize GStreamer */
gst_init (&argc, &argv);
2016-05-16 14:30:34 +00:00
/* Get a list of all visualization plugins */
2016-06-16 01:26:59 +00:00
list = gst_registry_feature_filter (gst_registry_get (), filter_vis_features, FALSE, NULL);
2016-05-16 14:30:34 +00:00
/* Print their names */
g_print("Available visualization plugins:\n");
for (walk = list; walk != NULL; walk = g_list_next (walk)) {
const gchar *name;
GstElementFactory *factory;
2016-05-16 14:30:34 +00:00
factory = GST_ELEMENT_FACTORY (walk->data);
name = gst_element_factory_get_longname (factory);
g_print(" %s\n", name);
2016-05-16 14:30:34 +00:00
if (selected_factory == NULL || g_str_has_prefix (name, "GOOM")) {
selected_factory = factory;
}
}
2016-05-16 14:30:34 +00:00
/* Don't use the factory if it's still empty */
/* e.g. no visualization plugins found */
if (!selected_factory) {
g_print ("No visualization plugins found!\n");
return -1;
}
2016-05-16 14:30:34 +00:00
/* We have now selected a factory for the visualization element */
g_print ("Selected '%s'\n", gst_element_factory_get_longname (selected_factory));
vis_plugin = gst_element_factory_create (selected_factory, NULL);
if (!vis_plugin)
return -1;
2016-05-16 14:30:34 +00:00
/* Build the pipeline */
2016-05-27 18:19:02 +00:00
pipeline = gst_parse_launch ("playbin uri=http://radio.hbr1.com:19800/ambient.ogg", NULL);
2016-05-16 14:30:34 +00:00
/* Set the visualization flag */
g_object_get (pipeline, "flags", &flags, NULL);
flags |= GST_PLAY_FLAG_VIS;
g_object_set (pipeline, "flags", flags, NULL);
2016-05-27 18:19:02 +00:00
/* set vis plugin for playbin */
2016-05-16 14:30:34 +00:00
g_object_set (pipeline, "vis-plugin", vis_plugin, NULL);
2016-05-16 14:30:34 +00:00
/* Start playing */
gst_element_set_state (pipeline, GST_STATE_PLAYING);
2016-05-16 14:30:34 +00:00
/* Wait until error or EOS */
bus = gst_element_get_bus (pipeline);
msg = gst_bus_timed_pop_filtered (bus, GST_CLOCK_TIME_NONE, GST_MESSAGE_ERROR | GST_MESSAGE_EOS);
2016-05-16 14:30:34 +00:00
/* Free resources */
if (msg != NULL)
gst_message_unref (msg);
gst_plugin_feature_list_free (list);
gst_object_unref (bus);
gst_element_set_state (pipeline, GST_STATE_NULL);
gst_object_unref (pipeline);
return 0;
}
```
2016-06-16 01:26:59 +00:00
> ![information] If you need help to compile this code, refer to the
> **Building the tutorials** section for your platform: [Mac] or
> [Windows] or use this specific command on Linux:
>
> `` gcc playback-tutorial-6.c -o playback-tutorial-6 `pkg-config --cflags --libs gstreamer-1.0` ``
>
> If you need help to run this code, refer to the **Running the
> tutorials** section for your platform: [Mac OS X], [Windows][1], for
> [iOS] or for [android].
>
> This tutorial plays music streamed from the [HBR1](http://www.hbr1.com/) Internet radio station. A window should open displaying somewhat psychedelic color patterns moving with the music. The media is fetched from the Internet, so the window might take a few seconds to appear, depending on your connection speed.
>
> Required libraries: `gstreamer-1.0`
## Walkthrough
First off, we indicate `playbin` that we want an audio visualization by
setting the `GST_PLAY_FLAG_VIS` flag. If the media already contains
2016-05-16 14:30:34 +00:00
video, this flag has no effect.
2016-06-06 00:58:09 +00:00
``` c
2016-05-16 14:30:34 +00:00
/* Set the visualization flag */
g_object_get (pipeline, "flags", &flags, NULL);
flags |= GST_PLAY_FLAG_VIS;
g_object_set (pipeline, "flags", flags, NULL);
```
2016-06-16 01:26:59 +00:00
If no visualization plugin is enforced by the user, `playbin` will use
`goom` (audio visualization will be disabled if `goom` is not
2016-05-16 14:30:34 +00:00
available). The rest of the tutorial shows how to find out the available
2016-05-27 18:19:02 +00:00
visualization elements and enforce one to `playbin`.
2016-05-16 14:30:34 +00:00
2016-06-06 00:58:09 +00:00
``` c
2016-05-16 14:30:34 +00:00
/* Get a list of all visualization plugins */
2016-06-16 01:26:59 +00:00
list = gst_registry_feature_filter (gst_registry_get (), filter_vis_features, FALSE, NULL);
2016-05-16 14:30:34 +00:00
```
2016-06-16 01:26:59 +00:00
`gst_registry_feature_filter()` examines all elements currently in the
2016-05-16 14:30:34 +00:00
GStreamer registry and selects those for which
2016-06-16 01:26:59 +00:00
the `filter_vis_features` function returns TRUE. This function selects
2016-05-16 14:30:34 +00:00
only the Visualization plugins:
2016-06-06 00:58:09 +00:00
``` c
2016-05-16 14:30:34 +00:00
/* Return TRUE if this is a Visualization element */
static gboolean filter_vis_features (GstPluginFeature *feature, gpointer data) {
GstElementFactory *factory;
2016-05-16 14:30:34 +00:00
if (!GST_IS_ELEMENT_FACTORY (feature))
return FALSE;
factory = GST_ELEMENT_FACTORY (feature);
if (!g_strrstr (gst_element_factory_get_klass (factory), "Visualization"))
return FALSE;
2016-05-16 14:30:34 +00:00
return TRUE;
}
```
A bit of theory regarding the organization of GStreamer elements is in
2016-06-16 01:26:59 +00:00
place: Each of the files that GStreamer loads at runtime is known as a
2016-05-16 14:30:34 +00:00
Plugin (`GstPlugin`). A Plugin can contain many Features
(`GstPluginFeature`). There are different kinds of Features, among them,
2016-06-16 01:26:59 +00:00
the Element Factories (`GstElementFactory`) that we have been using to
2016-05-16 14:30:34 +00:00
build Elements (`GstElement`).
This function simply disregards all Features which are not Factories,
and then all Factories whose class (obtained with
2016-06-16 01:26:59 +00:00
`gst_element_factory_get_klass()`) does not include “Visualization”. As
2016-05-16 14:30:34 +00:00
stated in the documentation for `GstElementFactory`, a Factorys class
is a “string describing the type of element, as an unordered list
separated with slashes (/)”. Examples of classes are “Source/Network”,
“Codec/Decoder/Video”, “Codec/Encoder/Audio” or “Visualization”.
2016-06-06 00:58:09 +00:00
``` c
2016-05-16 14:30:34 +00:00
/* Print their names */
g_print("Available visualization plugins:\n");
for (walk = list; walk != NULL; walk = g_list_next (walk)) {
const gchar *name;
GstElementFactory *factory;
2016-05-16 14:30:34 +00:00
factory = GST_ELEMENT_FACTORY (walk->data);
name = gst_element_factory_get_longname (factory);
g_print(" %s\n", name);
2016-05-16 14:30:34 +00:00
if (selected_factory == NULL || g_str_has_prefix (name, "GOOM")) {
selected_factory = factory;
}
}
```
Once we have the list of Visualization plugins, we print their names
2016-06-16 01:26:59 +00:00
(`gst_element_factory_get_longname()`) and choose one (in this case,
2016-05-16 14:30:34 +00:00
GOOM).
2016-06-06 00:58:09 +00:00
``` c
2016-05-16 14:30:34 +00:00
/* We have now selected a factory for the visualization element */
g_print ("Selected '%s'\n", gst_element_factory_get_longname (selected_factory));
vis_plugin = gst_element_factory_create (selected_factory, NULL);
if (!vis_plugin)
return -1;
```
2016-06-16 01:26:59 +00:00
The selected factory is used to instantiate an actual `GstElement` which
is then passed to `playbin` through the `vis-plugin` property:
2016-05-16 14:30:34 +00:00
2016-06-06 00:58:09 +00:00
``` c
2016-05-27 18:19:02 +00:00
/* set vis plugin for playbin */
2016-05-16 14:30:34 +00:00
g_object_set (pipeline, "vis-plugin", vis_plugin, NULL);
```
And we are done.
2016-06-16 01:26:59 +00:00
## Conclusion
2016-05-16 14:30:34 +00:00
This tutorial has shown:
2016-06-16 01:26:59 +00:00
- How to enable Audio Visualization in `playbin` with the
`GST_PLAY_FLAG_VIS` flag
2016-05-16 14:30:34 +00:00
- How to enforce one particular visualization element with the
`vis-plugin` `playbin` property
2016-05-16 14:30:34 +00:00
It has been a pleasure having you here, and see you soon\!
[information]: images/icons/emoticons/information.svg
[Mac]: installing/on-mac-osx.md
[Windows]: installing/on-windows.md
[Mac OS X]: installing/on-mac-osx.md#building-the-tutorials
[1]: installing/on-windows.md#running-the-tutorials
[iOS]: installing/for-ios-development.md#building-the-tutorials
[android]: installing/for-android-development.md#building-the-tutorials
[warning]: images/icons/emoticons/warning.svg