From 06563a56ef19b4b2c25db40b3bdc2a551fdeccfd Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Olivier=20Cr=C3=AAte?= Date: Wed, 15 Jun 2016 21:26:59 -0400 Subject: [PATCH] Update all playback tutorials --- TODO.md | 46 +-- sdk-playback-tutorial-audio-visualization.md | 113 +++---- sdk-playback-tutorial-color-balance.md | 106 +++---- sdk-playback-tutorial-custom-playbin-sinks.md | 122 +++----- ...ack-tutorial-digital-audio-pass-through.md | 53 ++-- ...ial-hardware-accelerated-video-decoding.md | 296 ++++-------------- sdk-playback-tutorial-playbin-usage.md | 139 ++++---- ...playback-tutorial-progressive-streaming.md | 179 +++++------ ...ack-tutorial-short-cutting-the-pipeline.md | 104 +++--- sdk-playback-tutorial-subtitle-management.md | 126 ++++---- sdk-playback-tutorials.md | 2 - 11 files changed, 507 insertions(+), 779 deletions(-) diff --git a/TODO.md b/TODO.md index f3ee87d4fb..86b3a43fbd 100644 --- a/TODO.md +++ b/TODO.md @@ -4,15 +4,6 @@ This is just a simple TODO list to follow progress of the port from gstreamer.com content to hotdoc Pages to review: - - sdk-playback-tutorials.md - - sdk-playback-tutorial-subtitle-management.md - - sdk-playback-tutorial-short-cutting-the-pipeline.md - - sdk-playback-tutorial-progressive-streaming.md - - sdk-playback-tutorial-color-balance.md - - sdk-playback-tutorial-audio-visualization.md - - sdk-playback-tutorial-custom-playbin-sinks.md - - sdk-playback-tutorial-hardware-accelerated-video-decoding.md - - sdk-playback-tutorial-digital-audio-pass-through.md - sdk-android-tutorials.md - sdk-android-tutorial-video.md - sdk-android-tutorial-media-player.md @@ -48,6 +39,19 @@ Code: Reviewed pages: - index.md - sdk-basic-tutorials.md + - sdk-basic-tutorial-concepts.md + - sdk-basic-tutorial-dynamic-pipelines.md + - sdk-basic-tutorial-time-management.md + - sdk-basic-tutorial-media-formats-and-pad-capabilities.md + - sdk-basic-tutorial-multithreading-and-pad-availability.md + - sdk-basic-tutorial-short-cutting-the-pipeline.md + - sdk-basic-tutorial-media-information-gathering.md + - sdk-basic-tutorial-gstreamer-tools.md + - sdk-basic-tutorial-debugging-tools.md + - sdk-basic-tutorial-streaming.md + - sdk-basic-tutorial-playback-speed.md + - sdk-basic-tutorial-handy-elements.md + - sdk-basic-tutorial-platform-specific-elements.md - sdk-installing.md - sdk-installing-for-android-development.md - sdk-building-from-source-using-cerbero.md @@ -56,23 +60,19 @@ Reviewed pages: - sdk-android-tutorial-link-against-gstreamer.md - sdk-android-tutorial-a-running-pipeline.md - sdk-api-reference.md - - sdk-playback-tutorial-playbin-usage.md + - sdk-playback-tutorials.md + - sdk-playback-tutorial-playbin-usage.md + - sdk-playback-tutorial-subtitle-management.md + - sdk-playback-tutorial-short-cutting-the-pipeline.md + - sdk-playback-tutorial-progressive-streaming.md + - sdk-playback-tutorial-color-balance.md + - sdk-playback-tutorial-audio-visualization.md + - sdk-playback-tutorial-custom-playbin-sinks.md + - sdk-playback-tutorial-hardware-accelerated-video-decoding.md + - sdk-playback-tutorial-digital-audio-pass-through.md - sdk-basic-tutorial-hello-world.md - sdk-gst-inspect.md - gst-launch.md - - sdk-basic-tutorial-concepts.md - - sdk-basic-tutorial-dynamic-pipelines.md - - sdk-basic-tutorial-time-management.md - - sdk-basic-tutorial-media-formats-and-pad-capabilities.md - - sdk-basic-tutorial-multithreading-and-pad-availability.md - - sdk-basic-tutorial-short-cutting-the-pipeline.md - - sdk-basic-tutorial-media-information-gathering.md - - sdk-basic-tutorial-gstreamer-tools.md - - sdk-basic-tutorial-debugging-tools.md - - sdk-basic-tutorial-streaming.md - - sdk-basic-tutorial-playback-speed.md - - sdk-basic-tutorial-handy-elements.md - - sdk-basic-tutorial-platform-specific-elements.md For-later pages: - sdk-qt-tutorials.md [tpm: this should all be rewritten from scratch with qmlglsink; QtGStreamer is outdated and unmaintained, we should not promote it] diff --git a/sdk-playback-tutorial-audio-visualization.md b/sdk-playback-tutorial-audio-visualization.md index f4f4c7caae..50e06b1d78 100644 --- a/sdk-playback-tutorial-audio-visualization.md +++ b/sdk-playback-tutorial-audio-visualization.md @@ -1,6 +1,6 @@ # Playback tutorial 6: Audio visualization -# Goal +## Goal GStreamer comes with a set of elements that turn audio into video. They can be used for scientific visualization or to spice up your music @@ -9,33 +9,24 @@ player, for example. This tutorial shows: - How to enable audio visualization - How to select the visualization element -# Introduction +## Introduction -Enabling audio visualization in `playbin` is actually very easy. Just -set the appropriate `playbin` flag and, when an audio-only stream is +Enabling audio visualization in `playbin` is actually very easy. Just +set the appropriate `playbin` flag and, when an audio-only stream is found, it will instantiate the necessary elements to create and display the visualization. If you want to specify the actual element that you want to use to generate the visualization, you instantiate it yourself and then tell -`playbin` about it through the `vis-plugin` property. +`playbin` about it through the `vis-plugin` property. This tutorial searches the GStreamer registry for all the elements of -the Visualization class, tries to select `goom` (or another one if it is +the Visualization class, tries to select `goom` (or another one if it is not available) and passes it to `playbin`. -# A fancy music player +## A fancy music player -Copy this code into a text file named `playback-tutorial-6.c`. - - - - - - - - -

This tutorial is included in the SDK since release 2012.7. If you cannot find it in the downloaded code, please install the latest release of the GStreamer SDK.

+Copy this code into a text file named `playback-tutorial-6.c`. **playback-tutorial-6.c** @@ -72,7 +63,7 @@ int main(int argc, char *argv[]) { gst_init (&argc, &argv); /* Get a list of all visualization plugins */ - list = gst_registry_feature_filter (gst_registry_get_default (), filter_vis_features, FALSE, NULL); + list = gst_registry_feature_filter (gst_registry_get (), filter_vis_features, FALSE, NULL); /* Print their names */ g_print("Available visualization plugins:\n"); @@ -131,34 +122,24 @@ int main(int argc, char *argv[]) { } ``` - - - - - - - -
-
-Need help? (Click to expand) -
-
-

If you need help to compile this code, refer to the Building the tutorials section for your platform: Linux, Mac OS X or Windows, or use this specific command on Linux:

-
-
-

gcc playback-tutorial-6.c -o playback-tutorial-6 `pkg-config --cflags --libs gstreamer-1.0`

-
-
-

If you need help to run this code, refer to the Running the tutorials section for your platform: Linux, Mac OS X or Windows

-

This tutorial plays music streamed from the HBR1 Internet radio station. A window should open displaying somewhat psychedelic color patterns moving with the music. The media is fetched from the Internet, so the window might take a few seconds to appear, depending on your connection speed.

-

Required libraries: gstreamer-1.0

-
-
+> ![information] If you need help to compile this code, refer to the +> **Building the tutorials** section for your platform: [Mac] or +> [Windows] or use this specific command on Linux: +> +> `` gcc playback-tutorial-6.c -o playback-tutorial-6 `pkg-config --cflags --libs gstreamer-1.0` `` +> +> If you need help to run this code, refer to the **Running the +> tutorials** section for your platform: [Mac OS X], [Windows][1], for +> [iOS] or for [android]. +> +> This tutorial plays music streamed from the [HBR1](http://www.hbr1.com/) Internet radio station. A window should open displaying somewhat psychedelic color patterns moving with the music. The media is fetched from the Internet, so the window might take a few seconds to appear, depending on your connection speed. +> +> Required libraries: `gstreamer-1.0` -# Walkthrough +## Walkthrough -First off, we indicate `playbin` that we want an audio visualization by -setting the `GST_PLAY_FLAG_VIS` flag. If the media already contains +First off, we indicate `playbin` that we want an audio visualization by +setting the `GST_PLAY_FLAG_VIS` flag. If the media already contains video, this flag has no effect. ``` c @@ -168,19 +149,19 @@ flags |= GST_PLAY_FLAG_VIS; g_object_set (pipeline, "flags", flags, NULL); ``` -If no visualization plugin is enforced by the user, `playbin` will use -`goom` (audio visualization will be disabled if `goom` is not +If no visualization plugin is enforced by the user, `playbin` will use +`goom` (audio visualization will be disabled if `goom` is not available). The rest of the tutorial shows how to find out the available visualization elements and enforce one to `playbin`. ``` c /* Get a list of all visualization plugins */ -list = gst_registry_feature_filter (gst_registry_get_default (), filter_vis_features, FALSE, NULL); +list = gst_registry_feature_filter (gst_registry_get (), filter_vis_features, FALSE, NULL); ``` -`gst_registry_feature_filter()` examines all elements currently in the +`gst_registry_feature_filter()` examines all elements currently in the GStreamer registry and selects those for which -the `filter_vis_features` function returns TRUE. This function selects +the `filter_vis_features` function returns TRUE. This function selects only the Visualization plugins: ``` c @@ -199,15 +180,15 @@ static gboolean filter_vis_features (GstPluginFeature *feature, gpointer data) { ``` A bit of theory regarding the organization of GStreamer elements is in -place: Each of the files that GStreamer loads at runtime is known as a +place: Each of the files that GStreamer loads at runtime is known as a Plugin (`GstPlugin`). A Plugin can contain many Features (`GstPluginFeature`). There are different kinds of Features, among them, -the Element Factories (`GstElementFactory`) that we have been using to +the Element Factories (`GstElementFactory`) that we have been using to build Elements (`GstElement`). This function simply disregards all Features which are not Factories, and then all Factories whose class (obtained with -`gst_element_factory_get_klass()`) does not include “Visualization”.  As +`gst_element_factory_get_klass()`) does not include “Visualization”. As stated in the documentation for `GstElementFactory`, a Factory’s class is a “string describing the type of element, as an unordered list separated with slashes (/)”. Examples of classes are “Source/Network”, @@ -231,7 +212,7 @@ for (walk = list; walk != NULL; walk = g_list_next (walk)) { ``` Once we have the list of Visualization plugins, we print their names -(`gst_element_factory_get_longname()`) and choose one (in this case, +(`gst_element_factory_get_longname()`) and choose one (in this case, GOOM). ``` c @@ -242,8 +223,8 @@ if (!vis_plugin) return -1; ``` -The selected factory is used to instantiate an actual `GstElement` which -is then passed to `playbin` through the `vis-plugin` property: +The selected factory is used to instantiate an actual `GstElement` which +is then passed to `playbin` through the `vis-plugin` property: ``` c /* set vis plugin for playbin */ @@ -252,20 +233,22 @@ g_object_set (pipeline, "vis-plugin", vis_plugin, NULL); And we are done. -# Conclusion +## Conclusion This tutorial has shown: - - How to enable Audio Visualization in `playbin` with the - `GST_PLAY_FLAG_VIS` flag + - How to enable Audio Visualization in `playbin` with the + `GST_PLAY_FLAG_VIS` flag - How to enforce one particular visualization element with the - `vis-plugin` `playbin` property  + `vis-plugin` `playbin` property It has been a pleasure having you here, and see you soon\! -## Attachments: - -![](images/icons/bullet_blue.gif) -[vs2010.zip](attachments/327802/2424878.zip) (application/zip) -![](images/icons/bullet_blue.gif) -[playback-tutorial-6.c](attachments/327802/2424879.c) (text/plain) + [information]: images/icons/emoticons/information.png + [Mac]: sdk-installing-on-mac-osx.md + [Windows]: Installing+on+Windows + [Mac OS X]: sdk-installing-on-mac-osx.md#building-the-tutorials + [1]: sdk-installing-on-windows.md#running-the-tutorials + [iOS]: sdk-installing-for-ios-development.md#building-the-tutorials + [android]: sdk-installing-for-android-development.md#building-the-tutorials + [warning]: images/icons/emoticons/warning.png diff --git a/sdk-playback-tutorial-color-balance.md b/sdk-playback-tutorial-color-balance.md index 63245dd213..eafbf81b2b 100644 --- a/sdk-playback-tutorial-color-balance.md +++ b/sdk-playback-tutorial-color-balance.md @@ -1,6 +1,6 @@ # Playback tutorial 5: Color Balance -# Goal +## Goal Brightness, Contrast, Hue and Saturation are common video adjustments, which are collectively known as Color Balance settings in GStreamer. @@ -9,43 +9,33 @@ This tutorial shows: - How to find out the available color balance channels - How to change them -# Introduction - -[Basic tutorial 5: GUI toolkit -integration](Basic%2Btutorial%2B5%253A%2BGUI%2Btoolkit%2Bintegration.html) has +## Introduction +[](sdk-basic-tutorial-toolkit-integration.md) has already explained the concept of GObject interfaces: applications use them to find out if certain functionality is available, regardless of the actual element which implements it. -`playbin` implements the Color Balance interface (`gstcolorbalance`), +`playbin` implements the Color Balance interface (`GstColorBalance`), which allows access to the color balance settings. If any of the -elements in the `playbin` pipeline support this interface, -`playbin` simply forwards it to the application, otherwise, a +elements in the `playbin` pipeline support this interface, +`playbin` simply forwards it to the application, otherwise, a colorbalance element is inserted in the pipeline. This interface allows querying for the available color balance channels -(`gstcolorbalancechannel`), along with their name and valid range of +(`GstColorBalanceChannel`), along with their name and valid range of values, and then modify the current value of any of them. -# Color balance example +## Color balance example -Copy this code into a text file named `playback-tutorial-5.c`. - - - - - - - - -

This tutorial is included in the SDK since release 2012.7. If you cannot find it in the downloaded code, please install the latest release of the GStreamer SDK.

+Copy this code into a text file named `playback-tutorial-5.c`. **playback-tutorial-5.c** ``` c #include +#include #include -#include +#include typedef struct _CustomData { GstElement *pipeline; @@ -161,7 +151,7 @@ int main(int argc, char *argv[]) { data.pipeline = gst_parse_launch ("playbin uri=http://docs.gstreamer.com/media/sintel_trailer-480p.webm", NULL); /* Add a keyboard watch so we get notified of keystrokes */ -#ifdef _WIN32 +#ifdef G_OS_WIN32 io_stdin = g_io_channel_win32_new_fd (fileno (stdin)); #else io_stdin = g_io_channel_unix_new (fileno (stdin)); @@ -190,36 +180,25 @@ int main(int argc, char *argv[]) { } ``` - - - - - - - -
-
-Need help? (Click to expand) -
-
-

If you need help to compile this code, refer to the Building the tutorials section for your platform: Linux, Mac OS X or Windows, or use this specific command on Linux:

-
-
-

gcc playback-tutorial-5.c -o playback-tutorial-5 `pkg-config --cflags --libs gstreamer-interfaces-1.0 gstreamer-1.0`

-
-
-

If you need help to run this code, refer to the Running the tutorials section for your platform: Linux, Mac OS X or Windows

-

-

This tutorial opens a window and displays a movie, with accompanying audio. The media is fetched from the Internet, so the window might take a few seconds to appear, depending on your connection speed.

-

The console should print all commands (Each command is a single upper-case or lower-case letter) and list all available Color Balance channels, typically, CONTRAST, BRIGHTNESS, HUE and SATURATION. Type each command (letter) followed by the Enter key.

-

-

Required libraries: gstreamer-interfaces-1.0 gstreamer-1.0

-
-
+> ![information] If you need help to compile this code, refer to the +> **Building the tutorials** section for your platform: [Mac] or +> [Windows] or use this specific command on Linux: +> +> `` gcc playback-tutorial-5.c -o playback-tutorial-5 `pkg-config --cflags --libs gstreamer-1.0 gstreamer-video-1.0` `` +> +> If you need help to run this code, refer to the **Running the +> tutorials** section for your platform: [Mac OS X], [Windows][1], for +> [iOS] or for [android]. +> +> This tutorial opens a window and displays a movie, with accompanying audio. The media is fetched from the Internet, so the window might take a few seconds to appear, depending on your connection speed. +> +>The console should print all commands (Each command is a single upper-case or lower-case letter) and list all available Color Balance channels, typically, CONTRAST, BRIGHTNESS, HUE and SATURATION. Type each command (letter) followed by the Enter key. +> +> Required libraries: `gstreamer-1.0 gstreamer-video-1.0` -# Walkthrough +## Walkthrough -The `main()` function is fairly simple. A `playbin` pipeline is +The `main()` function is fairly simple. A `playbin` pipeline is instantiated and set to run, and a keyboard watch is installed so keystrokes can be monitored. @@ -242,12 +221,12 @@ static void print_current_values (GstElement *pipeline) { This method prints the current value for all channels, and exemplifies how to retrieve the list of channels. This is accomplished through the -`gst_color_balance_list_channels()` method. It returns a `GList` which +`gst_color_balance_list_channels()` method. It returns a `GList` which needs to be traversed. -Each element in the list is a `GstColorBalanceChannel` structure, +Each element in the list is a `GstColorBalanceChannel` structure, informing of the channel’s name, minimum value and maximum value. -`gst_color_balance_get_value()` can then be called on each channel to +`gst_color_balance_get_value()` can then be called on each channel to retrieve the current value. In this example, the minimum and maximum values are used to output the @@ -300,26 +279,29 @@ stored and indexed by something more efficient than a string. The current value for the channel is then retrieved, changed (the increment is proportional to its dynamic range), clamped (to avoid -out-of-range values) and set using `gst_color_balance_set_value()`. +out-of-range values) and set using `gst_color_balance_set_value()`. And there is not much more to it. Run the program and observe the effect of changing each of the channels in real time. -# Conclusion +## Conclusion This tutorial has shown how to use the color balance interface. Particularly, it has shown: - How to retrieve the list of color available balance channels - with `gst_color_balance_list_channels()` + with `gst_color_balance_list_channels()` - How to manipulate the current value of each channel using - `gst_color_balance_get_value()` and `gst_color_balance_set_value()` + `gst_color_balance_get_value()` and `gst_color_balance_set_value()` It has been a pleasure having you here, and see you soon\! -## Attachments: -![](images/icons/bullet_blue.gif) -[playback-tutorial-5.c](attachments/327804/2424874.c) (text/plain) -![](images/icons/bullet_blue.gif) -[vs2010.zip](attachments/327804/2424875.zip) (application/zip) + [information]: images/icons/emoticons/information.png + [Mac]: sdk-installing-on-mac-osx.md + [Windows]: Installing+on+Windows + [Mac OS X]: sdk-installing-on-mac-osx.md#building-the-tutorials + [1]: sdk-installing-on-windows.md#running-the-tutorials + [iOS]: sdk-installing-for-ios-development.md#building-the-tutorials + [android]: sdk-installing-for-android-development.md#building-the-tutorials + [warning]: images/icons/emoticons/warning.png diff --git a/sdk-playback-tutorial-custom-playbin-sinks.md b/sdk-playback-tutorial-custom-playbin-sinks.md index 2c3cfde764..5249b5632b 100644 --- a/sdk-playback-tutorial-custom-playbin-sinks.md +++ b/sdk-playback-tutorial-custom-playbin-sinks.md @@ -1,55 +1,46 @@ # Playback tutorial 7: Custom playbin sinks -# Goal +## Goal -`playbin` can be further customized by manually selecting its audio and -video sinks. This allows applications to rely on `playbin` to retrieve +`playbin` can be further customized by manually selecting its audio and +video sinks. This allows applications to rely on `playbin` to retrieve and decode the media and then manage the final render/display themselves. This tutorial shows: - How to replace the sinks selected by `playbin`. - How to use a complex pipeline as a sink. -# Introduction +## Introduction -Two properties of `playbin` allow selecting the desired audio and video -sinks: `audio-sink` and `video-sink` (respectively). The application -only needs to instantiate the appropriate `GstElement` and pass it to -`playbin` through these properties. +Two properties of `playbin` allow selecting the desired audio and video +sinks: `audio-sink` and `video-sink` (respectively). The application +only needs to instantiate the appropriate `GstElement` and pass it to +`playbin` through these properties. This method, though, only allows using a single Element as sink. If a more complex pipeline is required, for example, an equalizer plus an audio sink, it needs to be wrapped in a Bin, so it looks to -`playbin` as if it was a single Element. +`playbin` as if it was a single Element. A Bin (`GstBin`) is a container that encapsulates partial pipelines so they can be managed as single elements. As an example, the -`GstPipeline` we have been using in all tutorials is a type of +`GstPipeline` we have been using in all tutorials is a type of `GstBin`, which does not interact with external Elements. Elements inside a Bin connect to external elements through Ghost Pads (`GstGhostPad`), this is, Pads on the surface of the Bin which simply forward data from an external Pad to a given Pad on an internal Element. -![](attachments/1441842/2424880.png) +![](images/bin-element-ghost.png) **Figure 1:** A Bin with two Elements and one Ghost Pad. -`GstBin`s are also a type of `GstElement`, so they can be used wherever -an Element is required, in particular, as sinks for `playbin` (and they +`GstBin`s are also a type of `GstElement`, so they can be used wherever +an Element is required, in particular, as sinks for `playbin` (and they are then known as **sink-bins**). -# An equalized player +## An equalized player -Copy this code into a text file named `playback-tutorial-7.c`. - - - - - - - - -

This tutorial is included in the SDK since release 2012.7. If you cannot find it in the downloaded code, please install the latest release of the GStreamer SDK.

+Copy this code into a text file named `playback-tutorial-7.c`. **playback-tutorial7.c** @@ -111,31 +102,21 @@ int main(int argc, char *argv[]) { } ``` - - - - - - - -
-
-Need help? (Click to expand) -
-
-

If you need help to compile this code, refer to the Building the tutorials section for your platform: Linux, Mac OS X or Windows, or use this specific command on Linux:

-
-
-

gcc playback-tutorial-7.c -o playback-tutorial-7 `pkg-config --cflags --libs gstreamer-1.0`

-
-
-

If you need help to run this code, refer to the Running the tutorials section for your platform: Linux, Mac OS X or Windows

-

This tutorial opens a window and displays a movie, with accompanying audio. The media is fetched from the Internet, so the window might take a few seconds to appear, depending on your connection speed. The higher frequency bands have been attenuated, so the movie sound should have a more powerful bass component.

-

Required libraries: gstreamer-1.0

-
-
+> ![information] If you need help to compile this code, refer to the +> **Building the tutorials** section for your platform: [Mac] or +> [Windows] or use this specific command on Linux: +> +> `` gcc playback-tutorial-7.c -o playback-tutorial-7 `pkg-config --cflags --libs gstreamer-1.0` `` +> +> If you need help to run this code, refer to the **Running the +> tutorials** section for your platform: [Mac OS X], [Windows][1], for +> [iOS] or for [android]. +> +> This tutorial opens a window and displays a movie, with accompanying audio. The media is fetched from the Internet, so the window might take a few seconds to appear, depending on your connection speed. The higher frequency bands have been attenuated, so the movie sound should have a more powerful bass component.< +> +> Required libraries: `gstreamer-1.0` -# Walkthrough +## Walkthrough ``` c /* Create the elements inside the sink bin */ @@ -149,7 +130,7 @@ if (!equalizer || !convert || !sink) { ``` All the Elements that compose our sink-bin are instantiated. We use an -`equalizer-3bands` and an `autoaudiosink`, with an `audioconvert` in +`equalizer-3bands` and an `autoaudiosink`, with an `audioconvert` in between, because we are not sure of the capabilities of the audio sink (since they are hardware-dependant). @@ -175,14 +156,13 @@ Now we need to create a Ghost Pad so this partial pipeline inside the Bin can be connected to the outside. This Ghost Pad will be connected to a Pad in one of the internal Elements (the sink pad of the equalizer), so we retrieve this Pad with `gst_element_get_static_pad()`. Remember -from [Basic tutorial 7: Multithreading and Pad -Availability](Basic%2Btutorial%2B7%253A%2BMultithreading%2Band%2BPad%2BAvailability.html) that +from [](sdk-basic-tutorial-multithreading-and-pad-availability.md) that if this was a Request Pad instead of an Always Pad, we would need to use `gst_element_request_pad()`. -The Ghost Pad is created with `gst_ghost_pad_new()` (pointing to the +The Ghost Pad is created with `gst_ghost_pad_new()` (pointing to the inner Pad we just acquired), and activated with `gst_pad_set_active()`. -It is then added to the Bin with `gst_element_add_pad()`, transferring +It is then added to the Bin with `gst_element_add_pad()`, transferring ownership of the Ghost Pad to the bin, so we do not have to worry about releasing it. @@ -190,14 +170,14 @@ Finally, the sink Pad we obtained from the equalizer needs to be release with `gst_object_unref()`. At this point, we have a functional sink-bin, which we can use as the -audio sink in `playbin`. We just need to instruct `playbin` to use it: +audio sink in `playbin`. We just need to instruct `playbin` to use it: ``` c /* Set playbin's audio sink to be our sink bin */ g_object_set (GST_OBJECT (pipeline), "audio-sink", bin, NULL); ``` -It is as simple as setting the `audio-sink` property on `playbin` to +It is as simple as setting the `audio-sink` property on `playbin` to the newly created sink. ``` c @@ -209,33 +189,33 @@ g_object_set (G_OBJECT (equalizer), "band2", (gdouble)-24.0, NULL); The only bit remaining is to configure the equalizer. For this example, the two higher frequency bands are set to the maximum attenuation so the bass is boosted. Play a bit with the values to feel the difference (Look -at the documentation for the `equalizer-3bands` element for the allowed +at the documentation for the `equalizer-3bands` element for the allowed range of values). -# Exercise +## Exercise Build a video bin instead of an audio bin, using one of the many interesting video filters GStreamer offers, like `solarize`, -`vertigotv` or any of the Elements in the `effectv` plugin. Remember to -use the color space conversion element `ffmpegcolorspace` if your +`vertigotv` or any of the Elements in the `effectv` plugin. Remember to +use the color space conversion element `videoconvert` if your pipeline fails to link due to incompatible caps. -# Conclusion +## Conclusion This tutorial has shown: - - How to set your own sinks to `playbin` using the audio-sink and + - How to set your own sinks to `playbin` using the audio-sink and video-sink properties. - - How to wrap a piece of pipeline into a `GstBin` so it can be used as - a **sink-bin** by `playbin`. + - How to wrap a piece of pipeline into a `GstBin` so it can be used as + a **sink-bin** by `playbin`. It has been a pleasure having you here, and see you soon\! -## Attachments: - -![](images/icons/bullet_blue.gif) -[bin-element-ghost.png](attachments/1441842/2424880.png) (image/png) -![](images/icons/bullet_blue.gif) -[playback-tutorial-7.c](attachments/1441842/2424881.c) (text/plain) -![](images/icons/bullet_blue.gif) -[vs2010.zip](attachments/1441842/2424882.zip) (application/zip) + [information]: images/icons/emoticons/information.png + [Mac]: sdk-installing-on-mac-osx.md + [Windows]: Installing+on+Windows + [Mac OS X]: sdk-installing-on-mac-osx.md#building-the-tutorials + [1]: sdk-installing-on-windows.md#running-the-tutorials + [iOS]: sdk-installing-for-ios-development.md#building-the-tutorials + [android]: sdk-installing-for-android-development.md#building-the-tutorials + [warning]: images/icons/emoticons/warning.png diff --git a/sdk-playback-tutorial-digital-audio-pass-through.md b/sdk-playback-tutorial-digital-audio-pass-through.md index 53adea9bc4..f0f5b88fba 100644 --- a/sdk-playback-tutorial-digital-audio-pass-through.md +++ b/sdk-playback-tutorial-digital-audio-pass-through.md @@ -1,10 +1,10 @@ # Playback tutorial 9: Digital audio pass-through -# Goal +## Goal This tutorial shows how GStreamer handles digital audio pass-through. -# Introduction +## Introduction Besides the common analog format, high-end audio systems usually also accept data in digital form, either compressed or uncompressed. This is @@ -23,7 +23,7 @@ In this scenario, GStreamer does not need to perform audio decoding; it can simply output the encoded data, acting in *pass-through* mode, and let the external audio system perform the decoding. -# Inner workings of GStreamer audio sinks +## Inner workings of GStreamer audio sinks First off, digital audio output must be enabled at the system level. The method to achieve this depend on the operating system, but it generally @@ -31,25 +31,24 @@ involves going to the audio control panel and activating a checkbox reading “Digital Audio Output” or similar. The main GStreamer audio sinks for each platform, Pulse Audio -(`pulsesink`) for Linux, `osxaudiosink` for OS X and Direct Sound +(`pulsesink`) for Linux, `osxaudiosink` for OS X and Direct Sound (`directsoundsink`) for Windows, detect when digital audio output is available and change their input caps accordingly to accept encoded -data. For example, these elements typically accept `audio/x-raw-int` or -`audio/x-raw-float` data: when digital audio output is enabled in the -system, they may also accept `audio/mpeg`, `audio/x-ac3`, -`audio/x-eac3` or `audio/x-dts`. +data. For example, these elements typically accept `audio/x-raw` data: +when digital audio output is enabled in the system, they may also +accept `audio/mpeg`, `audio/x-ac3`, `audio/x-eac3` or `audio/x-dts`. -Then, when `playbin` builds the decoding pipeline, it realizes that the +Then, when `playbin` builds the decoding pipeline, it realizes that the audio sink can be directly connected to the encoded data (typically coming out of a demuxer), so there is no need for a decoder. This process is automatic and does not need any action from the application. On Linux, there exist other audio sinks, like Alsa (`alsasink`) which work differently (a “digital device” needs to be manually selected -through the `device` property of the sink). Pulse Audio, though, is the +through the `device` property of the sink). Pulse Audio, though, is the commonly preferred audio sink on Linux. -# Precautions with digital formats +## Precautions with digital formats When Digital Audio Output is enabled at the system level, the GStreamer audio sinks automatically expose all possible digital audio caps, @@ -60,8 +59,8 @@ supported, and, in fact, the cable can even be disconnected during this process. For example, after enabling Digital Audio Output in the system’s Control -Panel,  `directsoundsink`  will automatically expose `audio/x-ac3`, -`audio/x-eac3` and `audio/x-dts` caps in addition to `audio/x-raw-int`. +Panel, `directsoundsink` will automatically expose `audio/x-ac3`, +`audio/x-eac3` and `audio/x-dts` caps in addition to `audio/x-raw`. However, one particular external decoder might only understand raw integer streams and would try to play the compressed data as such (a painful experience for your ears, rest assured). @@ -77,28 +76,26 @@ configuration panel, from the same place where Digital Audio Output is enabled, but, unfortunately, this option is not available in all audio drivers. -Another solution involves, using a custom sinkbin (see [Playback -tutorial 7: Custom playbin -sinks](Playback%2Btutorial%2B7%253A%2BCustom%2Bplaybin%2Bsinks.html)) -which includes a `capsfilter` element (see [Basic tutorial 14: Handy -elements](Basic%2Btutorial%2B14%253A%2BHandy%2Belements.html)) and an -audio sink. The caps that the external decoder supports are then set in -the capsfiler so the wrong format is not output. This allows the -application to enforce the appropriate format instead of relying on the -user to have the system correctly configured. Still requires user -intervention, but can be used regardless of the options the audio driver -offers. +Another solution involves, using a custom sinkbin (see +[](sdk-playback-tutorial-custom-playbin-sinks.md)) which includes a +`capsfilter` element (see [](sdk-basic-tutorial-handy-elements.md)) +and an audio sink. The caps that the external decoder supports are +then set in the capsfiler so the wrong format is not output. This +allows the application to enforce the appropriate format instead of +relying on the user to have the system correctly configured. Still +requires user intervention, but can be used regardless of the options +the audio driver offers. -Please do not use `autoaudiosink` as the audio sink, as it currently +Please do not use `autoaudiosink` as the audio sink, as it currently only supports raw audio, and will ignore any compressed format. -# Conclusion +## Conclusion This tutorial has shown a bit of how GStreamer deals with digital audio. In particular, it has shown that: - - Applications using `playbin` do not need to do anything special to + - Applications using `playbin` do not need to do anything special to enable digital audio output: it is managed from the audio control panel of the operating system. -It has been a pleasure having you here, and see you soon\! +It has been a pleasure having you here, and see you soon! diff --git a/sdk-playback-tutorial-hardware-accelerated-video-decoding.md b/sdk-playback-tutorial-hardware-accelerated-video-decoding.md index 25292cb40e..6ae8cc1440 100644 --- a/sdk-playback-tutorial-hardware-accelerated-video-decoding.md +++ b/sdk-playback-tutorial-hardware-accelerated-video-decoding.md @@ -1,6 +1,6 @@ # Playback tutorial 8: Hardware-accelerated video decoding -# Goal +## Goal Hardware-accelerated video decoding has rapidly become a necessity, as low-power devices grow more common. This tutorial (more of a lecture, @@ -11,7 +11,7 @@ Sneak peek: if properly setup, you do not need to do anything special to activate hardware acceleration; GStreamer automatically takes advantage of it. -# Introduction +## Introduction Video decoding can be an extremely CPU-intensive task, especially for higher resolutions like 1080p HDTV. Fortunately, modern graphics cards, @@ -20,152 +20,114 @@ allowing the CPU to concentrate on other duties. Having dedicated hardware becomes essential for low-power CPUs which are simply incapable of decoding such media fast enough. -In the current state of things (July-2012) each GPU manufacturer offers +In the current state of things (June 2016) each GPU manufacturer offers a different method to access their hardware (a different API), and a strong industry standard has not emerged yet. -As of July-2012, there exist at least 8 different video decoding +As of June 2016, there exist at least 8 different video decoding acceleration APIs: -[VAAPI](http://en.wikipedia.org/wiki/Video_Acceleration_API) (*Video + - [VAAPI](http://en.wikipedia.org/wiki/Video_Acceleration_API) (*Video Acceleration API*): Initially designed by -[Intel](http://en.wikipedia.org/wiki/Intel) in 2007, targeted at the X -Window System on Unix-based operating systems, now open-source. It is +[Intel](http://en.wikipedia.org/wiki/Intel) in 2007, targeted at the X +Window System on Unix-based operating systems, now open-source. It now also +supports Wayland through dmabuf. It is currently not limited to Intel GPUs as other manufacturers are free to use this API, for example, [Imagination Technologies](http://en.wikipedia.org/wiki/Imagination_Technologies) or [S3 Graphics](http://en.wikipedia.org/wiki/S3_Graphics). Accessible to -GStreamer through -the [gstreamer-vaapi](http://gitorious.org/vaapi/gstreamer-vaapi) and -[Fluendo](http://en.wikipedia.org/wiki/Fluendo)’s Video Acceleration -Decoder (fluvadec) plugins. +GStreamer through the [gstreamer-vaapi](https://cgit.freedesktop.org/gstreamer/gstreamer-vaapi/) package. -[VDPAU](http://en.wikipedia.org/wiki/VDPAU) (*Video Decode and +- [VDPAU](http://en.wikipedia.org/wiki/VDPAU) (*Video Decode and Presentation API for UNIX*): Initially designed by -[NVidia](http://en.wikipedia.org/wiki/NVidia) in 2008, targeted at the X +[NVidia](http://en.wikipedia.org/wiki/NVidia) in 2008, targeted at the X Window System on Unix-based operating systems, now open-source. Although it is also an open-source library, no manufacturer other than NVidia is -using it yet. Accessible to GStreamer through -the [vdpau](http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/sys/vdpau) element -in plugins-bad and [Fluendo](http://en.wikipedia.org/wiki/Fluendo)’s -Video Acceleration Decoder (fluvadec) plugins. +using it yet. Accessible to GStreamer through +the [vdpau](http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/sys/vdpau) element in plugins-bad. -[DXVA](http://en.wikipedia.org/wiki/DXVA) (*DirectX Video -Acceleration*): [Microsoft](http://en.wikipedia.org/wiki/Microsoft) API -specification for the Microsoft Windows and Xbox 360 -platforms. Accessible to GStreamer through -the [Fluendo](http://en.wikipedia.org/wiki/Fluendo)’s Video -Acceleration Decoder (fluvadec) plugin. - -[XVBA](http://en.wikipedia.org/wiki/Xvba) (*X-Video Bitstream -Acceleration*): Designed by [AMD -Graphics](http://en.wikipedia.org/wiki/AMD_Graphics), is an arbitrary -extension of the X video extension (Xv) for the X Window System on Linux -operating-systems. Currently only AMD's ATI Radeon graphics cards -hardware that have support for Unified Video Decoder version 2.0 or -later are supported by the proprietary ATI Catalyst device -driver. Accessible to GStreamer through -the [Fluendo](http://en.wikipedia.org/wiki/Fluendo)’s Video -Acceleration Decoder -(fluvadec) plugin. - -[VDA](http://developer.apple.com/library/mac/#technotes/tn2267/_index.html) -(*Video Decode Acceleration*): Available on [Mac OS -X](http://en.wikipedia.org/wiki/OS_X) v10.6.3 and later with Mac models -equipped with the NVIDIA GeForce 9400M, GeForce 320M, GeForce GT 330M, -ATI HD Radeon GFX, Intel HD Graphics and others. Only accelerates -decoding of H.264 media. Accessible to GStreamer through -the [Fluendo](http://en.wikipedia.org/wiki/Fluendo)’s Video -Acceleration Decoder (fluvadec) plugin. - -[OpenMAX](http://en.wikipedia.org/wiki/OpenMAX) (*Open Media -Acceleration*): Managed by the non-profit technology consortium [Khronos + - [OpenMAX](http://en.wikipedia.org/wiki/OpenMAX) (*Open Media +Acceleration*): Managed by the non-profit technology consortium [Khronos Group](http://en.wikipedia.org/wiki/Khronos_Group "Khronos Group"), -it is a "royalty-free, cross-platform set of C-language programming +it is a "royalty-free, cross-platform set of C-language programming interfaces that provides abstractions for routines especially useful for -audio, video, and still images". Accessible to GStreamer through -the [gstreamer-omx](http://git.freedesktop.org/gstreamer/gst-omx) plugin. +audio, video, and still images". Accessible to GStreamer through +the [gst-omx](http://git.freedesktop.org/gstreamer/gst-omx) plugin. -[OVD](http://developer.amd.com/sdks/AMDAPPSDK/assets/OpenVideo_Decode_API.PDF) + - [OVD](http://developer.amd.com/sdks/AMDAPPSDK/assets/OpenVideo_Decode_API.PDF) (*Open Video Decode*): Another API from [AMD -Graphics](http://en.wikipedia.org/wiki/AMD_Graphics), designed to be a +Graphics](http://en.wikipedia.org/wiki/AMD_Graphics), designed to be a platform agnostic method for softrware developers to leverage the [Universal Video Decode](http://en.wikipedia.org/wiki/Unified_Video_Decoder) (UVD) -hardware inside AMD Radeon graphics cards. Currently unavailable to -GStreamer. +hardware inside AMD Radeon graphics cards. Currently unavailable to +GStreamer . -[DCE](http://en.wikipedia.org/wiki/Distributed_Codec_Engine) -(*Distributed Codec Engine*): An open source software library ("libdce") -and API specification by [Texas + - [DCE](http://en.wikipedia.org/wiki/Distributed_Codec_Engine) +(*Distributed Codec Engine*): An open source software library ("libdce") +and API specification by [Texas Instruments](http://en.wikipedia.org/wiki/Texas_Instruments), targeted -at Linux systems and ARM platforms. Accessible to GStreamer through -the [gstreamer-ducati](https://github.com/robclark/gst-ducati) plugin. +at Linux systems and ARM platforms. Accessible to GStreamer through +the [gstreamer-ducati](https://github.com/robclark/gst-ducati) plugin. -There exist some GStreamer plugins, like the -[gstreamer-vaapi](http://gitorious.org/vaapi/gstreamer-vaapi) project or -the -[vdpau](http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/sys/vdpau) -element in plugins-bad, which target one particular hardware -acceleration API and expose its functionality through different -GStreamer elements. The application is then responsible for selecting -the appropriate plugin depending on the available APIs. + - [Android + MediaCodec](https://developer.android.com/reference/android/media/MediaCodec.html): This is Android's API to access the device's + hardware decoder and encoder if available. This is accessible through the + `androidmedia` plugin in gst-plugins-bad. This includes both encoding and + decoding. -Some other GStreamer plugins, like -[Fluendo](http://en.wikipedia.org/wiki/Fluendo)’s Video Acceleration -Decoder (fluvadec), detect at runtime the available APIs and select one -automatically. This makes any program using these plugins independent of -the API, or even the operating system. + - Apple VideoTool Box Framework: Apple's API to access h is available + through the `applemedia` plugin which includes both encoding through + the `vtenc` element and decoding through the `vtdec` element. -# Inner workings of hardware-accelerated video decoding plugins + - Video4Linux: Recent Linux kernels have a kernel API to expose + hardware codecs in a standard way, this is now supported by the + `v4l2` plugin in `gst-plugins-good`. This can support both decoding + and encoding depending on the platform. + +## Inner workings of hardware-accelerated video decoding plugins These APIs generally offer a number of functionalities, like video -decoding, post-processing, presentation of the decoded frames, or -download of such frames to system memory. Correspondingly, plugins -generally offer a different GStreamer element for each of these -functions, so pipelines can be built to accommodate any need. +decoding, post-processing, or presentation of the decoded +frames. Correspondingly, plugins generally offer a different GStreamer +element for each of these functions, so pipelines can be built to +accommodate any need. -For example, the `gstreamer-vaapi` plugin offers the `vaapidecode`, -`vaapiupload`, `vaapidownload` and `vaapisink` elements that allow +For example, the `gstreamer-vaapi` plugin offers the `vaapidecode`, +`vaapipostproc` and `vaapisink` elements that allow hardware-accelerated decoding through VAAPI, upload of raw video frames to GPU memory, download of GPU frames to system memory and presentation of GPU frames, respectively. It is important to distinguish between conventional GStreamer frames, which reside in system memory, and frames generated by -hardware-accelerated APIs. The latter reside in GPU memory and cannot be -touched by GStreamer. They can usually be downloaded to system memory -and treated as conventional GStreamer frames, but it is far more -efficient to leave them in the GPU and display them from there. +hardware-accelerated APIs. The latter reside in GPU memory and cannot +be touched by GStreamer. They can usually be downloaded to system +memory and treated as conventional GStreamer frames when they are +mapped, but it is far more efficient to leave them in the GPU and +display them from there. GStreamer needs to keep track of where these “hardware buffers” are -though, so conventional buffers still travel from element to element, -but their only content is a hardware buffer ID, or handler. If retrieved -with an `appsink`, for example, hardware buffers make no sense, since -they are meant to be handled only by the plugin that generated them. - -To indicate this, these buffers have special Caps, like -`video/x-vdpau-output` or `video/x-fluendo-va`. In this way, the -auto-plugging mechanism of GStreamer will not try to feed hardware -buffers to conventional elements, as they would not understand the -received buffers. Moreover, using these Caps, the auto-plugger is able -to automatically build pipelines that use hardware acceleration, since, -after a VAAPI decoder, a VAAPI sink is the only element that fits. +though, so conventional buffers still travel from element to +element. They look like regular buffers, but mapping their content is +much slower as it has to be retrieved from the special memory used by +hardware accelerated elements. This special memory types are +negotiated using the allocation query mechanism. This all means that, if a particular hardware acceleration API is present in the system, and the corresponding GStreamer plugin is also -available, auto-plugging elements like `playbin` are free to use +available, auto-plugging elements like `playbin` are free to use hardware acceleration to build their pipelines; the application does not need to do anything special to enable it. Almost: -When `playbin` has to choose among different equally valid elements, +When `playbin` has to choose among different equally valid elements, like conventional software decoding (through `vp8dec`, for example) or hardware accelerated decoding (through `vaapidecode`, for example), it uses their *rank* to decide. The rank is a property of each element that -indicates its priority; `playbin` will simply select the element that +indicates its priority; `playbin` will simply select the element that is able to build a complete pipeline and has the highest rank. -So, whether `playbin` will use hardware acceleration or not will depend +So, whether `playbin` will use hardware acceleration or not will depend on the relative ranks of all elements capable of dealing with that media type. Therefore, the easiest way to make sure hardware acceleration is enabled or disabled is by changing the rank of the associated element, @@ -195,9 +157,9 @@ static void enable_factory (const gchar *name, gboolean enable) { ``` The first parameter passed to this method is the name of the element to -modify, for example, `vaapidecode` or `fluvadec`. +modify, for example, `vaapidecode` or `fluvadec`. -The key method is `gst_plugin_feature_set_rank()`, which will set the +The key method is `gst_plugin_feature_set_rank()`, which will set the rank of the requested element factory to the desired level. For convenience, ranks are divided in NONE, MARGINAL, SECONDARY and PRIMARY, but any number will do. When enabling an element, we set it to @@ -205,132 +167,8 @@ PRIMARY+1, so it has a higher rank than the rest of elements which commonly have PRIMARY rank. Setting an element’s rank to NONE will make the auto-plugging mechanism to never select it. -# Hardware-accelerated video decoding and the GStreamer SDK - -There are no plugins deployed in the GStreamer SDK Amazon 2012.7 that -allow hardware-accelerated video decoding. The main reasons are that -some of them are not yet fully operational, or still have issues, or are -proprietary. Bear in mind that this situation is bound to change in the -near future, as this is a very active area of development. - -Some of these plugins can be built from their publicly available -sources, using the Cerbero build system (see [Installing on -Linux](Installing%2Bon%2BLinux.html)) or independently (linking against -the GStreamer SDK libraries, obviously). Some other plugins are readily -available in binary form from their vendors. - -The following sections try to summarize the current state of some of -these plugins. - -### vdpau in gst-plugins-bad - - - GStreamer element for VDPAU, present in - [gst-plugins-bad](http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/sys/vdpau). - - Supported codecs:  - - - - - - - - - - - -
MPEG2MPEG4H.264
- -### gstreamer-vaapi - - - GStreamer element for VAAPI. Standalone project hosted at - [gstreamer-vaapi](http://gitorious.org/vaapi/gstreamer-vaapi). - - Supported codecs: - - - - - - - - - - - - - -
MPEG2MPEG4H.264VC1WMV3
- - - Can interface directly with Clutter (See [Basic tutorial 15: Clutter - integration](Basic%2Btutorial%2B15%253A%2BClutter%2Bintegration.html)), - so frames do not need to leave the GPU. - - Compatible with `playbin`. - -### gst-omx - - - GStreamer element for OpenMAX. Standalone project hosted at - [gst-omx](http://git.freedesktop.org/gstreamer/gst-omx/). - - Supported codecs greatly vary depending on the underlying hardware. - -### fluvadec - - - GStreamer element for VAAPI, VDPAU, DXVA2, XVBA and VDA from - [Fluendo](http://en.wikipedia.org/wiki/Fluendo) (propietary). - - Supported codecs depend on the chosen API, which is selected at - runtime depending on what is available on the system: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
 MPEG2MPEG4H.264VC1
VAAPI✓✓✓✓
VDPAU✓✓✓✓
XVBA  ✓✓
DXVA2  ✓ 
VDA  ✓ 
- - - Can interface directly with Clutter (See [Basic tutorial 15: Clutter - integration](Basic%2Btutorial%2B15%253A%2BClutter%2Bintegration.html)), - so frames do not need to leave the GPU. - - Compatible with `playbin`. +> ![warning] The GStreamer developers often rank hardware decoders lower than +> the software ones when they are defective. This should act as a warning. # Conclusion @@ -343,4 +181,6 @@ accelerated video decoding. Particularly, - Hardware acceleration can be enabled or disabled by changing the rank of the decoding element with `gst_plugin_feature_set_rank()`. -It has been a pleasure having you here, and see you soon\! +It has been a pleasure having you here, and see you soon! + + [warning]: images/icons/emoticons/warning.png diff --git a/sdk-playback-tutorial-playbin-usage.md b/sdk-playback-tutorial-playbin-usage.md index 3023e396a4..329b27193a 100644 --- a/sdk-playback-tutorial-playbin-usage.md +++ b/sdk-playback-tutorial-playbin-usage.md @@ -1,10 +1,10 @@ # Playback tutorial 1: Playbin usage -# Goal +## Goal -We have already worked with the `playbin` element, which is capable of +We have already worked with the `playbin` element, which is capable of building a complete playback pipeline without much work on our side. -This tutorial shows how to further customize `playbin` in case its +This tutorial shows how to further customize `playbin` in case its default values do not suit our particular needs. We will learn: @@ -15,10 +15,10 @@ We will learn: - How to gather information regarding each stream. As a side note, even though its name is `playbin`, you can pronounce it -“playbin”, since the original `playbin` element is deprecated and nobody +“playbin”, since the original `playbin` element is deprecated and nobody should be using it. -# Introduction +## Introduction More often than not, multiple audio, video and subtitle streams can be found embedded in a single file. The most common case are regular @@ -52,9 +52,9 @@ The following code recovers the amount of streams in the file, their associated metadata, and allows switching the audio stream while the media is playing. -# The multilingual player +## The multilingual player -Copy this code into a text file named `playback-tutorial-1.c` (or find +Copy this code into a text file named `playback-tutorial-1.c` (or find it in the SDK installation). **playback-tutorial-1.c** @@ -123,7 +123,7 @@ int main(int argc, char *argv[]) { gst_bus_add_watch (bus, (GstBusFunc)handle_message, &data); /* Add a keyboard watch so we get notified of keystrokes */ -#ifdef _WIN32 +#ifdef G_OS_WIN32 io_stdin = g_io_channel_win32_new_fd (fileno (stdin)); #else io_stdin = g_io_channel_unix_new (fileno (stdin)); @@ -266,7 +266,7 @@ static gboolean handle_keyboard (GIOChannel *source, GIOCondition cond, CustomDa gchar *str = NULL; if (g_io_channel_read_line (source, &str, NULL, NULL, NULL) == G_IO_STATUS_NORMAL) { - int index = atoi (str); + int index = g_ascii_strtoull (str, NULL, 0); if (index < 0 || index >= data->n_audio) { g_printerr ("Index out of bounds\n"); } else { @@ -283,26 +283,27 @@ static gboolean handle_keyboard (GIOChannel *source, GIOCondition cond, CustomDa > ![information] If you need help to compile this code, refer to the > **Building the tutorials** section for your platform: [Mac] or > [Windows] or use this specific command on Linux: +> > `` gcc playback-tutorial-1.c -o playback-tutorial-1 `pkg-config --cflags --libs gstreamer-1.0` `` +> +> If you need help to run this code, refer to the **Running the +> tutorials** section for your platform: [Mac OS X], [Windows][1], for +> [iOS] or for [android]. +> +> This tutorial opens a window and displays a movie, with accompanying +> audio. The media is fetched from the Internet, so the window might take +> a few seconds to appear, depending on your connection speed. The number +> of audio streams is shown in the terminal, and the user can switch from +> one to another by entering a number and pressing enter. A small delay is +> to be expected. +> +> Bear in mind that there is no latency management (buffering), so on slow +> connections, the movie might stop after a few seconds. See how [Tutorial +> 12: Live streaming] solves this issue. +> +> Required libraries: `gstreamer-1.0` -If you need help to run this code, refer to the **Running the -tutorials** section for your platform: [Mac OS X], [Windows][1], for -[iOS] or for [android]. - -This tutorial opens a window and displays a movie, with accompanying -audio. The media is fetched from the Internet, so the window might take -a few seconds to appear, depending on your connection speed. The number -of audio streams is shown in the terminal, and the user can switch from -one to another by entering a number and pressing enter. A small delay is -to be expected. - -Bear in mind that there is no latency management (buffering), so on slow -connections, the movie might stop after a few seconds. See how [Tutorial -12: Live streaming] solves this issue. - -Required libraries: `gstreamer-1.0` - -# Walkthrough +## Walkthrough ``` c /* Structure to contain all our information, so we can pass it around */ @@ -338,9 +339,9 @@ typedef enum { Later we are going to set some of `playbin`'s flags. We would like to have a handy enum that allows manipulating these flags easily, but since -`playbin` is a plug-in and not a part of the GStreamer core, this enum +`playbin` is a plug-in and not a part of the GStreamer core, this enum is not available to us. The “trick” is simply to declare this enum in -our code, as it appears in the `playbin` documentation: `GstPlayFlags`. +our code, as it appears in the `playbin` documentation: `GstPlayFlags`. GObject allows introspection, so the possible values for these flags can be retrieved at runtime without using this trick, but in a far more cumbersome way. @@ -352,17 +353,17 @@ static gboolean handle_keyboard (GIOChannel *source, GIOCondition cond, CustomDa ``` Forward declarations for the two callbacks we will be using. -`handle_message` for the GStreamer messages, as we have already seen, -and `handle_keyboard` for key strokes, since this tutorial is +`handle_message` for the GStreamer messages, as we have already seen, +and `handle_keyboard` for key strokes, since this tutorial is introducing a limited amount of interactivity. We skip over the creation of the pipeline, the instantiation of -`playbin` and pointing it to our test media through the `uri` -property. `playbin` is in itself a pipeline, and in this case it is the +`playbin` and pointing it to our test media through the `uri` +property. `playbin` is in itself a pipeline, and in this case it is the only element in the pipeline, so we skip completely the creation of the -pipeline, and use directly the  `playbin` element. +pipeline, and use directly the `playbin` element. -We focus on some of the other properties of `playbin`, though: +We focus on some of the other properties of `playbin`, though: ``` c /* Set flags to show Audio and Video but ignore Subtitles */ @@ -372,8 +373,8 @@ flags &= ~GST_PLAY_FLAG_TEXT; g_object_set (data.playbin, "flags", flags, NULL); ``` -`playbin`'s behavior can be changed through its `flags` property, which -can have any combination of `GstPlayFlags`. The most interesting values +`playbin`'s behavior can be changed through its `flags` property, which +can have any combination of `GstPlayFlags`. The most interesting values are: | Flag | Description | @@ -389,7 +390,7 @@ are: In our case, for demonstration purposes, we are enabling audio and video and disabling subtitles, leaving the rest of flags to their default values (this is why we read the current value of the flags with -`g_object_get()` before overwriting it with `g_object_set()`). +`g_object_get()` before overwriting it with `g_object_set()`). ``` c /* Set connection speed. This will affect some internal decisions of playbin */ @@ -397,10 +398,10 @@ g_object_set (data.playbin, "connection-speed", 56, NULL); ``` This property is not really useful in this example. -`connection-speed` informs `playbin` of the maximum speed of our network +`connection-speed` informs `playbin` of the maximum speed of our network connection, so, in case multiple versions of the requested media are -available in the server, `playbin` chooses the most appropriate. This is -mostly used in combination with streaming protocols like `mms` or +available in the server, `playbin` chooses the most appropriate. This is +mostly used in combination with streaming protocols like `mms` or `rtsp`. We have set all these properties one by one, but we could have all of @@ -410,7 +411,7 @@ them with a single call to `g_object_set()`: g_object_set (data.playbin, "uri", "http://docs.gstreamer.com/media/sintel_cropped_multilingual.webm", "flags", flags, "connection-speed", 56, NULL); ``` -This is why `g_object_set()` requires a NULL as the last parameter. +This is why `g_object_set()` requires a NULL as the last parameter. ``` c /* Add a keyboard watch so we get notified of keystrokes */ @@ -438,13 +439,13 @@ g_main_loop_run (data.main_loop); To allow interactivity, we will no longer poll the GStreamer bus manually. Instead, we create a `GMainLoop`(GLib main loop) and set it running with `g_main_loop_run()`. This function blocks and will not -return until `g_main_loop_quit()` is issued. In the meantime, it will +return until `g_main_loop_quit()` is issued. In the meantime, it will call the callbacks we have registered at the appropriate -times: `handle_message` when a message appears on the bus, and -`handle_keyboard` when the user presses any key. +times: `handle_message` when a message appears on the bus, and +`handle_keyboard` when the user presses any key. There is nothing new in handle\_message, except that when the pipeline -moves to the PLAYING state, it will call the `analyze_streams` function: +moves to the PLAYING state, it will call the `analyze_streams` function: ``` c /* Extract some metadata from the streams and print it on the screen */ @@ -461,9 +462,9 @@ static void analyze_streams (CustomData *data) { ``` As the comment says, this function just gathers information from the -media and prints it on the screen. The number of video, audio and +media and prints it on the screen. The number of video, audio and subtitle streams is directly available through the `n-video`, -`n-audio` and `n-text` properties. +`n-audio` and `n-text` properties. ``` c for (i = 0; i < data->n_video; i++) { @@ -481,13 +482,14 @@ for (i = 0; i < data->n_video; i++) { ``` Now, for each stream, we want to retrieve its metadata. Metadata is -stored as tags in a `GstTagList` structure, which is a list of data -pieces identified by a name. The `GstTagList` associated with a stream -can be recovered with `g_signal_emit_by_name()`, and then individual -tags are extracted with the `gst_tag_list_get_*` functions -like `gst_tag_list_get_string()` for example. +stored as tags in a `GstTagList` structure, which is a list of data +pieces identified by a name. The `GstTagList` associated with a stream +can be recovered with `g_signal_emit_by_name()`, and then individual +tags are extracted with the `gst_tag_list_get_*` functions +like `gst_tag_list_get_string()` for example. -> ![information] This rather unintuitive way of retrieving the tag list +> ![information] +> This rather unintuitive way of retrieving the tag list > is called an Action Signal. Action signals are emitted by the > application to a specific element, which then performs an action and > returns a result. They behave like a dynamic function call, in which @@ -496,7 +498,7 @@ like `gst_tag_list_get_string()` for example. > documentation along with the regular signals, and are tagged “Action”. > See `playbin`, for example. -`playbin` defines 3 action signals to retrieve metadata: +`playbin` defines 3 action signals to retrieve metadata: `get-video-tags`, `get-audio-tags` and `get-text-tags`. The name if the tags is standardized, and the list can be found in the `GstTagList` documentation. In this example we are interested in the @@ -511,11 +513,11 @@ g_object_get (data->playbin, "current-text", &data->current_text, NULL); Once we have extracted all the metadata we want, we get the streams that are currently selected through 3 more properties of `playbin`: -`current-video`, `current-audio` and `current-text`.  +`current-video`, `current-audio` and `current-text`. It is interesting to always check the currently selected streams and never make any assumption. Multiple internal conditions can make -`playbin` behave differently in different executions. Also, the order in +`playbin` behave differently in different executions. Also, the order in which the streams are listed can change from one run to another, so checking the metadata to identify one particular stream becomes crucial. @@ -525,7 +527,7 @@ static gboolean handle_keyboard (GIOChannel *source, GIOCondition cond, CustomDa gchar *str = NULL; if (g_io_channel_read_line (source, &str, NULL, NULL, NULL) == G_IO_STATUS_NORMAL) { - int index = atoi (str); + int index = g_ascii_strtoull (str, NULL, 0); if (index < 0 || index >= data->n_audio) { g_printerr ("Index out of bounds\n"); } else { @@ -542,38 +544,36 @@ static gboolean handle_keyboard (GIOChannel *source, GIOCondition cond, CustomDa Finally, we allow the user to switch the running audio stream. This very basic function just reads a string from the standard input (the keyboard), interprets it as a number, and tries to set the -`current-audio` property of `playbin` (which previously we have only +`current-audio` property of `playbin` (which previously we have only read). Bear in mind that the switch is not immediate. Some of the previously decoded audio will still be flowing through the pipeline, while the new stream becomes active and is decoded. The delay depends on the particular multiplexing of the streams in the container, and the length -`playbin` has selected for its internal queues (which depends on the +`playbin` has selected for its internal queues (which depends on the network conditions). If you execute the tutorial, you will be able to switch from one language to another while the movie is running by pressing 0, 1 or 2 (and ENTER). This concludes this tutorial. -_________________________________________________________________________ - -# Conclusion +## Conclusion This tutorial has shown: - A few more of `playbin`'s properties: `flags`, `connection-speed`, - `n-video`, `n-audio`, `n-text`, `current-video`, `current-audio` and + `n-video`, `n-audio`, `n-text`, `current-video`, `current-audio` and `current-text`. - How to retrieve the list of tags associated with a stream - with `g_signal_emit_by_name()`. + with `g_signal_emit_by_name()`. - How to retrieve a particular tag from the list with - `gst_tag_list_get_string()`or `gst_tag_list_get_uint()` + `gst_tag_list_get_string()`or `gst_tag_list_get_uint()` - How to switch the current audio simply by writing to the - `current-audio` property. + `current-audio` property. The next playback tutorial shows how to handle subtitles, either embedded in the container or in an external file. @@ -583,7 +583,7 @@ code of the tutorial and any accessory files needed to build it. It has been a pleasure having you here, and see you soon! - [Playback tutorial 2: Subtitle management]: Playback%2Btutorial%2B2%253A%2BSubtitle%2Bmanagement.html + [Playback tutorial 2: Subtitle management]: sdk-playback-tutorial-subtitle-management.md [information]: images/icons/emoticons/information.png [Mac]: sdk-installing-on-mac-osx.md [Windows]: Installing+on+Windows @@ -591,6 +591,3 @@ It has been a pleasure having you here, and see you soon! [1]: sdk-installing-on-windows.md#running-the-tutorials [iOS]: sdk-installing-for-ios-development.md#building-the-tutorials [android]: sdk-installing-for-android-development.md#building-the-tutorials - [Tutorial 12: Live streaming]: http://docs.gstreamer.com/display/GstSDK/Tutorial+12%3A+Live+streaming - [Tutorial 17: DVD playback]: http://docs.gstreamer.com/display/GstSDK/Tutorial+17%3A+DVD+playback - [information]: images/icons/emoticons/information.png diff --git a/sdk-playback-tutorial-progressive-streaming.md b/sdk-playback-tutorial-progressive-streaming.md index 25bcad068e..44e6d02a4b 100644 --- a/sdk-playback-tutorial-progressive-streaming.md +++ b/sdk-playback-tutorial-progressive-streaming.md @@ -1,12 +1,11 @@ # Playback tutorial 4: Progressive streaming -# Goal +## Goal -[Basic tutorial 12: -Streaming](Basic%2Btutorial%2B12%253A%2BStreaming.html) showed how to +[](sdk-basic-tutorial-streaming.md) showed how to enhance the user experience in poor network conditions, by taking -buffering into account. This tutorial further expands [Basic tutorial -12: Streaming](Basic%2Btutorial%2B12%253A%2BStreaming.html) by enabling +buffering into account. This tutorial further expands +[](sdk-basic-tutorial-streaming.md) by enabling the local storage of the streamed media, and describes the advantages of this technique. In particular, it shows: @@ -15,11 +14,11 @@ this technique. In particular, it shows: - How to know where it has been downloaded - How to limit the amount of downloaded data that is kept -# Introduction +## Introduction When streaming, data is fetched from the network and a small buffer of -future-data is kept to ensure smooth playback (see [Basic tutorial 12: -Streaming](Basic%2Btutorial%2B12%253A%2BStreaming.html)). However, data +future-data is kept to ensure smooth playback (see +[](sdk-basic-tutorial-streaming.md)). However, data is discarded as soon as it is displayed or rendered (there is no past-data buffer). This means, that if a user wants to jump back and continue playback from a point in the past, data needs to be @@ -30,25 +29,16 @@ downloaded data stored locally for this contingency. A graphical widget is also normally used to show how much of the file has already been downloaded. -`playbin` offers similar functionalities through the `DOWNLOAD` flag +`playbin` offers similar functionalities through the `DOWNLOAD` flag which stores the media in a local temporary file for faster playback of already-downloaded chunks. This code also shows how to use the Buffering Query, which allows knowing what parts of the file are available. -# A network-resilient example with local storage +## A network-resilient example with local storage -Copy this code into a text file named `playback-tutorial-4.c`. - - - - - - - - -

This tutorial is included in the SDK since release 2012.7. If you cannot find it in the downloaded code, please install the latest release of the GStreamer SDK.

+Copy this code into a text file named `playback-tutorial-4.c`. **playback-tutorial-4.c** @@ -56,7 +46,7 @@ Copy this code into a text file named `playback-tutorial-4.c`. #include #include -#define GRAPH_LENGTH 80 +#define GRAPH_LENGTH 78 /* playbin flags */ typedef enum { @@ -74,6 +64,7 @@ static void got_location (GstObject *gstobject, GstObject *prop_object, GParamSp gchar *location; g_object_get (G_OBJECT (prop_object), "temp-location", &location, NULL); g_print ("Temporary file: %s\n", location); + g_free (location); /* Uncomment this line to keep the temporary file after the program exits */ /* g_object_set (G_OBJECT (prop_object), "temp-remove", FALSE, NULL); */ } @@ -131,7 +122,6 @@ static gboolean refresh_ui (CustomData *data) { if (result) { gint n_ranges, range, i; gchar graph[GRAPH_LENGTH + 1]; - GstFormat format = GST_FORMAT_TIME; gint64 position = 0, duration = 0; memset (graph, ' ', GRAPH_LENGTH); @@ -141,14 +131,14 @@ static gboolean refresh_ui (CustomData *data) { for (range = 0; range < n_ranges; range++) { gint64 start, stop; gst_query_parse_nth_buffering_range (query, range, &start, &stop); - start = start * GRAPH_LENGTH / 100; - stop = stop * GRAPH_LENGTH / 100; + start = start * GRAPH_LENGTH / (stop - start); + stop = stop * GRAPH_LENGTH / (stop - start); for (i = (gint)start; i < stop; i++) graph [i] = '-'; } - if (gst_element_query_position (data->pipeline, &format, &position) && + if (gst_element_query_position (data->pipeline, GST_TIME_FORMAT, &position) && GST_CLOCK_TIME_IS_VALID (position) && - gst_element_query_duration (data->pipeline, &format, &duration) && + gst_element_query_duration (data->pipeline, GST_TIME_FORMAT, &duration) && GST_CLOCK_TIME_IS_VALID (duration)) { i = (gint)(GRAPH_LENGTH * (double)position / (double)(duration + 1)); graph [i] = data->buffering_level < 100 ? 'X' : '>'; @@ -226,37 +216,34 @@ int main(int argc, char *argv[]) { } ``` - - - - - - - -
-
-Need help? (Click to expand) -
-
-

If you need help to compile this code, refer to the Building the tutorials section for your platform: Linux, Mac OS X or Windows, or use this specific command on Linux:

-
-
-

gcc playback-tutorial-3.c -o playback-tutorial-3 `pkg-config --cflags --libs gstreamer-1.0`

-
-
-

If you need help to run this code, refer to the Running the tutorials section for your platform: Linux, Mac OS X or Windows

-

This tutorial opens a window and displays a movie, with accompanying audio. The media is fetched from the Internet, so the window might take a few seconds to appear, depending on your connection speed. In the console window, you should see a message indicating where the media is being stored, and a text graph representing the downloaded portions and the current position. A buffering message appears whenever buffering is required, which might never happen is your network connection is fast enough

-

Required libraries: gstreamer-1.0

-
-
+> ![information] If you need help to compile this code, refer to the +> **Building the tutorials** section for your platform: [Mac] or +> [Windows] or use this specific command on Linux: +> +> `` gcc playback-tutorial-4.c -o playback-tutorial-4 `pkg-config --cflags --libs gstreamer-1.0` `` +> +> If you need help to run this code, refer to the **Running the +> tutorials** section for your platform: [Mac OS X], [Windows][1], for +> [iOS] or for [android]. +> +> This tutorial opens a window and displays a movie, with accompanying +> audio. The media is fetched from the Internet, so the window might +> take a few seconds to appear, depending on your connection +> speed. In the console window, you should see a message indicating +> where the media is being stored, and a text graph representing the +> downloaded portions and the current position. A buffering message +> appears whenever buffering is required, which might never happen is +> your network connection is fast enough +> +> Required libraries: `gstreamer-1.0` -# Walkthrough -This code is based on that of [Basic tutorial 12: -Streaming](Basic%2Btutorial%2B12%253A%2BStreaming.html). Let’s review +## Walkthrough + +This code is based on that of [](sdk-basic-tutorial-streaming.md). Let’s review only the differences. -#### Setup +### Setup ``` c /* Set the download flag */ @@ -265,18 +252,18 @@ flags |= GST_PLAY_FLAG_DOWNLOAD; g_object_set (pipeline, "flags", flags, NULL); ``` -By setting this flag, `playbin` instructs its internal queue (a -`queue2` element, actually) to store all downloaded +By setting this flag, `playbin` instructs its internal queue (a +`queue2` element, actually) to store all downloaded data. ``` c g_signal_connect (pipeline, "deep-notify::temp-location", G_CALLBACK (got_location), NULL); ``` -`deep-notify` signals are emitted by `GstObject` elements (like +`deep-notify` signals are emitted by `GstObject` elements (like `playbin`) when the properties of any of their children elements -change. In this case we want to know when the `temp-location` property -changes, indicating that the `queue2` has decided where to store the +change. In this case we want to know when the `temp-location` property +changes, indicating that the `queue2` has decided where to store the downloaded data. @@ -285,30 +272,25 @@ static void got_location (GstObject *gstobject, GstObject *prop_object, GParamSp gchar *location; g_object_get (G_OBJECT (prop_object), "temp-location", &location, NULL); g_print ("Temporary file: %s\n", location); + g_free (location); /* Uncomment this line to keep the temporary file after the program exits */ /* g_object_set (G_OBJECT (prop_object), "temp-remove", FALSE, NULL); */ } ``` -The `temp-location` property is read from the element that triggered the +The `temp-location` property is read from the element that triggered the signal (the `queue2`) and printed on screen. -When the pipeline state changes from `PAUSED` to `READY`, this file is +When the pipeline state changes from `PAUSED` to `READY`, this file is removed. As the comment reads, you can keep it by setting the -`temp-remove` property of the `queue2` to `FALSE`. +`temp-remove` property of the `queue2` to `FALSE`. - - - - - - - -

On Windows this file is usually created inside the Temporary Internet Files folder, which might hide it from Windows Explorer. If you cannot find the downloaded files, try to use the console.

+> ![warning] +> On Windows this file is usually created inside the `Temporary Internet Files` folder, which might hide it from Windows Explorer. If you cannot find the downloaded files, try to use the console. -#### User Interface +### User Interface -In `main` we also install a timer which we use to refresh the UI every +In `main` we also install a timer which we use to refresh the UI every second. ``` c @@ -316,7 +298,7 @@ second. g_timeout_add_seconds (1, (GSourceFunc)refresh_ui, &data); ``` -The `refresh_ui` method queries the pipeline to find out which parts of +The `refresh_ui` method queries the pipeline to find out which parts of the file have been downloaded and what the currently playing position is. It builds a graph to display this information (sort of a text-mode user interface) and prints it on screen, overwriting the previous one so @@ -324,9 +306,9 @@ it looks like it is animated: [---->------- ] -The dashes ‘`-`’ indicate the downloaded parts, and the greater-than +The dashes ‘`-`’ indicate the downloaded parts, and the greater-than sign ‘`>`’ shows the current position (turning into an ‘`X`’ when the -pipeline is paused). Keep in mind that if your network is fast enough, +pipeline is paused). Keep in mind that if your network is fast enough, you will not see the download bar (the dashes) advance at all; it will be completely full from the beginning. @@ -339,19 +321,18 @@ static gboolean refresh_ui (CustomData *data) { ``` The first thing we do in `refresh_ui` is construct a new Buffering -`GstQuery` with `gst_query_new_buffering()` and pass it to the pipeline -(`playbin`) with `gst_element_query()`. In [Basic tutorial 4: Time -management](Basic%2Btutorial%2B4%253A%2BTime%2Bmanagement.html) we have +`GstQuery` with `gst_query_new_buffering()` and pass it to the pipeline +(`playbin`) with `gst_element_query()`. In [](sdk-basic-tutorial-time-management.md) we have already seen how to perform simple queries like Position and Duration using specific methods. More complex queries, like Buffering, need to use the more general `gst_element_query()`. -The Buffering query can be made in different `GstFormat` (TIME, BYTES, +The Buffering query can be made in different `GstFormat` (TIME, BYTES, PERCENTAGE and a few more). Not all elements can answer the query in all the formats, so you need to check which ones are supported in your -particular pipeline. If `gst_element_query()` returns `TRUE`, the query -succeeded. The answer to the query is contained in the same -`GstQuery` structure we created, and can be retrieved using multiple +particular pipeline. If `gst_element_query()` returns `TRUE`, the query +succeeded. The answer to the query is contained in the same +`GstQuery` structure we created, and can be retrieved using multiple parse methods: ``` c @@ -359,8 +340,8 @@ n_ranges = gst_query_get_n_buffering_ranges (query); for (range = 0; range < n_ranges; range++) { gint64 start, stop; gst_query_parse_nth_buffering_range (query, range, &start, &stop); - start = start * GRAPH_LENGTH / 100; - stop = stop * GRAPH_LENGTH / 100; + start = start * GRAPH_LENGTH / (stop - start); + stop = stop * GRAPH_LENGTH / (stop - start); for (i = (gint)start; i < stop; i++) graph [i] = '-'; } @@ -369,13 +350,13 @@ for (range = 0; range < n_ranges; range++) { Data does not need to be downloaded in consecutive pieces from the beginning of the file: Seeking, for example, might force to start downloading from a new position and leave a downloaded chunk behind. -Therefore, `gst_query_get_n_buffering_ranges()` returns the number of +Therefore, `gst_query_get_n_buffering_ranges()` returns the number of chunks, or *ranges* of downloaded data, and then, the position and size of each range is retrieved with `gst_query_parse_nth_buffering_range()`. The format of the returned values (start and stop position for each range) depends on what we requested in the -`gst_query_new_buffering()` call. In this case, PERCENTAGE. These +`gst_query_new_buffering()` call. In this case, PERCENTAGE. These values are used to generate the graph. ``` c @@ -396,7 +377,7 @@ percentage. The current position is indicated with either a ‘`>`’ or an ‘`X`’ depending on the buffering level. If it is below 100%, the code in the -`cb_message` method will have set the pipeline to `PAUSED`, so we print +`cb_message` method will have set the pipeline to `PAUSED`, so we print an ‘`X`’. If the buffering level is 100% the pipeline is in the `PLAYING` state and we print a ‘`>`’. @@ -411,7 +392,7 @@ if (data->buffering_level < 100) { Finally, if the buffering level is below 100%, we report this information (and delete it otherwise). -#### Limiting the size of the downloaded file +### Limiting the size of the downloaded file ``` c /* Uncomment this line to limit the amount of downloaded data */ @@ -423,23 +404,25 @@ size of the temporary file, by overwriting already played regions. Observe the download bar to see which regions are kept available in the file. -# Conclusion +## Conclusion This tutorial has shown: - How to enable progressive downloading with the - `GST_PLAY_FLAG_DOWNLOAD` `playbin` flag - - How to know what has been downloaded using a Buffering `GstQuery` + `GST_PLAY_FLAG_DOWNLOAD` `playbin` flag + - How to know what has been downloaded using a Buffering `GstQuery` - How to know where it has been downloaded with the `deep-notify::temp-location` signal - How to limit the size of the temporary file with - the `ring-buffer-max-size` property of `playbin`. + the `ring-buffer-max-size` property of `playbin`. -It has been a pleasure having you here, and see you soon\! +It has been a pleasure having you here, and see you soon! -## Attachments: - -![](images/icons/bullet_blue.gif) -[playback-tutorial-4.c](attachments/327808/2424846.c) (text/plain) -![](images/icons/bullet_blue.gif) -[vs2010.zip](attachments/327808/2424847.zip) (application/zip) + [information]: images/icons/emoticons/information.png + [Mac]: sdk-installing-on-mac-osx.md + [Windows]: Installing+on+Windows + [Mac OS X]: sdk-installing-on-mac-osx.md#building-the-tutorials + [1]: sdk-installing-on-windows.md#running-the-tutorials + [iOS]: sdk-installing-for-ios-development.md#building-the-tutorials + [android]: sdk-installing-for-android-development.md#building-the-tutorials + [warning]: images/icons/emoticons/warning.png diff --git a/sdk-playback-tutorial-short-cutting-the-pipeline.md b/sdk-playback-tutorial-short-cutting-the-pipeline.md index bf913f5604..ffc5d38ce3 100644 --- a/sdk-playback-tutorial-short-cutting-the-pipeline.md +++ b/sdk-playback-tutorial-short-cutting-the-pipeline.md @@ -1,42 +1,30 @@ # Playback tutorial 3: Short-cutting the pipeline -# Goal +## Goal -[Basic tutorial 8: Short-cutting the -pipeline](Basic%2Btutorial%2B8%253A%2BShort-cutting%2Bthe%2Bpipeline.html) showed +[](sdk-basic-tutorial-short-cutting-the-pipeline.md) showed how an application can manually extract or inject data into a pipeline -by using two special elements called `appsrc` and `appsink`. -`playbin` allows using these elements too, but the method to connect -them is different. To connect an `appsink` to `playbin` see [Playback -tutorial 7: Custom playbin -sinks](Playback%2Btutorial%2B7%253A%2BCustom%2Bplaybin%2Bsinks.html). +by using two special elements called `appsrc` and `appsink`. +`playbin` allows using these elements too, but the method to connect +them is different. To connect an `appsink` to `playbin` see [](sdk-playback-tutorial-custom-playbin-sinks.md). This tutorial shows: - - How to connect `appsrc` with `playbin` - - How to configure the `appsrc` + - How to connect `appsrc` with `playbin` + - How to configure the `appsrc` -# A playbin waveform generator +## A playbin waveform generator -Copy this code into a text file named `playback-tutorial-3.c`. - - - - - - - - -

This tutorial is included in the SDK since release 2012.7. If you cannot find it in the downloaded code, please install the latest release of the GStreamer SDK.

+Copy this code into a text file named `playback-tutorial-3.c`. **playback-tutorial-3.c** ``` c #include +#include #include #define CHUNK_SIZE 1024 /* Amount of bytes we are sending in each buffer */ #define SAMPLE_RATE 44100 /* Samples per second we are sending */ -#define AUDIO_CAPS "audio/x-raw-int,channels=1,rate=%d,signed=(boolean)true,width=16,depth=16,endianness=BYTE_ORDER" /* Structure to contain all our information, so we can pass it to callbacks */ typedef struct _CustomData { @@ -59,6 +47,7 @@ static gboolean push_data (CustomData *data) { GstBuffer *buffer; GstFlowReturn ret; int i; + GstMapInfo map; gint16 *raw; gint num_samples = CHUNK_SIZE / 2; /* Because each sample is 16 bits */ gfloat freq; @@ -71,7 +60,8 @@ static gboolean push_data (CustomData *data) { GST_BUFFER_DURATION (buffer) = gst_util_uint64_scale (CHUNK_SIZE, GST_SECOND, SAMPLE_RATE); /* Generate some psychodelic waveforms */ - raw = (gint16 *)GST_BUFFER_DATA (buffer); + gst_buffer_map (buffer, &map, GST_MAP_WRITE); + raw = (gint16 *)map.data; data->c += data->d; data->d -= data->c / 1000; freq = 1100 + 1000 * data->d; @@ -80,6 +70,7 @@ static gboolean push_data (CustomData *data) { data->b -= data->a / freq; raw[i] = (gint16)(500 * data->a); } + gst_buffer_unmap (buffer, &map); data->num_samples += num_samples; /* Push the buffer into the appsrc */ @@ -133,16 +124,16 @@ static void error_cb (GstBus *bus, GstMessage *msg, CustomData *data) { /* This function is called when playbin has created the appsrc element, so we have * a chance to configure it. */ static void source_setup (GstElement *pipeline, GstElement *source, CustomData *data) { - gchar *audio_caps_text; + GstAudioInfo info; GstCaps *audio_caps; g_print ("Source has been created. Configuring.\n"); data->app_source = source; /* Configure appsrc */ - audio_caps_text = g_strdup_printf (AUDIO_CAPS, SAMPLE_RATE); - audio_caps = gst_caps_from_string (audio_caps_text); - g_object_set (source, "caps", audio_caps, NULL); + gst_audio_info_set_format (&info, GST_AUDIO_FORMAT_S16, SAMPLE_RATE, 1, NULL); + audio_caps = gst_audio_info_to_caps (&info); + g_object_set (source, "caps", audio_caps, "format", GST_FORMAT_TIME, NULL); g_signal_connect (source, "need-data", G_CALLBACK (start_feed), data); g_signal_connect (source, "enough-data", G_CALLBACK (stop_feed), data); gst_caps_unref (audio_caps); @@ -185,16 +176,16 @@ int main(int argc, char *argv[]) { } ``` -To use an `appsrc` as the source for the pipeline, simply instantiate a -`playbin` and set its URI to `appsrc://` +To use an `appsrc` as the source for the pipeline, simply instantiate a +`playbin` and set its URI to `appsrc://` ``` c /* Create the playbin element */ data.pipeline = gst_parse_launch ("playbin uri=appsrc://", NULL); ``` -`playbin` will create an internal `appsrc` element and fire the -`source-setup` signal to allow the application to configure +`playbin` will create an internal `appsrc` element and fire the +`source-setup` signal to allow the application to configure it: ``` c @@ -202,7 +193,7 @@ g_signal_connect (data.pipeline, "source-setup", G_CALLBACK (source_setup), &dat ``` In particular, it is important to set the caps property of `appsrc`, -since, once the signal handler returns, `playbin` will instantiate the +since, once the signal handler returns, `playbin` will instantiate the next element in the pipeline according to these caps: @@ -210,16 +201,16 @@ caps: /* This function is called when playbin has created the appsrc element, so we have * a chance to configure it. */ static void source_setup (GstElement *pipeline, GstElement *source, CustomData *data) { - gchar *audio_caps_text; + GstAudioInfo info; GstCaps *audio_caps; g_print ("Source has been created. Configuring.\n"); data->app_source = source; /* Configure appsrc */ - audio_caps_text = g_strdup_printf (AUDIO_CAPS, SAMPLE_RATE); - audio_caps = gst_caps_from_string (audio_caps_text); - g_object_set (source, "caps", audio_caps, NULL); + gst_audio_info_set_format (&info, GST_AUDIO_FORMAT_S16, SAMPLE_RATE, 1, NULL); + audio_caps = gst_audio_info_to_caps (&info); + g_object_set (source, "caps", audio_caps, "format", GST_FORMAT_TIME, NULL); g_signal_connect (source, "need-data", G_CALLBACK (start_feed), data); g_signal_connect (source, "enough-data", G_CALLBACK (stop_feed), data); gst_caps_unref (audio_caps); @@ -227,41 +218,28 @@ static void source_setup (GstElement *pipeline, GstElement *source, CustomData * } ``` -The configuration of the `appsrc` is exactly the same as in [Basic -tutorial 8: Short-cutting the -pipeline](Basic%2Btutorial%2B8%253A%2BShort-cutting%2Bthe%2Bpipeline.html): -the caps are set to `audio/x-raw-int`, and two callbacks are registered, +The configuration of the `appsrc` is exactly the same as in +[](sdk-basic-tutorial-short-cutting-the-pipeline.md): +the caps are set to `audio/x-raw`, and two callbacks are registered, so the element can tell the application when it needs to start and stop -pushing data. See [Basic tutorial 8: Short-cutting the -pipeline](Basic%2Btutorial%2B8%253A%2BShort-cutting%2Bthe%2Bpipeline.html) +pushing data. See [](sdk-basic-tutorial-short-cutting-the-pipeline.md) for more details. -From this point onwards, `playbin` takes care of the rest of the +From this point onwards, `playbin` takes care of the rest of the pipeline, and the application only needs to worry about generating more data when told so. -To learn how data can be extracted from `playbin` using the -`appsink` element, see [Playback tutorial 7: Custom playbin -sinks](Playback%2Btutorial%2B7%253A%2BCustom%2Bplaybin%2Bsinks.html). +To learn how data can be extracted from `playbin` using the +`appsink` element, see [](sdk-playback-tutorial-custom-playbin-sinks.md). -# Conclusion +## Conclusion -This tutorial applies the concepts shown in [Basic tutorial 8: -Short-cutting the -pipeline](Basic%2Btutorial%2B8%253A%2BShort-cutting%2Bthe%2Bpipeline.html) to +This tutorial applies the concepts shown in +[](sdk-basic-tutorial-short-cutting-the-pipeline.md) to `playbin`. In particular, it has shown: - - How to connect `appsrc` with `playbin` using the special - URI `appsrc://` - - How to configure the `appsrc` using the `source-setup` signal + - How to connect `appsrc` with `playbin` using the special + URI `appsrc://` + - How to configure the `appsrc` using the `source-setup` signal -It has been a pleasure having you here, and see you soon\! - -## Attachments: - -![](images/icons/bullet_blue.gif) -[playback-tutorial-3.c](attachments/1442200/2424850.c) (text/plain) -![](images/icons/bullet_blue.gif) -[vs2010.zip](attachments/1442200/2424849.zip) (application/zip) -![](images/icons/bullet_blue.gif) -[playback-tutorial-3.c](attachments/1442200/2424848.c) (text/plain) +It has been a pleasure having you here, and see you soon! \ No newline at end of file diff --git a/sdk-playback-tutorial-subtitle-management.md b/sdk-playback-tutorial-subtitle-management.md index a3dd57f7de..979c7e66af 100644 --- a/sdk-playback-tutorial-subtitle-management.md +++ b/sdk-playback-tutorial-subtitle-management.md @@ -1,6 +1,6 @@ # Playback tutorial 2: Subtitle management -# Goal +## Goal This tutorial is very similar to the previous one, but instead of switching among different audio streams, we will use subtitle streams. @@ -12,34 +12,35 @@ This will allow us to learn: - How to customize the font used for the subtitles -# Introduction +## Introduction We already know (from the previous tutorial) that container files can hold multiple audio and video streams, and that we can very easily -choose among them by changing the `current-audio` or -`current-video` `playbin` property. Switching subtitles is just as +choose among them by changing the `current-audio` or +`current-video` `playbin` property. Switching subtitles is just as easy. It is worth noting that, just like it happens with audio and video, -`playbin` takes care of choosing the right decoder for the subtitles, +`playbin` takes care of choosing the right decoder for the subtitles, and that the plugin structure of GStreamer allows adding support for new formats as easily as copying a file. Everything is invisible to the application developer. -Besides subtitles embedded in the container, `playbin` offers the +Besides subtitles embedded in the container, `playbin` offers the possibility to add an extra subtitle stream from an external URI. This tutorial opens a file which already contains 5 subtitle streams, and adds another one from another file (for the Greek language). -# The multilingual player with subtitles +## The multilingual player with subtitles -Copy this code into a text file named `playback-tutorial-2.c` (or find +Copy this code into a text file named `playback-tutorial-2.c` (or find it in the SDK installation). **playback-tutorial-2.c** ``` c +#include #include /* Structure to contain all our information, so we can pass it around */ @@ -103,7 +104,7 @@ int main(int argc, char *argv[]) { gst_bus_add_watch (bus, (GstBusFunc)handle_message, &data); /* Add a keyboard watch so we get notified of keystrokes */ -#ifdef _WIN32 +#ifdef G_OS_WIN32 io_stdin = g_io_channel_win32_new_fd (fileno (stdin)); #else io_stdin = g_io_channel_unix_new (fileno (stdin)); @@ -262,38 +263,38 @@ static gboolean handle_keyboard (GIOChannel *source, GIOCondition cond, CustomDa } ``` - - - - - - - -
-
-Need help? (Click to expand) -
-
-

If you need help to compile this code, refer to the Building the tutorials section for your platform: Linux, Mac OS X or Windows, or use this specific command on Linux:

-
-
-

gcc playback-tutorial-2.c -o playback-tutorial-2 `pkg-config --cflags --libs gstreamer-1.0`

-
-
-

If you need help to run this code, refer to the Running the tutorials section for your platform: Linux, Mac OS X or Windows

-

-

This tutorial opens a window and displays a movie, with accompanying audio. The media is fetched from the Internet, so the window might take a few seconds to appear, depending on your connection speed. The number of subtitle streams is shown in the terminal, and the user can switch from one to another by entering a number and pressing enter. A small delay is to be expected. Please read the note at the bottom of this page.

-

Bear in mind that there is no latency management (buffering), so on slow connections, the movie might stop after a few seconds. See how Tutorial 12: Live streaming solves this issue.

-

-

Required libraries: gstreamer-1.0

-
-
-# Walkthrough +> ![information] Need help? +> +> If you need help to compile this code, refer to the **Building the +> tutorials** section for your platform: [Linux], [Mac OS X] or +> [Windows], or use this specific command on Linux: +> +> `` gcc playback-tutorial-2.c -o playback-tutorial-2 `pkg-config --cflags --libs gstreamer-1.0` `` +> +> If you need help to run this code, refer to the **Running the +> tutorials** section for your platform: [Linux][1], [Mac OS X][2] or +> [Windows][3]. +> +> This tutorial opens a window and displays a movie, with accompanying +> audio. The media is fetched from the Internet, so the window might +> take a few seconds to appear, depending on your connection +> speed. The number of subtitle streams is shown in the terminal, and +> the user can switch from one to another by entering a number and +> pressing enter. A small delay is to be +> expected. _Please read the note at the bottom of this +> page._ Bear in mind that +> there is no latency management (buffering), so on slow connections, +> the movie might stop after a few seconds. See how +> [](sdk-basic-tutorial-streaming.md) solves this issue. +> +> Required libraries: `gstreamer-1.0` -This tutorial is copied from [Playback tutorial 1: Playbin -usage](Playback%2Btutorial%2B1%253A%2BPlaybin%2Busage.html) with some -changes, so let's review only the changes. +## Walkthrough + +This tutorial is copied from +[](sdk-playback-tutorial-playbin-usage.md) with some changes, so let's +review only the changes. ``` c /* Set the subtitle URI to play and some font description */ @@ -301,8 +302,8 @@ g_object_set (data.playbin, "suburi", "http://docs.gstreamer.com/media/sintel_tr g_object_set (data.playbin, "subtitle-font-desc", "Sans, 18", NULL); ``` -After setting the media URI, we set the `suburi` property, which points -`playbin` to a file containing a subtitle stream. In this case, the +After setting the media URI, we set the `suburi` property, which points +`playbin` to a file containing a subtitle stream. In this case, the media file already contains multiple subtitle streams, so the one provided in the `suburi` is added to the list, and will be the currently selected one. @@ -312,17 +313,17 @@ resides in the container file, therefore, subtitles not embedded in a container will not have metadata. When running this tutorial you will find that the first subtitle stream does not have a language tag. -The `subtitle-font-desc` property allows specifying the font to render -the subtitles. Since [Pango](http://www.pango.org/) is the library used +The `subtitle-font-desc` property allows specifying the font to render +the subtitles. Since [Pango](http://www.pango.org/) is the library used to render fonts, you can check its documentation to see how this font should be specified, in particular, the -[pango-font-description-from-string](http://developer.gnome.org/pango/stable/pango-Fonts.html#pango-font-description-from-string) function. +[pango-font-description-from-string](http://developer.gnome.org/pango/stable/pango-Fonts.html#pango-font-description-from-string) function. In a nutshell, the format of the string representation is `[FAMILY-LIST] -[STYLE-OPTIONS] [SIZE]` where `FAMILY-LIST` is a comma separated list of -families optionally terminated by a comma, `STYLE_OPTIONS` is a +[STYLE-OPTIONS] [SIZE]` where `FAMILY-LIST` is a comma separated list of +families optionally terminated by a comma, `STYLE_OPTIONS` is a whitespace separated list of words where each word describes one of -style, variant, weight, or stretch, and `SIZE` is an decimal number +style, variant, weight, or stretch, and `SIZE` is an decimal number (size in points). For example the following are all valid string representations: @@ -333,7 +334,7 @@ representations: The commonly available font families are: Normal, Sans, Serif and Monospace. -The available styles are: Normal (the font is upright), Oblique (the +The available styles are: Normal (the font is upright), Oblique (the font is slanted, but in a roman style), Italic (the font is slanted in an italic style). @@ -344,10 +345,10 @@ The available variants are: Normal, Small\_Caps (A font with the lower case characters replaced by smaller variants of the capital characters) The available stretch styles -are: Ultra-Condensed, Extra-Condensed, Condensed, Semi-Condensed, Normal, Semi-Expanded, Expanded, +are: Ultra-Condensed, Extra-Condensed, Condensed, Semi-Condensed, Normal, Semi-Expanded, Expanded, Extra-Expanded, Ultra-Expanded -  + ``` c /* Set flags to show Audio, Video and Subtitles */ @@ -356,17 +357,16 @@ flags |= GST_PLAY_FLAG_VIDEO | GST_PLAY_FLAG_AUDIO | GST_PLAY_FLAG_TEXT; g_object_set (data.playbin, "flags", flags, NULL); ``` -We set the `flags` property to allow Audio, Video and Text (Subtitles). +We set the `flags` property to allow Audio, Video and Text (Subtitles). -The rest of the tutorial is the same as [Playback tutorial 1: Playbin -usage](Playback%2Btutorial%2B1%253A%2BPlaybin%2Busage.html), except -that the keyboard input changes the `current-text` property instead of +The rest of the tutorial is the same as [](sdk-playback-tutorial-playbin-usage.md), except +that the keyboard input changes the `current-text` property instead of the `current-audio`. As before, keep in mind that stream changes are not immediate, since there is a lot of information flowing through the pipeline that needs to reach the end of it before the new stream shows up. -# Conclusion +## Conclusion This tutorial showed how to handle subtitles from `playbin`, whether they are embedded in the container or in a different file: @@ -374,10 +374,10 @@ they are embedded in the container or in a different file: - Subtitles are chosen using the `current-tex`t and `n-tex`t properties of `playbin`. - - External subtitle files can be selected using the `suburi` property. + - External subtitle files can be selected using the `suburi` property. - Subtitle appearance can be customized with the - `subtitle-font-desc` property. + `subtitle-font-desc` property. The next playback tutorial shows how to change the playback speed. @@ -385,14 +385,4 @@ Remember that attached to this page you should find the complete source code of the tutorial and any accessory files needed to build it. It has been a pleasure having you here, and see you soon\! - - - - - - - -

There is a bug in the current version of the sdk https://bugzilla.gnome.org/show_bug.cgi?id=638168:

-

Switching subtitle tracks while there is a subtitle on the screen gives this warning:

-

WARN                 katedec gstkatedec.c:309:gst_kate_dec_chain:<katedec1> failed to push buffer: wrong-state

-

And after a while it freezes.

+ [information]: images/icons/emoticons/information.png diff --git a/sdk-playback-tutorials.md b/sdk-playback-tutorials.md index d824f6fc04..37aff16dd5 100644 --- a/sdk-playback-tutorials.md +++ b/sdk-playback-tutorials.md @@ -1,6 +1,4 @@ # Playback tutorials -# Welcome to the GStreamer SDK Playback tutorials - These tutorials explain everything you need to know to produce a media playback application using GStreamer.