41 KiB
GStreamer SDK documentation : Android tutorial 2: A running pipeline
This page last changed on May 07, 2013 by xartigas.
Goal ![](/mirrors/gstreamer/media/commit/7f0ee500c1d5c193d7306463e80fb7dc6a460cf0/attachments/thumbnails/2687063/2654324)
The tutorials seen in the Basic and
Playback sections are intended for Desktop
platforms and, therefore, their main thread is allowed to block (using
gst_bus_pop_filtered()
) or relinquish control to a GLib main loop. On
Android this would lead to the application being tagged as
non-responsive and probably closed.
This tutorial shows how to overcome this problem. In particular, we will learn:
- How to move the native code to its own thread
- How to allow threads created from C code to communicate with Java
- How to access Java code from C
- How to allocate a
CustomData
structure from C and have Java host it
Introduction
When using a Graphical User Interface (UI), if the application waits for
GStreamer calls to complete the user experience will suffer. The usual
approach, with the GTK+ toolkit for example, is to
relinquish control to a GLib GMainLoop
and let it control the events
coming from the UI or GStreamer.
This approach can be very cumbersome when GStreamer and the Android UI communicate through the JNI interface, so we take a cleaner route: We use a GLib main loop, and move it to its own thread, so it does not block the application. This simplifies the GStreamer-Android integration, and we only need to worry about a few inter-process synchronization bits, which are detailed in this tutorial.
Additionally, this tutorial shows how to obtain, from any thread, the JNI Environment pointer required to make JNI calls. This is necessary, for example, to call Java code from callbacks in threads spawned deep within GStreamer, which never received this pointer directly.
Finally, this tutorial explains how to call Java methods from native C code, which involves locating the desired method’s ID in the class. These IDs never change, so they are cached as global variables in the C code and obtained in the static initializer of the class.
The code below builds a pipeline with an audiotestsrc
and an
autoaudiosink
(it plays an audible tone). Two buttons in the UI allow
setting the pipeline to PLAYING or PAUSED. A TextView in the UI shows
messages sent from the C code (for errors and state changes).
A pipeline on Android [Java code]
src/com/gst_sdk_tutorials/tutorial_2/Tutorial2.java
package com.gst_sdk_tutorials.tutorial_2;
import android.app.Activity;
import android.os.Bundle;
import android.util.Log;
import android.view.View;
import android.view.View.OnClickListener;
import android.widget.ImageButton;
import android.widget.TextView;
import android.widget.Toast;
import com.gstreamer.GStreamer;
public class Tutorial2 extends Activity {
private native void nativeInit(); // Initialize native code, build pipeline, etc
private native void nativeFinalize(); // Destroy pipeline and shutdown native code
private native void nativePlay(); // Set pipeline to PLAYING
private native void nativePause(); // Set pipeline to PAUSED
private static native boolean nativeClassInit(); // Initialize native class: cache Method IDs for callbacks
private long native_custom_data; // Native code will use this to keep private data
private boolean is_playing_desired; // Whether the user asked to go to PLAYING
// Called when the activity is first created.
@Override
public void onCreate(Bundle savedInstanceState)
{
super.onCreate(savedInstanceState);
// Initialize GStreamer and warn if it fails
try {
GStreamer.init(this);
} catch (Exception e) {
Toast.makeText(this, e.getMessage(), Toast.LENGTH_LONG).show();
finish();
return;
}
setContentView(R.layout.main);
ImageButton play = (ImageButton) this.findViewById(R.id.button_play);
play.setOnClickListener(new OnClickListener() {
public void onClick(View v) {
is_playing_desired = true;
nativePlay();
}
});
ImageButton pause = (ImageButton) this.findViewById(R.id.button_stop);
pause.setOnClickListener(new OnClickListener() {
public void onClick(View v) {
is_playing_desired = false;
nativePause();
}
});
if (savedInstanceState != null) {
is_playing_desired = savedInstanceState.getBoolean("playing");
Log.i ("GStreamer", "Activity created. Saved state is playing:" + is_playing_desired);
} else {
is_playing_desired = false;
Log.i ("GStreamer", "Activity created. There is no saved state, playing: false");
}
// Start with disabled buttons, until native code is initialized
this.findViewById(R.id.button_play).setEnabled(false);
this.findViewById(R.id.button_stop).setEnabled(false);
nativeInit();
}
protected void onSaveInstanceState (Bundle outState) {
Log.d ("GStreamer", "Saving state, playing:" + is_playing_desired);
outState.putBoolean("playing", is_playing_desired);
}
protected void onDestroy() {
nativeFinalize();
super.onDestroy();
}
// Called from native code. This sets the content of the TextView from the UI thread.
private void setMessage(final String message) {
final TextView tv = (TextView) this.findViewById(R.id.textview_message);
runOnUiThread (new Runnable() {
public void run() {
tv.setText(message);
}
});
}
// Called from native code. Native code calls this once it has created its pipeline and
// the main loop is running, so it is ready to accept commands.
private void onGStreamerInitialized () {
Log.i ("GStreamer", "Gst initialized. Restoring state, playing:" + is_playing_desired);
// Restore previous playing state
if (is_playing_desired) {
nativePlay();
} else {
nativePause();
}
// Re-enable buttons, now that GStreamer is initialized
final Activity activity = this;
runOnUiThread(new Runnable() {
public void run() {
activity.findViewById(R.id.button_play).setEnabled(true);
activity.findViewById(R.id.button_stop).setEnabled(true);
}
});
}
static {
System.loadLibrary("gstreamer_android");
System.loadLibrary("tutorial-2");
nativeClassInit();
}
}
As usual, the first bit that gets executed is the static initializer of the class:
static {
System.loadLibrary("gstreamer_android");
System.loadLibrary("tutorial-2");
nativeClassInit();
}
As explained in the previous tutorial, the two native libraries are
loaded and their JNI_OnLoad()
methods are executed. Here, we also call
the native method nativeClassInit()
, previously declared with the
native
keyword in line 19. We will later see what its purpose is
In the onCreate()
method GStreamer is initialized as in the previous
tutorial with GStreamer.init(this)
, and then the layout is inflated
and listeners are setup for the two UI buttons:
ImageButton play = (ImageButton) this.findViewById(R.id.button_play);
play.setOnClickListener(new OnClickListener() {
public void onClick(View v) {
is_playing_desired = true;
nativePlay();
}
});
ImageButton pause = (ImageButton) this.findViewById(R.id.button_stop);
pause.setOnClickListener(new OnClickListener() {
public void onClick(View v) {
is_playing_desired = false;
nativePause();
}
});
Each button instructs the native code to set the pipeline to the desired
state, and also remembers this state in the
is_playing_desired
variable. This is required so, when the
application is restarted (for example, due to an orientation change), it
can set the pipeline again to the desired state. This approach is easier
and safer than tracking the actual pipeline state, because orientation
changes can happen before the pipeline has moved to the desired state,
for example.
if (savedInstanceState != null) {
is_playing_desired = savedInstanceState.getBoolean("playing");
Log.i ("GStreamer", "Activity created. Saved state is playing:" + is_playing_desired);
} else {
is_playing_desired = false;
Log.i ("GStreamer", "Activity created. There is no saved state, playing: false");
}
Restore the previous playing state (if any) from savedInstanceState
.
We will first build the GStreamer pipeline (below) and only when the
native code reports itself as initialized we will use
is_playing_desired
.
nativeInit();
As will be shown in the C code, nativeInit()
creates a dedicated
thread, a GStreamer pipeline, a GLib main loop, and, right before
calling g_main_loop_run()
and going to sleep, it warns the Java code
that the native code is initialized and ready to accept commands.
This finishes the onCreate()
method and the Java initialization. The
UI buttons are disabled, so nothing will happen until native code is
ready and onGStreamerInitialized()
is called:
private void onGStreamerInitialized () {
Log.i ("GStreamer", "Gst initialized. Restoring state, playing:" + is_playing_desired);
This is called by the native code when its main loop is finally running.
We first retrieve the desired playing state from is_playing_desired
,
and then set that state:
// Restore previous playing state
if (is_playing_desired) {
nativePlay();
} else {
nativePause();
}
Here comes the first caveat, when re-enabling the UI buttons:
// Re-enable buttons, now that GStreamer is initialized
final Activity activity = this;
runOnUiThread(new Runnable() {
public void run() {
activity.findViewById(R.id.button_play).setEnabled(true);
activity.findViewById(R.id.button_stop).setEnabled(true);
}
});
This method is being called from the thread that the native code created
to run its main loop, and is not allowed to issue UI-altering commands:
Only the UI thread can do that. The solution is easy though: Android
Activities have a handy
runOnUiThread()
method which lets bits of code to be executed from the correct thread. A
Runnable
instance has to be constructed and any parameter can be passed either by
sub-classing
Runnable
and adding a dedicated constructor, or by using the final
modifier, as
shown in the above snippet.
The same problem exists when the native code wants to output a string in
our TextView using the setMessage()
method: it has to be done from the
UI thread. The solution is the same:
private void setMessage(final String message) {
final TextView tv = (TextView) this.findViewById(R.id.textview_message);
runOnUiThread (new Runnable() {
public void run() {
tv.setText(message);
}
});
}
Finally, a few remaining bits:
protected void onSaveInstanceState (Bundle outState) {
Log.d ("GStreamer", "Saving state, playing:" + is_playing_desired);
outState.putBoolean("playing", is_playing_desired);
}
This method stores the currently desired playing state when Android is about to shut us down, so next time it restarts (after an orientation change, for example), it can restore the same state.
protected void onDestroy() {
nativeFinalize();
super.onDestroy();
}
And this is called before Android destroys our application. We call the
nativeFinalize()
method to exit the main loop, destroy its thread and
all allocated resources.
This concludes the UI part of the tutorial.
A pipeline on Android [C code]
jni/tutorial-2.c
#include <string.h>
#include <jni.h>
#include <android/log.h>
#include <gst/gst.h>
#include <pthread.h>
GST_DEBUG_CATEGORY_STATIC (debug_category);
#define GST_CAT_DEFAULT debug_category
/*
* These macros provide a way to store the native pointer to CustomData, which might be 32 or 64 bits, into
* a jlong, which is always 64 bits, without warnings.
*/
#if GLIB_SIZEOF_VOID_P == 8
# define GET_CUSTOM_DATA(env, thiz, fieldID) (CustomData *)(*env)->GetLongField (env, thiz, fieldID)
# define SET_CUSTOM_DATA(env, thiz, fieldID, data) (*env)->SetLongField (env, thiz, fieldID, (jlong)data)
#else
# define GET_CUSTOM_DATA(env, thiz, fieldID) (CustomData *)(jint)(*env)->GetLongField (env, thiz, fieldID)
# define SET_CUSTOM_DATA(env, thiz, fieldID, data) (*env)->SetLongField (env, thiz, fieldID, (jlong)(jint)data)
#endif
/* Structure to contain all our information, so we can pass it to callbacks */
typedef struct _CustomData {
jobject app; /* Application instance, used to call its methods. A global reference is kept. */
GstElement *pipeline; /* The running pipeline */
GMainContext *context; /* GLib context used to run the main loop */
GMainLoop *main_loop; /* GLib main loop */
gboolean initialized; /* To avoid informing the UI multiple times about the initialization */
} CustomData;
/* These global variables cache values which are not changing during execution */
static pthread_t gst_app_thread;
static pthread_key_t current_jni_env;
static JavaVM *java_vm;
static jfieldID custom_data_field_id;
static jmethodID set_message_method_id;
static jmethodID on_gstreamer_initialized_method_id;
/*
* Private methods
*/
/* Register this thread with the VM */
static JNIEnv *attach_current_thread (void) {
JNIEnv *env;
JavaVMAttachArgs args;
GST_DEBUG ("Attaching thread %p", g_thread_self ());
args.version = JNI_VERSION_1_4;
args.name = NULL;
args.group = NULL;
if ((*java_vm)->AttachCurrentThread (java_vm, &env, &args) < 0) {
GST_ERROR ("Failed to attach current thread");
return NULL;
}
return env;
}
/* Unregister this thread from the VM */
static void detach_current_thread (void *env) {
GST_DEBUG ("Detaching thread %p", g_thread_self ());
(*java_vm)->DetachCurrentThread (java_vm);
}
/* Retrieve the JNI environment for this thread */
static JNIEnv *get_jni_env (void) {
JNIEnv *env;
if ((env = pthread_getspecific (current_jni_env)) == NULL) {
env = attach_current_thread ();
pthread_setspecific (current_jni_env, env);
}
return env;
}
/* Change the content of the UI's TextView */
static void set_ui_message (const gchar *message, CustomData *data) {
JNIEnv *env = get_jni_env ();
GST_DEBUG ("Setting message to: %s", message);
jstring jmessage = (*env)->NewStringUTF(env, message);
(*env)->CallVoidMethod (env, data->app, set_message_method_id, jmessage);
if ((*env)->ExceptionCheck (env)) {
GST_ERROR ("Failed to call Java method");
(*env)->ExceptionClear (env);
}
(*env)->DeleteLocalRef (env, jmessage);
}
/* Retrieve errors from the bus and show them on the UI */
static void error_cb (GstBus *bus, GstMessage *msg, CustomData *data) {
GError *err;
gchar *debug_info;
gchar *message_string;
gst_message_parse_error (msg, &err, &debug_info);
message_string = g_strdup_printf ("Error received from element %s: %s", GST_OBJECT_NAME (msg->src), err->message);
g_clear_error (&err);
g_free (debug_info);
set_ui_message (message_string, data);
g_free (message_string);
gst_element_set_state (data->pipeline, GST_STATE_NULL);
}
/* Notify UI about pipeline state changes */
static void state_changed_cb (GstBus *bus, GstMessage *msg, CustomData *data) {
GstState old_state, new_state, pending_state;
gst_message_parse_state_changed (msg, &old_state, &new_state, &pending_state);
/* Only pay attention to messages coming from the pipeline, not its children */
if (GST_MESSAGE_SRC (msg) == GST_OBJECT (data->pipeline)) {
gchar *message = g_strdup_printf("State changed to %s", gst_element_state_get_name(new_state));
set_ui_message(message, data);
g_free (message);
}
}
/* Check if all conditions are met to report GStreamer as initialized.
* These conditions will change depending on the application */
static void check_initialization_complete (CustomData *data) {
JNIEnv *env = get_jni_env ();
if (!data->initialized && data->main_loop) {
GST_DEBUG ("Initialization complete, notifying application. main_loop:%p", data->main_loop);
(*env)->CallVoidMethod (env, data->app, on_gstreamer_initialized_method_id);
if ((*env)->ExceptionCheck (env)) {
GST_ERROR ("Failed to call Java method");
(*env)->ExceptionClear (env);
}
data->initialized = TRUE;
}
}
/* Main method for the native code. This is executed on its own thread. */
static void *app_function (void *userdata) {
JavaVMAttachArgs args;
GstBus *bus;
CustomData *data = (CustomData *)userdata;
GSource *bus_source;
GError *error = NULL;
GST_DEBUG ("Creating pipeline in CustomData at %p", data);
/* Create our own GLib Main Context and make it the default one */
data->context = g_main_context_new ();
g_main_context_push_thread_default(data->context);
/* Build pipeline */
data->pipeline = gst_parse_launch("audiotestsrc ! audioconvert ! audioresample ! autoaudiosink", &error);
if (error) {
gchar *message = g_strdup_printf("Unable to build pipeline: %s", error->message);
g_clear_error (&error);
set_ui_message(message, data);
g_free (message);
return NULL;
}
/* Instruct the bus to emit signals for each received message, and connect to the interesting signals */
bus = gst_element_get_bus (data->pipeline);
bus_source = gst_bus_create_watch (bus);
g_source_set_callback (bus_source, (GSourceFunc) gst_bus_async_signal_func, NULL, NULL);
g_source_attach (bus_source, data->context);
g_source_unref (bus_source);
g_signal_connect (G_OBJECT (bus), "message::error", (GCallback)error_cb, data);
g_signal_connect (G_OBJECT (bus), "message::state-changed", (GCallback)state_changed_cb, data);
gst_object_unref (bus);
/* Create a GLib Main Loop and set it to run */
GST_DEBUG ("Entering main loop... (CustomData:%p)", data);
data->main_loop = g_main_loop_new (data->context, FALSE);
check_initialization_complete (data);
g_main_loop_run (data->main_loop);
GST_DEBUG ("Exited main loop");
g_main_loop_unref (data->main_loop);
data->main_loop = NULL;
/* Free resources */
g_main_context_pop_thread_default(data->context);
g_main_context_unref (data->context);
gst_element_set_state (data->pipeline, GST_STATE_NULL);
gst_object_unref (data->pipeline);
return NULL;
}
/*
* Java Bindings
*/
/* Instruct the native code to create its internal data structure, pipeline and thread */
static void gst_native_init (JNIEnv* env, jobject thiz) {
CustomData *data = g_new0 (CustomData, 1);
SET_CUSTOM_DATA (env, thiz, custom_data_field_id, data);
GST_DEBUG_CATEGORY_INIT (debug_category, "tutorial-2", 0, "Android tutorial 2");
gst_debug_set_threshold_for_name("tutorial-2", GST_LEVEL_DEBUG);
GST_DEBUG ("Created CustomData at %p", data);
data->app = (*env)->NewGlobalRef (env, thiz);
GST_DEBUG ("Created GlobalRef for app object at %p", data->app);
pthread_create (&gst_app_thread, NULL, &app_function, data);
}
/* Quit the main loop, remove the native thread and free resources */
static void gst_native_finalize (JNIEnv* env, jobject thiz) {
CustomData *data = GET_CUSTOM_DATA (env, thiz, custom_data_field_id);
if (!data) return;
GST_DEBUG ("Quitting main loop...");
g_main_loop_quit (data->main_loop);
GST_DEBUG ("Waiting for thread to finish...");
pthread_join (gst_app_thread, NULL);
GST_DEBUG ("Deleting GlobalRef for app object at %p", data->app);
(*env)->DeleteGlobalRef (env, data->app);
GST_DEBUG ("Freeing CustomData at %p", data);
g_free (data);
SET_CUSTOM_DATA (env, thiz, custom_data_field_id, NULL);
GST_DEBUG ("Done finalizing");
}
/* Set pipeline to PLAYING state */
static void gst_native_play (JNIEnv* env, jobject thiz) {
CustomData *data = GET_CUSTOM_DATA (env, thiz, custom_data_field_id);
if (!data) return;
GST_DEBUG ("Setting state to PLAYING");
gst_element_set_state (data->pipeline, GST_STATE_PLAYING);
}
/* Set pipeline to PAUSED state */
static void gst_native_pause (JNIEnv* env, jobject thiz) {
CustomData *data = GET_CUSTOM_DATA (env, thiz, custom_data_field_id);
if (!data) return;
GST_DEBUG ("Setting state to PAUSED");
gst_element_set_state (data->pipeline, GST_STATE_PAUSED);
}
/* Static class initializer: retrieve method and field IDs */
static jboolean gst_native_class_init (JNIEnv* env, jclass klass) {
custom_data_field_id = (*env)->GetFieldID (env, klass, "native_custom_data", "J");
set_message_method_id = (*env)->GetMethodID (env, klass, "setMessage", "(Ljava/lang/String;)V");
on_gstreamer_initialized_method_id = (*env)->GetMethodID (env, klass, "onGStreamerInitialized", "()V");
if (!custom_data_field_id || !set_message_method_id || !on_gstreamer_initialized_method_id) {
/* We emit this message through the Android log instead of the GStreamer log because the later
* has not been initialized yet.
*/
__android_log_print (ANDROID_LOG_ERROR, "tutorial-2", "The calling class does not implement all necessary interface methods");
return JNI_FALSE;
}
return JNI_TRUE;
}
/* List of implemented native methods */
static JNINativeMethod native_methods[] = {
{ "nativeInit", "()V", (void *) gst_native_init},
{ "nativeFinalize", "()V", (void *) gst_native_finalize},
{ "nativePlay", "()V", (void *) gst_native_play},
{ "nativePause", "()V", (void *) gst_native_pause},
{ "nativeClassInit", "()Z", (void *) gst_native_class_init}
};
/* Library initializer */
jint JNI_OnLoad(JavaVM *vm, void *reserved) {
JNIEnv *env = NULL;
java_vm = vm;
if ((*vm)->GetEnv(vm, (void**) &env, JNI_VERSION_1_4) != JNI_OK) {
__android_log_print (ANDROID_LOG_ERROR, "tutorial-2", "Could not retrieve JNIEnv");
return 0;
}
jclass klass = (*env)->FindClass (env, "com/gst_sdk_tutorials/tutorial_2/Tutorial2");
(*env)->RegisterNatives (env, klass, native_methods, G_N_ELEMENTS(native_methods));
pthread_key_create (¤t_jni_env, detach_current_thread);
return JNI_VERSION_1_4;
}
Let’s start with the CustomData
structure. We have seen it in most of
the basic tutorials, and it is used to hold all our information in one
place, so we can easily pass it around to
callbacks:
/* Structure to contain all our information, so we can pass it to callbacks */
typedef struct _CustomData {
jobject app; /* Application instance, used to call its methods. A global reference is kept. */
GstElement *pipeline; /* The running pipeline */
GMainContext *context; /* GLib context used to run the main loop */
GMainLoop *main_loop; /* GLib main loop */
gboolean initialized; /* To avoid informing the UI multiple times about the initialization */
} CustomData;
We will see the meaning of each member as we go. What is interesting now
is that CustomData
belongs to the application, so a pointer is kept in
the Tutorial2 Java class in the private long native_custom_data
attribute. Java only holds this pointer for us; it
is completely handled in C code.
From C, this pointer can be set and retrieved with the
SetLongField()
and
GetLongField()
JNI functions, but two convenience macros have been defined,
SET_CUSTOM_DATA
and GET_CUSTOM_DATA
. These macros are handy because
the long
type used in Java is always 64 bits wide, but the pointer
used in C can be either 32 or 64 bits wide. The macros take care of the
conversion without warnings.
/* Library initializer */
jint JNI_OnLoad(JavaVM *vm, void *reserved) {
JNIEnv *env = NULL;
java_vm = vm;
if ((*vm)->GetEnv(vm, (void**) &env, JNI_VERSION_1_4) != JNI_OK) {
__android_log_print (ANDROID_LOG_ERROR, "tutorial-2", "Could not retrieve JNIEnv");
return 0;
}
jclass klass = (*env)->FindClass (env, "com/gst_sdk_tutorials/tutorial_2/Tutorial2");
(*env)->RegisterNatives (env, klass, native_methods, G_N_ELEMENTS(native_methods));
pthread_key_create (¤t_jni_env, detach_current_thread);
return JNI_VERSION_1_4;
}
The JNI_OnLoad
function is almost the same as the previous tutorial.
It registers the list of native methods (which is longer in this
tutorial). It also
uses pthread_key_create()
to be able to store per-thread information, which is crucial to properly
manage the JNI Environment, as shown later.
/* Static class initializer: retrieve method and field IDs */
static jboolean gst_native_class_init (JNIEnv* env, jclass klass) {
custom_data_field_id = (*env)->GetFieldID (env, klass, "native_custom_data", "J");
set_message_method_id = (*env)->GetMethodID (env, klass, "setMessage", "(Ljava/lang/String;)V");
on_gstreamer_initialized_method_id = (*env)->GetMethodID (env, klass, "onGStreamerInitialized", "()V");
if (!custom_data_field_id || !set_message_method_id || !on_gstreamer_initialized_method_id) {
/* We emit this message through the Android log instead of the GStreamer log because the later
* has not been initialized yet.
*/
__android_log_print (ANDROID_LOG_ERROR, "tutorial-2", "The calling class does not implement all necessary interface methods");
return JNI_FALSE;
}
return JNI_TRUE;
}
This method is called from the static initializer of the Java class,
which is passed as a parameter (since this is called from a static
method, it receives a class object instead of an instance object). In
order for C code to be able to call a Java method, it needs to know the
method’s
MethodID.
This ID is obtained from the method’s name and signature and can be
cached. The purpose of the gst_native_class_init()
function is to
obtain the IDs of all the methods and fields that the C code will need.
If some ID cannot be retrieved, the calling Java class does not offer
the expected interface and execution should halt (which is not currently
done for simplicity).
Let’s review now the first native method which can be directly called from Java:
gst_native_init()
(nativeInit()
from Java)
This method is called at the end of Java's onCreate()
.
static void gst_native_init (JNIEnv* env, jobject thiz) {
CustomData *data = g_new0 (CustomData, 1);
SET_CUSTOM_DATA (env, thiz, custom_data_field_id, data);
It first allocates memory for the CustomData
structure and passes the
pointer to the Java class with SET_CUSTOM_DATA
, so it is remembered.
data->app = (*env)->NewGlobalRef (env, thiz);
A pointer to the application class (the Tutorial2
class) is also kept
in CustomData
(a Global
Reference
is used) so its methods can be called later.
pthread_create (&gst_app_thread, NULL, &app_function, data);
Finally, a thread is created and it starts running the
app_function()
method.
app_function()
/* Main method for the native code. This is executed on its own thread. */
static void *app_function (void *userdata) {
JavaVMAttachArgs args;
GstBus *bus;
CustomData *data = (CustomData *)userdata;
GSource *bus_source;
GError *error = NULL;
GST_DEBUG ("Creating pipeline in CustomData at %p", data);
/* Create our own GLib Main Context and make it the default one */
data->context = g_main_context_new ();
g_main_context_push_thread_default(data->context);
It first creates a GLib context so all GSource
are kept in the same
place. This also helps cleaning after GSources created by other
libraries which might not have been properly disposed of. A new context
is created with g_main_context_new()
and then it is made the default
one for the thread with
g_main_context_push_thread_default()
.
data->pipeline = gst_parse_launch("audiotestsrc ! audioconvert ! audioresample ! autoaudiosink", &error);
if (error) {
gchar *message = g_strdup_printf("Unable to build pipeline: %s", error->message);
g_clear_error (&error);
set_ui_message(message, data);
g_free (message);
return NULL;
}
It then creates a pipeline the easy way, with gst-parse-launch()
. In
this case, it is simply an audiotestsrc
(which produces a continuous
tone) and an autoaudiosink
, with accompanying adapter elements.
bus = gst_element_get_bus (data->pipeline);
bus_source = gst_bus_create_watch (bus);
g_source_set_callback (bus_source, (GSourceFunc) gst_bus_async_signal_func, NULL, NULL);
g_source_attach (bus_source, data->context);
g_source_unref (bus_source);
g_signal_connect (G_OBJECT (bus), "message::error", (GCallback)error_cb, data);
g_signal_connect (G_OBJECT (bus), "message::state-changed", (GCallback)state_changed_cb, data);
gst_object_unref (bus);
These lines create a bus signal watch and connect to some interesting
signals, just like we have been doing in the basic tutorials. The
creation of the watch is done step by step instead of using
gst_bus_add_signal_watch()
to exemplify how to use a custom GLib
context.
GST_DEBUG ("Entering main loop... (CustomData:%p)", data);
data->main_loop = g_main_loop_new (data->context, FALSE);
check_initialization_complete (data);
g_main_loop_run (data->main_loop);
GST_DEBUG ("Exited main loop");
g_main_loop_unref (data->main_loop);
data->main_loop = NULL;
Finally, the main loop is created and set to run. When it exits (because
somebody else calls g_main_loop_quit()
) the main loop is disposed of.
Before entering the main loop, though,
check_initialization_complete()
is called. This method checks if all
conditions are met to consider the native code “ready” to accept
commands. Since having a running main loop is one of the conditions,
check_initialization_complete()
is called here. This method is
reviewed below.
Once the main loop has quit, all resources are freed in lines 178 to 181.
check_initialization_complete()
static void check_initialization_complete (CustomData *data) {
JNIEnv *env = get_jni_env ();
if (!data->initialized && data->main_loop) {
GST_DEBUG ("Initialization complete, notifying application. main_loop:%p", data->main_loop);
(*env)->CallVoidMethod (env, data->app, on_gstreamer_initialized_method_id);
if ((*env)->ExceptionCheck (env)) {
GST_ERROR ("Failed to call Java method");
(*env)->ExceptionClear (env);
}
data->initialized = TRUE;
}
}
This method does not do much in this tutorial, but it will also be used in the next ones, with progressively more complex functionality. Its purpose is to check if the native code is ready to accept commands, and, if so, notify the UI code.
In tutorial 2, the only conditions are 1) the code is not already
initialized and 2) the main loop is running. If these two are met, the
Java onGStreamerInitialized()
method is called via the
CallVoidMethod()
JNI call.
Here comes a tricky bit. JNI calls require a JNI Environment, which is
different for every thread. C methods called from Java receive a
JNIEnv
pointer as a parameter, but this is not the situation with
check_initialization_complete()
. Here, we are in a thread which has
never been called from Java, so we have no JNIEnv
. We need to use the
JavaVM
pointer (passed to us in the JNI_OnLoad()
method, and shared
among all threads) to attach this thread to the Java Virtual Machine and
obtain a JNIEnv
. This JNIEnv
is stored in the Thread-Local
Storage (TLS) using
the pthread key we created in JNI_OnLoad()
, so we do not need to
attach the thread anymore.
This behavior is implemented in the get_jni_env()
method, used for
example in check_initialization_complete()
as we have just seen. Let’s
see how it works, step by step:
get_jni_env()
static JNIEnv *get_jni_env (void) {
JNIEnv *env;
if ((env = pthread_getspecific (current_jni_env)) == NULL) {
env = attach_current_thread ();
pthread_setspecific (current_jni_env, env);
}
return env;
}
It first retrieves the current JNIEnv
from the TLS using
pthread_getspecific()
and the key we obtained from
pthread_key_create().
If it returns NULL, we never attached this thread, so we do now with
attach_current_thread()
and then store the new JNIEnv
into the TLS
with
pthread_setspecific().
attach_current_thread()
This method is simply a convenience wrapper around AttachCurrentThread() to deal with its parameters.
detach_current_thread()
This method is called by the pthreads library when a TLS key is deleted, meaning that the thread is about to be destroyed. We simply detach the thread from the JavaVM with DetachCurrentThread().
Let's now review the rest of the native methods accessible from Java:
gst_native_finalize()
(nativeFinalize()
from Java)
static void gst_native_finalize (JNIEnv* env, jobject thiz) {
CustomData *data = GET_CUSTOM_DATA (env, thiz, custom_data_field_id);
if (!data) return;
GST_DEBUG ("Quitting main loop...");
g_main_loop_quit (data->main_loop);
GST_DEBUG ("Waiting for thread to finish...");
pthread_join (gst_app_thread, NULL);
GST_DEBUG ("Deleting GlobalRef for app object at %p", data->app);
(*env)->DeleteGlobalRef (env, data->app);
GST_DEBUG ("Freeing CustomData at %p", data);
g_free (data);
SET_CUSTOM_DATA (env, thiz, custom_data_field_id, NULL);
GST_DEBUG ("Done finalizing");
}
This method is called from Java in onDestroy()
, when the activity is
about to be destroyed. Here, we:
- Instruct the GLib main loop to quit with
g_main_loop_quit()
. This call returns immediately, and the main loop will terminate at its earliest convenience. - Wait for the thread to finish with
pthread_join().
This call blocks until the
app_function()
method returns, meaning that the main loop has exited, and the thread has been destroyed. - Dispose of the global reference we kept for the Java application
class (
Tutorial2
) inCustomData
. - Free
CustomData
and set the Java pointer inside theTutorial2
class to NULL withSET_CUSTOM_DATA()
.
gst_native_play
and gst_native_pause()
(nativePlay
and nativePause()
from Java)
These two simple methods retrieve CustomData
from the passed-in object
with GET_CUSTOM_DATA()
and set the pipeline found inside CustomData
to the desired state, returning immediately.
Finally, let’s see how the GStreamer callbacks are handled:
error_cb
and state_changed_cb
This tutorial does not do much in these callbacks. They simply parse the
error or state changed message and display a message in the UI using the
set_ui_message()
method:
set_ui_message()
static void set_ui_message (const gchar *message, CustomData *data) {
JNIEnv *env = get_jni_env ();
GST_DEBUG ("Setting message to: %s", message);
jstring jmessage = (*env)->NewStringUTF(env, message);
(*env)->CallVoidMethod (env, data->app, set_message_method_id, jmessage);
if ((*env)->ExceptionCheck (env)) {
GST_ERROR ("Failed to call Java method");
(*env)->ExceptionClear (env);
}
(*env)->DeleteLocalRef (env, jmessage);
}
This is the other method (besides check_initialization_complete()
)
that needs to call a Java function from a thread which never received an
JNIEnv
pointer directly. Notice how all the complexities of attaching
the thread to the JavaVM and storing the JNI environment in the TLS are
hidden in the simple call to get_jni_env()
.
The desired message (received in ASCII, or modified UTF8), is converted to UTF16 as required by Java using the NewStringUTF() JNI call.
The setMessage()
Java method is called via the JNI
CallVoidMethod()
using the global reference to the class we are keeping in
CustomData
(data->app
) and the set_message_method_id
we cached in
gst_native_class_init()
.
We check for exceptions with the JNI ExceptionCheck() method and free the UTF16 message with DeleteLocalRef().
A pipeline on Android [Android.mk]
jni/Android.mk
LOCAL_PATH := $(call my-dir)
include $(CLEAR_VARS)
LOCAL_MODULE := tutorial-2
LOCAL_SRC_FILES := tutorial-2.c
LOCAL_SHARED_LIBRARIES := gstreamer_android
LOCAL_LDLIBS := -llog
include $(BUILD_SHARED_LIBRARY)
ifndef GSTREAMER_SDK_ROOT
ifndef GSTREAMER_SDK_ROOT_ANDROID
$(error GSTREAMER_SDK_ROOT_ANDROID is not defined!)
endif
GSTREAMER_SDK_ROOT := $(GSTREAMER_SDK_ROOT_ANDROID)
endif
GSTREAMER_NDK_BUILD_PATH := $(GSTREAMER_SDK_ROOT)/share/gst-android/ndk-build/
include $(GSTREAMER_NDK_BUILD_PATH)/plugins.mk
GSTREAMER_PLUGINS := $(GSTREAMER_PLUGINS_CORE) $(GSTREAMER_PLUGINS_SYS)
include $(GSTREAMER_NDK_BUILD_PATH)/gstreamer.mk
Notice how the required GSTREAMER_PLUGINS
are now
$(GSTREAMER_PLUGINS_CORE)
(For the test source and converter elements)
and $(GSTREAMER_PLUGINS_SYS)
(for the audio sink).
And this is it! This has been a rather long tutorial, but we covered a lot of territory. Building on top of this one, the following ones are shorter and focus only on the new topics.
Conclusion
This tutorial has shown:
- How to manage multiple threads from C code and have them interact with java.
- How to access Java code from any C thread using AttachCurrentThread().
- How to allocate a CustomData structure from C and have Java host it, so it is available to all threads.
Most of the methods introduced in this tutorial, like get_jni_env()
,
check_initialization_complete()
, app_function()
and the API methods
gst_native_init()
, gst_native_finalize()
and
gst_native_class_init()
will continue to be used in the following
tutorials with minimal modifications, so better get used to them!
As usual, it has been a pleasure having you here, and see you soon!
Attachments:
tutorial2-screenshot.png
(image/png)
tutorial2-screenshot.png
(image/png)
tutorial2-screenshot.png
(image/png)
tutorial2-screenshot.png
(image/png)
Document generated by Confluence on Oct 08, 2015 10:27