This chapter will discuss how you can manipulate your pipeline in several ways from your application on. Parts of this chapter are downright hackish, so be assured that you'll need some programming knowledge before you start reading this. Topics that will be discussed here include how you can insert data into a pipeline from your application, how to read data from a pipeline, how to manipulate the pipeline's speed, length, starting point and how to listen to a pipeline's data processing. Probing is best envisioned as a pad listener. Technically, a probe is nothing more than a signal callback that can be attached to a pad. Those signals are by default not fired at all (since that may have a negative impact on performance), but can be enabled by attaching a probe using gst_pad_add_data_probe () or one of the similar functions. Those functions attach the signal handler and enable the actual signal emission. Similarly, one can use the gst_pad_remove_data_probe () or related functions to remove the signal handlers again. It is also possible to only listen to events or only to buffers (and ignore the other). Probes run in pipeline threading context, so callbacks should try to not block and generally not do any weird stuff, since this could have a negative impact on pipeline performance or, in case of bugs, cause deadlocks or crashes. However, most common buffer operations that elements can do in _chain () functions, can be done in probe callbacks as well. The example below gives a short impression on how to use them. #include <gst/gst.h> static gboolean cb_have_data (GstPad *pad, GstBuffer *buffer, gpointer u_data) { gint x, y; guint16 *data = (guint16 *) GST_BUFFER_DATA (buffer), t; /* invert data */ for (y = 0; y < 288; y++) { for (x = 0; x < 384 / 2; x++) { t = data[384 - 1 - x]; data[384 - 1 - x] = data[x]; data[x] = t; } data += 384; } return TRUE; } gint main (gint argc, gchar *argv[]) { GMainLoop *loop; GstElement *pipeline, *src, *sink, *filter, *csp; GstPad *pad; /* init GStreamer */ gst_init (&argc, &argv); loop = g_main_loop_new (NULL, FALSE); /* build */ pipeline = gst_pipeline_new ("my-pipeline"); src = gst_element_factory_make ("videotestsrc", "src"); filter = gst_element_factory_make ("capsfilter", "filter"); csp = gst_element_factory_make ("ffmpegcolorspace", "csp"); sink = gst_element_factory_make ("xvimagesink", "sink"); gst_bin_add_many (GST_BIN (pipeline), src, filter, csp, sink, NULL); gst_element_link_many (src, filter, csp, sink, NULL); g_object_set (G_OBJECT (filter), "filter-caps", gst_caps_new_simple ("video/x-raw-rgb", "width", G_TYPE_INT, 384, "height", G_TYPE_INT, 288, "framerate", G_TYPE_DOUBLE, (gdouble) 25.0, "bpp", G_TYPE_INT, 16, "depth", G_TYPE_INT, 16, "endianness", G_TYPE_INT, G_BYTE_ORDER, NULL), NULL); pad = gst_element_get_pad (src, "src"); gst_pad_add_buffer_probe (pad, G_CALLBACK (cb_have_data), NULL); gst_object_unref (pad); /* run */ gst_element_set_state (pipeline, GST_STATE_PLAYING); g_main_loop_run (loop); /* exit */ gst_element_set_state (pipeline, GST_STATE_NULL); gst_object_unref (pipeline); return 0; } Compare that output with the output of gst-launch-0.9 videotestsrc ! xvimagesink, just so you know what you're looking for. Many people have expressed the wish to use their own sources to inject data into a pipeline. Some people have also expressed the wish to grab the output in a pipeline and take care of the actual output inside their application. While either of these methods are stongly discouraged, &GStreamer; offers hacks to do this. However, there is no support for those methods. If it doesn't work, you're on your own. Also, synchronization, thread-safety and other things that you've been able to take for granted so far are no longer guanranteed if you use any of those methods. It's always better to simply write a plugin and have the pipeline schedule and manage it. See the Plugin Writer's Guide for more information on this topic. Also see the next section, which will explain how to embed plugins statically in your application. After all those disclaimers, let's start. There's three possible elements that you can use for the above-mentioned purposes. Those are called fakesrc (an imaginary source), fakesink (an imaginary sink) and identity (an imaginary filter). The same method applies to each of those elements. Here, we will discuss how to use those elements to insert (using fakesrc) or grab (using fakesink or identity) data from a pipeline, and how to set negotiation. Those who're paying close attention, will notice that the purpose of identity is almost identical to that of probes. Indeed, this is true. Probes allow for the same purpose, and a bunch more, and with less overhead plus dynamic removing/adding of handlers, but apart from those, probes and identity have the same purpose, just in a completely different implementation type. The three before-mentioned elements (fakesrc, fakesink and identity) each have a handoff signal that will be called in the _get ()- (fakesrc) or _chain ()-function (identity, fakesink). In the signal handler, you can set (fakesrc) or get (identity, fakesink) data to/from the provided buffer. Note that in the case of fakesrc, you have to set the size of the provided buffer using the sizemax property. For both fakesrc and fakesink, you also have to set the signal-handoffs property for this method to work. Note that your handoff function should not block, since this will block pipeline iteration. Also, do not try to use all sort of weird hacks in such functions to accomplish something that looks like synchronization or so; it's not the right way and will lead to issues elsewhere. If you're doing any of this, you're basically misunderstanding the &GStreamer; design. Sometimes, when using fakesrc as a source in your pipeline, you'll want to set a specific format, for example a video size and format or an audio bitsize and number of channels. You can do this by forcing a specific GstCaps on the pipeline, which is possible by using filtered caps. You can set a filtered caps on a link by using the capsfilter element in between the two elements, and specifying a GstCaps as filter-caps property on this element. It will then only allow types matching that specified capability set for negotiation. This example application will generate black/white (it switches every second) video to an X-window output by using fakesrc as a source and using filtered caps to force a format. Since the depth of the image depends on your X-server settings, we use a colorspace conversion element to make sure that the output to your X server will have the correct bitdepth. You can also set timestamps on the provided buffers to override the fixed framerate. #include <string.h> /* for memset () */ #include <gst/gst.h> static void cb_handoff (GstElement *fakesrc, GstBuffer *buffer, GstPad *pad, gpointer user_data) { static gboolean white = FALSE; /* this makes the image black/white */ memset (GST_BUFFER_DATA (buffer), white ? 0xff : 0x0, GST_BUFFER_SIZE (buffer)); white = !white; } gint main (gint argc, gchar *argv[]) { GstElement *pipeline, *fakesrc, *flt, *conv, *videosink; GMainLoop *loop; /* init GStreamer */ gst_init (&argc, &argv); loop = g_main_loop_new (NULL, FALSE); /* setup pipeline */ pipeline = gst_pipeline_new ("pipeline"); fakesrc = gst_element_factory_make ("fakesrc", "source"); flt = gst_element_factory_make ("capsfilter", "flt"); conv = gst_element_factory_make ("ffmpegcolorspace", "conv"); videosink = gst_element_factory_make ("xvimagesink", "videosink"); /* setup */ g_object_set (G_OBJECT (flt), "filter-caps", gst_caps_new_simple ("video/x-raw-rgb", "width", G_TYPE_INT, 384, "height", G_TYPE_INT, 288, "framerate", G_TYPE_DOUBLE, (gdouble) 1.0, "bpp", G_TYPE_INT, 16, "depth", G_TYPE_INT, 16, "endianness", G_TYPE_INT, G_BYTE_ORDER, NULL), NULL); gst_bin_add_many (GST_BIN (pipeline), fakesrc, flt, conv, videosink, NULL); gst_element_link_many (fakesrc, flt, conv, videosink, NULL); /* setup fake source */ g_object_set (G_OBJECT (fakesrc), "signal-handoffs", TRUE, "sizemax", 384 * 288 * 2, "sizetype", 2, NULL); g_signal_connect (fakesrc, "handoff", G_CALLBACK (cb_handoff), NULL); /* play */ gst_element_set_state (pipeline, GST_STATE_PLAYING); g_main_loop_run (loop); /* clean up */ gst_element_set_state (pipeline, GST_STATE_NULL); gst_object_unref (GST_OBJECT (pipeline)); return 0; } The Plugin Writer's Guide describes in great detail how to write elements for the &GStreamer; framework. In this section, we will solely discuss how to embed such elements statically in your application. This can be useful for application-specific elements that have no use elsewhere in &GStreamer;. Dynamically loaded plugins contain a structure that's defined using GST_PLUGIN_DEFINE (). This structure is loaded when the plugin is loaded by the &GStreamer; core. The structure contains an initialization function (usually called plugin_init) that will be called right after that. It's purpose is to register the elements provided by the plugin with the &GStreamer; framework. If you want to embed elements directly in your application, the only thing you need to do is to manually run this structure using _gst_plugin_register_static (). The initialization will then be called, and the elements will from then on be available like any other element, without them having to be dynamically loadable libraries. In the example below, you would be able to call gst_element_factory_make ("my-element-name", "some-name") to create an instance of the element. /* * Here, you would write the actual plugin code. */ [..] static gboolean register_elements (GstPlugin *plugin) { return gst_element_register (plugin, "my-element-name", GST_RANK_NONE, MY_PLUGIN_TYPE); } static GstPluginDesc plugin_desc = { GST_VERSION_MAJOR, GST_VERSION_MINOR, "my-private-plugins", "Private elements of my application", register_elements, NULL, "0.0.1", "LGPL", "my-application", "http://www.my-application.net/", GST_PADDING_INIT }; /* * Call this function right after calling gst_init (). */ void my_elements_init (void) { _gst_plugin_register_static (&plugin_desc); }