Linux has historically lagged behind other operating systems in the multimedia arena. Microsoft's Windows and Apple's MacOS both have strong support for multimedia devices, multimedia content creation, playback, and realtime processing. Linux, on the other hand, has a poorly integrated collection of multimedia utilities and applications available, which can hardly compete with the professional level of software available for MS Windows and MacOS. GStreamer was designed to provide a solution to the current Linux media problems. We describe the typical problems in today's media handling on Linux. The Linux user who wishes to hear a sound file must hunt through their collection of sound file players in order to play the tens of sound file formats in wide use today. Most of these players basically reimplement the same code over and over again. The Linux developer who wishes to embed a video clip in their application must use crude hacks to run an external video player. There is no library available that a developer can use to create a custom media player. Your typical MPEG player was designed to play MPEG video and audio. Most of these players have implemented a complete infrastructure focused on achieving their only goal: playback. No provisions were made to add filters or special effects to the video or audio data. If you want to convert an MPEG-2 video stream into an AVI file, your best option would be to take all of the MPEG-2 decoding algorithms out of the player and duplicate them into your own AVI encoder. These algorithms cannot easily be shared across applications. Attempts have been made to create libraries for handling various media types. Because they focus on a very specific media type (avifile, libmpeg2, ...), significant work is needed to integrate them due to a lack of a common API. &GStreamer; allows you to wrap these libraries with a common API, which significantly simplifies integration and reuse. Your typical media player might have a plugin for different media types. Two media players will typically implement their own plugin mechanism so that the codecs cannot be easily exchanged. The plugin system of the typical media player is also very tailored to the specific needs of the application. The lack of a unified plugin mechanism also seriously hinders the creation of binary only codecs. No company is willing to port their code to all the different plugin mechanisms. While &GStreamer; also uses it own plugin system it offers a very rich framework for the plugin developer and ensures the plugin can be used in a wide range of applications, transparently interacting with other plugins. The framework that &GStreamer; provides for the plugins is flexible enough to host even the most demanding plugins. Because of the problems mentioned above, application authors have so far often been urged to spend a considerable amount of time in writing their own backends, plugin mechanisms and so on. The result has often been, unfortunately, that both the backend as well as the user interface were only half-finished. Demotivated, the application authors would start rewriting the whole thing and complete the circle. This leads to a poor end user experience. No infrastructure is present to allow network transparent media handling. A distributed MPEG encoder will typically duplicate the same encoder algorithms found in a non-distributed encoder. No provisions have been made for technologies such as the GNOME object embedding using Bonobo. The &GStreamer; core does not use network transparent technologies at the lowest level as it only adds overhead for the local case. That said, it shouldn't be hard to create a wrapper around the core components. There are tcp plugins now that implement a &GStreamer; Data Protocol that allows pipelines to be slit over TCP. These are located in the gst-plugins module directory gst/tcp. We need solid media handling if we want to see Linux succeed on the desktop. We must clear the road for commercially backed codecs and multimedia applications so that Linux can become an option for doing multimedia. We describe what we try to achieve with &GStreamer;. &GStreamer; wants to provide a clean interface to: The application programmer who wants to build a media pipeline. The programmer can use an extensive set of powerful tools to create media pipelines without writing a single line of code. Performing complex media manipulations becomes very easy. The plugin programmer. Plugin programmers are provided a clean and simple API to create self-contained plugins. An extensive debugging and tracing mechanism has been integrated. GStreamer also comes with an extensive set of real-life plugins that serve as examples too. &GStreamer; adheres to the GLib 2.0 object model. A programmer familiar with GLib 2.0 or older versions of GTK+ will be comfortable with &GStreamer;. &GStreamer; uses the mechanism of signals and object properties. All objects can be queried at runtime for their various properties and capabilities. &GStreamer; intends to be similar in programming methodology to GTK+. This applies to the object model, ownership of objects, reference counting, ... All &GStreamer; Objects can be extended using the GObject inheritance methods. All plugins are loaded dynamically and can be extended and upgraded independently. Plugins are shared libraries that are loaded at runtime. Since all the properties of the plugin can be set using the GObject properties, there is no need (and in fact no way) to have any header files installed for the plugins. Special care has been taken to make plugins completely self-contained. All relevant aspects of plugins can be queried at run-time. High performance is obtained by: using GLib's g_mem_chunk and fast non-blocking allocation algorithms where possible to minimize dynamic memory allocation. extremely light-weight links between plugins. Data can travel the pipeline with minimal overhead. Data passing between plugins only involves a pointer dereference in a typical pipeline. providing a mechanism to directly work on the target memory. A plugin can for example directly write to the X server's shared memory space. Buffers can also point to arbitrary memory, such as a sound card's internal hardware buffer. refcounting and copy on write minimize usage of memcpy. Sub-buffers efficiently split buffers into manageable pieces. the use of cothreads to minimize the threading overhead. Cothreads are a simple and fast user-space method for switching between subtasks. Cothreads were measured to consume as little as 600 cpu cycles. allowing hardware acceleration by using specialized plugins. using a plugin registry with the specifications of the plugins so that the plugin loading can be delayed until the plugin is actually used. all critical data passing is free of locks and mutexes. The core of &GStreamer; is essentially media-agnostic. It only knows about bytes and blocks, and only contains basic elements. The core of &GStreamer; is functional enough to even implement low-level system tools, like cp. All of the media handling functionality is provided by plugins external to the core. These tell the core how to handle specific types of media. &GStreamer; also wants to be an easy framework where codec developers can experiment with different algorithms, speeding up the development of open and free multimedia codecs like Theora and Vorbis.