gstreamer/sdk-using-appsink-appsrc-in-qt.md

# Using appsink/appsrc in Qt

## Goal

For those times when you need to stream data into or out of GStreamer
through your application, GStreamer includes two helpful elements:

  - `appsink` - Allows applications to easily extract data from a
    GStreamer pipeline
  - `appsrc` - Allows applications to easily stream data into a
    GStreamer pipeline

This tutorial will demonstrate how to use both of them by constructing a
pipeline to decode an audio file, stream it into an application's code,
then stream it back into your audio output device. All this, using
QtGStreamer.

## Steps

First, the files. These are also available in the
`examples/appsink-src` directory of the QGstreamer SDK.

**CMakeLists.txt**

```
project(qtgst-example-appsink-src)
find_package(QtGStreamer REQUIRED)
find_package(Qt4 REQUIRED)
include_directories(${QTGSTREAMER_INCLUDES} ${QT_QTCORE_INCLUDE_DIRS})
add_definitions(${QTGSTREAMER_DEFINITIONS})
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${QTGSTREAMER_FLAGS}")
add_executable(appsink-src main.cpp)
target_link_libraries(appsink-src ${QTGSTREAMER_UTILS_LIBRARIES} ${QT_QTCORE_LIBRARIES})
```

**main.cpp**

``` c
#include <iostream>
#include <QtCore/QCoreApplication>
#include <QGlib/Error>
#include <QGlib/Connect>
#include <QGst/Init>
#include <QGst/Bus>
#include <QGst/Pipeline>
#include <QGst/Parse>
#include <QGst/Message>
#include <QGst/Utils/ApplicationSink>
#include <QGst/Utils/ApplicationSource>

class MySink : public QGst::Utils::ApplicationSink
{
public:
    MySink(QGst::Utils::ApplicationSource *src)
        : QGst::Utils::ApplicationSink(), m_src(src) {}
protected:
    virtual void eos()
    {
        m_src->endOfStream();
    }
    virtual QGst::FlowReturn newBuffer()
    {
        m_src->pushBuffer(pullBuffer());
        return QGst::FlowOk;
    }
private:
    QGst::Utils::ApplicationSource *m_src;
};

class Player : public QCoreApplication
{
public:
    Player(int argc, char **argv);
    ~Player();
private:
    void onBusMessage(const QGst::MessagePtr & message);
private:
    QGst::Utils::ApplicationSource m_src;
    MySink m_sink;
    QGst::PipelinePtr pipeline1;
    QGst::PipelinePtr pipeline2;
};
Player::Player(int argc, char **argv)
    : QCoreApplication(argc, argv), m_sink(&m_src)
{
    QGst::init(&argc, &argv);
    if (argc <= 1) {
        std::cerr << "Usage: " << argv[0] << " <audio_file>" << std::endl;
        std::exit(1);
    }
    const char *caps = "audio/x-raw-int,channels=1,rate=8000,"
                       "signed=(boolean)true,width=16,depth=16,endianness=1234";
    /* source pipeline */
    QString pipe1Descr = QString("filesrc location=\"%1\" ! "
                                 "decodebin2 ! "
                                 "audioconvert ! "
                                 "audioresample ! "
                                 "appsink name=\"mysink\" caps=\"%2\"").arg(argv[1], caps);
    pipeline1 = QGst::Parse::launch(pipe1Descr).dynamicCast<QGst::Pipeline>();
    m_sink.setElement(pipeline1->getElementByName("mysink"));
    QGlib::connect(pipeline1->bus(), "message::error", this, &Player::onBusMessage);
    pipeline1->bus()->addSignalWatch();
    /* sink pipeline */
    QString pipe2Descr = QString("appsrc name=\"mysrc\" caps=\"%1\" ! autoaudiosink").arg(caps);
    pipeline2 = QGst::Parse::launch(pipe2Descr).dynamicCast<QGst::Pipeline>();
    m_src.setElement(pipeline2->getElementByName("mysrc"));
    QGlib::connect(pipeline2->bus(), "message", this, &Player::onBusMessage);
    pipeline2->bus()->addSignalWatch();
    /* start playing */
    pipeline1->setState(QGst::StatePlaying);
    pipeline2->setState(QGst::StatePlaying);
}
Player::~Player()
{
    pipeline1->setState(QGst::StateNull);
    pipeline2->setState(QGst::StateNull);
}
void Player::onBusMessage(const QGst::MessagePtr & message)
{
    switch (message->type()) {
    case QGst::MessageEos:
        quit();
        break;
    case QGst::MessageError:
        qCritical() << message.staticCast<QGst::ErrorMessage>()->error();
        break;
    default:
        break;
    }
}

int main(int argc, char **argv)
{
    Player p(argc, argv);
    return p.exec();
}
```

### Walkthrough

As this is a very simple example, most of the action happens in the
`Player`'s constructor. First, GStreamer is initialized through
`QGst::init()`:

**GStreamer Initialization**

``` c
    QGst::init(&argc, &argv);
```

Now we can construct the first half of the pipeline:

**Pipeline Setup**

``` c
    const char *caps = "audio/x-raw-int,channels=1,rate=8000,"
                       "signed=(boolean)true,width=16,depth=16,endianness=1234";
 
    /* source pipeline */
    QString pipe1Descr = QString("filesrc location=\"%1\" ! "
                                 "decodebin2 ! "
                                 "audioconvert ! "
                                 "audioresample ! "
                                 "appsink name=\"mysink\" caps=\"%2\"").arg(argv[1], caps);
    pipeline1 = QGst::Parse::launch(pipe1Descr).dynamicCast<QGst::Pipeline>();
    m_sink.setElement(pipeline1->getElementByName("mysink"));
    QGlib::connect(pipeline1->bus(), "message::error", this, &Player::onBusMessage);
    pipeline1->bus()->addSignalWatch();
```

`QGst::Parse::launch()` parses the text description of a pipeline and
returns a `QGst::PipelinePtr`. In this case, the pipeline is composed
of:

  - A `filesrc` element to read the file
  - `decodebin2` to automatically examine the stream and pick the right
    decoder(s)
  - `audioconvert` and `audioresample` to convert the output of the
    `decodebin2` into the caps specified for the `appsink`
  - An `appsink` element with specific caps

Next, we tell our `MySink` class (which is a subclass
of `QGst::Utils::ApplicationSink`) what `appsink` element to use.

The second half of the pipeline is created similarly:

**Second Pipeline**

``` c
    /* sink pipeline */
    QString pipe2Descr = QString("appsrc name=\"mysrc\" caps=\"%1\" ! autoaudiosink").arg(caps);
    pipeline2 = QGst::Parse::launch(pipe2Descr).dynamicCast<QGst::Pipeline>();
    m_src.setElement(pipeline2->getElementByName("mysrc"));
    QGlib::connect(pipeline2->bus(), "message", this, &Player::onBusMessage);
    pipeline2->bus()->addSignalWatch();
```

Finally, the pipeline is started:

**Starting the pipeline**

``` c
 /* start playing */
    pipeline1->setState(QGst::StatePlaying);
    pipeline2->setState(QGst::StatePlaying);
```

Once the pipelines are started, the first one begins pushing buffers
into the `appsink` element. Our `MySink` class implements the
`newBuffer()` method, which is called by QGStreamer when a new buffer is
ready for processing:

**MySink::newBuffer()**

``` c
    virtual QGst::FlowReturn newBuffer()
    {
        m_src->pushBuffer(pullBuffer());
        return QGst::FlowOk;
    }
```

Our implementation takes the new buffer and pushes it into the
`appsrc` element, which got assigned in the `Player` constructor:

**Player::Player()**

``` c
Player::Player(int argc, char **argv)
    : QCoreApplication(argc, argv), m_sink(&m_src)
```

From there, buffers flow into the `autoaudiosink` element, which
automatically figures out a way to send it to your speakers.

## Conclusion

You should now have an understanding of how to push and pull arbitrary
data into and out of a GStreamer pipeline.

It has been a pleasure having you here, and see you soon\!