gstreamer-cheat-sheet/sharing_and_splitting_pipelines.md

# Sharing and splitting pipelines (GStreamer command-line cheat sheet)

There are various reasons why you might want your video (or audio) to leave the pipeline, such as:

* To enter a separate application, such as [Snowmix](http://snowmix.sourceforge.net/)
* To use multiple processes (perhaps for security reasons)
* To split into multiple pipelines, so that a failure in one part does not alter another
* To split into multiple pipelines, so that you can 'seek' (jump to a certain point) in one without affecting another

To quote from http://blog.nirbheek.in/2018/02/decoupling-gstreamer-pipelines.html:

> In some applications, you want even greater decoupling of parts of your pipeline.
> For instance, if you're reading data from the network, you don't want a network error
> to bring down our entire pipeline, or if you're working with a hotpluggable device,
> device removal should be recoverable without needing to restart the pipeline.

There are many elements that can achieve this, each with their own pros and cons.

## Summary of methods to share and split pipelines

_As with the rest of this site, this is a rough guide, and is probably not complete or accurate!_

| Name | Description | Points to note | Further reading |
| ---- | ----------- | -------------- | --------------- |
| *shmsink and shmsrc* | Allows video to be read/written from shared memory | Used to send/receive from Snowmix | See below |
| *appsrc/appsink* | Allows video data to leave/enter the pipeline from your own application | n/a | https://thiblahute.github.io/GStreamer-doc/app-1.0/index.html?gi-language=c |
| *fdsrc/fdsink* | Allows communication via a file descriptor | n/a | https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gstreamer-plugins/html/gstreamer-plugins-fdsrc.html |
| *interpipe* | Allows simple communication between two or more independent pipelines. Very powerful. | * Not part of GStreamer (though it is open-source... I'm not sure why it's not been included) | * Well-documented at https://developer.ridgerun.com/wiki/index.php?title=GstInterpipe<br>* https://gstreamer.freedesktop.org/data/events/gstreamer-conference/2015/Melissa%20Montero%20-%20GST%20Daemon%20and%20Interpipes:%20A%20simpler%20way%20to%20get%20your%20applications%20done%20.pdf |
| *inter* (intervideosink, etc) | Send/receive AV between two pipelines in the same process | Only support raw audio or video, and drop events and queries at the boundary (source: [Nirbheek's blog](http://blog.nirbheek.in/2018/02/decoupling-gstreamer-pipelines.html)) | * https://thiblahute.github.io/GStreamer-doc/inter-1.0/index.html?gi-language=c<br>* Pros and cons discussed here: http://gstreamer-devel.966125.n4.nabble.com/How-to-connect-intervideosink-and-intervideosrc-for-IPC-pipelines-td4684567.html |
| *ipcpipeline* | Allows communication between pipelines *in different processes*. | Arrived with GStreamer 1.14 (Spring 2018) |  https://www.collabora.com/news-and-blog/blog/2017/11/17/ipcpipeline-splitting-a-gstreamer-pipeline-into-multiple-processes/ |
| *gstproxy (proxysink and proxysrc)* | Send/receive AV between two pipelines in the same process. | Arrived with GStreamer 1.14 (Spring 2018) | See below |


## Sharing via memory - shmsink and shmsrc

The [`shmsink`](https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-bad/html/gst-plugins-bad-plugins-shmsink.html) element allows you to write video into shared memory, from which another gstreamer application can read it with [`shmsrc`](https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-bad/html/gst-plugins-bad-plugins-shmsrc.html).


### Putting a stream into memory

Put a test video source into memory:

```
gst-launch-1.0 -v videotestsrc ! \
    'video/x-raw, format=(string)I420,  width=(int)320, height=(int)240, framerate=(fraction)30/1' ! \
    queue !  identity ! \
    shmsink wait-for-connection=1 socket-path=/tmp/tmpsock  shm-size=20000000 sync=true  
```

Another example, this time from a file rather than test source, and keeping the audio local:

```
gst-launch-1.0 filesrc location=$SRC ! \
    qtdemux name=demux  demux.audio_0 ! queue ! decodebin ! audioconvert ! audioresample ! \
    autoaudiosink \
    demux.video_0 ! queue ! \
    decodebin ! videoconvert ! videoscale ! videorate ! \
    'video/x-raw, format=(string)I420,  width=(int)320, height=(int)240, framerate=(fraction)30/1' ! \
    queue !  identity ! \
    shmsink wait-for-connection=0 socket-path=/tmp/tmpsock  shm-size=20000000 sync=true  
```

### Reading a stream from memory

This will display the video of a stream locally:

```
gst-launch-1.0 shmsrc socket-path=/tmp/tmpsock ! \
    'video/x-raw, format=(string)I420, width=(int)320, height=(int)240, framerate=(fraction)30/1' ! \
    autovideosink
````

## gstproxy (proxysink and proxysrc)

I've used `proxysink` and `proxysrc` to split large pipelines into smaller ones. That way, if a part fails, the rest can continue.


### gstproxy documentation

* Introduced by the blog mentioned above (http://blog.nirbheek.in/2018/02/decoupling-gstreamer-pipelines.html)
* Example code on proxysrc here: https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-bad-plugins/html/gst-plugins-bad-plugins-proxysrc.html
* Equivalent proxysink: https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-bad-plugins/html/gst-plugins-bad-plugins-proxysink.html


### gstproxy examples

It's not possible to use them via the command-line, because you connect them by having the receiver (`proxysrc`) reference the sender (`proxysink`).

A very simple example would be:

```
1st pipeline:   audiotestsrc is-live=1 ! proxysink
2nd pipeline:   proxysrc ! autoaudiosink
```

This would achieve the same as `audiotestsrc | autoaudiosink`, but in two pipelines.

An Python example of this can be found at [/python_examples/gstproxy_01_audiotestsrc.py](/python_examples/gstproxy_01_audiotestsrc.py).

A slightly more interesting example can be found at

[/python_examples/gstproxy_02_playbin.py](/python_examples/gstproxy_02_playbin.py). This plays a video file (e.g. mp4). It shows:

* that `proxysink` can work with [`playbin`](https://gstreamer.freedesktop.org/data/doc/gstreamer/head/gst-plugins-base-plugins/html/gst-plugins-base-plugins-playbin.html)
* separate proxies for audio and video
* that when the video ends, the other pipelines continue.