mirror of https://gitlab.freedesktop.org/gstreamer/gstreamer.git synced 2024-11-08 18:39:54 +00:00

Julien e9f5d94c93 gs: add source and sink for Google Cloud Storage

Useful when having a service that runs a GStreamer pipeline
or application in Google Cloud to avoid storing the inputs
and outputs in the running container or service. For example
when analyzing a video from a Google Cloud Storage bucket
and extracting images or converting the video and then uploading
the results into another Google Cloud Storage bucket.

- gssrc allows to read from a file located in Google Cloud
Storage and it supports seeking.
- gssink allows to write to a file located in Google Cloud
Storage. There are 2 modes, one similar to multifilesink and
the other similar to filesink.

Example:
  gst-launch-1.0 gssrc location=gs://mybucket/videos/sample.mp4 ! decodebin ! glimagesink
  gst-launch-1.0 playbin uri=gs://mybucket/videos/sample.mp4
  gst-launch-1.0 videotestsrc num-buffers=5 ! pngenc ! gssink object-name="img/img%05d.png" bucket-name="mybucket" next-file=buffer
  gst-launch-1.0 filesrc location=sample.mp4 ! gssink object-name="videos/video.mp4" bucket-name="mybucket" next-file=none

When running locally simply set GOOGLE_APPLICATION_CREDENTIALS. But
when running in Google Cloud Run or Google Cloud Engine, just set the
"service-account-email" property on each element.

Closes #1264

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-plugins-bad/-/merge_requests/1369>

2021-03-18 22:32:48 +00:00

2.1 KiB

Raw Blame History

Install the Google Cloud Storage dependencies.

sudo apt-get install \
    cmake \
    libcurl3-gnutls-dev \
    libgrpc++-dev \
    libprotobuf-dev \
    protobuf-compiler-grpc

Build the Google Cloud Storage library

git clone https://github.com/google/crc32c.git
cd crc32c && git checkout -b 1.1.1
mkdir build && cd build
cmake .. \
    -GNinja \
    -DCMAKE_INSTALL_PREFIX:PATH=~/dev/gst-build/prefix \
    -DCMAKE_INSTALL_LIBDIR:PATH=lib \
    -DBUILD_SHARED_LIBS=YES \
    -DCRC32C_USE_GLOG=NO \
    -DCRC32C_BUILD_TESTS=NO \
    -DCRC32C_BUILD_BENCHMARKS=NO
ninja && ninja install

git clone https://github.com/abseil/abseil-cpp.git
git checkout master
mkdir build && cd build
cmake .. \
    -GNinja \
    -DBUILD_TESTING=NO \
    -DCMAKE_INSTALL_PREFIX:PATH=~/dev/gst-build/prefix \
    -DCMAKE_INSTALL_LIBDIR:PATH=lib \
    -DBUILD_SHARED_LIBS=YES
ninja && ninja install

git clone https://github.com/googleapis/google-cloud-cpp.git
git checkout -b v1.25.0
mkdir build && cd build
cmake .. \
    -GNinja \
    -DCMAKE_INSTALL_PREFIX:PATH=~/dev/gst-build/prefix \
    -DCMAKE_INSTALL_LIBDIR:PATH=lib \
    -DBUILD_SHARED_LIBS=YES \
    -DBUILD_TESTING=NO \
    -DGOOGLE_CLOUD_CPP_ENABLE=storage
ninja && ninja install

Running the gs elements locally

When running from the command line or in a container running locally, simply set the credentials by exporting GOOGLE_APPLICATION_CREDENTIALS. If you are not familiar with this environment variable, check the documentation https://cloud.google.com/docs/authentication/getting-started Note that you can restrict a service account to the role Storage Admin or Storage Object Creator instead of the Project Owner role from the above documentation.

Running the gs elements in Google Cloud Run

Add the Storage Object Viewer role to the service account assigned to the Cloud Run service where gssrc runs. For gssink add the role Storage Object Creator. Then just set the service-account-email property on the element.

Running the gs elements in Google Cloud Kubernetes

You need to set GOOGLE_APPLICATION_CREDENTIALS in the container and ship the json file to which the environment variable points to.

2.1 KiB Raw Blame History