installing: Update the documentation for building with Cerbero

This was horrendously out of date. Copy over the instructions from
Cerbero's README file.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-docs/-/merge_requests/151>
This commit is contained in:
Nirbheek Chauhan 2021-04-09 11:33:23 +05:30
parent 0b8b6f83ad
commit 35b3a4ab25
9 changed files with 378 additions and 163 deletions

Binary file not shown.

After

Width:  |  Height:  |  Size: 19 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 35 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 76 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 75 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 31 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 84 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 190 KiB

View file

@ -1,244 +1,459 @@
---
short-description: Setting up a development environment the modern way
authors:
- name: Nirbheek Chauhan
email: nirbheek@centricular.com
years: [2021]
...
# Building from source using Cerbero
> ![Warning] This section is intended for advanced users.
**If you just want to use GStreamer, please visit [the download page](https://gstreamer.freedesktop.org/download/).
We provide pre-built binaries for Windows, macOS, Android, and iOS**.
## Build requirements
Cerbero is a cross-platform build aggregator for Open Source projects that
builds and creates native packages for different platforms, architectures and
distributions. It supports both native compilation and cross compilation and
can run on macOS, Linux, and Windows.
The GStreamer build system provides bootstrapping facilities for all
platforms, but it still needs a minimum base to bootstrap:
You should use Cerbero to build GStreamer if you:
- python >= 3.5
- git
1. Want to do GStreamer development for Android, iOS, or UWP, or
1. Have to build GStreamer packages for distribution or deployment, or
1. Need plugins with external dependencies without Meson ports
### Windows users
However, if you are a developer who wants to work on the GStreamer code itself
on Linux, Windows, or macOS, it is much more convenient to use gst-build.
Please refer to [Building using Meson](installing/building-from-source-using-meson.md).
Cerbero can be used on Windows using the Msys/MinGW shell (a Unix-like
shell for Windows). There is a bit of setup that you need to do before
Cerbero can take control.
## Minimum Requirements
You need to install the following programs:
Cerbero provides bootstrapping facilities for all platforms, but it still needs a
minimum base to bootstrap on top of.
- [Python 3.5+]
* First page of the installer
- Check "Add Python 3.x to PATH"
- Click "Customize installation"
* Second page, check "pip"
* Third page, select:
- Install for all users
- Associate files with Python
- Add Python to environment variables
- Customize install location: C:\Python3
- [Git]
* Select the install option "Checkout as-is, Commit as-is"
* Ensure that git is installed in PATH, but no other tools are
- [Msys/MinGW] (Install it with all the options enabled)
- [CMake] (Select the option "Add CMake in system path for the
current user")
- [Yasm] (Download the win32 or win64 version for your platform, name
it `yasm.exe`, and place it in your MinGW `bin` directory,
typically, `C:\MinGW\bin`)
- [WiX 3.11.1]
### Linux Setup
Several packages that have Meson build files are now built by default with
Visual Studio, so you need to install Visual Studio 2015 or newer in the
default location. The Visual Studio Community build which is free for
open-source use can be installed at:
On Linux, you will only need a distribution with python >= 3.6. Cerbero will
use your package manager to install all other required packages during
[bootstrap](#bootstrap-to-setup-environment).
* https://visualstudio.microsoft.com/vs/older-downloads/
### macOS Setup
You should add the cerbero git directory to the list of excluded folders in your
anti-virus, or you will get random build failures when Autotools does file
operations such as renames and deletions. It will also slow your build by
about 3-4x.
On macOS you will need to have install the following software:
Your user ID can't have spaces (eg: John Smith). Paths with spaces are
not correctly handled in the build system and msys uses the user ID for
the home folder.
* XCode
* Python 3.6+ https://www.python.org/downloads/
Cerbero must be run in the MinGW shell, which is accessible from the
main menu once MinGW is installed.
Cerbero will build all other required packages during [bootstrap](#bootstrap-to-setup-environment).
(Note that inside the shell, / is mapped to c:\Mingw\msys\1.0 )
### Windows Setup
### macOS users
To use cerbero on macOS you need to install the "Command Line Tools" from
XCode. They are available from the "Preferences" dialog under
"Downloads".
### iOS developers
If you want to build GStreamer for iOS, you also need the iOS
SDK. The minimum required iOS SDK version is 6.0 and is included in
[XCode] since version 4.
The initial setup on Windows is somewhat longer since the required packages
must be installed manually. Detailed steps on what you need to install are
**[at the bottom of the page](#installing-minimum-requirements-on-windows)**.
## Download the sources
To build GStreamer, you first need to download **Cerbero**.
Cerbero is a multi-platform build system for Open Source projects that
builds and creates native packages for different platforms,
architectures and distributions.
To build GStreamer using Cerbero, you first need to download **Cerbero**:
Get a copy of Cerbero by cloning the git repository:
```sh
$ git clone https://gitlab.freedesktop.org/gstreamer/cerbero
```
git clone https://gitlab.freedesktop.org/gstreamer/cerbero
Despite the presence of `setup.py` this tool does not need installation. It is
invoked via the `cerbero-uninstalled` script, which should be invoked as
`./cerbero-uninstalled`, or you can create an alias to it in your `.bashrc`
file.
Cerbero can be run uninstalled and for convenience you can create an
alias in your `.bashrc` file*. *If you prefer to skip this step,
remember that you need to replace the calls to `cerbero` with
`./cerbero-uninstalled` in the next steps.
## Bootstrap to setup environment
echo "alias cerbero='~/git/cerbero/cerbero-uninstalled'" >> ~/.bashrc
Before using cerbero for the first time, you will need to run the bootstrap
command. This command installs the missing parts of the build system using the
packages manager when available, and also downloads the necessary toolchains
when building for Windows/MinGW or Android.
## Setup environment
Note that this will take a while (a couple hours or even more on Windows).
After Cerbero and the base requirements are in place, you need to setup
the build environment.
```sh
$ ./cerbero-uninstalled bootstrap
```
Cerbero reads the configuration file `$HOME/.cerbero/cerbero.cbc` to
determine the build options. This file is a python code which allows
overriding/defining some options.
On Linux and macOS, this will use `sudo` to make changes to the system.
If the file does not exist, Cerbero will try to determine the distro you
are running and will use default build options such as the default build
directory. The default options should work fine on the supported
distributions.
To fire up the bootstrapping process, go to the directory where you
cloned/unpacked Cerbero and type:
cerbero bootstrap
Enter the superuser/root password when prompted.
The bootstrap process will then install all packages required to build
The bootstrap process will then install or build all packages required to build
GStreamer.
## Build GStreamer
To generate GStreamer binaries, use the following command:
cerbero package gstreamer-1.0
```sh
$ ./cerbero-uninstalled package gstreamer-1.0
```
This should build all required GStreamer components and create packages for
your distribution at the Cerbero source directory.
This will fetch and build all required GStreamer components and create packages
for your distribution, then place them in the Cerbero source directory.
A list of supported packages to build can be retrieved using:
cerbero list-packages
```sh
$ ./cerbero-uninstalled list-packages
```
Packages are composed of 0 (in case of a meta package) or more
components that can be built separately if desired. The components are
defined as individual recipes and can be listed with:
cerbero list
```sh
$ ./cerbero-uninstalled list
```
To build an individual recipe and its dependencies, do the following:
cerbero build <recipe_name>
```sh
$ ./cerbero-uninstalled build <recipe_name>
```
Or to build or force a rebuild of a recipe without building its
dependencies use:
cerbero buildone <recipe_name>
```sh
$ ./cerbero-uninstalled buildone <recipe_name>
```
To wipe everything and start from scratch:
cerbero wipe
```sh
$ ./cerbero-uninstalled wipe
```
Once built, the output of the recipes will be installed at the prefix
defined in the Cerbero configuration file `$HOME/.cerbero/cerbero.cbc`
or at `$HOME/cerbero/dist` if no prefix is defined.
Once built, the binaries built by all the recipes will be installed inside
a auto-detected prefix inside the `build` directory in the Cerbero source tree.
### Build a single project with GStreamer
## Cross Compilation
Rebuilding the whole GStreamer is relatively fast on Linux and OS X, but it
can be very slow on Windows, so if you only need to rebuild a single
project (eg: gst-plugins-good to patch qtdemux) there is a much faster
way of doing it. You will need to follow the steps detailed in this
page, but skipping the step "**Build GStreamer**", and installing the
GStreamer's development files as explained in [Installing GStreamer].
If you're using Cerbero to cross-compile to iOS, Android, Cross-MinGW, or UWP,
you must select the appropriate config file and pass it to all steps:
bootstrap, build, package, etc.
By default, Cerbero uses as prefix a folder in the user directory with
the following schema \~/cerbero/dist/$platform\_$arch, but for GStreamer
we must change this prefix to use its installation directory. This can
be done with a custom configuration file named *custom.cbc*:
For example if you're on Linux and you want to build for Android Universal, you
must run:
# For Windows x86
prefix='/c/gstreamer/1.0/x86/'
```sh
# Bootstrap for Android Universal on Linux
$ ./cerbero-uninstalled -c config/cross-android-universal.cbc bootstrap
# For Windows x86_64
#prefix='/c/gstreamer/1.0/x86_64'
# Build everything and package for Android Universal
$ ./cerbero-uninstalled -c config/cross-android-universal.cbc package gstreamer-1.0
```
# For Linux
#prefix='/opt/gstreamer'
Here's a list of config files for each target machine:
# For OS X
#prefix='/Library/Frameworks/GStreamer.framework/Versions/1.0'
#### Linux Targets
The prefix path might not be writable by your current user. Make sure
you fix it before, for instance with:
Target | Config file
:-----------------|:-----------
MinGW 32-bit | `cross-win32.cbc`
MinGW 64-bit | `cross-win64.cbc`
Android Universal | `cross-android-universal.cbc`
Android ARM64 | `cross-android-arm64.cbc`
Android ARMv7 | `cross-android-armv7.cbc`
Android x86 | `cross-android-x86.cbc`
Android x86_64 | `cross-android-x86-64.cbc`
$ sudo chown -R <username> /Library/Frameworks/GStreamer.framework/
#### macOS Targets
Cerbero has a shell command that starts a new shell with all the
environment set up to target GStreamer. You can start a new shell using
the installation prefix defined in *custom.cbc *with the following
command:
Target | Config file
:----------------------|:-----------
macOS System Framework | `osx-x86-64.cbc`
iOS Universal | `cross-ios-universal.cbc`
iOS ARM64 | `cross-ios-arm64.cbc`
iOS ARMv7 | `cross-ios-armv7.cbc`
iOS x86 | `cross-ios-x86.cbc`
iOS x86_64 | `cross-ios-x86-64.cbc`
$ cerbero -c custom.cbc shell
#### Windows Targets
Once you are in Cerbero's shell you can compile new projects targeting
GStreamer using the regular build process:
On Windows, config files are used to select the architecture and variants are
used to select the toolchain (MinGW, MSVC, UWP):
$ git clone https://gitlab.freedesktop.org/gstreamer/gst-plugins-good; cd gst-plugins-good
$ sh autogen.sh --disable-gtk-doc --prefix=<prefix>
$ make -C gst/isomp4
Target | Config file | Variant
:---------------|:--------------------------|:-------
MinGW x86 | `win32.cbc` |
MinGW x86_64 | `win64.cbc` |
MSVC x86 | `win32.cbc` | visualstudio
MSVC x86_64 | `win64.cbc` | visualstudio
UWP x86 | `win32.cbc` | uwp
UWP x86_64 | `win64.cbc` | uwp
UWP ARM64 | `cross-win-arm64.cbc` | uwp
UWP Universal | `cross-uwp-universal.cbc | (implicitly uwp)
### Cross-compilation of GStreamer
Example usage:
Cerbero can be used to cross-compile GStreamer to other platforms like
Android or Windows. You only need to use a configuration file that sets
the target platform, but we also provide a set of pre-defined
configuration files for the supported platforms (you will find them in
the `config` folder with the `.cbc` extension.
```sh
# Target MinGW 32-bit
$ ./cerbero-uninstalled -c config/win32.cbc package gstreamer-1.0
#### Android
# Target MSVC 64-bit
$ ./cerbero-uninstalled -c config/win64.cbc -v visualstudio package gstreamer-1.0
You can cross-compile GStreamer for Android from a Linux host using the
configuration file `config/cross-android.cbc`. Replace all the previous
commands with:
# Target UWP, x86_64
$ ./cerbero-uninstalled -c config/win64.cbc -v uwp package gstreamer-1.0
cerbero -c config/cross-android.cbc <command>
# Target UWP, Cross ARM64
$ ./cerbero-uninstalled -c config/cross-win-arm64.cbc -v uwp package gstreamer-1.0
#### Windows
# Target UWP, All Supported Arches
$ ./cerbero-uninstalled -c config/cross-uwp-universal.cbc package gstreamer-1.0
```
GStreamer can also be cross-compiled to Windows from Linux, but you should
only use it for testing purpose. The DirectShow plugins cannot be
cross-compiled yet and WiX can't be used with Wine yet, so packages can
only be created from Windows.
## Enabling Optional Features with Variants
Replace all the above commands for Windows 32bits with:
Cerbero controls optional and platform-specific features with `variants`. You
can see a full list of available variants by running:
cerbero -c config/cross-win32.cbc <command>
```sh
$ ./cerbero-uninstalled --list-variants
```
Or with using the following for Windows 64bits:
Some variants are enabled by default while others are not. You can enable
a particular variant by doing one of the following:
cerbero -c config/cross-win64.cbc <command>
* Either invoke `cerbero-uninstalled` with the `-v` argument, for example:
#### iOS
```sh
$ ./cerbero-uninstalled -v variantname [-c ...] package gstreamer-1.0
```
To cross compile for iOS from OS X, use the configuration file
`config/cross-ios-universal.cbc`. Replace all previous commands with:
* Or, edit `~/.cerbero/cerbero.cbc` and add `variants = ['variantname']` at the
bottom. Create the file if it doesn't exist.
cerbero -c config/cross-ios-universal.cbc <command>
Multiple variants can either be separated by a comma or with multiple `-v`
arguments, for example the following are equivalent:
[Warning]: images/icons/emoticons/warning.svg
[Python 3.5+]: https://www.python.org/downloads/
[Git]: https://github.com/git-for-windows/git/releases/latest
[Msys/MinGW]: https://sourceforge.net/projects/mingw/files/Installer/mingw-get-inst/
[CMake]: http://www.cmake.org/cmake/resources/software.htm
[Yasm]: http://yasm.tortall.net/Download.html
[WiX 3.11.1]: https://github.com/wixtoolset/wix3/releases/tag/wix3111rtm
[XCode]: https://developer.apple.com/devcenter/ios/index.action#downloads
[Installing GStreamer]: installing/index.md
```sh
$ ./cerbero-uninstalled -v variantname1,variantname2 [-c ...] package gstreamer-1.0
$ ./cerbero-uninstalled -v variantname1 -v variantname2 [-c ...] package gstreamer-1.0
```
To explicitly disable a variant, use `novariantname` instead.
In the case of multiple enabling/disable of the same variant, then the last
condition on the command line will take effect. e.g. if novariantname is last
then variantname is disabled.
### Enabling Qt5 Support
Starting with version 1.15.2, Cerbero has built-in support for building the Qt5
QML GStreamer plugin. You can toggle that on by
[enabling the `qt5` variant](#enabling-optional-features-with-variants).
You must also tell Cerbero where your Qt5 installation prefix is. You can do it
by setting the `QMAKE` environment variable to point to the `qmake` that you
want to use, f.ex. `/path/to/Qt5.12.0/5.12.0/ios/bin/qmake`
When building for Android Universal with Qt < 5.14, instead of `QMAKE`, you
**must** set the `QT5_PREFIX` environment variable pointed to the directory
inside your prefix which contains all the android targets, f.ex.
`/path/to/Qt5.12.0/5.12.0`.
Next, run `package`:
```sh
$ export QMAKE='/path/to/Qt5.12.0/5.12.0/<target>/bin/qmake'
$ ./cerbero-uninstalled -v qt5 [-c ...] package gstreamer-1.0
```
This will try to build the Qt5 QML plugin and error out if Qt5 could not be
found or if the plugin could not be built. The plugin will be automatically
added to the package outputted.
**NOTE:** The package outputted will not contain a copy of the Qt5 libraries in
it. You must link to them while building your app yourself.
## Enabling Hardware Codec Support
Starting with version 1.15.2, Cerbero has built-in support for building and
packaging hardware codecs for Intel and Nvidia. If the appropriate variant is
enabled, the plugin will either be built or Cerbero will error out if that's
not possible.
### Intel Hardware Codecs
For Intel, the [variant to enable](#enabling-optional-features-with-variants)
is `intelmsdk` which will build the `msdk` plugin.
You must set the `INTELMEDIASDKROOT` env var to point to your [Intel Media
SDK](https://software.intel.com/en-us/media-sdk) prefix, or you must have the
SDK's pkgconfig prefix in `PKG_CONFIG_PATH`
On Windows, `INTELMEDIASDKROOT` automatically set by the installer. On Linux,
if you need to set this, you must set it to point to the directory that
contains the mediasdk `include` and `lib64` dirs.
For VA-API, the [variant to enable](#enabling-optional-features-with-variants)
is `vaapi` which will build the gstreamer-vaapi plugins with all
options enabled if possible.
### Nvidia Hardware Codecs
Since 1.17.1, the `nvcodec` plugin does not need access to the Nvidia Video SDK
or the CUDA SDK. It now loads everything at runtime. Hence, it is now enabled
by default on all platforms.
## Enabling Visual Studio Support
Starting with version 1.15.2, Cerbero supports building all GStreamer recipes,
all mandatory dependencies (such as glib, libffi, zlib, etc), and some external
dependencies with Visual Studio. You must explicitly opt-in to this by [enabling
the `visualstudio` variant](#enabling-optional-features-with-variants):
```sh
$ python ./cerbero-uninstalled -v visualstudio package gstreamer-1.0
```
If you already have a Cerbero build, it is highly recommended to run the `wipe`
command before switching to building with Visual Studio.
[Some plugins that require external dependencies will be automatically
disabled](https://gitlab.freedesktop.org/gstreamer/cerbero/issues/121) when
running in this mode.
Currently, most recipes that use Meson (`btype = BuildType.MESON`) and those
that have the `can_msvc` recipe property set to `True` are built with Visual
Studio.
## Installing Minimum Requirements on Windows
**IMPORTANT:** Using cerbero on Windows with the [GCC/MinGW
toolchain](https://gitlab.freedesktop.org/gstreamer/cerbero/-/blob/master/docs/toolchains.md#windows)
requires a 64-bit operating system. The toolchain is only available for 64-bit
and it can produce 32-bit or 64-bit binaries.
These steps are necessary for using Cerbero on Windows.
#### Install Python 3.6 or newer (either 32-bit or 64-bit)
Download the [Windows executable installer](https://www.python.org/downloads/)
and run it.
* On the first page of the installer, select the following:
![Enable Add Python to PATH, then click Customize Installation](images/cerbero/py-installer-page1.png)
* On the second page, the defaults are fine
* Third page, you must select the following options:
![Enable Install for all users, associate files with Python, add Python to environment variables, and customize the install location to not have any spaces in it](images/cerbero/py-installer-page3.png)
* Enabled or Install [.NET 3.5.1 Framework](https://docs.microsoft.com/en-us/dotnet/framework/install/dotnet-35-windows-10)
* On Windows 10, remove the Windows Store path entry from the PATH variable in the system settings. Otherwise, Cerbero will try to use the dummy Windows Store version of Python
#### Install Git for Windows
Download the [Git for Windows installer](https://gitforwindows.org/) and run it.
* First page is the license
* Next page is `Select Components`, the defaults are fine, enable whatever else you prefer
* Next `Choosing the default editor used by Git`, select whatever you prefer
* Next `Adjusting your PATH environment`, you *must* select as shown in the screenshot
![Select "Git from the command line and also from 3rd-party software"](images/cerbero/git-installer-PATH.png)
* Next `Choosing HTTPS transport backend`, default is fine
* Next `Configuring the line ending conversions`, you *must* select as shown in the screenshot
![Select "Git from the command line and also from 3rd-party software"](images/cerbero/git-installer-line-endings.png)
* Next `Configuring the terminal emulator`, default is fine
* Next `Configuring extra options`, defaults are fine
Git will be installed at `C:\Program Files\Git`.
#### Install MSYS/MinGW
Download the [`mingw-get-setup` executable installer](http://sourceforge.net/projects/mingw/files/Installer/mingw-get-setup.exe/download) and run it.
* First page, keep all the options as-is
* Second page will download the latest package catalogue and base packages
* Once done, the MinGW Installation Manager will open, select the following
packages under Basic Setup:
![Under Basic Setup, select mingw-developer-toolkit, mingw32-base, and msys-base](images/cerbero/msys-install-packages.png)
Then, click on the `Installation` menu and select `Apply Changes`. MSYS will be
installed at `C:\MinGW`.
**IMPORTANT:** After installation, you must create a shortcut on the desktop to
`C:\MinGW\msys\1.0\msys.bat` which will run the MinGW shell. **You must run
Cerbero from inside that**.
**NOTE**: Cerbero does not use the MinGW compiler toolchain shipped with MSYS.
We download our own custom [GCC toolchain](https://gitlab.freedesktop.org/gstreamer/cerbero/-/blob/master/docs/toolchains.md#gcc-mingw)
during [bootstrap](#bootstrap-to-setup-environment).
**NOTE**: MSYS is not the same as [MSYS2](https://www.msys2.org/), and the
GStreamer project does not support running Cerbero inside the MSYS2
environment. Things may work or they may break, and you get to keep the pieces.
#### Install Visual Studio 2015 or newer
This is needed for correctly generating import libraries for recipes built with
MinGW. Both the Community build and the Professional build are supported.
You must install the latest Windows 10 SDK when installing Visual Studio as
shown below. You do not need any older Windows SDKs.
![Select the 'Desktop development with C++' workload](images/cerbero/vs2017-installer-workloads.png)
If you want to build for UWP (aka Universal Windows Platform), you have to use
VS 2017 or newer, and you must *also* select the Universal Windows Platform
workload:
![Select both 'Desktop development with C++' and 'Universal Windows Platform development' workloads](images/cerbero/vs-installer-uwp-workload.png)
You can find all versions of Visual Studio at:
https://visualstudio.microsoft.com/vs/older-downloads/
#### Install other tools
* CMake: http://www.cmake.org/cmake/resources/software.html
* WiX 3.11.1 installer: https://github.com/wixtoolset/wix3/releases/tag/wix3111rtm
#### Important Windows-specific Notes
You should add the cerbero git directory to the list of excluded folders in your
anti-virus, or you will get random build failures when Autotools does file
operations such as renames and deletions. It will also slow your build by
about 3-4x.
Cerbero must be run in the MingGW shell, which is accessible from the main menu
or desktop. If it is not, create a shortcut on the desktop to `C:\MinGW\msys\1.0\msys.bat`
The path to your `$HOME` must not contain spaces. If your Windows username
contains spaces, you can create a new directory in `/home` and execute:
If you are using Windows 10, it is also highly recommended to enable "Developer
Mode" in Windows Settings as shown below.
![Enable Developer Mode in Windows Settings](images/cerbero/windows-settings-developer-mode.png)
```cmd
$ echo 'export HOME=/home/newdir' > ~/.profile
```
Then restart your shell and type `cd` to go to the new home directory.
Note that inside the shell, `/` is mapped to `C:\Mingw\msys\1.0\`