Commit graph

15 commits

Author SHA1 Message Date
Nirbheek Chauhan
9c7a2df4a8 cerbero: Always fetch sources with four parallel jobs
The default number of parallel jobs is two, which is too few. We can
easily use four or more. Should speed up image builds and also
downloading of (new) sources that aren't already cached in the image.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/411>
2021-08-17 16:04:42 +05:30
Nirbheek Chauhan
2c0ab5f8a1 gitlab/cerbero: Don't fail when there's no macOS cache
`test -d "${CERBERO_HOME}/dist/${ARCH}"` will cause the build to error
out if the dist dir doesn't exist, which will happen if there's no
cache, for example:
https://gitlab.freedesktop.org/seungha.yang/gst-plugins-bad/-/jobs/7405565

What we actually want to do, is to not perform the rsync if there's no
cache, since it means we want to rebuild from scratch; for instance,
if the current MR uses a custom cerbero branch of the same name which
won't have a prebuilt cache.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/398>
2021-02-18 14:52:32 +05:30
Nirbheek Chauhan
26a990454c gitlab: Allow du to fail in cerbero scripts
It's purely for informative reasons. `du` will fail on the sources dir
if a branch name has unicode in it due to an MSYS/MinGW bug. The long
term fix is to from MSYS/MinGW to MSYS/MinGW-W64 or MSYS2/MinGW-W64.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/395>
2021-02-03 14:05:19 +05:30
Jordan Petridis
34dec61d86
Revert "cerbero: Never use cached cerbero git repo"
This reverts commit c3b5693736.

This broke the build and needs further investigation

https://gitlab.freedesktop.org/gstreamer/gst-ci/-/jobs/5630095
2020-11-16 15:41:03 +02:00
Nirbheek Chauhan
c3b5693736 cerbero: Never use cached cerbero git repo
Sometimes the git repo cached on the docker image gets corrupted on
Windows, in which case there is no way to clean things and resolve
that error. For example: https://gitlab.freedesktop.org/bilboed/gst-plugins-bad/-/jobs/5517037

```
++ cp -a C:/cerbero/.git .
cp: cannot create regular file `./.git/objects/pack/pack-cbdde9dc4dd07bff6761e6715e20d7c0abe62b9e.idx': Permission denied
cp: cannot create regular file `./.git/objects/pack/pack-cbdde9dc4dd07bff6761e6715e20d7c0abe62b9e.pack': Permission denied
```

Always delete the cached git repo and copy a pristine copy from the
image. This will also avoid accumulation of garbage in the git repo.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/376>
2020-11-12 14:04:47 +05:30
Nirbheek Chauhan
42696e1654 cerbero ci: Make before_script wiping more thorough
Will avoid packaging errors due to the old package file still being
around from a previous job, such as in:

https://gitlab.freedesktop.org/nirbheek/cerbero/-/jobs/4804147

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/361>
2020-10-02 22:27:11 +05:30
Nirbheek Chauhan
b8e112ed74 cerbero ci: Wipe in before_script because gitlab doesn't
Noticed in this job, which failed because we rebuilt on top of an
existing builddir:

https://gitlab.freedesktop.org/nirbheek/cerbero/-/jobs/4772728

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/358>
2020-10-01 10:00:49 +05:30
Nirbheek Chauhan
6208a61946 cerbero ci: Add hacks for build tools
Instead of putting a hammer-like symlink to ../../gstreamer/cerbero,
do recipe-specific relocation hacks. Fixes random errors like:

```
++ ln -sf /builds/nirbheek/cerbero ../../gstreamer/cerbero
ln: ../../gstreamer/cerbero/cerbero: cannot overwrite directory
```

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/350>
2020-09-23 12:50:16 +05:30
Nirbheek Chauhan
98e3c7d8b8 cerbero ci: Refactor cerbero_before_script
We were doing a few things suboptimally:

1. Making a symlink from the current directory to
   ../../gstreamer/cerbero so that build-tools kept working, which
   should not be necessary since cerbero relocates all binaries inside
   the cerbero homedir after extracting the cache.

2. Hard-coding the cerbero builddir when building on Windows because
   `pwd` was outputting an MSYS-style path which Python can't read

3. rsync-ing all the contents of $CERBERO_HOST_DIR, which is
   bad because we should be building on top of the artifact cache.
   The only thing we want from $CERBERO_HOST_DIR is cerbero-sources
   (the network cache), and we can setup the toolchain during
   bootstrap with the new --system and --toolchains arguments which
   were split out of --system-only.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/350>
2020-09-22 21:58:59 +05:30
Nirbheek Chauhan
eed018920c cerbero ci: Sprinkle some 'time' commands in the script
Sprinkle some 'time' commands so we can measure how long things take
and we can know where to spend time optimizing first.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/346>
2020-09-19 12:00:15 +05:30
Nirbheek Chauhan
dab27ee564 cerbero ci: Try to fix deps build on the gstreamer namespace
Can't ln twice: it does a cp on MSYS.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/344>
2020-09-17 23:23:02 +05:30
Nirbheek Chauhan
f24e7b721a cerbero ci: Don't expand the ssh private key
Because of set -x we were printing the key into the CI logs:
https://gitlab.freedesktop.org/gstreamer/cerbero/-/jobs/4572836#L797

Don't ever expand the variable in shell.
2020-09-17 23:18:30 +05:30
Jordan Petridis
b857a5ada1 citemplate: add initial cerbero msvc builds
This commit adds the last bits needed to have a working
CI job of cerbero using mingw and msvc17 toolchain.

It adds a new stage to the docker image that mirrors
the setup of the cerbero fedora docker build, and
tweaks the cerbero_setup script run by jobs with
a couple windows specific fixes.

Close #39

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/334>
2020-09-16 20:56:14 +03:00
Jordan Petridis
6ffc323ff5 gitlab/cerbero_setup.sh: refactor some tests to if/else statments
and replace the ccache check

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/329>
2020-09-15 16:42:58 +00:00
Jordan Petridis
7d46f1ac5e ci_template: move the cerbero scripts from yaml to a new file
Its hard to manage ever growing bash scripts in yaml, and
its even harder to run them locally to reproduce the environment.

This is essentially a copy-paste of the scripts, to make review
easier. We can refactor later.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/329>
2020-09-15 16:42:58 +00:00