Previously we were optimizing for cpu time, so we where building
gst-build once and then exporting that to be used by the test jobs.
However this meant that we where uploading 200mb (previously 600mb)
zipped of artifacts and then re-downloading them for each test job.
This caused big costs in terms of cloud egress since the runners
aren't hosted on the same cloud as the storage/artifacts instance.
Instead we are going to be rebuilding gst-build for each test
job from now, it also doesn't take more time than the network
i/o would of downloading the artifacts, so the impact of rebuilding
shouldn't be noticebly.
We are also using pinned git refs the modules we rebuild from
the manifest, so the binaries should be reproducible for the most
part (minus things like .pyc files).
Close#68
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/280>
Previously we where accidently exporting the whole repo of
gst-integration-testsuites which includes 350mb of raw media
files and made the artifacts storage explode through the roof
along with the CI bills fd.o had to pay for uploading and
redownloading the artifacts
To deal with this, we clean all the media files from the builddir
and when needed we copy them over from the cache in the docker image,
and then git fetch the repo.
Close#69
We don't run the libnice testsuite, and when binaries are built
they consume ~45mb of space. This increases the size of the artifacts
we export from the gst-build job for the testsuite and drives up
the storage and bandwith costs when re-downloading the artifacts.
Similary disable the test targets of couple other subprojects as well
We have notice that a lot of CI activity is cause by user pushing to their
branch after having created an MR. To reduce our CI foot-print, the CI will
now only be automatically triggered when a reviewer assign the MR to the merge
bot. It will still be possible to run the CI manually but the result of that
CI won't be used by Marge.
It seems to be timing out with high frequency only on Windows runners.
```
Version: 12.8.0
00:47
Git revision: 1b659122
Git branch: 12-8-stable
GO version: go1.13.7
Built: 2020-02-22T03:03:07+0000
OS/Arch: windows/amd64
Uploading artifacts...
gst-build/build/meson-logs/: found 2 matching files
WARNING: Failed to load system CertPool: crypto/x509: system root pool is not available on Windows
ERROR: Job failed (system failure): aborted: <nil>
```
See: https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/261
This might be related to the same issue described in the previous
commit: Till we can update the container image to the Feb 11 security
update, x86 executables and in general the container image will behave
badly because of:
https://support.microsoft.com/en-us/help/4542617/you-might-encounter-issues-when-using-windows-server-containers-with-t
vs2017 x86 has been failing with a runner system failure while
uploading artifacts / submitting job status:
```
Uploading artifacts...
gst-build/build/meson-logs/: found 2 matching files
WARNING: Failed to load system CertPool: crypto/x509: system root pool is not available on Windows
ERROR: Job failed (system failure): aborted: <nil>
```
https://gitlab.freedesktop.org/slomo/gst-plugins-good/-/jobs/2084184
Disable it for now.
This will reduce the excessive load on the runners which are having issues
with this job in particuliar. We will revisit when we better understand the
runners issues.
Passing regex as variable does not really works, we ended up matching the
regex as a string instead. Replace all REGEX variable with rules: override.
It is longer but more reliable.
Related to !247Fixes#63
Rules is a new feature that replaces only/except and allow for finer grain
control on the workflow. With rules, we gain finer grain to pipeline and merge
request pipelines.
The windows runner has become a bit unstable lately, might be
due to some recent update. It frequently timeouts while waiting
to pick up a job or sometimes it goes missing in the middle of a job.
This is where the WINEPREFIX is now in Cerbero. This used to be
share/wine, but was moved to var/tmp/wine for clarity. It was causing
two problems:
1. The size of these generated files are ~1GB, which were ~500MB after
tar.gz, and they were completely useless since they can just be
regenerated by Wine the next time it's run. Let's not waste egress
bandwidth.
2. Random build failures because wineserver and associated processes
would not always exit before we started tarring up the prefix, then
write to the directory on exit while tar was reading the directory
causing `tar -czf` to fail:
```
$ tar -C ${CERBERO_HOME} -czf $CERBERO_DEPS build-tools build-tools.cache dist/${ARCH} ${ARCH}.cache
tar: build-tools/share/wine: file changed as we read it
Uploading artifacts...
manifest.xml: found 1 matching files
cerbero-build/logs: found 461 matching files
cerbero-build/cerbero-deps.log: found 1 matching files
cerbero-deps.tar.gz: found 1 matching files
Uploading artifacts to coordinator... ok id=1807197 responseStatus=201 Created token=4_qFUP8z
ERROR: Job failed: exit code 1
```
This is slightly weird, cause I am not sure what causes the clone
to be there, since gitlab-runner supposedly always either use a
clean volume or at least runs git clean on the existing ones.
But its there and so we have to deal with failures like so
https://gitlab.freedesktop.org/tpm/gstreamer-sharp/-/jobs/1672137
