The idea behind local jobs was that you would have jobs that
share the same setup as the citemplate ones but would always
be pointing to your forked registry with your custom build of
the image, since the citemplate hardcoded the image it was
running against.
With the changes introduced in the previous commit
we now mirror the setup from freedesktop/ci-templates and the idea
is that jobs always run from the registry in your fork. If the
image sha/id matches the one from the upstream registry, its copied
over else a new one is build, pushed and tested.
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/308>
This is the inital step towards migrating our docker images setup
to something closer and eventually freedesktop/citemplates [1]
The idea is that jobs always run from the registry in your fork. If the
image sha/id matches the one from the upstream registry, its copied
over else a new one is build, pushed and tested.
Only change the fedora job for now while testing.
[1]: https://gitlab.freedesktop.org/freedesktop/ci-templates
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/308>
Previously we were optimizing for cpu time, so we where building
gst-build once and then exporting that to be used by the test jobs.
However this meant that we where uploading 200mb (previously 600mb)
zipped of artifacts and then re-downloading them for each test job.
This caused big costs in terms of cloud egress since the runners
aren't hosted on the same cloud as the storage/artifacts instance.
Instead we are going to be rebuilding gst-build for each test
job from now, it also doesn't take more time than the network
i/o would of downloading the artifacts, so the impact of rebuilding
shouldn't be noticebly.
We are also using pinned git refs the modules we rebuild from
the manifest, so the binaries should be reproducible for the most
part (minus things like .pyc files).
Close#68
Part-of: <https://gitlab.freedesktop.org/gstreamer/gst-ci/-/merge_requests/280>
docker builds can be big, and other shared runner have smaller
storage space which we should avoid filling if can be avoided.
The gst tagged runners are a better fit for such builds since
their disk storage is about 1-2 terabytes.
This avoids the need of using privilledged namespaces and dind
as buildah are able to build images unprivilledged.
One thing to note is that buildah inside docker is not a
supported configuration and not tested in upstream podman,
but the possible fallout is still easier to deal with than
dind and requiring privileged runners.
Rules is a new feature that replaces only/except and allow for finer grain
control on the workflow. With rules, we gain finer grain to pipeline and merge
request pipelines.
Declare an docker build-arg [1] and use it
whenever cloning one of our repositories. If the buildarg
is not specified, the variable defaults back to 'master'
and thus the current behavior doesn't change.
From the .gitlab-ci.yml file, when building pass
the GST_UPSTREAM_BRANCH that's defined from the ci_template
as the buildarg so we will be building the corresponding branches
for the docker images.
Close#33
[1] https://docs.docker.com/engine/reference/builder/#arg
Previously we tried using the git commit ref as a UID for the
images. This does not work though cause multiple jobs that rebuild
the image can be triggered and override the same image tag. Instead
use the CI_JOB_ID to provide better semantics wrt naming conflicts.
Additionally add the date as part of the tag to better indicate
the age of the image. Gitlab WEBGUI doens't indicate the age
in a good way nor makes it easy to short the images.
There is no point in doing a different docker image for each build we
are going to do inside an Ubuntu distro. We can later use the same image
for native ubuntu builds, or other cross builds (e.g. android API 21,
etc).
It was installed in '/' which doesn't feel right. Installing it in /root
is also not correct because we want to run the build as user instead of
root in the future and cleanup.sh removes everything in /root. /opt
seems the best place because that's also the default location when
installing Android Studio.
While they are very useful, each time we create a branch,
gitlab tries to build all the images which is very resource
intesinve. Thus make all the local images and everything that
depends upon them a manual job and only trigger them before
merging an MR.
Add a fedora job that runs the 'check' tests with
gst-validate-launcher. Its fairly well abstracted so the same
template can be used to add the rest of the test-suites fairly
soon.
