Commit graph

50 commits

Author SHA1 Message Date
Nirbheek Chauhan
e2a8aac2b0 ci/windows: Remove cargo-vendor sources from the cerbero source cache
Copying the source cache to a running Cerbero CI job on Windows causes
strange / corrupted errors that talk about hard links like:

++ cp -a C:/cerbero/cerbero-sources .
cp: will not create hard link `./cerbero-sources/gstreamer-1.0/subprojects/gst-plugins-good/sys/osxaudio' to directory `./cerbero-sources/cargo-c-0.9.12/cargo-vendor/adler/benches'
[...]

So, delete it. The vendored sources are cached by cargo in its own
cache inside CARGO_HOME, which is build/rust/cargo/ and is preserved
in the image. We won't need to redownload any of this.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3805>
2023-01-26 16:03:58 +00:00
Mathieu Duponchelle
f33f728b82 ci/README: basic instructions for running locally
These should be extended upon by someone more knowledgeable with the
topic, but that's the basic information I would want to find there
in order to start debugging CI issues locally.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3681>
2023-01-09 22:26:52 +00:00
Nirbheek Chauhan
5e3967fbff ci: pin hotdoc to 0.13.7 for now, 0.14 has a regression
```
ERROR: [links]: (mandatory-link-not-found): Mandatory link Link GstGLSinkBin -> None (GstGLSinkBin) could not be resolved
ERROR: [links]: (mandatory-link-not-found): Mandatory link Link GstRTPMux -> None (GstRTPMux) could not be resolved
ERROR: [links]: (mandatory-link-not-found): Mandatory link Link GstSRTSink -> None (GstSRTSink) could not be resolved
ERROR: [links]: (mandatory-link-not-found): Mandatory link Link GstSRTSrc -> None (GstSRTSrc) could not be resolved
```

Same change was already made in the 1.20 branch:

https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/1582#note_1669723

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3527>
2022-12-10 15:13:09 +00:00
Xavier Claessens
4c3bb001f5 Install meson from git when MESON_COMMIT variable is set
When starting manually a CI pipeline with MESON_COMMIT variable defined,
install the specified version of Meson. This can be used to help the
Meson project smoke testing RC releases or investigate eventual
regressions.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3036>
2022-11-24 16:17:08 +00:00
Nirbheek Chauhan
9fc753a4eb ci: Fix traceback when user doesn't have a cerbero fork
```
Traceback (most recent call last):
  File "ci/gitlab/trigger_cerbero_pipeline.py", line 59, in <module>
    if cerbero is None:
NameError: name 'cerbero' is not defined
```

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3063>
2022-10-26 14:49:14 +05:30
Nirbheek Chauhan
5468d4bf5b ci: Fetch all user cerbero branches when matching branch names
Fixes CI on coordinated merges when the user has more than 20 branches
in their fork, which will happen very easily since new forks will
always have all the branches of the original remote.

```
ci/gitlab/trigger_cerbero_pipeline.py:48: UserWarning: Calling a `list()` method without specifying `get_all=True` or `iterator=True` will return a maximum of 20 items. Your query returned 20 of 37 items. See https://python-gitlab.readthedocs.io/en/v3.9.0/api-usage.html#pagination for more details. If this was done intentionally, then this warning can be supressed by adding the argument `get_all=False` to the `list()` call. (python-gitlab: /usr/local/lib/python3.7/site-packages/gitlab/client.py:979)
  if os.environ["CI_COMMIT_REF_NAME"] in [b.name for b in cerbero.branches.list()]:
```
2022-10-25 20:27:23 +05:30
Nirbheek Chauhan
46b3f48bdb ci: Bump meson to 0.62.2 in Fedora docker images
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/3061>
2022-09-22 09:56:05 +05:30
Nirbheek Chauhan
895290f411 ci: Bootstrap with visualstudio variant enabled
visualstudio variant now pulls in more build-tools deps on top of
mingw since it enables the rust variant by default. It's also the
variant that's tested on Cerbero CI.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2507>
2022-09-17 22:41:40 +00:00
Thibault Saunier
339e5916c6 Build documentation for rust plugins
- Update the docker image we use, starting using the standard one adding
  `gtk4-doc` as required by rust plugins
- Update the plugins_doc_caches as required, some more plugins are built
  with the new image
- Install ninja from pip as the version from F31 is too old
- Avoid buildings all GSreamer plugins when building the doc as it takes
  time and resources for no good reason
- Stop linking to `GInstanceInitFunc` as it is not present in latest GLib
  documentation, leading to warnings in hotdoc.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2954>
2022-09-15 20:11:47 +00:00
Jordan Petridis
21ac55bac2 docker/windows: Pass a couple installargs to python3
From these, only PrependPath isn't enabled by default,
but its nice to explicitly enable the ones we need anyway.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2933>
2022-08-26 15:56:47 +00:00
Jordan Petridis
d3dfdeaf47 windows/Dockerfile: replace ADD arguments with Invoke-WebRequest
Have the windows susbsystem handle the networking layer and avoid
layer invalidation until the strings/urls are changed.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2812>
2022-07-29 19:56:11 +00:00
Jordan Petridis
f3a324c1a5 ci: Update the windows image
Mainly to get newer meson

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2812>
2022-07-29 19:56:11 +00:00
Nirbheek Chauhan
885d4b84a5 ci: Fix project URL when triggering cerbero pipelines
When merge request pipelines are triggered on the gstreamer namespace,
CI_PROJECT_URL will be gitlab.[...]/gstreamer/gstreamer but we need to
use gitlab.[...]/$USER/gstreamer because that's where the source
branch is located.

This exhibits as cerbero pipelines failing because it can't find the
specified branch:

https://gitlab.freedesktop.org/gstreamer/cerbero/-/pipelines/639379

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2761>
2022-07-16 06:16:17 +00:00
Xavier Claessens
ea990ff817 ci: Make sure to always update subprojects
The command was not run in Windows jobs, better keep subprojects
handling in a single script.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2509>
2022-05-27 14:33:53 +00:00
Jordan Petridis
12442748aa ci: use the correct vs toolcahin version for cerbero
This seems to be a leftover from when we upgraded from vs15
to vs16.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2395>
2022-05-18 09:14:08 +00:00
Jordan Petridis
6adb7eb22c ci: use LONGPATH for the windows container
Sometimes the CI might create paths that are bigger than the 260
character limit.

ex:
```
error: unable to create file subprojects/gst-integration-testsuites/testsuites/validate/h264/parse.trickmode_predicted.seek_trickmode_predicted/flow-expectations/log-parse-src-expected: Filename too long
```

https://docs.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2395>
2022-05-18 09:14:08 +00:00
Jordan Petridis
5cd3f68842 ci: remove windows-rust jobs
These are moved now into gstreamer-rs repository.

https://gitlab.freedesktop.org/gstreamer/gstreamer-rs/-/merge_requests/701

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2395>
2022-05-18 09:14:08 +00:00
Jordan Petridis
02c277258b ci: Install Media Foundations on the windows container
In order to do this, we need to switch to the Server container
image, instead of the ServerCore we were using till now.

This image also only has Server2022 base variants.

This is a continuation of:

https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1783

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2395>
2022-05-18 09:14:08 +00:00
Jordan Petridis
fff82f5650 ci: remove unused files
These are leftovers of the gst-ci repo, before the mono repo
merge and can be removed now.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2403>
2022-05-14 11:13:10 +00:00
Jordan Petridis
1d627e29cd ci: fix the windows-rust image builds
There was a rule gated on the project name which wasn't removed
once we moved to the monorepo and this job was silently broken
since.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2302>
2022-05-05 21:33:43 +00:00
Thibault Saunier
956270f3ab ci: Trigger cerbero on user project if a branch with the same name exists there
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2264>
2022-04-21 12:50:43 -04:00
Matthew Waters
bd5d24822a build/fuzzing: integrate fuzz targets into the build system
Currently disabled but may be enabled later.

Updates the existing fuzzing to use shared libraries as that's easier
for meson to deal with if there is a mix of static and shared libraries
on the system.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2123>
2022-04-07 08:17:35 +10:00
Andoni Morales Alastruey
f89dff303a gstreamer-sharp: ensure a consitent code format
Use EditorConfig and dotnet-format to ensure a consistent
code style.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/983>
2022-04-05 08:28:50 +00:00
Matthew Waters
f0e5c8e0f2 ci/fuzzing: update glib version to 2.72.0
Fixes an issue where on configure, glib's internal gnulib was tripping
up the undefined behaviour sanitizer with a divide by 0.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1976>
2022-03-18 18:35:05 +11:00
Matthew Waters
03ff52ee69 ci/fuzzling: fix oss-fuzz build
gst-plugins-base doesn't have a cairo option anymore.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1975>
2022-03-18 13:16:42 +11:00
Xavier Claessens
69863131bd Bump Meson requirement to >=0.60.0
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1934>
2022-03-14 08:56:54 -04:00
Tim-Philipp Müller
e3ae87a648 ci: remove jenkins files which are no longer used
Fixes #1031

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1763>
2022-02-22 10:49:49 +00:00
Nirbheek Chauhan
a10bcd1671 ci: Trigger all jobs on gitlab CI script changes
Also fix typo in trigger variable that was causing triggered cerbero
pipelines to run deps builds and upload cache.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1641>
2022-02-05 09:21:48 +05:30
Nirbheek Chauhan
66933d1f05 ci: Add a variable to identify triggered cerbero pipelines
This commit is required for proper functioning of the following
cerbero merge request: https://gitlab.freedesktop.org/gstreamer/cerbero/-/merge_requests/800

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1638>
2022-02-04 20:19:57 +05:30
Nirbheek Chauhan
673b742751 ci: Add a Windows native cross-arm64 job with a new image
Needed an update to the windows Dockerfile to:

1. Install the 'UniversalBuildTools' workload for Cerbero
2. Install ARM and ARM64 workloads for cross-uwp-universal in Cerbero
3. Install VS 2019 since we need that for ARM64 NEON support in Opus

We can't test UWP in gstreamer.git because glib needs custom patches
for that. It will be tested in Cerbero.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1570>
2022-01-28 02:01:39 +05:30
Nirbheek Chauhan
72bd408c39 ci: Remove windows-sdk-8.1 from docker image
We don't need this, we use the Windows 10 SDK provided by Visual
Studio Build Tools.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1570>
2022-01-28 02:01:39 +05:30
Jordan Petridis
b0370cdf00 ci: replace license notice
This file was copied from microsoft documentation [1]
and its header was refering to the license on that repo.

This change was lost during gstreamer/gstreamer!1000

[1] 9ec230dfd4/LICENSE-CODE

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1487>
2022-01-03 12:43:15 +02:00
Nicolas Dufresne
06f1dfc724 ci: fedora-docker: Cache media files
As of now all integration tests endup fetching 600MB of media every run. This
will increase the docker side, but this is to the traceof that the docker
image will quickly be cached on all runners.

Fixes #841

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1413>
2021-12-06 12:18:25 +00:00
Matthew Waters
fdb16cddb1 ci/fuzzing: update for mono repo
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1253>
2021-10-26 14:09:24 +00:00
Mathieu Duponchelle
8d7dde2587 ci: fix plugins cache diffs directory
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/867>
2021-10-18 22:34:11 +00:00
Tim-Philipp Müller
d4bdbeaa0d meson: add 'gpl' option and only build plugins with (A)GPL deps if explicitly enabled
Require explicit opt-in to build plugins with (A)GPL dependencies.

Keep ugly/bad options on 'auto' for now so cerbero doesn't fail.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1084>
2021-10-18 18:03:19 +01:00
Andoni Morales Alastruey
4be5c7ce0a Fix GStreamer msys2 builds in the CI
Install ninja and the toolchain in the docker image.
Use the UCRT64 environment for building

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1000>
2021-10-14 10:13:51 +00:00
Andoni Morales Alastruey
a1e332bf7e ci: prepare the Windows image for MSYS2 cerbero builds
* Install MSYS2 and VS with choco
 * Install Windows SDK 8.1 requied for libvpx
 * Improve build time of the windows image using as much RUN layers as
   possible

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1000>
2021-10-14 10:13:51 +00:00
Xavier Claessens
d4296b7d3d trigger_cerbero_pipeline.py: Do not hardcode gitlab instance
When GStreamer is forked into a private GitLab instance we should
trigger cerbero into that private instance too. Otherwise the token
won't be accepted.

Also do not hardcode the cerbero project ID because it is instance
specific.

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1056>
2021-10-06 09:04:30 +00:00
Xavier Claessens
275c540984 ci: Update fedora image for cerbero trigger
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/931>
2021-10-05 23:54:13 +03:00
Xavier Claessens
bc40073836 ci: Fix subprojects caching in Windows image
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/931>
2021-10-05 23:44:03 +03:00
Jordan Petridis
66066ffa03 ci/docker/fedora: update rustc and meson versions
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/931>
2021-10-05 23:44:03 +03:00
Jordan Petridis
67c8e634e0 ci: port to fd.o citemplates
https://gitlab.freedesktop.org/freedesktop/ci-templates

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/931>
2021-10-05 23:44:03 +03:00
Xavier Claessens
c4e14ef426 handle-subprojects-cache.py: Fix shebang 2021-10-04 20:00:40 -04:00
Jordan Petridis
e7ef042400 ci: Rebuild windows image
Get updates and newer ca-certificates in.

Also update the way we install chocolatey

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/1002>
2021-10-01 14:53:53 +03:00
Thibault Saunier
9270f072cc ci:doc: Rewrite script to import doc on fdo ensuring the right job is used
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/999>
2021-09-30 20:05:00 +00:00
Thibault Saunier
a619f2cedf docs: Always build documentation on gstreamer/gstreamer main branch
So artifacts can the be imported on gst.fd.o.

And update the importation script accordingly

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/952>
2021-09-29 16:40:00 +00:00
Thibault Saunier
37d7e9a22d ci: Wait for cerbero pipeline to finish
So we are sure the pipeline is marked as failed if the cerbero sub pipeline fails

See https://gitlab.com/gitlab-org/gitlab/-/issues/341737 for details

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/946>
2021-09-28 10:58:25 -03:00
Thibault Saunier
c6b9c81fdd ci: Remove now useless .gitlab-ci.yml files
Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/891>
2021-09-24 16:21:18 -03:00
Thibault Saunier
091946a478 ci: Port CI to the new monorepo
Main differences with previous setup are:
- No manifest creation
- gst-indent is executed only when the bot is assigned (instead of the manifest task)
- Cerbero jobs are triggered in the cerbero repo
- Remove cerbero and android related files as they now are in cerbero
  itself.
- Update `container.ps1` to the new file layout

Part-of: <https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/891>
2021-09-24 16:21:18 -03:00