From 0c0549be6845bfc121221b554732ecaa5785dee7 Mon Sep 17 00:00:00 2001 From: Robert Kaussow Date: Mon, 24 Feb 2025 23:29:19 +0100 Subject: [PATCH] Refactor admin docs --- docs/docs/10-intro/index.md | 2 +- docs/docs/30-administration/00-general.md | 43 ++++++++++++++ .../30-administration/00-getting-started.md | 59 ------------------- .../30-administration/04-image-variants.md | 30 ---------- .../05-deployment-methods/20-kubernetes.md | 50 ---------------- .../05-deployment-methods/30-third-party.md | 12 ---- .../10-docker-compose.md | 36 ++--------- .../05-installation/20-helm-chart.md | 45 ++++++++++++++ .../30-packages.md} | 33 +++++++---- .../_category_.yaml | 2 +- 10 files changed, 118 insertions(+), 194 deletions(-) create mode 100644 docs/docs/30-administration/00-general.md delete mode 100644 docs/docs/30-administration/00-getting-started.md delete mode 100644 docs/docs/30-administration/04-image-variants.md delete mode 100644 docs/docs/30-administration/05-deployment-methods/20-kubernetes.md delete mode 100644 docs/docs/30-administration/05-deployment-methods/30-third-party.md rename docs/docs/30-administration/{05-deployment-methods => 05-installation}/10-docker-compose.md (72%) create mode 100644 docs/docs/30-administration/05-installation/20-helm-chart.md rename docs/docs/30-administration/{05-deployment-methods/40-nixos.md => 05-installation/30-packages.md} (69%) rename docs/docs/30-administration/{05-deployment-methods => 05-installation}/_category_.yaml (54%) diff --git a/docs/docs/10-intro/index.md b/docs/docs/10-intro/index.md index 7d9ced179..29f3a3eb9 100644 --- a/docs/docs/10-intro/index.md +++ b/docs/docs/10-intro/index.md @@ -23,4 +23,4 @@ Then you might want to jump directly into it and [start creating your first pipe ## Want to start from scratch and deploy your own Woodpecker instance? -Woodpecker is [pretty lightweight](../30-administration/00-getting-started.md#hardware-requirements) and will even run on your Raspberry Pi. You can follow the [deployment guide](../30-administration/00-getting-started.md) to set up your own Woodpecker instance. +Woodpecker is [pretty lightweight](../30-administration/00-general.md#hardware-requirements) and will even run on your Raspberry Pi. You can follow the [deployment guide](../30-administration/00-general.md) to set up your own Woodpecker instance. diff --git a/docs/docs/30-administration/00-general.md b/docs/docs/30-administration/00-general.md new file mode 100644 index 000000000..e8a92795c --- /dev/null +++ b/docs/docs/30-administration/00-general.md @@ -0,0 +1,43 @@ +# General + +Woodpecker consists of essential components (`server` and `agent`) and an optional component (`autoscaler). + +The **server** provides the user interface, processes webhook requests to the underlying forge, serves the API and analyzes the pipeline configurations from the YAML files. + +The **agent** executes the [workflows](../20-usage/15-terminology/index.md) via a specific [backend](../20-usage/15-terminology/index.md) (Docker, Kubernetes, local) and connects to the server via GRPC. Multiple agents can coexist so that the job limits, choice of backend and other agent-related settings can be fine-tuned for a single instance. + +The **autoscaler** allows spinning up new VMs on a cloud provider of choice to process pending builds. After the builds finished, the VMs are destroyed again (after a short transition time). + +:::tip +You can add more agents to increase the number of parallel workflows or set the agent's `WOODPECKER_MAX_WORKFLOWS=1` environment variable to increase the number of parallel workflows per agent. +::: + +## Database + +Woodpecker uses a SQLite database by default, which requires no installation or configuration. For larger instances it is recommended to use it with a Postgres or MariaDB instance. For more details take a look at the [database settings](./10-database.md) page. + +## Forge + +What would a CI/CD system be without any code. By connecting Woodpecker to your [forge](../20-usage/15-terminology/index.md), you can start pipelines on events like pushes or pull requests. Woodpecker will also use your forge to authenticate and report back the status of your pipelines. For more details take a look at the [forge settings](./11-forges/11-overview.md) page. + +## Container images + +:::info +No `latest` tag exists to prevent accidental major version upgrades. Either use a SemVer tag or one of the rolling major/minor version tags. Alternatively, the `next` tag can be used for rolling builds from the `main` branch. +::: + +- `vX.Y.Z`: SemVer tags for specific releases, no entrypoint shell (scratch image) + - `vX.Y` + - `vX` +- `vX.Y.Z-alpine`: SemVer tags for specific releases, rootless for Server and CLI (as of v3.0). + - `vX.Y-alpine` + - `vX-alpine` +- `next`: Built from the `main` branch +- `pull_`: Images built from Pull Request branches. + +Images are pushed to DockerHub and Quay. + +- woodpecker-server ([DockerHub](https://hub.docker.com/repository/docker/woodpeckerci/woodpecker-server) or [Quay](https://quay.io/repository/woodpeckerci/woodpecker-server)) +- woodpecker-agent ([DockerHub](https://hub.docker.com/repository/docker/woodpeckerci/woodpecker-agent) or [Quay](https://quay.io/repository/woodpeckerci/woodpecker-agent)) +- woodpecker-cli ([DockerHub](https://hub.docker.com/repository/docker/woodpeckerci/woodpecker-cli) or [Quay](https://quay.io/repository/woodpeckerci/woodpecker-cli)) +- woodpecker-autoscaler ([DockerHub](https://hub.docker.com/repository/docker/woodpeckerci/autoscaler)) diff --git a/docs/docs/30-administration/00-getting-started.md b/docs/docs/30-administration/00-getting-started.md deleted file mode 100644 index e5d573e56..000000000 --- a/docs/docs/30-administration/00-getting-started.md +++ /dev/null @@ -1,59 +0,0 @@ -# Getting started - -A Woodpecker deployment consists of two parts: - -- A server which is the heart of Woodpecker and ships the web interface. -- Next to one server, you can deploy any number of agents which will run the pipelines. - -Each agent is able to process one [workflow](../20-usage/15-terminology/index.md) by default. If you have 4 agents installed and connected to the Woodpecker server, your system will process four workflows (not pipelines) in parallel. - -:::tip -You can add more agents to increase the number of parallel workflows or set the agent's `WOODPECKER_MAX_WORKFLOWS=1` environment variable to increase the number of parallel workflows per agent. -::: - -## Which version of Woodpecker should I use? - -Woodpecker is having two different kinds of releases: **stable** and **next**. - -Find more information about the different versions [here](/versions). - -## Hardware Requirements - -Below are minimal resources requirements for Woodpecker components itself: - -| Component | Memory | CPU | -| --------- | ------ | --- | -| Server | 200 MB | 1 | -| Agent | 32 MB | 1 | - -Note, that those values do not include the operating system or workload (pipelines execution) resource consumption. - -In addition you need at least some kind of database which requires additional resources depending on the selected database system. - -## Installation - -You can install Woodpecker on multiple ways. If you are not sure which one to choose, we recommend using the [docker compose](./05-deployment-methods/10-docker-compose.md) method for the beginning: - -- Using [docker compose](./05-deployment-methods/10-docker-compose.md) with the official [container images](./05-deployment-methods/10-docker-compose.md#docker-images) -- Using [Kubernetes](./05-deployment-methods/20-kubernetes.md) via the Woodpecker Helm chart -- Using binaries, DEBs or RPMs you can download from [latest release](https://github.com/woodpecker-ci/woodpecker/releases/latest) -- Or using a [third-party installation method](./05-deployment-methods/30-third-party.md) - -## Database - -By default Woodpecker uses a SQLite database which requires zero installation or configuration. See the [database settings](./10-database.md) page if you want to use a different database system like MySQL or PostgreSQL. - -## Forge - -What would be a CI/CD system without any code? By connecting Woodpecker to your [forge](../20-usage/15-terminology/index.md) like GitHub or Gitea you can start running pipelines on events like pushes or pull requests. Woodpecker will also use your forge for authentication and to report back the status of your pipelines. See the [forge settings](./11-forges/11-overview.md) to connect it to Woodpecker. - -## Configuration - -Check the [server configuration](./10-server-config.md) and [agent configuration](./15-agent-config.md) pages to see if you need to adjust any additional parts and after that you should be ready to start with [your first pipeline](../20-usage/10-intro.md). - -## Agent - -The agent is the worker which executes the [workflows](../20-usage/15-terminology/index.md). -Woodpecker agents can execute work using a [backend](../20-usage/15-terminology/index.md) like [docker](./22-backends/10-docker.md) or [kubernetes](./22-backends/40-kubernetes.md). -By default if you choose to deploy an agent using [docker compose](./05-deployment-methods/10-docker-compose.md) the agent simply use docker for the backend as well. -So nothing to worry about here. If you still prefer to adjust the agent to your needs, check the [agent configuration](./15-agent-config.md) page. diff --git a/docs/docs/30-administration/04-image-variants.md b/docs/docs/30-administration/04-image-variants.md deleted file mode 100644 index 2d946a9df..000000000 --- a/docs/docs/30-administration/04-image-variants.md +++ /dev/null @@ -1,30 +0,0 @@ -# Image variants - -:::info -The `latest` tag has been deprecated as of v3.0 and will be completely removed in the future. -This was done to prevent accidental major version upgrades. -::: - -- `vX.Y.Z`: SemVer tags for specific releases, no entrypoint shell (scratch image) - - `vX.Y` - - `vX` -- `vX.Y.Z-alpine`: SemVer tags for specific releases, based on Alpine, rootless for Server and CLI (as of v3.0). - - `vX.Y-alpine` - - `vX-alpine` -- `next`: Built from the `main` branch -- `pull_`: Images built from Pull Request branches. - -## Image registries - -Images are pushed to DockerHub and Quay. - -[woodpecker-server (DockerHub)](https://hub.docker.com/repository/docker/woodpeckerci/woodpecker-server) -[woodpecker-server (Quay)](https://quay.io/repository/woodpeckerci/woodpecker-server) - -[woodpecker-agent (DockerHub)](https://hub.docker.com/repository/docker/woodpeckerci/woodpecker-agent) -[woodpecker-agent (Quay)](https://quay.io/repository/woodpeckerci/woodpecker-agent) - -[woodpecker-cli (DockerHub)](https://hub.docker.com/repository/docker/woodpeckerci/woodpecker-cli) -[woodpecker-cli (Quay)](https://quay.io/repository/woodpeckerci/woodpecker-cli) - -[woodpecker-autoscaler (DockerHub)](https://hub.docker.com/repository/docker/woodpeckerci/autoscaler) diff --git a/docs/docs/30-administration/05-deployment-methods/20-kubernetes.md b/docs/docs/30-administration/05-deployment-methods/20-kubernetes.md deleted file mode 100644 index a3a92e043..000000000 --- a/docs/docs/30-administration/05-deployment-methods/20-kubernetes.md +++ /dev/null @@ -1,50 +0,0 @@ -# Kubernetes - -We recommended to deploy Woodpecker using the [Woodpecker helm chart](https://github.com/woodpecker-ci/helm). -Have a look at the [`values.yaml`](https://github.com/woodpecker-ci/helm/blob/main/charts/woodpecker/values.yaml) config files for all available settings. - -The chart contains two sub-charts, `server` and `agent` which are automatically configured as needed. -The chart started off with two independent charts but was merged into one to simplify the deployment at start of 2023. - -A couple of backend-specific config env vars exists which are described in the [kubernetes backend docs](../22-backends/40-kubernetes.md). - -## Metrics - -Please see [Prometheus](../40-advanced/90-prometheus.md) for general information on configuration and usage. - -For Kubernetes, you must set the following values when deploying via Helm chart to enable in-cluster metrics gathering: - -```yaml -metrics: - enabled: true - port: 9001 -``` - -This activates the `/metrics` endpoint on port `9001` without authentication. This port is not exposed externally by default. Use the instructions at [Prometheus](../40-advanced/90-prometheus.md) if you want to enable authenticated external access to metrics. - -To enable Prometheus pod monitoring discovery, you must also make the following settings: - - - -```yaml -prometheus: - podmonitor: - enabled: true - interval: 60s - labels: {} -``` - - - -### Troubleshooting Metrics - -If you are not receiving metrics despite the steps above, ensure that in your Prometheus configuration either your namespace is explicitly configured in `podMonitorNamespaceSelector` or the selectors are disabled. - -```yaml -# Search all available namespaces -podMonitorNamespaceSelector: - matchLabels: {} -# Enable all available pod monitors -podMonitorSelector: - matchLabels: {} -``` diff --git a/docs/docs/30-administration/05-deployment-methods/30-third-party.md b/docs/docs/30-administration/05-deployment-methods/30-third-party.md deleted file mode 100644 index 8eabded39..000000000 --- a/docs/docs/30-administration/05-deployment-methods/30-third-party.md +++ /dev/null @@ -1,12 +0,0 @@ -# Distribution packages - -:::info -Woodpecker itself is not responsible for creating these packages. Please reach out to the people responsible for packaging Woodpecker for the individual distributions. -::: - -- [NixOS](./40-nixos.md) via the [NixOS module](https://search.nixos.org/options?channel=unstable&size=200&sort=relevance&query=woodpecker) -- [Alpine (Edge)](https://pkgs.alpinelinux.org/packages?name=woodpecker&branch=edge&repo=&arch=&maintainer=) -- [Arch Linux](https://archlinux.org/packages/?q=woodpecker) -- [openSUSE](https://software.opensuse.org/package/woodpecker) -- [YunoHost](https://apps.yunohost.org/app/woodpecker) -- [Cloudron](https://www.cloudron.io/store/org.woodpecker_ci.cloudronapp.html) diff --git a/docs/docs/30-administration/05-deployment-methods/10-docker-compose.md b/docs/docs/30-administration/05-installation/10-docker-compose.md similarity index 72% rename from docs/docs/30-administration/05-deployment-methods/10-docker-compose.md rename to docs/docs/30-administration/05-installation/10-docker-compose.md index c8f9000d9..9da953701 100644 --- a/docs/docs/30-administration/05-deployment-methods/10-docker-compose.md +++ b/docs/docs/30-administration/05-installation/10-docker-compose.md @@ -1,8 +1,10 @@ -# docker compose +# Docker Compose -The below [docker compose](https://docs.docker.com/compose/) configuration can be used to start a Woodpecker server with a single agent. +This example [docker-compose](https://docs.docker.com/compose/) setup shows the deployment of a Woodpecker instance connected to GitHub (`WOODPECKER_GITHUB=true`). If you are using another forge, please change this including the respective secret settings. -It relies on a number of environment variables that you must set before running `docker compose up`. The variables are described below. +It creates persistent volumes for the server and agent config directories. The bundled SQLite DB is stored in `/var/lib/woodpecker` and is the most important part to be persisted as it holds all users and repository information. + +The server uses the default port `8000` and gets exposed to the host here, so WoodpeckerWO can be accessed through this port on the host or by a reverse proxy sitting in front of it. ```yaml title="docker-compose.yaml" services: @@ -110,31 +112,3 @@ The server and agents use a shared secret to authenticate communication. This sh - [...] + - WOODPECKER_AGENT_SECRET=${WOODPECKER_AGENT_SECRET} ``` - -## Docker images - -Image variants: - -- The `vX.X.X` images are stable releases -- The `vX.X` images are based on the current release branch (e.g. `release/v1.0`) and can be used to get bug fixes asap -- The `vX` same as `vX.X` variant but also includes feature releases -- The `next` images are based on the current `main` branch - -:::note -The `latest` tag is not available on purpose (and has been dropped with the 3.x release) to prevent accidental major version upgrades. -Hence, users are forced to specify a fixed or rolling tag, omitting the tag identifier (which equals to pulling `latest` implicitly) won't work. -::: - -```bash -# server -docker pull woodpeckerci/woodpecker-server:v3 -docker pull woodpeckerci/woodpecker-server:v3-alpine - -# agent -docker pull woodpeckerci/woodpecker-agent:v3 -docker pull woodpeckerci/woodpecker-agent:v3-alpine - -# cli -docker pull woodpeckerci/woodpecker-cli:v3 -docker pull woodpeckerci/woodpecker-cli:v3-alpine -``` diff --git a/docs/docs/30-administration/05-installation/20-helm-chart.md b/docs/docs/30-administration/05-installation/20-helm-chart.md new file mode 100644 index 000000000..d421f8959 --- /dev/null +++ b/docs/docs/30-administration/05-installation/20-helm-chart.md @@ -0,0 +1,45 @@ +# Helm Chart + +Woodpecker provides a [Helm chart](https://github.com/woodpecker-ci/helm) for Kubernetes environments: + +```bash +helm repo add woodpecker oci://ghcr.io/woodpecker-ci/helm +helm install woodpecker woodpecker/woodpecker +``` + +## Metrics + +To enable metrics gathering, set the following in values.yml: + +```yaml +metrics: + enabled: true + port: 9001 +``` + +This activates the `/metrics` endpoint on port `9001` without authentication. This port is not exposed externally by default. Use the instructions at Prometheus if you want to enable authenticated external access to metrics. + +To enable both Prometheus pod monitoring discovery, set: + + + +```yaml +prometheus: + podmonitor: + enabled: true + interval: 60s + labels: {} +``` + + + +If you are not receiving metrics after following the steps above, verify that your Prometheus configuration includes your namespace explicitly in the podMonitorNamespaceSelector or that the selectors are disabled: + +```yaml +# Search all available namespaces +podMonitorNamespaceSelector: + matchLabels: {} +# Enable all available pod monitors +podMonitorSelector: + matchLabels: {} +``` diff --git a/docs/docs/30-administration/05-deployment-methods/40-nixos.md b/docs/docs/30-administration/05-installation/30-packages.md similarity index 69% rename from docs/docs/30-administration/05-deployment-methods/40-nixos.md rename to docs/docs/30-administration/05-installation/30-packages.md index 063016985..8852c7659 100644 --- a/docs/docs/30-administration/05-deployment-methods/40-nixos.md +++ b/docs/docs/30-administration/05-installation/30-packages.md @@ -1,15 +1,32 @@ -# NixOS +# Distribution packages + +## Official packages + +- DEB +- RPM + +## Community packages :::info -Note that this module is not maintained by the Woodpecker developers. +Woodpecker itself is not responsible for creating these packages. Please reach out to the people responsible for packaging Woodpecker for the individual distributions. +::: + +- [Alpine (Edge)](https://pkgs.alpinelinux.org/packages?name=woodpecker&branch=edge&repo=&arch=&maintainer=) +- [Arch Linux](https://archlinux.org/packages/?q=woodpecker) +- [openSUSE](https://software.opensuse.org/package/woodpecker) +- [YunoHost](https://apps.yunohost.org/app/woodpecker) +- [Cloudron](https://www.cloudron.io/store/org.woodpecker_ci.cloudronapp.html) + +### NixOS + +:::info +This module is not maintained by the Woodpecker developers. If you experience issues please open a bug report in the [nixpkgs repo](https://github.com/NixOS/nixpkgs/issues/new/choose) where the module is maintained. ::: -The NixOS install is in theory quite similar to the binary install and supports multiple backends. +In theory, the NixOS installation is very similar to the binary installation and supports multiple backends. In practice, the settings are specified declaratively in the NixOS configuration and no manual steps need to be taken. -## General Configuration - ```nix @@ -83,8 +100,4 @@ in } ``` -All configuration options can be found via [NixOS Search](https://search.nixos.org/options?channel=unstable&size=200&sort=relevance&query=woodpecker) - -## Tips and tricks - -There are some resources on how to utilize Woodpecker more effectively with NixOS on the [Awesome Woodpecker](/awesome) page, like using the runners nix-store in the pipeline. +All configuration options can be found via [NixOS Search](https://search.nixos.org/options?channel=unstable&size=200&sort=relevance&query=woodpecker). There are also some additional resources on how to utilize Woodpecker more effectively with NixOS on the [Awesome Woodpecker](/awesome) page, like using the runners nix-store in the pipeline. diff --git a/docs/docs/30-administration/05-deployment-methods/_category_.yaml b/docs/docs/30-administration/05-installation/_category_.yaml similarity index 54% rename from docs/docs/30-administration/05-deployment-methods/_category_.yaml rename to docs/docs/30-administration/05-installation/_category_.yaml index 3907838b0..59d884127 100644 --- a/docs/docs/30-administration/05-deployment-methods/_category_.yaml +++ b/docs/docs/30-administration/05-installation/_category_.yaml @@ -1,3 +1,3 @@ -label: 'Deployment methods' +label: 'Installation' collapsible: true collapsed: true