mirror of
https://github.com/woodpecker-ci/woodpecker.git
synced 2024-06-13 02:49:37 +00:00
Anbraten
GitHub
parent
bbc64c36b3
commit
03870c5583
27
README.md
27
README.md
|
|
@ -56,29 +56,6 @@ Please consider to donate and become a backer. 🙏 [[Become a backer](https://o
|
|||
|
||||
<a href="https://opencollective.com/woodpecker-ci" target="_blank"><img src="https://opencollective.com/woodpecker-ci/backers.svg?width=890"></a>
|
||||
|
||||
## 🚀 Usage
|
||||
|
||||
### .woodpecker.yml
|
||||
|
||||
- Place your pipeline in a file named `.woodpecker.yml` in your repository
|
||||
- Pipeline steps can be named as you like
|
||||
- Run any command in the commands section
|
||||
|
||||
[Read More](https://woodpecker-ci.org/docs/usage/intro)
|
||||
|
||||
### Build steps are containers
|
||||
|
||||
- Define any Docker image as context
|
||||
- Install the needed tools in custom Docker images, use them as context
|
||||
|
||||
[Read More](https://woodpecker-ci.org/docs/usage/pipeline-syntax#steps)
|
||||
|
||||
### Plugins
|
||||
|
||||
Woodpecker has [official plugins](https://woodpecker-ci.org/plugins), but you can also use your own.
|
||||
|
||||
[Read More](https://woodpecker-ci.org/docs/usage/plugins/plugins)
|
||||
|
||||
## 📖 Documentation
|
||||
|
||||
https://woodpecker-ci.org/
|
||||
|
|
@ -99,9 +76,9 @@ We use an own [Weblate](https://weblate.org/en/) instance at [translate.woodpeck
|
|||
|
||||
## 👋 Who uses Woodpecker?
|
||||
|
||||
[Codeberg](https://codeberg.org), the Woodpecker project itself, and many others.
|
||||
Woodpecker is used by [itself](https://ci.woodpecker-ci.org/woodpecker/woodpecker-ci/) multiple well-known companies, organizations like [Codeberg](https://codeberg.org), hobbyist and many others.
|
||||
|
||||
Leave a [comment](https://github.com/woodpecker-ci/woodpecker/issues/122) if you're using it as well.
|
||||
Leave a [comment](https://github.com/woodpecker-ci/woodpecker/discussions/2149) if you're using it as well.
|
||||
|
||||
Also consider using the topic `WoodpeckerCI` in your repository, so others can learn from your config and use the hashtag `#WoodpeckerCI` when talking about the project on social media!
|
||||
|
||||
|
|
|
|||
1549
docs/docs/20-usage/15-terminiology/architecture.excalidraw
Normal file
1549
docs/docs/20-usage/15-terminiology/architecture.excalidraw
Normal file
File diff suppressed because it is too large
Load diff
17
docs/docs/20-usage/15-terminiology/architecture.svg
Normal file
17
docs/docs/20-usage/15-terminiology/architecture.svg
Normal file
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 31 KiB |
|
|
@ -1,5 +1,13 @@
|
|||
# Terminology
|
||||
|
||||
## Woodpecker architecture
|
||||
|
||||
|
||||
|
||||
## Pipeline, workflow & step
|
||||
|
||||
|
||||
|
||||
## Glossary
|
||||
|
||||
- **Woodpecker CI**: The project name around Woodpecker.
|
||||
|
|
@ -24,19 +32,20 @@
|
|||
- **Dependency**: [Workflows][Workflow] can depend on each other, and if possible, they are executed in parallel.
|
||||
- **Status**: Status refers to the outcome of a step or [workflow][Workflow] after it has been executed, determined by the internal command exit code. At the end of a [workflow][Workflow], its status is sent to the [forge][Forge].
|
||||
|
||||
## Terms
|
||||
## Conventions
|
||||
|
||||
Sometimes there exist multiple terms that can be used for a thing, we try to define it here once and stick to it.
|
||||
Sometimes there are multiple terms that can be used to describe something. This section lists the preferred terms to use in Woodpecker:
|
||||
|
||||
- environment variables `*_LINK` should be `*_URL`, also in code, use `URL()` instead of `Link` ([Vote](https://framadate.org/jVSQHwIGfJYy82IL))
|
||||
- **Pipelines** were previously called **builds**
|
||||
- **Steps** were previously called **jobs**
|
||||
- Environment variables `*_LINK` should be called `*_URL`. In the code use `URL()` instead of `Link()`
|
||||
- Use the term **pipelines** instead of the previous **builds**
|
||||
- Use the term **steps** instead of the previous **jobs**
|
||||
|
||||
[Pipeline]: ./20-pipeline-syntax.md
|
||||
[Workflow]: ./25-workflows.md
|
||||
[Forge]: ../30-administration/11-forges/10-overview.md
|
||||
[Plugin]: ./51-plugins/10-plugins.md
|
||||
[Workspace]: ./20-pipeline-syntax.md#workspace
|
||||
[Matrix]: ./30-matrix-workflows.md
|
||||
[Docker]: ../30-administration/22-backends/10-docker.md
|
||||
[Local]: ../30-administration/22-backends/20-local.md
|
||||
<!-- References -->
|
||||
[Pipeline]: ../20-workflow-syntax.md
|
||||
[Workflow]: ../25-workflows.md
|
||||
[Forge]: ../../30-administration/11-forges/10-overview.md
|
||||
[Plugin]: ../51-plugins/10-plugins.md
|
||||
[Workspace]: ../20-workflow-syntax.md#workspace
|
||||
[Matrix]: ../30-matrix-workflows.md
|
||||
[Docker]: ../../30-administration/22-backends/10-docker.md
|
||||
[Local]: ../../30-administration/22-backends/20-local.md
|
||||
1077
docs/docs/20-usage/15-terminiology/pipeline-workflow-step.excalidraw
Normal file
1077
docs/docs/20-usage/15-terminiology/pipeline-workflow-step.excalidraw
Normal file
File diff suppressed because it is too large
Load diff
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 21 KiB |
|
|
@ -1,6 +1,6 @@
|
|||
# Pipeline syntax
|
||||
# Workflow syntax
|
||||
|
||||
The pipeline section defines a list of steps to build, test and deploy your code. Pipeline steps are executed serially, in the order in which they are defined. If a step returns a non-zero exit code, the pipeline immediately aborts and returns a failure status.
|
||||
The workflow section defines a list of steps to build, test and deploy your code. Steps are executed serially, in the order in which they are defined. If a step returns a non-zero exit code, the workflow and therefore all other workflows and the pipeline immediately aborts and returns a failure status.
|
||||
|
||||
Example steps:
|
||||
|
||||
|
|
@ -19,7 +19,7 @@ steps:
|
|||
- npm run build
|
||||
```
|
||||
|
||||
In the above example we define two pipeline steps, `frontend` and `backend`. The names of these steps are completely arbitrary.
|
||||
In the above example we define two steps, `frontend` and `backend`. The names of these steps are completely arbitrary.
|
||||
|
||||
Another way to name a step is by using the name keyword:
|
||||
|
||||
|
|
@ -50,8 +50,8 @@ git commit -m "updated README [CI SKIP]"
|
|||
|
||||
## Steps
|
||||
|
||||
Every step of your pipeline executes arbitrary commands inside a specified container. The defined commands are executed serially.
|
||||
The associated commit of a current pipeline run is checked out with git to a workspace which is mounted to every step of the pipeline as the working directory.
|
||||
Every step of your workflow executes commands inside a specified container. The defined commands are executed serially.
|
||||
The associated commit is checked out with git to a workspace which is mounted to every step of the workflow as the working directory.
|
||||
|
||||
```diff
|
||||
steps:
|
||||
|
|
@ -64,7 +64,7 @@ The associated commit of a current pipeline run is checked out with git to a wor
|
|||
|
||||
### File changes are incremental
|
||||
|
||||
- Woodpecker clones the source code in the beginning pipeline
|
||||
- Woodpecker clones the source code in the beginning of the workflow
|
||||
- Changes to files are persisted through steps as the same volume is mounted to all steps
|
||||
|
||||
```yaml
|
||||
|
|
@ -82,7 +82,7 @@ steps:
|
|||
|
||||
### `image`
|
||||
|
||||
Woodpecker pulls the defined image and uses it as environment to execute the pipeline step commands, for plugins and for service containers.
|
||||
Woodpecker pulls the defined image and uses it as environment to execute the workflow step commands, for plugins and for service containers.
|
||||
|
||||
When using the `local` backend, the `image` entry is used to specify the shell, such as Bash or Fish, that is used to run the commands.
|
||||
|
||||
|
|
@ -122,75 +122,11 @@ Woodpecker does not automatically upgrade container images. Example configuratio
|
|||
+ pull: true
|
||||
```
|
||||
|
||||
##### Images from private registries
|
||||
|
||||
You must provide registry credentials on the UI in order to pull private pipeline images defined in your YAML configuration file.
|
||||
|
||||
These credentials are never exposed to your pipeline, which means they cannot be used to push, and are safe to use with pull requests, for example. Pushing to a registry still require setting credentials for the appropriate plugin.
|
||||
|
||||
Example configuration using a private image:
|
||||
|
||||
```diff
|
||||
steps:
|
||||
build:
|
||||
+ image: gcr.io/custom/golang
|
||||
commands:
|
||||
- go build
|
||||
- go test
|
||||
```
|
||||
|
||||
Woodpecker matches the registry hostname to each image in your YAML. If the hostnames match, the registry credentials are used to authenticate to your registry and pull the image. Note that registry credentials are used by the Woodpecker agent and are never exposed to your build containers.
|
||||
|
||||
Example registry hostnames:
|
||||
|
||||
- Image `gcr.io/foo/bar` has hostname `gcr.io`
|
||||
- Image `foo/bar` has hostname `docker.io`
|
||||
- Image `qux.com:8000/foo/bar` has hostname `qux.com:8000`
|
||||
|
||||
Example registry hostname matching logic:
|
||||
|
||||
- Hostname `gcr.io` matches image `gcr.io/foo/bar`
|
||||
- Hostname `docker.io` matches `golang`
|
||||
- Hostname `docker.io` matches `library/golang`
|
||||
- Hostname `docker.io` matches `bradyrydzewski/golang`
|
||||
- Hostname `docker.io` matches `bradyrydzewski/golang:latest`
|
||||
|
||||
##### Global registry support
|
||||
|
||||
To make a private registry globally available check the [server configuration docs](../30-administration/10-server-config.md#global-registry-setting).
|
||||
|
||||
##### GCR registry support
|
||||
|
||||
For specific details on configuring access to Google Container Registry, please view the docs [here](https://cloud.google.com/container-registry/docs/advanced-authentication#using_a_json_key_file).
|
||||
|
||||
##### Local Images
|
||||
|
||||
It's possible to build a local image by mounting the docker socket as a volume.
|
||||
|
||||
With a `Dockerfile` at the root of the project:
|
||||
|
||||
```diff
|
||||
steps:
|
||||
build-image:
|
||||
image: docker
|
||||
commands:
|
||||
- docker build --rm -t local/project-image .
|
||||
volumes:
|
||||
- /var/run/docker.sock:/var/run/docker.sock
|
||||
|
||||
build-project:
|
||||
image: local/project-image
|
||||
commands:
|
||||
- ./build.sh
|
||||
```
|
||||
|
||||
:::warning
|
||||
For this privileged rights are needed only available to admins. In addition this only works when using a single agent.
|
||||
:::
|
||||
Learn more how you can use images from [different registries](./41-registries.md).
|
||||
|
||||
### `commands`
|
||||
|
||||
Commands of every pipeline step are executed serially as if you would enter them into your local shell.
|
||||
Commands of every step are executed serially as if you would enter them into your local shell.
|
||||
|
||||
```diff
|
||||
steps:
|
||||
|
|
@ -221,19 +157,19 @@ docker run --entrypoint=build.sh golang
|
|||
|
||||
### `environment`
|
||||
|
||||
Woodpecker provides the ability to pass environment variables to individual pipeline steps.
|
||||
Woodpecker provides the ability to pass environment variables to individual steps.
|
||||
|
||||
For more details check the [environment docs](./50-environment.md).
|
||||
|
||||
### `secrets`
|
||||
|
||||
Woodpecker provides the ability to store named parameters external to the YAML configuration file, in a central secret store. These secrets can be passed to individual steps of the pipeline at runtime.
|
||||
Woodpecker provides the ability to store named parameters external to the YAML configuration file, in a central secret store. These secrets can be passed to individual steps of the workflow at runtime.
|
||||
|
||||
For more details check the [secrets docs](./40-secrets.md).
|
||||
|
||||
### `failure`
|
||||
|
||||
Some of the pipeline steps may be allowed to fail without causing the whole pipeline to report a failure (e.g., a step executing a linting check). To enable this, add `failure: ignore` to your pipeline step. If Woodpecker encounters an error while executing the step, it will report it as failed but still execute the next steps of the pipeline, if any, without affecting the status of the pipeline.
|
||||
Some of the steps may be allowed to fail without causing the whole workflow and therefore pipeline to report a failure (e.g., a step executing a linting check). To enable this, add `failure: ignore` to your step. If Woodpecker encounters an error while executing the step, it will report it as failed but still executes the next steps of the workflow, if any, without affecting the status of the workflow.
|
||||
|
||||
```diff
|
||||
steps:
|
||||
|
|
@ -247,7 +183,7 @@ Some of the pipeline steps may be allowed to fail without causing the whole pipe
|
|||
|
||||
### `when` - Conditional Execution
|
||||
|
||||
Woodpecker supports defining a list of conditions for a pipeline step by using a `when` block. If at least one of the conditions in the `when` block evaluate to true the step is executed, otherwise it is skipped. A condition can be a check like:
|
||||
Woodpecker supports defining a list of conditions for a step by using a `when` block. If at least one of the conditions in the `when` block evaluate to true the step is executed, otherwise it is skipped. A condition can be a check like:
|
||||
|
||||
```diff
|
||||
steps:
|
||||
|
|
@ -368,7 +304,7 @@ when:
|
|||
|
||||
#### `ref`
|
||||
|
||||
The `ref` filter compares the git reference against which the pipeline is executed.
|
||||
The `ref` filter compares the git reference against which the workflow is executed.
|
||||
This allows you to filter, for example, tags that must start with **v**:
|
||||
|
||||
```yaml
|
||||
|
|
@ -379,7 +315,7 @@ when:
|
|||
|
||||
#### `status`
|
||||
|
||||
There are use cases for executing pipeline steps on failure, such as sending notifications for failed pipelines. Use the status constraint to execute steps even when the pipeline fails:
|
||||
There are use cases for executing steps on failure, such as sending notifications for failed workflow / pipeline. Use the status constraint to execute steps even when the workflow fails:
|
||||
|
||||
```diff
|
||||
steps:
|
||||
|
|
@ -394,7 +330,7 @@ steps:
|
|||
#### `platform`
|
||||
|
||||
:::note
|
||||
This condition should be used in conjunction with a [matrix](./30-matrix-workflows.md#example-matrix-pipeline-using-multiple-platforms) pipeline as a regular pipeline will only executed by a single agent which only has one arch.
|
||||
This condition should be used in conjunction with a [matrix](./30-matrix-workflows.md#example-matrix-pipeline-using-multiple-platforms) workflow as a regular workflow will only be executed by a single agent which only has one arch.
|
||||
:::
|
||||
|
||||
Execute a step for a specific platform:
|
||||
|
|
@ -510,7 +446,7 @@ when:
|
|||
|
||||
### `group` - Parallel execution
|
||||
|
||||
Woodpecker supports parallel step execution for same-machine fan-in and fan-out. Parallel steps are configured using the `group` attribute. This instructs the pipeline runner to execute the named group in parallel.
|
||||
Woodpecker supports parallel step execution for same-machine fan-in and fan-out. Parallel steps are configured using the `group` attribute. This instructs the agent to execute the named group in parallel.
|
||||
|
||||
Example parallel configuration:
|
||||
|
||||
|
|
@ -534,7 +470,7 @@ Example parallel configuration:
|
|||
repo: octocat/hello-world
|
||||
```
|
||||
|
||||
In the above example, the `frontend` and `backend` steps are executed in parallel. The pipeline runner will not execute the `publish` step until the group completes.
|
||||
In the above example, the `frontend` and `backend` steps are executed in parallel. The agent will not execute the `publish` step until the group completes.
|
||||
|
||||
### `volumes`
|
||||
|
||||
|
|
@ -544,7 +480,7 @@ For more details check the [volumes docs](./70-volumes.md).
|
|||
|
||||
### `detach`
|
||||
|
||||
Woodpecker gives the ability to detach steps to run them in background until the pipeline finishes.
|
||||
Woodpecker gives the ability to detach steps to run them in background until the workflow finishes.
|
||||
|
||||
For more details check the [service docs](./60-services.md#detachment).
|
||||
|
||||
|
|
@ -554,13 +490,13 @@ Using `directory`, you can set a subdirectory of your repository or an absolute
|
|||
|
||||
## `services`
|
||||
|
||||
Woodpecker can provide service containers. They can for example be used to run databases or cache containers during the execution of pipeline.
|
||||
Woodpecker can provide service containers. They can for example be used to run databases or cache containers during the execution of workflow.
|
||||
|
||||
For more details check the [services docs](./60-services.md).
|
||||
|
||||
## `workspace`
|
||||
|
||||
The workspace defines the shared volume and working directory shared by all pipeline steps. The default workspace matches the below pattern, based on your repository URL.
|
||||
The workspace defines the shared volume and working directory shared by all workflow steps. The default workspace matches the below pattern, based on your repository URL.
|
||||
|
||||
```txt
|
||||
/woodpecker/src/github.com/octocat/hello-world
|
||||
|
|
@ -581,7 +517,7 @@ The workspace can be customized using the workspace block in the YAML file:
|
|||
- go test
|
||||
```
|
||||
|
||||
The base attribute defines a shared base volume available to all pipeline steps. This ensures your source code, dependencies and compiled binaries are persisted and shared between steps.
|
||||
The base attribute defines a shared base volume available to all steps. This ensures your source code, dependencies and compiled binaries are persisted and shared between steps.
|
||||
|
||||
```diff
|
||||
workspace:
|
||||
|
|
@ -630,12 +566,12 @@ For more details check the [matrix build docs](./30-matrix-workflows.md).
|
|||
|
||||
## `labels`
|
||||
|
||||
You can set labels for your pipeline to select an agent to execute the pipeline on. An agent will pick up and run a pipeline when **every** label assigned to a pipeline matches the agents labels.
|
||||
You can set labels for your workflow to select an agent to execute the workflow on. An agent will pick up and run a workflow when **every** label assigned to it matches the agents labels.
|
||||
|
||||
To set additional agent labels check the [agent configuration options](../30-administration/15-agent-config.md#woodpecker_filter_labels). Agents will have at least four default labels: `platform=agent-os/agent-arch`, `hostname=my-agent`, `backend=docker` (type of the agent backend) and `repo=*`. Agents can use a `*` as a wildcard for a label. For example `repo=*` will match every repo.
|
||||
|
||||
Pipeline labels with an empty value will be ignored.
|
||||
By default each pipeline has at least the `repo=your-user/your-repo-name` label. If you have set the [platform attribute](#platform) for your pipeline it will have a label like `platform=your-os/your-arch` as well.
|
||||
Workflow labels with an empty value will be ignored.
|
||||
By default each workflow has at least the `repo=your-user/your-repo-name` label. If you have set the [platform attribute](#platform) for your workflow it will have a label like `platform=your-os/your-arch` as well.
|
||||
|
||||
You can add additional labels as a key value map:
|
||||
|
||||
|
|
@ -655,12 +591,12 @@ steps:
|
|||
|
||||
### Filter by platform
|
||||
|
||||
To configure your pipeline to only be executed on an agent with a specific platform, you can use the `platform` key.
|
||||
To configure your workflow to only be executed on an agent with a specific platform, you can use the `platform` key.
|
||||
Have a look at the official [go docs](https://go.dev/doc/install/source) for the available platforms. The syntax of the platform is `GOOS/GOARCH` like `linux/arm64` or `linux/amd64`.
|
||||
|
||||
Example:
|
||||
|
||||
Assuming we have two agents, one `linux/arm` and one `linux/amd64`. Previously this pipeline would have executed on **either agent**, as Woodpecker is not fussy about where it runs the pipelines. By setting the following option it will only be executed on an agent with the platform `linux/arm64`.
|
||||
Assuming we have two agents, one `linux/arm` and one `linux/amd64`. Previously this workflow would have executed on **either agent**, as Woodpecker is not fussy about where it runs the workflows. By setting the following option it will only be executed on an agent with the platform `linux/arm64`.
|
||||
|
||||
```diff
|
||||
+labels:
|
||||
|
|
@ -672,15 +608,15 @@ steps:
|
|||
|
||||
## `variables`
|
||||
|
||||
Woodpecker supports [YAML anchors & aliases](https://yaml.org/spec/1.2.2/#3222-anchors-and-aliases) in the pipeline configuration. These can be used as variables to not repeat yourself.
|
||||
Woodpecker supports using [YAML anchors & aliases](https://yaml.org/spec/1.2.2/#3222-anchors-and-aliases) as variables in the workflow configuration.
|
||||
|
||||
For more details and examples check the [Advanced YAML syntax docs](./35-advanced-yaml-syntax.md)
|
||||
For more details and examples check the [Advanced usage docs](./90-advanced-usage.md)
|
||||
|
||||
## `clone`
|
||||
|
||||
Woodpecker automatically configures a default clone step if not explicitly defined. When using the `local` backend, the [plugin-git](https://github.com/woodpecker-ci/plugin-git) binary must be on your `$PATH` for the default clone step to work. If not, you can still write a manual clone step.
|
||||
|
||||
You can manually configure the clone step in your pipeline for customization:
|
||||
You can manually configure the clone step in your workflow for customization:
|
||||
|
||||
```diff
|
||||
+clone:
|
||||
|
|
@ -758,9 +694,9 @@ By default Woodpecker is automatically adding a clone step. This clone step can
|
|||
skip_clone: true
|
||||
```
|
||||
|
||||
## `when` - Global pipeline conditions
|
||||
## `when` - Global workflow conditions
|
||||
|
||||
Woodpecker gives the ability to skip whole pipelines (not just steps #when---conditional-execution-1) based on certain conditions by a `when` block. If all conditions in the `when` block evaluate to true the pipeline is executed, otherwise it is skipped, but treated as successful and other pipelines depending on it will still continue.
|
||||
Woodpecker gives the ability to skip whole workflows (not just steps #when---conditional-execution-1) based on certain conditions by a `when` block. If all conditions in the `when` block evaluate to true the workflow is executed, otherwise it is skipped, but treated as successful and other workflows depending on it will still continue.
|
||||
|
||||
### `repo`
|
||||
|
||||
|
|
@ -21,7 +21,7 @@ You can also set some custom path like `.my-ci/pipelines/` instead of `.woodpeck
|
|||
## Example workflow definition
|
||||
|
||||
:::warning
|
||||
Please note that files are only shared between steps of the same workflow (see [File changes are incremental](./20-pipeline-syntax.md#file-changes-are-incremental)). That means you cannot access artifacts e.g. from the `build` workflow in the `deploy` workflow.
|
||||
Please note that files are only shared between steps of the same workflow (see [File changes are incremental](./20-workflow-syntax.md#file-changes-are-incremental)). That means you cannot access artifacts e.g. from the `build` workflow in the `deploy` workflow.
|
||||
If you still need to pass artifacts between the workflows you need use some storage [plugin](./51-plugins/10-plugins.md) (e.g. one which stores files in an Amazon S3 bucket).
|
||||
:::
|
||||
|
||||
|
|
@ -126,5 +126,5 @@ depends_on:
|
|||
|
||||
:::info
|
||||
Some workflows don't need the source code, like creating a notification on failure.
|
||||
Read more about `skip_clone` at [pipeline syntax](./20-pipeline-syntax.md#skip_clone)
|
||||
Read more about `skip_clone` at [pipeline syntax](./20-workflow-syntax.md#skip_clone)
|
||||
:::
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# Matrix workflows
|
||||
|
||||
Woodpecker has integrated support for matrix workflows. Woodpecker executes a separate workflow for each combination in the matrix, allowing you to build and test a single commit against multiple configurations.
|
||||
Woodpecker has integrated support for matrix workflows. Woodpecker executes a separate workflow for each combination in the matrix, allowing you to build and test against multiple configurations.
|
||||
|
||||
Example matrix definition:
|
||||
|
||||
|
|
|
|||
|
|
@ -1,3 +1,71 @@
|
|||
# Registries
|
||||
|
||||
Woodpecker provides the ability to add container registries in the settings of your repository. Adding a registry allows you to authenticate and pull private images from a container registry when using these images as a step inside your pipeline.
|
||||
Woodpecker provides the ability to add container registries in the settings of your repository. Adding a registry allows you to authenticate and pull private images from a container registry when using these images as a step inside your pipeline. Using registry credentials can also help you avoid rate limiting when pulling images from public registries.
|
||||
|
||||
## Images from private registries
|
||||
|
||||
You must provide registry credentials in the UI in order to pull private container images defined in your YAML configuration file.
|
||||
|
||||
These credentials are never exposed to your steps, which means they cannot be used to push, and are safe to use with pull requests, for example. Pushing to a registry still requires setting credentials for the appropriate plugin.
|
||||
|
||||
Example configuration using a private image:
|
||||
|
||||
```diff
|
||||
steps:
|
||||
build:
|
||||
+ image: gcr.io/custom/golang
|
||||
commands:
|
||||
- go build
|
||||
- go test
|
||||
```
|
||||
|
||||
Woodpecker matches the registry hostname to each image in your YAML. If the hostnames match, the registry credentials are used to authenticate to your registry and pull the image. Note that registry credentials are used by the Woodpecker agent and are never exposed to your build containers.
|
||||
|
||||
Example registry hostnames:
|
||||
|
||||
- Image `gcr.io/foo/bar` has hostname `gcr.io`
|
||||
- Image `foo/bar` has hostname `docker.io`
|
||||
- Image `qux.com:8000/foo/bar` has hostname `qux.com:8000`
|
||||
|
||||
Example registry hostname matching logic:
|
||||
|
||||
- Hostname `gcr.io` matches image `gcr.io/foo/bar`
|
||||
- Hostname `docker.io` matches `golang`
|
||||
- Hostname `docker.io` matches `library/golang`
|
||||
- Hostname `docker.io` matches `bradyrydzewski/golang`
|
||||
- Hostname `docker.io` matches `bradyrydzewski/golang:latest`
|
||||
|
||||
## Global registry support
|
||||
|
||||
To make a private registry globally available, check the [server configuration docs](../30-administration/10-server-config.md#global-registry-setting).
|
||||
|
||||
## GCR registry support
|
||||
|
||||
For specific details on configuring access to Google Container Registry, please view the docs [here](https://cloud.google.com/container-registry/docs/advanced-authentication#using_a_json_key_file).
|
||||
|
||||
## Local Images
|
||||
|
||||
:::warning
|
||||
For this, privileged rights are needed only available to admins. In addition, this only works when using a single agent.
|
||||
:::
|
||||
|
||||
It's possible to build a local image by mounting the docker socket as a volume.
|
||||
|
||||
With a `Dockerfile` at the root of the project:
|
||||
|
||||
```diff
|
||||
steps:
|
||||
build-image:
|
||||
image: docker
|
||||
commands:
|
||||
- docker build --rm -t local/project-image .
|
||||
volumes:
|
||||
- /var/run/docker.sock:/var/run/docker.sock
|
||||
|
||||
build-project:
|
||||
image: local/project-image
|
||||
commands:
|
||||
- ./build.sh
|
||||
```
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -1,6 +1,10 @@
|
|||
# Advanced YAML syntax
|
||||
# Advanced usage
|
||||
|
||||
## Anchors & aliases
|
||||
## Advanced YAML syntax
|
||||
|
||||
YAML has some advanced syntax features that can be used like variables to reduce duplication in your pipeline config:
|
||||
|
||||
### Anchors & aliases
|
||||
|
||||
You can use [YAML anchors & aliases](https://yaml.org/spec/1.2.2/#3222-anchors-and-aliases) as variables in your pipeline config.
|
||||
|
||||
|
|
@ -32,7 +36,7 @@ Just add a new section called **variables** like this:
|
|||
commands: build
|
||||
```
|
||||
|
||||
## Map merges and overwrites
|
||||
### Map merges and overwrites
|
||||
|
||||
```yaml
|
||||
variables:
|
||||
|
|
@ -62,7 +66,7 @@ steps:
|
|||
branch: main
|
||||
```
|
||||
|
||||
## Sequence merges
|
||||
### Sequence merges
|
||||
|
||||
```yaml
|
||||
variables:
|
||||
|
|
@ -89,7 +93,41 @@ steps:
|
|||
- <<: *post_cmds
|
||||
```
|
||||
|
||||
## References
|
||||
### References
|
||||
|
||||
- [Official specification](https://yaml.org/spec/1.2.2/#3222-anchors-and-aliases)
|
||||
- [Cheatsheet](https://learnxinyminutes.com/docs/yaml)
|
||||
- [Official YAML specification](https://yaml.org/spec/1.2.2/#3222-anchors-and-aliases)
|
||||
- [YAML Cheatsheet](https://learnxinyminutes.com/docs/yaml)
|
||||
|
||||
## Persisting environment data between steps
|
||||
|
||||
One can create a file containing environment variables, and then source it in each step that needs them.
|
||||
|
||||
```yml
|
||||
steps:
|
||||
init:
|
||||
image: bash
|
||||
commands:
|
||||
echo "FOO=hello" >> envvars
|
||||
echo "BAR=world" >> envvars
|
||||
|
||||
debug:
|
||||
image: bash
|
||||
commands:
|
||||
- source envvars
|
||||
- echo $FOO
|
||||
```
|
||||
|
||||
## Declaring global variables in `docker-compose.yml`
|
||||
|
||||
As described in [Global environment variables](./50-environment.md#global-environment-variables), one can define global variables:
|
||||
|
||||
```yml
|
||||
services:
|
||||
woodpecker-server:
|
||||
# ...
|
||||
environment:
|
||||
- WOODPECKER_ENVIRONMENT=first_var:value1,second_var:value2
|
||||
# ...
|
||||
```
|
||||
|
||||
Note that this tightly couples the server and app configurations (where the app is a completely separate application). But this is a good option for truly global variables which should apply to all steps in all pipelines for all apps.
|
||||
|
|
@ -1,84 +0,0 @@
|
|||
# Advanced pipeline management
|
||||
|
||||
## Using variables
|
||||
|
||||
Once your pipeline starts to grow in size, it will become important to keep it DRY ("Don't Repeat Yourself") by using variables and environment variables. Depending on your specific need, there are a number of options.
|
||||
|
||||
### YAML extensions
|
||||
|
||||
As described in [Advanced YAML syntax](./35-advanced-yaml-syntax.md).
|
||||
|
||||
```yml
|
||||
variables:
|
||||
- &golang_image 'golang:1.18'
|
||||
|
||||
steps:
|
||||
build:
|
||||
image: *golang_image
|
||||
commands: build
|
||||
```
|
||||
|
||||
Note that the `golang_image` alias cannot be used with string interpolation. But this is otherwise a good option for most cases.
|
||||
|
||||
### YAML extensions (alternate form)
|
||||
|
||||
Another approach using YAML extensions:
|
||||
|
||||
```yml
|
||||
variables:
|
||||
- global_env: &global_env
|
||||
- BASH_VERSION=1.2.3
|
||||
- PATH_SRC=src/
|
||||
- PATH_TEST=test/
|
||||
- FOO=something
|
||||
|
||||
steps:
|
||||
build:
|
||||
image: bash:${BASH_VERSION}
|
||||
directory: ${PATH_SRC}
|
||||
commands:
|
||||
- make ${FOO} -o ${PATH_TEST}
|
||||
environment: *global_env
|
||||
|
||||
test:
|
||||
image: bash:${BASH_VERSION}
|
||||
commands:
|
||||
- test ${PATH_TEST}
|
||||
environment:
|
||||
- <<:*global_env
|
||||
- ADDITIONAL_LOCAL="var value"
|
||||
```
|
||||
|
||||
### Persisting environment data between steps
|
||||
|
||||
One can create a file containing environment variables, and then source it in each step that needs them.
|
||||
|
||||
```yml
|
||||
steps:
|
||||
init:
|
||||
image: bash
|
||||
commands:
|
||||
echo "FOO=hello" >> envvars
|
||||
echo "BAR=world" >> envvars
|
||||
|
||||
debug:
|
||||
image: bash
|
||||
commands:
|
||||
- source envvars
|
||||
- echo $FOO
|
||||
```
|
||||
|
||||
### Declaring global variables in `docker-compose.yml`
|
||||
|
||||
As described in [Global environment variables](./50-environment.md#global-environment-variables), one can define global variables:
|
||||
|
||||
```yml
|
||||
services:
|
||||
woodpecker-server:
|
||||
# ...
|
||||
environment:
|
||||
- WOODPECKER_ENVIRONMENT=first_var:value1,second_var:value2
|
||||
# ...
|
||||
```
|
||||
|
||||
Note that this tightly couples the server and app configurations (where the app is a completely separate application). But this is a good option for truly global variables which should apply to all steps in all pipelines for all apps.
|
||||
|
|
@ -35,7 +35,7 @@ In addition you need at least some kind of database which requires additional re
|
|||
You can install Woodpecker on multiple ways:
|
||||
|
||||
- Using [docker-compose](#docker-compose) with the official [container images](../80-downloads.md#docker-images)
|
||||
- Using [Kubernetes](./#kubernetes) via the Woodpeckers Helm chart
|
||||
- Using [Kubernetes](#kubernetes) via the Woodpeckers Helm chart
|
||||
- Using [binaries](../80-downloads.md)
|
||||
|
||||
### docker-compose
|
||||
|
|
@ -170,7 +170,7 @@ services:
|
|||
+ - WOODPECKER_AGENT_SECRET=${WOODPECKER_AGENT_SECRET}
|
||||
```
|
||||
|
||||
### Kubernetes
|
||||
### Kubernetes
|
||||
|
||||
We recommended to deploy Woodpecker using the [Woodpecker helm chart](https://github.com/woodpecker-ci/helm).
|
||||
Have a look at the [`values.yaml`](https://github.com/woodpecker-ci/helm/blob/main/values.yaml) config files for all available settings.
|
||||
|
|
|
|||
|
|
@ -9,6 +9,6 @@
|
|||
| Event: Pull-Request | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: |
|
||||
| Event: Deploy | :white_check_mark: | :x: | :x: | :x: |
|
||||
| [Multiple workflows](../../20-usage/25-workflows.md) | :white_check_mark: | :white_check_mark: | :white_check_mark: | :white_check_mark: |
|
||||
| [when.path filter](../../20-usage/20-pipeline-syntax.md#path) | :white_check_mark: | :white_check_mark:¹ | :white_check_mark: | :x: |
|
||||
| [when.path filter](../../20-usage/20-workflow-syntax.md#path) | :white_check_mark: | :white_check_mark:¹ | :white_check_mark: | :x: |
|
||||
|
||||
¹ for pull requests at least Gitea version 1.17 is required
|
||||
|
|
|
|||
|
|
@ -21,9 +21,9 @@ The following are automatically set and can be overridden:
|
|||
- WOODPECKER_HOSTNAME if not set, becomes the OS' hostname
|
||||
- WOODPECKER_MAX_WORKFLOWS if not set, defaults to 1
|
||||
|
||||
## Processes per agent
|
||||
## Workflows per agent
|
||||
|
||||
By default the maximum processes that are run per agent is 1. If required you can add `WOODPECKER_MAX_WORKFLOWS` to increase your parallel processing on a per-agent basis.
|
||||
By default, the maximum workflows that are executed in parallel on an agent is 1. If required, you can add `WOODPECKER_MAX_WORKFLOWS` to increase your parallel processing for an agent.
|
||||
|
||||
```diff
|
||||
# docker-compose.yml
|
||||
|
|
@ -38,38 +38,35 @@ services:
|
|||
+ - WOODPECKER_MAX_WORKFLOWS=4
|
||||
```
|
||||
|
||||
## Agent registration on server
|
||||
## Agent registration
|
||||
|
||||
When the agent starts, it connects to the server using token from `WOODPECKER_AGENT_SECRET`. The server identifies agent and, if such agent doesn't exist, register him.
|
||||
There are two types of token, so would be two ways of agent registration.
|
||||
When the agent starts it connects to the server using the token from `WOODPECKER_AGENT_SECRET`. The server identifies the agent and registers the agent in its database if it wasn't connected before.
|
||||
|
||||
There are two types of tokens to connect an agent to the server:
|
||||
|
||||
### Using system token
|
||||
|
||||
_System token_ is a token that is used system-wide, e. g. when you set the same token in `WOODPECKER_AGENT_SECRET` on both the server and the agents.
|
||||
A _system token_ is a token that is used system-wide, e.g. when you set the same token in `WOODPECKER_AGENT_SECRET` on both the server and the agents.
|
||||
|
||||
In that case registration process would be as follows:
|
||||
In that case registration process would be as following:
|
||||
|
||||
1. First time Agent communicates with Server using system token;
|
||||
2. Server registers Agent in DB, generates ID and sends this ID back to Agent;
|
||||
3. Agent stores ID in a file configured by `WOODPECKER_AGENT_CONFIG_FILE`.
|
||||
|
||||
At the following startups Agent uses system token **and** ID.
|
||||
1. The first time the agent communicates with the server, it is using the system token
|
||||
1. The server registers the agent in its database if not done before and generates a unique ID which is then sent back to the agent
|
||||
1. The agent stores the received ID in a file (configured by `WOODPECKER_AGENT_CONFIG_FILE`)
|
||||
1. At the following startups, the agent uses the system token **and** its received ID to identify itself to the server
|
||||
|
||||
### Using agent token
|
||||
|
||||
_Agent token_ is a token that is used by only particular agent. This unique token also configured by `WOODPECKER_AGENT_SECRET`, but only on the agent side.
|
||||
An _agent token_ is a token that is used by only one particular agent. This unique token is applied to the agent by `WOODPECKER_AGENT_SECRET`.
|
||||
|
||||
In that case you probably doesn't configure `WOODPECKER_AGENT_SECRET` on the server side. The registration process would be as follows:
|
||||
To get an _agent token_ you have to register the agent manually in the server using the UI:
|
||||
|
||||
1. Administrator registers Agent manually in _Server settings - Agents - Add agent_;
|
||||
|
||||
|
||||
2. The token generated in previous step have to be provided to Agent in `WOODPECKER_AGENT_SECRET`;
|
||||
3. First time Agent communicates with Server using agent token;
|
||||
4. Server identifies Agent by the token and fills additional information provided by Agent;
|
||||
|
||||
|
||||
At following startups Agent uses own token only.
|
||||
1. The administrator registers a new agent manually at `Settings -> Agents -> Add agent`
|
||||
|
||||
|
||||
1. The generated token from the previous step has to be provided to the agent using `WOODPECKER_AGENT_SECRET`
|
||||
1. The agent will connect to the server using the provided token and will update its status in the UI:
|
||||
|
||||
|
||||
## All agent configuration options
|
||||
|
||||
|
|
@ -128,7 +125,7 @@ Configures the number of parallel workflows.
|
|||
### `WOODPECKER_FILTER_LABELS`
|
||||
> Default: empty
|
||||
|
||||
Configures labels to filter pipeline pick up. Use a list of key-value pairs like `key=value,second-key=*`. `*` can be used as a wildcard. By default agents provide three additional labels `platform=os/arch`, `hostname=my-agent` and `repo=*` which can be overwritten if needed. To learn how labels work check out the [pipeline syntax page](../20-usage/20-pipeline-syntax.md#labels).
|
||||
Configures labels to filter pipeline pick up. Use a list of key-value pairs like `key=value,second-key=*`. `*` can be used as a wildcard. By default, agents provide three additional labels `platform=os/arch`, `hostname=my-agent` and `repo=*` which can be overwritten if needed. To learn how labels work, check out the [pipeline syntax page](../20-usage/20-workflow-syntax.md#labels).
|
||||
|
||||
### `WOODPECKER_HEALTHCHECK`
|
||||
> Default: `true`
|
||||
|
|
|
|||
|
|
@ -96,14 +96,14 @@ steps:
|
|||
image: /usr/bin/tree
|
||||
```
|
||||
|
||||
If no commands are provided, we treat them as plugins in the usual manner.
|
||||
If no commands are provided, we treat them as plugins in the usual manner.
|
||||
In the context of the local backend, plugins are simply executable binaries, which can be located using their name if they are listed in `$PATH`, or through an absolute path.
|
||||
|
||||
### Using labels to filter tasks
|
||||
|
||||
You can use the [agent configuration
|
||||
options](../15-agent-config.md#woodpecker_filter_labels) and the
|
||||
[pipeline syntax](../../20-usage/20-pipeline-syntax.md#labels) to only run certain
|
||||
[pipeline syntax](../../20-usage/20-workflow-syntax.md#labels) to only run certain
|
||||
pipelines on certain agents. Example:
|
||||
|
||||
Define a `label` `type` with value `exec` for a particular agent:
|
||||
|
|
|
|||
|
|
@ -1,6 +1,15 @@
|
|||
# Secrets encryption
|
||||
|
||||
By default, Woodpecker does not encrypt secrets in its database. You can enable encryption
|
||||
:::danger
|
||||
Secrets encryption is currently broken and therefore disabled by default. It will be fixed in an upcoming release.
|
||||
|
||||
Check:
|
||||
- https://github.com/woodpecker-ci/woodpecker/issues/1541 and
|
||||
- https://github.com/woodpecker-ci/woodpecker/pull/2300
|
||||
:::
|
||||
|
||||
|
||||
By default, Woodpecker does not encrypt secrets in its database. You can enable encryption
|
||||
using simple AES key or more advanced [Google TINK](https://developers.google.com/tink) encryption.
|
||||
|
||||
## Common
|
||||
|
|
@ -8,7 +17,7 @@ using simple AES key or more advanced [Google TINK](https://developers.google.co
|
|||
### Enabling secrets encryption
|
||||
|
||||
To enable secrets encryption and encrypt all existing secrets in database set
|
||||
`WOODPECKER_ENCRYPTION_KEY`, `WOODPECKER_ENCRYPTION_KEY_FILE` or `WOODPECKER_ENCRYPTION_TINK_KEYSET_PATH` environment
|
||||
`WOODPECKER_ENCRYPTION_KEY`, `WOODPECKER_ENCRYPTION_KEY_FILE` or `WOODPECKER_ENCRYPTION_TINK_KEYSET_PATH` environment
|
||||
variable depending on encryption method of your choice.
|
||||
|
||||
After encryption is enabled you will be unable to start Woodpecker server without providing valid encryption key!
|
||||
|
|
@ -45,7 +54,7 @@ You will need plaintext AEAD-compatible Google TINK keyset to encrypt your data.
|
|||
|
||||
To generate it and then rotate keys if needed, install `tinkey`([installation guide](https://developers.google.com/tink/install-tinkey))
|
||||
|
||||
Keyset contains one or more keys, used to encrypt or decrypt your data, and primary key ID, used to determine which key
|
||||
Keyset contains one or more keys, used to encrypt or decrypt your data, and primary key ID, used to determine which key
|
||||
to use while encrypting new data.
|
||||
|
||||
Keyset generation example:
|
||||
|
|
|
|||
|
|
@ -2,17 +2,17 @@
|
|||
|
||||
Some versions need some changes to the server configuration or the pipeline configuration files.
|
||||
|
||||
## next (1.1.0)
|
||||
## next (2.0.0)
|
||||
|
||||
- Drop deprecated `CI_BUILD_*`, `CI_PREV_BUILD_*`, `CI_JOB_*`, `*_LINK`, `CI_SYSTEM_ARCH`, `CI_REPO_REMOTE` built-in environment variables
|
||||
- Drop deprecated `pipeline:` keyword for steps in yaml config
|
||||
- Drop deprecated `branches:` keyword for global branch filter
|
||||
- Deprecate `platform:` filter in favor of `labels:`, [read more](./20-usage/20-pipeline-syntax.md#filter-by-platform)
|
||||
- Dropped deprecated `CI_BUILD_*`, `CI_PREV_BUILD_*`, `CI_JOB_*`, `*_LINK`, `CI_SYSTEM_ARCH`, `CI_REPO_REMOTE` built-in environment variables
|
||||
- Dropped deprecated `pipeline:` keyword in favor of `steps:` in pipeline config
|
||||
- Dropped deprecated `branches:` filter in favor of global [`when.branch`](./20-usage/20-workflow-syntax.md#branch-1) filter
|
||||
- Deprecated `platform:` filter in favor of `labels:`, [read more](./20-usage/20-workflow-syntax.md#filter-by-platform)
|
||||
|
||||
## 1.0.0
|
||||
|
||||
- The signature used to verify extensions calls (like those used for the [config-extension](./30-administration/100-external-configuration-api.md)) done by the Woodpecker server switched from using a shared-secret HMac to an ed25519 key-pair. Read more about it at the [config-extensions](./30-administration/100-external-configuration-api.md) documentation.
|
||||
- Refactored support of old agent filter labels and expression. Learn how to use the new [filter](./20-usage/20-pipeline-syntax.md#labels)
|
||||
- Refactored support of old agent filter labels and expression. Learn how to use the new [filter](./20-usage/20-workflow-syntax.md#labels)
|
||||
- Renamed step environment variable `CI_SYSTEM_ARCH` to `CI_SYSTEM_PLATFORM`. Same applies for the cli exec variable.
|
||||
- Renamed environment variables `CI_BUILD_*` and `CI_PREV_BUILD_*` to `CI_PIPELINE_*` and `CI_PREV_PIPELINE_*`, old ones are still available but deprecated
|
||||
- Renamed environment variables `CI_JOB_*` to `CI_STEP_*`, old ones are still available but deprecated
|
||||
|
|
|
|||
Loading…
Reference in a new issue