mirrors/woodpecker
1
0
Fork 0
mirror of https://github.com/woodpecker-ci/woodpecker.git synced 2024-11-27 04:11:03 +00:00
Code Issues Projects Releases Wiki Activity

Update Agent and Pipeline syntax documentation (#506)

Browse source 
- Creates `Agent Config` with examples of agents and configuration options
- Updates `Pipeline Syntax`
  - About the different types of conditionals
  - Sort Order (Global -> Step -> Advanced)
- Move Conditional Steps to own page

Co-authored-by: 6543 <6543@obermui.de>
This commit is contained in:
Tim Coombs 2021-11-21 06:45:59 +11:00 committed by GitHub
parent 07b32fa0fe
commit 745090b3af
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
5 changed files with 444 additions and 260 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

418
docs/docs/20-usage/20-pipeline-syntax.md
View file

@ -21,6 +21,154 @@ pipeline:
In the above example we define two pipeline steps, `frontend` and `backend`. The names of these steps are completely arbitrary. In the above example we define two pipeline steps, `frontend` and `backend`. The names of these steps are completely arbitrary.
## Global Pipeline Conditionals
Woodpecker gives the ability to skip whole pipelines (not just steps) when based on certain conditions.
### `branches`
Woodpecker can skip commits based on the target branch. If the branch matches the `branches:` block the pipeline is executed, otherwise it is skipped.
Example skipping a commit when the target branch is not master:
```diff
pipeline:
build:
image: golang
commands:
- go build
- go test
+branches: master
```
Example matching multiple target branches:
```diff
pipeline:
build:
image: golang
commands:
- go build
- go test
+branches: [ master, develop ]
```
Example uses glob matching:
```diff
pipeline:
build:
image: golang
commands:
- go build
- go test
+branches: [ master, feature/* ]
```
Example includes branches:
```diff
pipeline:
build:
image: golang
commands:
- go build
- go test
+branches:
+ include: [ master, feature/* ]
```
Example excludes branches:
```diff
pipeline:
build:
image: golang
commands:
- go build
- go test
+branches:
+ exclude: [ develop, feature/* ]
```
### `when`
If required, Woodpecker can be made to skip whole pipelines based on `when`. This could be utilised to ensure compliance that only certain jobs run on certain agents (regional restrictions). Or targeting architectures.
This is achieved by ensuring the `when` block is on the root level. Rather than
See [when](#step-when---step-conditional-execution) above to understand all the different types of conditions that can be used.
> Note: You may need to set the agent environment settings, as these are not set automatically. See: [agent configuration](/docs/administration/agent-config) for more details.
Example targeting a specific platform:
```diff
pipeline:
build:
image: golang
commands:
- go build
- go test
-when:
-platform: [ linux/arm* ]
+when:
+ platform: [ linux/arm* ]
```
Assuming we have two agents, one `arm` and one `amd64`. Previously this pipeline would have executed on **either agent**, as Woodpecker is not fussy about where it runs the pipelines.
Because we had our original `when` block underneath the `build` block, if it was run on the `linux/amd64` agent. It would have cloned the repository, and then skipped the build step. Resulting in a Successful build.
Moving the when block to the root level will ensure that the whole pipeline will run be targeted to agents that match all of the conditions.
This can be utilised in conjunction with other when blocks as well.
Example `when` pipeline & step block:
```yml
pipeline:
build:
image: golang
commands:
- go build
- go test
publish:
image: plugins/docker
repo: foo/bar
+when:
+tag: release*
+when:
+ platform: [ linux/arm* ]
```
### Skip Commits
Woodpecker gives the ability to skip individual commits by adding `[CI SKIP]` to the commit message. Note this is case-insensitive.
```diff
git commit -m "updated README [CI SKIP]"
```
## `services`
Woodpecker can provide service containers. They can for example be used to run databases or cache containers during the execution of pipeline.
For more details check the [services docs](/docs/usage/services/).
## Steps ## Steps
Every step of your pipeline executes arbitrary commands inside a specified docker container. The defined commands are executed serially. Every step of your pipeline executes arbitrary commands inside a specified docker container. The defined commands are executed serially.
@ -35,7 +183,7 @@ pipeline:
+ - go test + - go test
``` ```
## Step `image` ### `image`
Woodpecker uses Docker images for the build environment, for plugins and for service containers. The image field is exposed in the container blocks in the Yaml: Woodpecker uses Docker images for the build environment, for plugins and for service containers. The image field is exposed in the container blocks in the Yaml:
@ -116,7 +264,7 @@ To make a private registry globally available check the [server configuration do
For specific details on configuring access to Google Container Registry, please view the docs [here](https://cloud.google.com/container-registry/docs/advanced-authentication#using_a_json_key_file). For specific details on configuring access to Google Container Registry, please view the docs [here](https://cloud.google.com/container-registry/docs/advanced-authentication#using_a_json_key_file).
## Step `commands` ### `commands`
Commands of every pipeline step are executed serially as if you would enter them into your local shell. Commands of every pipeline step are executed serially as if you would enter them into your local shell.
@ -147,183 +295,25 @@ docker run --entrypoint=build.sh golang
> Please note that only build steps can define commands. You cannot use commands with plugins or services. > Please note that only build steps can define commands. You cannot use commands with plugins or services.
## Step `environment` ### `environment`
Woodpecker provides the ability to pass environment variables to individual pipeline steps. Woodpecker provides the ability to pass environment variables to individual pipeline steps.
For more details check the [environment docs](/docs/usage/environment/). For more details check the [environment docs](/docs/usage/environment/).
## Step `secrets` ### `secrets`
Woodpecker provides the ability to store named parameters external to the Yaml configuration file, in a central secret store. These secrets can be passed to individual steps of the pipeline at runtime. Woodpecker provides the ability to store named parameters external to the Yaml configuration file, in a central secret store. These secrets can be passed to individual steps of the pipeline at runtime.
For more details check the [secrets docs](/docs/usage/secrets/). For more details check the [secrets docs](/docs/usage/secrets/).
## Step `when` - Conditional Execution ### `when` - Conditional Execution
Woodpecker supports defining conditional pipeline steps in the `when` block. If all conditions in the `when` block evaluate to true the step is executed, otherwise it is skipped. Woodpecker supports defining conditional pipeline steps in the `when` block.
### `branch` For more details check the [Conditional Step Execution](/docs/usage/conditional-execution/).
Example conditional execution by branch: ### `group` - Parallel execution
```diff
pipeline:
slack:
image: plugins/slack
channel: dev
+ when:
+ branch: master
```
> The step now triggers on master, but also if the target branch of a pull request is `master`. Add an event condition to limit it further to pushes on master only.
Execute a step if the branch is `master` or `develop`:
```diff
when:
branch: [master, develop]
```
Execute a step if the branch starts with `prefix/*`:
```diff
when:
branch: prefix/*
```
Execute a step using custom include and exclude logic:
```diff
when:
branch:
include: [ master, release/* ]
exclude: [ release/1.0.0, release/1.1.* ]
```
### `event`
Execute a step if the build event is a `tag`:
```diff
when:
event: tag
```
Execute a step if the build event is a `tag` created from the specified branch:
```diff
when:
event: tag
+ branch: master
```
Execute a step for all non-pull request events:
```diff
when:
event: [push, tag, deployment]
```
Execute a step for all build events:
```diff
when:
event: [push, pull_request, tag, deployment]
```
### `tag`
Execute a step if the tag name starts with `release`:
```diff
when:
tag: release*
```
### `status`
There are use cases for executing pipeline steps on failure, such as sending notifications for failed pipelines. Use the status constraint to execute steps even when the pipeline fails:
```diff
pipeline:
slack:
image: plugins/slack
channel: dev
+ when:
+ status: [ success, failure ]
```
### `platform`
Execute a step for a specific platform:
```diff
when:
platform: linux/amd64
```
Execute a step for a specific platform using wildcards:
```diff
when:
platform: [ linux/*, windows/amd64 ]
```
### `environment`
Execute a step for deployment events matching the target deployment environment:
```diff
when:
environment: production
event: deployment
```
### `matrix`
Execute a step for a single matrix permutation:
```diff
when:
matrix:
GO_VERSION: 1.5
REDIS_VERSION: 2.8
```
### `instance`
Execute a step only on a certain Woodpecker instance matching the specified hostname:
```diff
when:
instance: stage.woodpecker.company.com
```
### `path`
> NOTE: This feature is currently only available for GitHub and Gitea repositories.
Execute a step only on a pipeline with certain files being changed:
```diff
when:
path: "src/*"
```
You can use [glob patterns](https://github.com/bmatcuk/doublestar#patterns) to match the changed files and specify if the step should run if a file matching that pattern has been changed `include` or if some files have **not** been changed `exclude`.
```diff
when:
path:
include: [ '.woodpecker/*.yml', '*.ini' ]
exclude: [ '*.md', 'docs/**' ]
ignore_message: "[ALL]"
```
** Hint: ** Passing a defined ignore-message like `[ALL]` inside the commit message will ignore all path conditions.
## Step `group` - Parallel execution
Woodpecker supports parallel step execution for same-machine fan-in and fan-out. Parallel steps are configured using the `group` attribute. This instructs the pipeline runner to execute the named group in parallel. Woodpecker supports parallel step execution for same-machine fan-in and fan-out. Parallel steps are configured using the `group` attribute. This instructs the pipeline runner to execute the named group in parallel.
@ -351,84 +341,16 @@ pipeline:
In the above example, the `frontend` and `backend` steps are executed in parallel. The pipeline runner will not execute the `publish` step until the group completes. In the above example, the `frontend` and `backend` steps are executed in parallel. The pipeline runner will not execute the `publish` step until the group completes.
## Step `volumes` ### `volumes`
Woodpecker gives the ability to define Docker volumes in the Yaml. You can use this parameter to mount files or folders on the host machine into your containers. Woodpecker gives the ability to define Docker volumes in the Yaml. You can use this parameter to mount files or folders on the host machine into your containers.
For more details check the [volumes docs](/docs/usage/volumes/). For more details check the [volumes docs](/docs/usage/volumes/).
## `branches`
Woodpecker gives the ability to skip commits based on the target branch. If the branch matches the `branches:` block the pipeline is executed, otherwise it is skipped. ## Advanced Configurations
Example skipping a commit when the target branch is not master: ### `workspace`
```diff
pipeline:
build:
image: golang
commands:
- go build
- go test
+branches: master
```
Example matching multiple target branches:
```diff
pipeline:
build:
image: golang
commands:
- go build
- go test
+branches: [ master, develop ]
```
Example uses glob matching:
```diff
pipeline:
build:
image: golang
commands:
- go build
- go test
+branches: [ master, feature/* ]
```
Example includes branches:
```diff
pipeline:
build:
image: golang
commands:
- go build
- go test
+branches:
+ include: [ master, feature/* ]
```
Example excludes branches:
```diff
pipeline:
build:
image: golang
commands:
- go build
- go test
+branches:
+ exclude: [ develop, feature/* ]
```
## `workspace`
The workspace defines the shared volume and working directory shared by all pipeline steps. The default workspace matches the below pattern, based on your repository url. The workspace defines the shared volume and working directory shared by all pipeline steps. The default workspace matches the below pattern, based on your repository url.
@ -492,13 +414,13 @@ git clone https://github.com/octocat/hello-world \
/go/src/github.com/octocat/hello-world /go/src/github.com/octocat/hello-world
``` ```
## `matrix` ### `matrix`
Woodpecker has integrated support for matrix builds. Woodpecker executes a separate build task for each combination in the matrix, allowing you to build and test a single commit against multiple configurations. Woodpecker has integrated support for matrix builds. Woodpecker executes a separate build task for each combination in the matrix, allowing you to build and test a single commit against multiple configurations.
For more details check the [matrix build docs](/docs/usage/matrix-builds/). For more details check the [matrix build docs](/docs/usage/matrix-builds/).
## `clone` ### `clone`
Woodpecker automatically configures a default clone step if not explicitly defined. You can manually configure the clone step in your pipeline for customization: Woodpecker automatically configures a default clone step if not explicitly defined. You can manually configure the clone step in your pipeline for customization:
@ -541,7 +463,7 @@ clone:
+ path: bitbucket.org/foo/bar + path: bitbucket.org/foo/bar
``` ```
### Git Submodules #### Git Submodules
To use the credentials that cloned the repository to clone it's submodules, update `.gitmodules` to use `https` instead of `git`: To use the credentials that cloned the repository to clone it's submodules, update `.gitmodules` to use `https` instead of `git`:
@ -566,21 +488,7 @@ pipeline:
... ...
``` ```
## `services` ### Privileged mode
Woodpecker can provide service containers. They can for example be used to run databases or cache containers during the execution of pipeline.
For more details check the [services docs](/docs/usage/services/).
## Skip Commits
Woodpecker gives the ability to skip individual commits by adding `[CI SKIP]` to the commit message. Note this is case-insensitive.
```diff
git commit -m "updated README [CI SKIP]"
```
## Privileged mode
Woodpecker gives the ability to configure privileged mode in the Yaml. You can use this parameter to launch containers with escalated capabilities. Woodpecker gives the ability to configure privileged mode in the Yaml. You can use this parameter to launch containers with escalated capabilities.

165
docs/docs/20-usage/22-conditional-execution.md Normal file
View file

@ -0,0 +1,165 @@
# Conditional Step Execution
Woodpecker supports defining conditional pipeline steps in the `when` block. If all conditions in the `when` block evaluate to true the step is executed, otherwise it is skipped.
This can also be utilised on a playbook level if you have multi-arch agents and require specific pipelines to be run on specific architectures. See [platform](#platform) below.
## `branch`
Example conditional execution by branch:
```diff
pipeline:
slack:
image: plugins/slack
channel: dev
+ when:
+ branch: master
```
> The step now triggers on master, but also if the target branch of a pull request is `master`. Add an event condition to limit it further to pushes on master only.
Execute a step if the branch is `master` or `develop`:
```diff
when:
branch: [master, develop]
```
Execute a step if the branch starts with `prefix/*`:
```diff
when:
branch: prefix/*
```
Execute a step using custom include and exclude logic:
```diff
when:
branch:
include: [ master, release/* ]
exclude: [ release/1.0.0, release/1.1.* ]
```
## `event`
Execute a step if the build event is a `tag`:
```diff
when:
event: tag
```
Execute a step if the build event is a `tag` created from the specified branch:
```diff
when:
event: tag
+ branch: master
```
Execute a step for all non-pull request events:
```diff
when:
event: [push, tag, deployment]
```
Execute a step for all build events:
```diff
when:
event: [push, pull_request, tag, deployment]
```
## `tag`
Execute a step if the tag name starts with `release`:
```diff
when:
tag: release*
```
## `status`
There are use cases for executing pipeline steps on failure, such as sending notifications for failed pipelines. Use the status constraint to execute steps even when the pipeline fails:
```diff
pipeline:
slack:
image: plugins/slack
channel: dev
+ when:
+ status: [ success, failure ]
```
## `platform`
Execute a step for a specific platform:
```diff
when:
platform: linux/amd64
```
Execute a step for a specific platform using wildcards:
```diff
when:
platform: [ linux/*, windows/amd64 ]
```
## `environment`
Execute a step for deployment events matching the target deployment environment:
```diff
when:
environment: production
event: deployment
```
## `matrix`
Execute a step for a single matrix permutation:
```diff
when:
matrix:
GO_VERSION: 1.5
REDIS_VERSION: 2.8
```
## `instance`
Execute a step only on a certain Woodpecker instance matching the specified hostname:
```diff
when:
instance: stage.woodpecker.company.com
```
## `path`
> NOTE: This feature is currently only available for GitHub and Gitea repositories.
Execute a step only on a pipeline with certain files being changed:
```diff
when:
path: "src/*"
```
You can use [glob patterns](https://github.com/bmatcuk/doublestar#patterns) to match the changed files and specify if the step should run if a file matching that pattern has been changed `include` or if some files have **not** been changed `exclude`.
```diff
when:
path:
include: [ '.woodpecker/*.yml', '*.ini' ]
exclude: [ '*.md', 'docs/**' ]
ignore_message: "[ALL]"
```
** Hint: ** Passing a defined ignore-message like `[ALL]` inside the commit message will ignore all path conditions.

0
docs/docs/20-usage/21-multi-pipeline.md → docs/docs/20-usage/25-multi-pipeline.md
View file

8
docs/docs/20-usage/30-matrix-builds.md
View file

@ -60,8 +60,8 @@ Example Yaml file after injecting the matrix parameters:
```diff ```diff
pipeline: pipeline:
build: build:
- image: golang:1.4 - image: golang:${GO_VERSION}
+ image: golang:${GO_VERSION} + image: golang:1.4
commands: commands:
- go get - go get
- go build - go build
@ -72,8 +72,8 @@ pipeline:
services: services:
database: database:
- image: mysql:5.5 - image: ${DATABASE}
+ image: ${DATABASE} + image: mysql:5.5
``` ```
## Examples ## Examples

113
docs/docs/30-administration/11-agent-config.md
View file

@ -1,3 +1,114 @@
# Agent configuration # Agent configuration
TODO Agents are configured by the command line or environement variables. At the minimum you need the following information:
```yaml
# docker-compose.yml
version: '3'
services:
woodpecker-agent:
[...]
environment:
+ - WOODPECKER_SERVER=localhost:9000
+ - WOODPECKER_AGENT_SECRET="your-shared-secret-goes-here"
```
The following are automatically set and can be overridden:
- WOODPECKER_HOSTNAME if not set, becomes the OS' hostname
- WOODPECKER_PLATFORM if not set, is the architecture eg: `linux/amd64`
- WOODPECKER_MAX_PROCS if not set, defaults to 1
## Processes per agent
By default the maximum processes that are run per agent is 1. If required you can add `WOODPECKER_MAX_PROCS` to increase your parellel processing on a per-agent basis.
```yaml
# docker-compose.yml
version: '3'
services:
woodpecker-agent:
[...]
environment:
- WOODPECKER_SERVER=localhost:9000
- WOODPECKER_AGENT_SECRET="your-shared-secret-goes-here"
+ - WOODPECKER_MAX_PROCS=4
```
## Filtering agents
When building your pipelines as long as you have set the platform or filter, builds can be made to only run code on certain agents.
```
- WOODPECKER_HOSTNAME=mycompany-ci-01.example.com
- WOODPECKER_PLATFORM=linux/amd64
- WOODPECKER_FILTER=
```
### Filter on Platform
Only want certain pipelines or steps to run on certain platforms? Such as arm vs amd64?
```diff
# docker-compose.yml
version: '3'
services:
woodpecker-agent:
[...]
environment:
- WOODPECKER_SERVER=localhost:9000
- WOODPECKER_AGENT_SECRET=""
+ - WOODPECKER_PLATFORM=linux/arm64
```
```yaml
# .woodpecker.yml
pipeline:
build:
image: golang
commands:
- go build
- go test
when:
platform: linux/amd64
testing:
image: golang
commands:
- go build
- go test
when:
platform: linux/arm*
```
See [Conditionals Pipeline](/docs/usage/pipeline-syntax#step-when---conditional-execution) syntax for more
## All agent configuration options
Here is the full list of configuration options and their default variables.
```yaml
- WOODPECKER_SERVER=localhost:9000
- WOODPECKER_AGENT_SECRET=""
- WOODPECKER_USERNAME=x-oauth-basic
- WOODPECKER_LOG_LEVEL=""
- WOODPECKER_DEBUG_PRETTY=""
- WOODPECKER_DEBUG_NOCOLOR=true
- WOODPECKER_HOSTNAME=""
- WOODPECKER_PLATFORM="linux/amd64"
- WOODPECKER_FILTER=""
- WOODPECKER_MAX_PROCS=1
- WOODPECKER_HEALTHCHECK=true
- WOODPECKER_KEEPALIVE_TIME=10
- WOODPECKER_KEEPALIVE_TIMEOUT=time.Second * 20
- WOODPECKER_GRPC_SECURE=""
- WOODPECKER_GRPC_VERIFY=true
```