I had experienced some issues running Woodpecker behind a reverse-proxy,
resulting from not defining the `WOODPECKER_ROOT_PATH` environment
variable in #2477.
As suggested by @qwerty287, specifying `WOODPECKER_ROOT_PATH=/foo`
*mostly* solved the issue of running the woodpecker server at an url
like `https://example.org/foo`.
However, the webhook urls and badge urls were generated excluding the
configured `WOODPECKER_ROOT_PATH`.
This PR (mostly) fixes issues related to non-empty
`WOODPECKER_ROOT_PATH`.
---------
Co-authored-by: qwerty287 <80460567+qwerty287@users.noreply.github.com>
- Slimify
- Add required permissions (missing so far)
- More detailed instructions on app settings
- Align env var options to style used in k8s section (UL instead of
headers)
---------
Co-authored-by: Lauris BH <lauris@nix.lv>
The `runs_on` section is missing from the syntax docs. So I added a
section for it, and linked to an example.
It's good enough. But I think it might be a good idea for someone (with
more knowledge than me) to add more info in this section. How to use it,
when to use it, etc. The example is useful but not very informative.
The current documentation mentions that when one uses woodpecker on the
same host as Gitlab, you might need to set the `Allow requests to the
local network from webhooks and integrations` option on the gitlab
server.
This option not only needs to be set when running on the same host, but
also needs to be set when setting up woodpecker with Gitlab on any
RFC1918 net and on any non standard TLD like `.local` or `.internal`.
official spec linked at top of page is inaccessible for most readers
(it's too dry and academic)
so added famous cheatsheet (heavily promoted on StackOverflow)
As suggested in
https://github.com/woodpecker-ci/woodpecker/discussions/2160.
- Simplified wording
- Referenced helm chart
- Removed "experimental" banner as the k8s backend is quite stable I'd
say (after using it for months now)
- Add example for `resources` definition`
---------
Co-authored-by: Anbraten <anton@ju60.de>
1. new translation docs
2. lazy-load TimeAgo locales (used for "x min ago" messages). This 1.
reduces size and 2. provides all languages without adding them manually.
3. Remove DayJS locales, they're unused.
Related-to: https://github.com/woodpecker-ci/woodpecker/pull/2078
Remaining CVEs:
```
❯ trivy fs --exit-code 1 --skip-dirs node_modules/,plugins/woodpecker-plugins/node_modules/ docs/
2023-08-01T10:02:36.911+0200 INFO Vulnerability scanning is enabled
2023-08-01T10:02:36.911+0200 INFO Secret scanning is enabled
2023-08-01T10:02:36.911+0200 INFO If your scanning is slow, please try '--scanners vuln' to disable secret scanning
2023-08-01T10:02:36.911+0200 INFO Please see also https://aquasecurity.github.io/trivy/v0.43/docs/scanner/secret/#recommendation for faster secret detection
2023-08-01T10:02:36.963+0200 INFO Number of language-specific files: 1
2023-08-01T10:02:36.963+0200 INFO Detecting pnpm vulnerabilities...
pnpm-lock.yaml (pnpm)
Total: 2 (UNKNOWN: 0, LOW: 0, MEDIUM: 1, HIGH: 1, CRITICAL: 0)
┌─────────┬────────────────┬──────────┬───────────────────┬────────────────┬──────────────────────────────────────────────────────────────┐
│ Library │ Vulnerability │ Severity │ Installed Version │ Fixed Version │ Title │
├─────────┼────────────────┼──────────┼───────────────────┼────────────────┼──────────────────────────────────────────────────────────────┤
│ got │ CVE-2022-33987 │ MEDIUM │ 9.6.0 │ 11.8.5, 12.1.0 │ missing verification of requested URLs allows redirects to │
│ │ │ │ │ │ UNIX sockets │
│ │ │ │ │ │ https://avd.aquasec.com/nvd/cve-2022-33987 │
├─────────┼────────────────┼──────────┼───────────────────┼────────────────┼──────────────────────────────────────────────────────────────┤
│ trim │ CVE-2020-7753 │ HIGH │ 0.0.1 │ 0.0.3 │ nodejs-trim: Regular Expression Denial of Service (ReDoS) in │
│ │ │ │ │ │ trim function │
│ │ │ │ │ │ https://avd.aquasec.com/nvd/cve-2020-7753 │
└─────────┴────────────────┴──────────┴───────────────────┴────────────────┴──────────────────────────────────────────────────────────────┘
```
- `trim` is pulled in by `@docusaurus/theme-classic` and can be ignored
due to
https://github.com/facebook/docusaurus/issues/7275#issuecomment-1113997259
- `got` can be ignored as well, see `trim`
Various ways to factor out common data in a pipeline file - having them
in one place rather than spread out over many pages, will help newbies
like me.
This PR introduces two new server configuration options, for providing a
custom .JS and .CSS file.
These can be used to show custom banner messages, add
environment-dependent signals, or simply a corporate logo.
### Motivation (what problem I try to solve)
I'm operating Woodpecker in multiple k8s clusters for different
environments.
When having multiple browser tabs open, I prefer strong indicators for
each environment.
E.g. a red "PROD" banner, or just a blue "QA" banner.
Also, we sometimes need to have the chance for maintenance, and instead
of broadcasting emails,
I prefer a banner message, stating something like: "Heads-up: there's a
planned downtime, next Friday, blabla...".
Also, I like to have the firm's logo visible, which makes Woodpecker
look more like an integral part of our platform.
### Implementation notes
* Two new config options are introduced ```WOODPECKER_CUSTOM_CSS_FILE```
and ```WOODPECKER_CUSTOM_JS_FILE```
* I've piggy-bagged the existing handler for assets, as it seemed to me
a minimally invasive approach
* the option along with an example is documented
* a simple unit test for the Gin-handler ensures some regression safety
* no extra dependencies are introduced
### Visual example
The documented example will look like this.
### Areas of uncertainty
This is my first contribution to Woodpecker and I tried my best to align
with your conventions.
That said, I found myself uncertain about these things and would be glad
about getting feedback.
* The handler tests are somewhat different than the other ones because I
wanted to keep them simple - I hope that still matches your coding
guidelines
* caching the page sometimes will let the browser not recognize changes
and a user must reload. I'm not fully into the details of how caching is
implemented and neither can judge if it's a real problem. Another pair
of eyes would be good.
Add Kubernetes Deployments and StatefulSet update and Dockle Scan Plugins.
For Kubernetes plugin, I based on the Drone unmaintened Kubernetes
plugin and took the statefulset management evolutions. I added sync/wait
and force redeploy capabilities + updates dependencies
For Dockle plugin, I took example on Trivy plugin.
Add plugin [Nextcloud Upload](https://github.com/Ellpeck/WoodpeckerPlugins/tree/main/nextcloud-upload) to the official plugin list.
there's already an official plugin that allows uploading
files using WebDAV, but my plugin has two Nextcloud-specific additions
that aren't part of the regular WebDAV spec:
- The ability to chunk uploads, which is necessary for larger files if
Nextcloud is hosted behind Cloudflare (which restricts uploads to a
maximum of 100MB)
- The ability to apply Nextcloud tags, which allows automatically
categorizing items and using Nextcloud's Retention plugin to easily
auto-remove older artifacts.
This add a simple implementation of requests/limits for individual
steps. There is no validation of what the resource actually is beyond
checking that it can successfully be converted to a Quantity, so it can
be used for things other than just memory/CPU.
close#1809
# Summary
This PR drops the outdated former swagger.yaml/json and introduced
automatic API document generation from Go code.
The generated code is also used to generate documentation/markdown for
the community page,
as well as enable the Woodpecker server to serve a Swagger Web UI for
manual tinkering.
I did opt-in for gin-swagger, a middleware for the Gin framework, to
ease implementation and have a sophisticated output.
This middleware only produces Swagger v2 specs. AFAIK the newer OpenApi
3x tooling is not yet that mature,
so I guess that's fine for now.
## Implemenation notes
- former swagger.json files removed
- former // swagger godocs removed
- introduced new dependency gin-swagger, which uses godoc annotations on
top of Gin Handler functions.
- reworked Makefile to automatically generate Go code for the server
- introduce new dependency go-swagger, to generate Markdown for
documentation purposes
- add a Swagger Web UI, incl. capabilities for manual API exploration
- consider relative root paths in the implementation
- write documentation for all exposed API endpoints
- incl. API docs in the community website (auto-generated)
- provide developer documentation, for the Woodpecker authors
- no other existing logic/code was intentionally changed
---------
close#292
---------
Co-authored-by: qwerty287 <80460567+qwerty287@users.noreply.github.com>
Co-authored-by: 6543 <6543@obermui.de>
it did make sense to have it still supported within v0.15.0,
but as we move future away and with the release of v1.0.0
we should not give the appearance of still support the original drone
v0.8 config
Gogs support is broken (and we won't fix it because we don't care about
it...) because it does not support OAuth, at least after we introduced
the new Vue UI.
See:
77d830d5b5/server/forge/gogs/gogs.go (L84)
This route is not present in the new UI.
closes#1636closes#1429
supersedes #1586
Uses a different approach: just take the index.html compiled by vite and
replace the paths to js and other files using regex. This is not
compatible with the dev proxy which is also the reason why we can't use
go templates for this.
# Changes
- Adds an admin view to see the whole work-queue of the server.
- The admin can also pause / resume the queue.
- The view is reloading data every 5 seconds automatically.
- The task model from queue got removed in favor of the one from models.
Coding support is likely broken and nobody will ever fix it. Also it
looks like nobody wants to use it, otherwise we would have get some bug
reports.
---------
Co-authored-by: 6543 <6543@obermui.de>
When a server such as Codeberg has unusually high response time, three
seconds may not be enough to fetch the configuration.
Signed-off-by: Earl Warren <contact@earl-warren.org>
Co-authored-by: 6543 <6543@obermui.de>
closes#101
Added secrets encryption in database
- Google TINK or simple AES as encryption mechanisms
- Keys rotation support on TINK
- Existing SecretService is wrapped by encryption layer
- Encryption can be enabled and disabled at any time
Co-authored-by: Kuzmin Ilya <ilia.kuzmin@indrive.com>
Co-authored-by: 6543 <6543@obermui.de>
As discussed in the comments in PR #1197. Also add documenation
accordingly.
One thing I'm not sure about is the simple check in health.go if the
address is usable in the GET request or not. From reading
https://pkg.go.dev/net#Dial it seems that the only non-standard address
format that would work in the `net` package but not in a GET url would
likely only be `:port`, as the others listed here are actually also
valid urls:
`For TCP, UDP and IP networks, if the host is empty or a literal
unspecified IP address, as in ":80", "0.0.0.0:80" or "[::]:80" for TCP
and UDP, "", "0.0.0.0" or "::" for IP, the local system is assumed.`
One additional thing I noticed is that while `WOODPECKER_SERVER_ADDR`
and `WOODPECKER_SERVER_ADDR` use the default value format of `:PORT`,
`WOODPECKER_SERVER` actually uses `localhost:9000`. I guess it makes a
bit of sense, considering the server might not be local to the agent,
but it looks a bit inconsistent this way. I don't think it would hurt to
make the `WOODPECKER_HEALTHCHECK_ADDR` in this format too, but then it's
different from the server flags again... :-)
closes#1181closes#834
Adds `ignore_failure` to pipeline steps. When it's set to true,
if the step fails the following steps continue to execute as if no failure had occurred.
---
failure enums idea:
* fail (default) = if other steps run in parallel, wait for them and
then let workflow fail
* cancel = if other steps run in parallel, kill them
* ignore = we mark the step as failed but it wont have any impact
This implements #1073, adds .yaml to the accepted endings for woodpecker configs.
This currently adds some more lines to the duplication (tried to compensate by fixing the other duplication in the configFetcher) as the CLI and Server are still separate.
Did used [cspell](https://www.npmjs.com/package/cspell) to find typos and fixed it.
Also add cspell to gitpod.
Signed-off-by: Yarden Shoham <hrsi88@gmail.com>
breakout from #934
when new events are added you don't have to worry that pipeline will behave different as it does now with this
Co-authored-by: Anbraten <anton@ju60.de>
Adding [Gitpod](https://github.com/gitpod-io/gitpod) allows us and others to easily start a complete Woodpecker setup and development cloud IDE. It starts a Woodpecker server, agent and a preconfigured Gitea instance. You can login at Gitea with `woodpecker` and `password`.
* make global environment variables available for pipeline substitution
* lint fixes
* global env support in cli exec; procBuilder tests
* drop GLOBAL_ prefix
* docs
* documentation typo
* Update docs/docs/20-usage/50-environment.md
as suggested by anbraten
Co-authored-by: Anbraten <anton@ju60.de>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Anbraten <anton@ju60.de>
use yaml aliases (https://yaml.org/spec/1.2.2/#3222-anchors-and-aliases) to have pipeline `variables`
Co-authored-by: qwerty287 <80460567+qwerty287@users.noreply.github.com>
Co-authored-by: Anbraten <anton@ju60.de>
to make it easier for devs to find the right place for code
close#655
Co-authored-by: Anbraten <anton@ju60.de>
Co-authored-by: qwerty287 <80460567+qwerty287@users.noreply.github.com>
Officially support labels for pipelines and agents to improve pipeline picking.
* add pipeline labels
* update, improve docs and add migration
* update proto file
---
closes#304 & #860
Clarify the `when.tag` example:
- The `tag` filter is only used if it's also a `tag` event, so this makes the example clearer for new users.
- mention glob
closes#11
Added support:
1. Environment variable `WOODPECKER_DELETE_MULTIPLE_RUNS_ON_EVENTS` (Default pull_request, push)
2. Builds will be marked as killed when they "override" another build
environment in docker-compose files is an array: lines start with a dash (-) and key value assignments are done using an equal sign (=)
Co-authored-by: 6543 <6543@obermui.de>
- migrate step conditions back into pipeline syntax, but show 2-4 level in toc to be able to see `when` keywords
- create new backend section in admin docs
- update docusaurus
- remove prefix docker of container / container-image where possible
- replace terms SCM, VCS, Github with [forge](https://en.wikipedia.org/wiki/Forge_(software))
- add darkmode favicon variant
With systems like docker swarm or docker compose it is usually a little awkward to manage secrets.
There is no way to directly inject them into the environment config. So you often have to write your secrets directly into the compose file
There are hacky workarounds such as overriding the entry-point of the container and loading a script which then fetches secrets from /run/secrets and replaces the environment variables, but this becomes very difficult once we are using docker images built from "scratch" (which is a really great practice otherwise) as there is no shell or standard tooling available
This adds a *_FILE variant of their Environment config values to work around this issue.
Signed-off-by: Lukas Bachschwell <lukas@lbsfilm.at>
The old docker hub link directs you either to a login page or to your profile. I think it makes more sense to redirect the user directly to available images in the explore tab
* Added documentation of all configuration options.
* sort some flags
* adjust config docs to current flags
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Anton Bracke <anton@ju60.de>
Some flags where unused and / or unnecessary as they are covered by alternatives implemented in PRs of milestone 0.15.0 and just complicated the setup.
closes#681
* use flag value
* fix test
* sed -i 's/STATUS_CONTEXT/WOODPECKER_STATUS_CONTEXT/g'
* docs
* Update docs/docs/91-migrations.md
Co-authored-by: Anbraten <anton@ju60.de>
* correct minor spellings in the docs
* add warning about artifacts not being shared in multi-pipelines
* highlight note on multi-pipelines docs page
* update mentions of GitLab to use its official notation (camel case)
Refactor
- use constants for strings
- more tests
- move constraint code into own package
Enhance
- all constrains use doublestart (glob pattern matching) now
Co-authored-by: Anbraten <anton@ju60.de>
In the Codeberg CI Matrix channel, there was confusion about the `image` specifying Docker containers. I changed some phrases and hopefully made it clearer to new users.
* Add "Stars over time to README
* Move info from README into docs & link to it
* New CI location
* New screenshot
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
Dropped support for `DRONE_*` environment variables in pipeline steps. Pipeline meta-data can be accessed with `CI_*` variables.
- `CI_*` prefix replaces `DRONE_*`
- `CI` value is now `woodpecker`
- `DRONE=true` has been removed
- Creates `Agent Config` with examples of agents and configuration options
- Updates `Pipeline Syntax`
- About the different types of conditionals
- Sort Order (Global -> Step -> Advanced)
- Move Conditional Steps to own page
Co-authored-by: 6543 <6543@obermui.de>
The goal here is to make consistent use of configuration environment variables prefixed `WOODPECKER_`. Where several variants existed, this PR aims to remove all but one option, leaving the most explicit.
This PR only changes server and agent code, but not documentation, in order to keep the PR digestible. Once we have consensus that this is correct, I'll change docs accordingly.
User (rather: admin) facing changes in this PR:
- In general, support for all server and agent config environment variables (env vars) starting with `DRONE_` is removed. The according `WOODPECKER_*` variables must be used instead.
- The env var `WOODPECKER_HOST` replaces `DRONE_HOST`, and `DRONE_SERVER_HOST`.
- The env var `WOODPECKER_AGENT_SECRET` is used to configure the shared secret which agents use to authenticate against the server. It replaces `WOODPECKER_SECRET`, `DRONE_SECRET`, `WOODPECKER_PASSWORD`, `DRONE_PASSWORD`, and `DRONE_AGENT_SECRET`.
- The env var `WOODPECKER_DATABASE_DRIVER` replaces `DRONE_DATABASE_DRIVER` and `DATABASE_DRIVER`.
- The env var `WOODPECKER_DATABASE_DATASOURCE` replaces `DRONE_DATABASE_DATASOURCE` and `DATABASE_CONFIG`.
Updated license files to be automatically be parseable by tools like [licensechecker](https://github.com/google/licensecheck) or Github. This is for example needed for godoc.
* docs(status): rewrite
status: changed doesn't exist in the current codebase and the
description was just confusing.
* Use consistent language
Co-authored-by: Anbraten <anton@ju60.de>
Co-authored-by: Nulo <git@nulo.in>
Co-authored-by: Anbraten <anton@ju60.de>
# Config resolution
- pipeline-config setting empty / not specified (default): `.woodpecker/` => `.woodpecker.yml` => `.drone.yml`
- pipeline-config setting defined by user: try that file / folder and no fallback (if a user sets some special value that is normally done for some reason)
# Changes
- pipeline-config setting will be empty by default
- remove fallback setting for config loading (simplifies config)
---
closes#133
---
* adjust config fetching mechanism
* default path empty
* remove fallback flag from ui and db
- Replace mkdocs with docosaurus (improved menu structure, ...)
- Structure docs into `Usage` and `Server Setup / Administration`
- Update favicon
- Create new pipeline-syntax page with all options and links to more detailed docs if available
- Add ci to publish to `woodpecker-ci.github.io`
- Deploy docs preview to surge for review
- Update start-page
Co-authored-by: 6543 <6543@obermui.de>
Example:
```yaml
when:
path: '*.md'
```
should match only builds in which the commit added/removed or modified files with the *.md extension
Co-authored-by: 6543 <6543@obermui.de>
