closes#1801closes#1815closes#1144
closes #983
closes #557closes#1827
regression of #1791
# TODO
- [x] adjust log model
- [x] add migration for logs
- [x] send log line via grpc using step-id
- [x] save log-line to db
- [x] stream log-lines to UI
- [x] use less structs for log-data
- [x] make web UI work
- [x] display logs loaded from db
- [x] display streaming logs
- [ ] ~~make migration work~~ -> dedicated pull (#1828)
# TESTED
- [x] new logs are stored in database
- [x] log retrieval via cli (of new logs) works
- [x] log streaming works (tested via curl & webui)
- [x] log retrieval via web (of new logs) works
---------
Co-authored-by: 6543 <6543@obermui.de>
# Summary
This PR drops the outdated former swagger.yaml/json and introduced
automatic API document generation from Go code.
The generated code is also used to generate documentation/markdown for
the community page,
as well as enable the Woodpecker server to serve a Swagger Web UI for
manual tinkering.
I did opt-in for gin-swagger, a middleware for the Gin framework, to
ease implementation and have a sophisticated output.
This middleware only produces Swagger v2 specs. AFAIK the newer OpenApi
3x tooling is not yet that mature,
so I guess that's fine for now.
## Implemenation notes
- former swagger.json files removed
- former // swagger godocs removed
- introduced new dependency gin-swagger, which uses godoc annotations on
top of Gin Handler functions.
- reworked Makefile to automatically generate Go code for the server
- introduce new dependency go-swagger, to generate Markdown for
documentation purposes
- add a Swagger Web UI, incl. capabilities for manual API exploration
- consider relative root paths in the implementation
- write documentation for all exposed API endpoints
- incl. API docs in the community website (auto-generated)
- provide developer documentation, for the Woodpecker authors
- no other existing logic/code was intentionally changed
---------
close#292
---------
Co-authored-by: qwerty287 <80460567+qwerty287@users.noreply.github.com>
Co-authored-by: 6543 <6543@obermui.de>
Gogs support is broken (and we won't fix it because we don't care about
it...) because it does not support OAuth, at least after we introduced
the new Vue UI.
See:
77d830d5b5/server/forge/gogs/gogs.go (L84)
This route is not present in the new UI.
closes#1636closes#1429
supersedes #1586
Uses a different approach: just take the index.html compiled by vite and
replace the paths to js and other files using regex. This is not
compatible with the dev proxy which is also the reason why we can't use
go templates for this.
Do not sync repos with forge if the repo is not necessary in DB.
In the DB, only repos that were active once or repos that are currently
active are stored. When trying to enable new repos, the repos list is
fetched from the forge instead and displayed directly. In addition to
this, the forge func `Perm` was removed and is now merged with `Repo`.
Solves a TODO on RepoBatch.
---------
Co-authored-by: Anbraten <anton@ju60.de>
Coding support is likely broken and nobody will ever fix it. Also it
looks like nobody wants to use it, otherwise we would have get some bug
reports.
---------
Co-authored-by: 6543 <6543@obermui.de>
When a server such as Codeberg has unusually high response time, three
seconds may not be enough to fetch the configuration.
Signed-off-by: Earl Warren <contact@earl-warren.org>
Co-authored-by: 6543 <6543@obermui.de>
closes#101
Added secrets encryption in database
- Google TINK or simple AES as encryption mechanisms
- Keys rotation support on TINK
- Existing SecretService is wrapped by encryption layer
- Encryption can be enabled and disabled at any time
Co-authored-by: Kuzmin Ilya <ilia.kuzmin@indrive.com>
Co-authored-by: 6543 <6543@obermui.de>
closes#11
Added support:
1. Environment variable `WOODPECKER_DELETE_MULTIPLE_RUNS_ON_EVENTS` (Default pull_request, push)
2. Builds will be marked as killed when they "override" another build
With systems like docker swarm or docker compose it is usually a little awkward to manage secrets.
There is no way to directly inject them into the environment config. So you often have to write your secrets directly into the compose file
There are hacky workarounds such as overriding the entry-point of the container and loading a script which then fetches secrets from /run/secrets and replaces the environment variables, but this becomes very difficult once we are using docker images built from "scratch" (which is a really great practice otherwise) as there is no shell or standard tooling available
This adds a *_FILE variant of their Environment config values to work around this issue.
Signed-off-by: Lukas Bachschwell <lukas@lbsfilm.at>
We previously got the machine hostname for Netrc from the url of the remote, but in cases where the clone-url does not match the api url this can lead to errors.
* Added documentation of all configuration options.
* sort some flags
* adjust config docs to current flags
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Anton Bracke <anton@ju60.de>
Some flags where unused and / or unnecessary as they are covered by alternatives implemented in PRs of milestone 0.15.0 and just complicated the setup.
closes#681
* use flag value
* fix test
* sed -i 's/STATUS_CONTEXT/WOODPECKER_STATUS_CONTEXT/g'
* docs
* Update docs/docs/91-migrations.md
Co-authored-by: Anbraten <anton@ju60.de>
Resolve some todos in server/model:
* Move persistent queue into its own package
* Create Types: StatusValue, SCMKind, RepoVisibly
* Rename struct Repo fields: SCMKind, IsSCMPrivate
#494 introduced a bug, where a migration function can remove the sqlite3 file outside of the mounted docker volume.
that would result in a data lose after a container recreate.
this fix it by only rename the file if in same folder else just use the old path as fallback and put warnings into the log
Co-authored-by: Anbraten <anton@ju60.de>
This is a preparation for #245 and adds a new endpoint (`.js``file) which can be included by the web-ui to get some config and credentials like the user profile or access-token.
The goal here is to make consistent use of configuration environment variables prefixed `WOODPECKER_`. Where several variants existed, this PR aims to remove all but one option, leaving the most explicit.
This PR only changes server and agent code, but not documentation, in order to keep the PR digestible. Once we have consensus that this is correct, I'll change docs accordingly.
User (rather: admin) facing changes in this PR:
- In general, support for all server and agent config environment variables (env vars) starting with `DRONE_` is removed. The according `WOODPECKER_*` variables must be used instead.
- The env var `WOODPECKER_HOST` replaces `DRONE_HOST`, and `DRONE_SERVER_HOST`.
- The env var `WOODPECKER_AGENT_SECRET` is used to configure the shared secret which agents use to authenticate against the server. It replaces `WOODPECKER_SECRET`, `DRONE_SECRET`, `WOODPECKER_PASSWORD`, `DRONE_PASSWORD`, and `DRONE_AGENT_SECRET`.
- The env var `WOODPECKER_DATABASE_DRIVER` replaces `DRONE_DATABASE_DRIVER` and `DATABASE_DRIVER`.
- The env var `WOODPECKER_DATABASE_DATASOURCE` replaces `DRONE_DATABASE_DATASOURCE` and `DATABASE_CONFIG`.
By adding a new ENV variable called `WOODPECKER_WWW_PROXY` it is possible to serve a webinterface via a proxy configured by the `WOODPECKER_WWW_PROXY` value for development instead of serving the interface from the bundled code or from some folder location as the old `WOODPECKER_WWW` option allowed. Using a proxy allows developing the UI with hot-reloading.
* Move cncd/{logging,pubsub,queue}/ to server/{logging,pubsub,queue}/
* Update REAMDEs and include history
Co-authored-by: Anbraten <anton@ju60.de>
Co-authored-by: Anbraten <anton@ju60.de>
* move api code to server/api
* move grpc server for agent communication to server/grpc
* move server.Config to server/config.go as it is used by both server/api and server/grpc
* move shared code used by server/api and server/grpc to server/shared
