Do not sync repos with forge if the repo is not necessary in DB.
In the DB, only repos that were active once or repos that are currently
active are stored. When trying to enable new repos, the repos list is
fetched from the forge instead and displayed directly. In addition to
this, the forge func `Perm` was removed and is now merged with `Repo`.
Solves a TODO on RepoBatch.
---------
Co-authored-by: Anbraten <anton@ju60.de>
Save which agent is running a task. This is now visible in the admin UI
in the queue and in the agent details screen.
# changes
- [x] save id of agent executing a task
- [x] add endpoint to get tasks of an agent for #999
- [x] show assigned agent-id in queue
- [x] (offtopic) use same colors for queue stats and icons (similar to
the ones used by pipelines)
- [x] (offtopic) use badges for queue labels & dependencies
# Changes
- Adds an admin view to see the whole work-queue of the server.
- The admin can also pause / resume the queue.
- The view is reloading data every 5 seconds automatically.
- The task model from queue got removed in favor of the one from models.
close#1114
As long as the `VersionResponse` type is not changed the check will
fail/pass gracefully
example output:
```
{"level":"error","error":"GRPC version mismatch","time":"2023-03-19T19:49:09+01:00","message":"Server version next-6923e7ab does report grpc version 2 but we only understand 1"}
GRPC version mismatch
```
- allow repo names to be case-insensitive
- improve backend error handling on DB get errors (record not found ->
404, else -> 500)
- replace magic numbers of http response codes
- unify the look and feel of cancel / save buttons on forms and view
them in one line
---------
Co-authored-by: Lauris BH <lauris@nix.lv>
Coding support is likely broken and nobody will ever fix it. Also it
looks like nobody wants to use it, otherwise we would have get some bug
reports.
---------
Co-authored-by: 6543 <6543@obermui.de>
Closes#1582
When `WOODPECKER_FLAT_PERMISSIONS=true` workaround is applied all
permissions are set to false (default) and query never returns any
matches.
This fixes it by always assigning Pull/Push/Admin to true when flatPermissions is enabled.
When a server such as Codeberg has unusually high response time, three
seconds may not be enough to fetch the configuration.
Signed-off-by: Earl Warren <contact@earl-warren.org>
Co-authored-by: 6543 <6543@obermui.de>
closes#101
Added secrets encryption in database
- Google TINK or simple AES as encryption mechanisms
- Keys rotation support on TINK
- Existing SecretService is wrapped by encryption layer
- Encryption can be enabled and disabled at any time
Co-authored-by: Kuzmin Ilya <ilia.kuzmin@indrive.com>
Co-authored-by: 6543 <6543@obermui.de>
This implements #1073, adds .yaml to the accepted endings for woodpecker configs.
This currently adds some more lines to the duplication (tried to compensate by fixing the other duplication in the configFetcher) as the CLI and Server are still separate.
Crude fix to allow to correctly list workspaces for bitbucket cloud
(https://bitbucket.org) and so run a pipeline.
Last year they removed a bunch of deprecated APIs and replaced them with
new ones.
Signed-off-by: Martin Herren <martin.herren@gmail.com>
Co-authored-by: Martin Herren <martin.herren@ecorobotix.com>
Co-authored-by: 6543 <6543@obermui.de>
Closes#1169
Replaces structs that were added inline in hook structs with structs of
the corresponding SDKs. This makes it more readable and error-proof.
Use IDs of the forge to fetch repositories instead of their names and owner names. This improves handling of renamed and transferred repos.
TODO
- [ ] try to support as many forges as possible
- [x] Gogs (no API)
- [ ] Bitbucket Server
- [x] Coding (no API?)
- [x] update repo every time it is fetched or received from the forge
- [x] if repo remote IDs are not available, use owner / name to get it
- [x] handle redirections (redirect a renamed repo to its new path)
- [x] ~~pull all repos once during migration to update ID (?)~~ issue fixed by on-demand loading of remote IDs
- [x] handle redirections in web UI
- [ ] improve handling of hooks after a repo was renamed (currently it checks for a redirection to the repo)
- [x] tests
- [x] `UNIQUE` constraint for remote IDs after migration shouldn't work (all repos have an empty string as remote ID)
close#854close#648 partial
close https://codeberg.org/Codeberg-CI/feedback/issues/46
Possible follow-up PRs
- apply the same scheme on everything fetched from the remote (currently only users)
Co-authored-by: 6543 <6543@obermui.de>
breakout from #934
when new events are added you don't have to worry that pipeline will behave different as it does now with this
Co-authored-by: Anbraten <anton@ju60.de>
at some point (~7years ago) the oauth2 implementation was copied into the code-base and never touched.
We only use it for gitlab the rest is already back using std.
This migrates to the std oauth2 implementation
* Implement database changes and store methods for global and organization secrets
* Add tests for new store methods
* Add organization secret API and UI
* Add global secrets API and UI
* Add suggestions
* Update warning style
* Apply suggestions from code review
Co-authored-by: Anbraten <anton@ju60.de>
* Fix lint warning
Co-authored-by: Anbraten <anton@ju60.de>
* make global environment variables available for pipeline substitution
* lint fixes
* global env support in cli exec; procBuilder tests
* drop GLOBAL_ prefix
* docs
* documentation typo
* Update docs/docs/20-usage/50-environment.md
as suggested by anbraten
Co-authored-by: Anbraten <anton@ju60.de>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Anbraten <anton@ju60.de>
to make it easier for devs to find the right place for code
close#655
Co-authored-by: Anbraten <anton@ju60.de>
Co-authored-by: qwerty287 <80460567+qwerty287@users.noreply.github.com>
- refactor
- create new errors to handle on them
- dedup code
- split server pipeline functionality's into dedicated functions
- add code comments to document what goes on
- add TODOs for next refactor
Officially support labels for pipelines and agents to improve pipeline picking.
* add pipeline labels
* update, improve docs and add migration
* update proto file
---
closes#304 & #860
closes#11
Added support:
1. Environment variable `WOODPECKER_DELETE_MULTIPLE_RUNS_ON_EVENTS` (Default pull_request, push)
2. Builds will be marked as killed when they "override" another build
* Do not filter on linux/amd64 per default & add tests
Tasks with no platform would otherwise not perform on runners with different OS/ARCH combos
Signed-off-by: Lukas Bachschwell <lukas@lbsfilm.at>
Co-authored-by: 6543 <6543@obermui.de>
We previously got the machine hostname for Netrc from the url of the remote, but in cases where the clone-url does not match the api url this can lead to errors.
If the repo was renamed, there's an issue with Gitea: it redirects the /api/v1/repos/<owner>/<repo>/hooks POST request to a GET request at the same URL.
This URL returns the list of all hooks, thus the Gitea SDK can't parse the response into a single gitea.Hook type.
A better error is also visisble if the repo was deleted.
* only calculate time on running builds
* Add updated timestamp into database and use it in frontend
* add more trace logging
* refactor (move grpc unrelated func into related package)
* fix xorm schema
* add todo
Some flags where unused and / or unnecessary as they are covered by alternatives implemented in PRs of milestone 0.15.0 and just complicated the setup.
closes#681
* use flag value
* fix test
* sed -i 's/STATUS_CONTEXT/WOODPECKER_STATUS_CONTEXT/g'
* docs
* Update docs/docs/91-migrations.md
Co-authored-by: Anbraten <anton@ju60.de>
- Add field for image list in Secrets Repo Settings (Web UI)
Simple comma separated input field, split into images array
- validate secret images in backend
- trim spaces and filter empty list items
Signed-off-by: 6543 <6543@obermui.de>
Co-authored-by: 6543 <6543@obermui.de>
- link to specific proc (only general build before)
- set status for all procs (before: only for the whole build on some SCMs)
- set status after restart
- set status to pending after waiting for approval
- make status of gitlab, gitea & github equal
- dedupe status update code
- dedupe `PostBuild` code
close#410, close#297, close#459, close#521
Benefits:
- the webhook delivery history of the drone-ci-service is broken (no way to check if a webhook was successfully delivered by Gitlab)
- drone-ci-service has limited events support (for example no comment or branch deleted event)
- independent from drone integration in general
Fixed problems:
- `c.AbortWithError()` stops the chain, but does not return a message as http response
- only return woodpeckers own messages and hide errors from http response (could be dangerous)
- added missing returns
- log all messages send to the user (at least on debug level)
* write back to webhook caller what happend
* skip sound like an error - it is none change that
* improve hook func
* dedup code & fix bugs that only existed on gated builds
* startBuild use std context
* wordings
Co-authored-by: Anbraten <anton@ju60.de>
* nit
* todo done
Co-authored-by: Anbraten <anton@ju60.de>
* UI: let remove be a remove
* UI: add deactivate repo btn
* Store: DeleteRepo also delete related
* Store: more test coverage
Co-authored-by: 6543 <6543@obermui.de>
Dropped support for `DRONE_*` environment variables in pipeline steps. Pipeline meta-data can be accessed with `CI_*` variables.
- `CI_*` prefix replaces `DRONE_*`
- `CI` value is now `woodpecker`
- `DRONE=true` has been removed
Resolve some todos in server/model:
* Move persistent queue into its own package
* Create Types: StatusValue, SCMKind, RepoVisibly
* Rename struct Repo fields: SCMKind, IsSCMPrivate
Rewrite of the UI using Typescript, Vue3, Windicss and Vite. The design should be close to the current one with some changes:
- latest pipeline in a sidebar on the right
- secrets and registry as part of the repo-settings (secrets and registry entries shouldn't be used as much so they can be "hidden" under settings IMO)
- start page shows list of active repositories with button to enable / add new ones (currently you see all repositories and in most cases you only add new repositories once in a while)
test for a bug where pipelines that use `depends_on` were not built in multi-pipeline builds. The problem is that pipelines names keep a leading `'/'` when the pipeline path does not have a trailing `'/'`.
This is a preparation for #245 and adds a new endpoint (`.js``file) which can be included by the web-ui to get some config and credentials like the user profile or access-token.
The goal here is to make consistent use of configuration environment variables prefixed `WOODPECKER_`. Where several variants existed, this PR aims to remove all but one option, leaving the most explicit.
This PR only changes server and agent code, but not documentation, in order to keep the PR digestible. Once we have consensus that this is correct, I'll change docs accordingly.
User (rather: admin) facing changes in this PR:
- In general, support for all server and agent config environment variables (env vars) starting with `DRONE_` is removed. The according `WOODPECKER_*` variables must be used instead.
- The env var `WOODPECKER_HOST` replaces `DRONE_HOST`, and `DRONE_SERVER_HOST`.
- The env var `WOODPECKER_AGENT_SECRET` is used to configure the shared secret which agents use to authenticate against the server. It replaces `WOODPECKER_SECRET`, `DRONE_SECRET`, `WOODPECKER_PASSWORD`, `DRONE_PASSWORD`, and `DRONE_AGENT_SECRET`.
- The env var `WOODPECKER_DATABASE_DRIVER` replaces `DRONE_DATABASE_DRIVER` and `DATABASE_DRIVER`.
- The env var `WOODPECKER_DATABASE_DATASOURCE` replaces `DRONE_DATABASE_DATASOURCE` and `DATABASE_CONFIG`.
* pass context down to remote clients
* make tests work
* add ctx to Refresh() and use it
* bitbucketserver
* code format
* plugin interface: add todo context
* solve todo
* RM TODO by using context.WithTimeout
* refactor & fix
* Apply suggestions from code review
Co-authored-by: Anbraten <anton@ju60.de>
* go fmt
* Update server/remote/coding/coding.go
Co-authored-by: Anbraten <anton@ju60.de>
Co-authored-by: Anbraten <anton@ju60.de>
By adding a new ENV variable called `WOODPECKER_WWW_PROXY` it is possible to serve a webinterface via a proxy configured by the `WOODPECKER_WWW_PROXY` value for development instead of serving the interface from the bundled code or from some folder location as the old `WOODPECKER_WWW` option allowed. Using a proxy allows developing the UI with hot-reloading.
#299 who closed#133 did not take into account, that that gitea (and eventually) other forges do return 200 and empty string if file was not found - this make configFetcher more resilient
* Move cncd/{logging,pubsub,queue}/ to server/{logging,pubsub,queue}/
* Update REAMDEs and include history
Co-authored-by: Anbraten <anton@ju60.de>
Co-authored-by: Anbraten <anton@ju60.de>
* move api code to server/api
* move grpc server for agent communication to server/grpc
* move server.Config to server/config.go as it is used by both server/api and server/grpc
* move shared code used by server/api and server/grpc to server/shared
* delete obsolete cncd code
jsonrpc2 client+server, and the utils that used them (piped, pipec)
are not used anymore.
jsonrpc2 was replaced with grpc
piped+pipec were replaced by agents
* delete duplicate section in swagger file
* comment typos
* go mod tidy (websocket+jsonrpc2 no longer used)
* go mod vendor
Co-authored-by: Anbraten <anton@ju60.de>
With #299 the `sanitizePath` has a bug now. If no special config (empty string) has been set it falls back to default values, but the `sanitizePath` function the used the pipelince-config-path to sanitize pipeline names.
# Config resolution
- pipeline-config setting empty / not specified (default): `.woodpecker/` => `.woodpecker.yml` => `.drone.yml`
- pipeline-config setting defined by user: try that file / folder and no fallback (if a user sets some special value that is normally done for some reason)
# Changes
- pipeline-config setting will be empty by default
- remove fallback setting for config loading (simplifies config)
---
closes#133
---
* adjust config fetching mechanism
* default path empty
* remove fallback flag from ui and db
closes#271
- filter pipeline config folders for `.yml` and `.yaml` files
- improve `fetchConfig` tests
- update remote mock and correct wrong folder name `mock` => `mocks` to match package name
- fix: return correct filename for fallback
- improve config loading by checking if folder or not before sending api call
Example:
```yaml
when:
path: '*.md'
```
should match only builds in which the commit added/removed or modified files with the *.md extension
Co-authored-by: 6543 <6543@obermui.de>
Woodpecker should use configured server URL when creating self
URL-s for VCS UI and not depend on URL read from request. In
case of internal calls like answers for webhooks from Gitea
on backend, address in URL may be internal and this causes
Gitea to display CI status for commit with intenal URL
to woodpecker not external one.
