Commit graph

134 commits

Author SHA1 Message Date
Anbraten
c464f857ae
Remove unused file system api (#1791)
Co-authored-by: 6543 <6543@obermui.de>
2023-05-31 18:03:03 +02:00
qwerty287
6d2240b2e6
Identify users using their remote ID (#1732) 2023-05-11 05:19:35 +02:00
qwerty287
0f9188597e
Initiate Pagination Implementation for API and Infinite Scroll in UI (#1651)
- Add pagination support to the API endpoints that return lists of items
- Adjust UI to enable infinite scrolling via pagination
2023-04-30 03:40:13 +02:00
qwerty287
9b32d021b3
Consistent status on delete (#1703)
Closes #1675
2023-04-15 15:22:39 +02:00
qwerty287
a06d3e1a61
Stable sort order for DB lists (#1702) 2023-04-11 09:33:27 +02:00
qwerty287
0970f35df5
Do not store inactive repos (#1658)
Do not sync repos with forge if the repo is not necessary in DB.

In the DB, only repos that were active once or repos that are currently
active are stored. When trying to enable new repos, the repos list is
fetched from the forge instead and displayed directly. In addition to
this, the forge func `Perm` was removed and is now merged with `Repo`.

Solves a TODO on RepoBatch.

---------

Co-authored-by: Anbraten <anton@ju60.de>
2023-03-21 23:01:59 +01:00
Anbraten
f13ffc2c8f
Save agent-id for tasks and add endpoint to get agent tasks (#1631)
Save which agent is running a task. This is now visible in the admin UI
in the queue and in the agent details screen.

# changes
- [x] save id of agent executing a task
- [x] add endpoint to get tasks of an agent for #999 
- [x] show assigned agent-id in queue
- [x] (offtopic) use same colors for queue stats and icons (similar to
the ones used by pipelines)
- [x] (offtopic) use badges for queue labels & dependencies


![image](https://user-images.githubusercontent.com/6918444/226541271-23f3b7b2-7a08-45c2-a2e6-1c7fc31b6f1d.png)
2023-03-21 15:10:43 +02:00
Anbraten
2337f1854a
Add queue details UI for admins (#1632)
# Changes
- Adds an admin view to see the whole work-queue of the server. 
- The admin can also pause / resume the queue. 
- The view is reloading data every 5 seconds automatically.
- The task model from queue got removed in favor of the one from models.
2023-03-20 04:50:56 +01:00
6543
92614dfb1e
Agent check gRPC version against server (#1653)
close #1114

As long as the `VersionResponse` type is not changed the check will
fail/pass gracefully

example output:
```
{"level":"error","error":"GRPC version mismatch","time":"2023-03-19T19:49:09+01:00","message":"Server version next-6923e7ab does report grpc version 2 but we only understand 1"}
GRPC version mismatch
```
2023-03-19 22:42:21 +01:00
qwerty287
f582ad3159
Various enhancements in configuration (#1645)
- backends: move to cli flags instead of os.Getenv
- ssh: support 2fa with key and password
- allow to set grpc jwt secret (solves todo)
- allow to set default and max timeout (solves todo)

Closes https://github.com/woodpecker-ci/woodpecker/issues/896
Closes https://github.com/woodpecker-ci/woodpecker/issues/1131
2023-03-19 20:24:43 +01:00
qwerty287
ade8e6d010
Various fixes and improvements (#1643)
- allow repo names to be case-insensitive
- improve backend error handling on DB get errors (record not found ->
404, else -> 500)
- replace magic numbers of http response codes
- unify the look and feel of cancel / save buttons on forms and view
them in one line

---------

Co-authored-by: Lauris BH <lauris@nix.lv>
2023-03-19 13:52:58 +01:00
qwerty287
42a115e19e
Add PR pipeline list (#1641)
Instead of viewing PR pipelines in the branches lists, add a separate
list for them. The API endpoint for PRs supports pagination (thus I
added a lot of pagination-related stuff), the UI doesn't yet though.


![wp](https://user-images.githubusercontent.com/80460567/226099133-bb4935d6-c357-4d73-8465-9157e25c0d82.png)

Closes #1619 

Extends this part of #1640

---------

Co-authored-by: Anbraten <anton@ju60.de>
2023-03-19 10:43:57 +01:00
Anbraten
277a839157
Add users UI for admins (#1634)
Co-authored-by: Lauris BH <lauris@nix.lv>
2023-03-18 21:21:20 +01:00
Anbraten
fa5b0fb96e
Fix linter (#1647) 2023-03-18 20:35:27 +01:00
Lukasz
dc90f97524
Persist DepStatus of tasks (#1610) 2023-03-11 01:58:28 +01:00
6543
18d3139e9e
Use modern error handling and enforce it via lint (#1327)
Co-authored-by: Anbraten <anton@ju60.de>
2023-02-02 00:08:02 +01:00
Anbraten
d96032349a
Store an agents list and add agent heartbeats (#1189)
Co-authored-by: 6543 <6543@obermui.de>
2023-01-28 14:13:04 +01:00
6543
c276bc8937
use same sql for set default statement on mysql/mariadb and postgres (#1540)
introduced by #1027 so it does not need a changelog or backport
2023-01-14 12:52:41 +02:00
antomy-gc
6516a28cdd
Secrets encryption in database (#1475)
closes #101

Added secrets encryption in database

- Google TINK or simple AES as encryption mechanisms
- Keys rotation support on TINK
- Existing SecretService is wrapped by encryption layer
- Encryption can be enabled and disabled at any time

Co-authored-by: Kuzmin Ilya <ilia.kuzmin@indrive.com>
Co-authored-by: 6543 <6543@obermui.de>
2023-01-12 20:59:07 +01:00
Joaquin
482678daf5
Fix pipeline feed not updated by new events (#1424)
Co-authored-by: Anbraten <anton@ju60.de>
2022-11-18 10:59:24 +01:00
Anbraten
de8ea95dd3
Rename forge-id to forge-remote-id (#1418) 2022-11-15 15:01:23 +01:00
Josh Soref
023d03dd61
Spelling (#1405)
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2022-11-09 08:12:17 +01:00
6543
18311d4360
Split and refactor (#1394)
Closes #974
2022-11-06 12:44:04 +01:00
qwerty287
3372d1a87c
Rename remote to forge (#1357)
As of #745

Co-authored-by: Anbraten <anton@ju60.de>
2022-11-05 00:35:06 +01:00
Joonhyeok Ahn (Joon)
aed3a80287
Migrate sql querys to xorm query builder (#1356)
Co-authored-by: 6543 <6543@obermui.de>
2022-10-31 16:08:57 +01:00
qwerty287
8f183c82a8
Support changed files for Gitea PRs (#1342)
- add tests to fetch changed files
- ignore error if gitea version is to low
- adjust docs accordingly

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lauris BH <lauris@nix.lv>
2022-10-28 19:17:30 +02:00
Harikesh00
36e42914fa
Renamed procs/jobs to steps in code (#1331)
Renamed `procs` to `steps` in code for the issue #1288

Co-authored-by: Harikesh Prajapati <harikesh.prajapati@druva.com>
Co-authored-by: qwerty287 <ndev@web.de>
Co-authored-by: qwerty287 <80460567+qwerty287@users.noreply.github.com>
Co-authored-by: 6543 <6543@obermui.de>
2022-10-28 17:38:53 +02:00
qwerty287
f88c70b55e
Rename to pipeline in DB and JSONs (#1296)
Closes #1282 

Follow-up to #1224, addresses #745

- changes JSON fields
- adds migration to rename columns
- fixes some comments
2022-10-22 15:54:43 +02:00
Joonhyeok Ahn (Joon)
186aee61cf
Check if repo exists before creating pipeline (#1297)
Close #1257

make sure the repo exists first before triggering the pipeline
2022-10-22 01:34:11 +02:00
qwerty287
849e05bb8b
Rename build to pipeline in code (#1224)
Ref:  #745

Co-authored-by: Anbraten <anton@ju60.de>
Co-authored-by: 6543 <6543@obermui.de>
2022-10-18 03:24:12 +02:00
Avinil Bedarkar
493ec45be6
Return return 404 if registry to delete do not exist (#1278)
Closes #524

Co-authored-by: 6543 <6543@obermui.de>
2022-10-18 01:48:04 +02:00
Avinil Bedarkar
11f37f4649
Converting all inputs which are coming as secret to lowercase (#1276)
Closes #926

Co-authored-by: qwerty287 <80460567+qwerty287@users.noreply.github.com>
2022-10-16 14:58:13 +02:00
6543
612eee8269
Sort secret lists and events (#1223)
close  #1176
2022-09-29 22:46:20 +02:00
qwerty287
52d3652f2e
Fetch repositories with remote ID if possible (#1078)
Use IDs of the forge to fetch repositories instead of their names and owner names. This improves handling of renamed and transferred repos.

TODO

- [ ] try to support as many forges as possible
    - [x] Gogs (no API)
    - [ ] Bitbucket Server
    - [x] Coding (no API?)
- [x] update repo every time it is fetched or received from the forge
- [x] if repo remote IDs are not available, use owner / name to get it
- [x] handle redirections (redirect a renamed repo to its new path)
- [x] ~~pull all repos once during migration to update ID (?)~~ issue fixed by on-demand loading of remote IDs
- [x] handle redirections in web UI
- [ ] improve handling of hooks after a repo was renamed (currently it checks for a redirection to the repo)
- [x] tests
- [x] `UNIQUE` constraint for remote IDs after migration shouldn't work (all repos have an empty string as remote ID)

close #854
close #648 partial
close https://codeberg.org/Codeberg-CI/feedback/issues/46

Possible follow-up PRs
- apply the same scheme on everything fetched from the remote (currently only users)

Co-authored-by: 6543 <6543@obermui.de>
2022-09-05 17:08:51 +02:00
Anbraten
5ca7ede9e4
Improve logging (#1158)
* switch default log level to info add start message and cleanup server start
* refactor code
* fix agent debug / trace logging
2022-09-03 20:41:23 +02:00
6543
18c1807f4f
Create mock for store and add unit tests for cron (#1146)
* pass remote on init as argument like store

* mock store

* TestCreateBuild
2022-09-01 01:19:49 +02:00
6543
383f273392
Add cron feature (#934)
https://woodpecker-ci.org/docs/usage/cron

Co-authored-by: Anbraten <anton@ju60.de>
Co-authored-by: qwerty287 <80460567+qwerty287@users.noreply.github.com>
2022-09-01 00:36:32 +02:00
6543
08a99152d6
Dedup code and migrate away from deprecated funcs (#1141)
Co-authored-by: Anbraten <anton@ju60.de>
2022-08-30 01:14:07 +02:00
6543
ae771a9931
Fix migration number (#1134) 2022-08-27 23:58:52 +02:00
6543
f21d854114
Move away from deprecated go funcs (#1123) 2022-08-25 08:39:19 +02:00
Lauris BH
1ac2c42652
Add global and organization secrets (#1027)
* Implement database changes and store methods for global and organization secrets

* Add tests for new store methods
* Add organization secret API and UI
* Add global secrets API and UI

* Add suggestions

* Update warning style

* Apply suggestions from code review

Co-authored-by: Anbraten <anton@ju60.de>

* Fix lint warning

Co-authored-by: Anbraten <anton@ju60.de>
2022-08-14 13:48:53 +02:00
Lauris BH
550aefead4
Make returned proc list to be returned always in correct order (#1060) 2022-08-02 18:34:16 +02:00
Simon Vieille
6351684070
Update type of 'log_data' from blob to longblob (#1050)
* Update type of 'log_data' from blob to longblob

* Add migration task to update log_data type
2022-07-29 14:57:18 +02:00
6543
08479390ff
Use editorconfig-checker (#982) 2022-06-17 12:03:34 +02:00
Anbraten
cc30db44ac
Use asym key to sign webhooks (#916)
* use async key pair for webhooks

* fix tests

* fix linter

* improve code

* add key pair to database

* undo some changes

* more undo

* improve docs

* add api-endpoint

* add signaturne api endpoint

* fix error

* fix linting and test

* fix lint

* add test

* migration 006

* no need for migration

* replace httsign lib

* fix lint

Co-authored-by: 6543 <6543@obermui.de>
2022-06-01 20:06:27 +02:00
Anbraten
0b2b776ed8
Remove senders model (#923)
* remove senders
* drop table
2022-05-17 21:27:44 +02:00
6543
c350af645b
Move Badge & CCMenue into own packages (#908)
* move badge generation into own package

* move ccmenue away from models

* fix misspell
2022-05-14 14:30:09 +02:00
Anbraten
bdb007e064
Auto cancel blocked pipelines (#905) 2022-05-11 07:45:21 +02:00
Zav Shotan
7313de2b1d
Add support for superseding runs (#831)
closes #11

Added support:
1. Environment variable `WOODPECKER_DELETE_MULTIPLE_RUNS_ON_EVENTS` (Default pull_request, push)
2. Builds will be marked as killed when they "override" another build
2022-05-09 11:26:09 +02:00
Johan Van de Wauw
8fa2a5efe4
Fix various typos (#835) 2022-03-09 01:44:08 +01:00
6543
0d463ca467
Let non required migration tasks fail and continue (#729) 2022-01-31 15:50:10 +01:00
6543
071bd7418a
Fix tests and pipeline (#732)
* fix pipeline

* use EqualStringSlice to compare slices in tests

* full test coverage

* tests: wait for mysql/postgres to sync ...
2022-01-31 14:39:53 +01:00
Anbraten
401072abb1
Support ChangedFiles for Github & Gitlab PRs and Gitlab pushes (#697) 2022-01-17 23:46:59 +01:00
6543
c186e2fa8c
Update dev-docs & improve migration code (#670) 2022-01-08 20:21:22 +01:00
6543
acbde30c3b
Lint code with gofumpt (#664) 2022-01-06 07:44:14 +01:00
6543
2f91bdd4a0
gofumpt -w -l -extra . (#661) 2022-01-05 21:50:23 +01:00
6543
0aefdc9978
Add tests framework for storage migration (#630)
* add test sqlite db

* migration tests for sqlite

* fix pipeline config

* fix tests to other dbms

* fix postgres & mysql
2021-12-20 16:15:21 +01:00
6543
b2ce1f5da5
fix sqlite migration on column drop of uncommon schemas (#629)
* fix sqlite schema normalising

* sqlite robuster alg to extract columns form schema
2021-12-20 09:59:27 +01:00
6543
f4c73792f3
Calculate build number on creation (#615)
* Calculate build number on creation
* Delete repo.Counter (form code and database)
2021-12-18 20:37:51 +01:00
Anbraten
fe6c999160
Let remove be a remove (#593)
* UI: let remove be a remove
* UI: add deactivate repo btn
* Store: DeleteRepo also delete related
* Store: more test coverage

Co-authored-by: 6543 <6543@obermui.de>
2021-12-11 16:03:14 +01:00
6543
4cbdacb21c
Nits Collected over last month (#595)
- add coverage.out
- add context queue
- fix misspell
- sanitize config: WOODPECKER_GITEA_URL
- storage backend migration should have no xorm session within migration function
2021-12-11 13:15:04 +01:00
6543
7ab33f62f7
fix unit tests 2021-12-11 02:41:28 +01:00
Anbraten
cb97b39c82
Fix pr secret event names (#592)
* Fix pr secret event names
* Add validation for secret events
2021-12-11 02:37:40 +01:00
Lukas
680d003a29
Add linter revive (#554)
* Add linter revive

* Add underscore to variable name to prevent shadowing

* Remove unnecessary leading underscore

* Revert changes to vendor file

* export ConfigFetcher as interface

* no 'yoda conditions'

* rename envsubst

Co-authored-by: 6543 <6543@obermui.de>
2021-12-01 14:22:06 +01:00
6543
34a17f8219
Postgress can exec Sync2 after migration only in own transactions (#544)
(postgress do not allow us to use sync2 twice in one transaction at all)
2021-11-27 12:02:52 +01:00
6543
ff8ad5bb83
Fix registry delete (#532)
- Make make it possible to delete `"§4fda`
- Reject non urls for registry-address
- Fix #520
2021-11-25 19:04:36 +01:00
Lukas
25bf91bd37
Add linter whitespace (#531) 2021-11-24 01:31:11 +01:00
6543
f454371e56
store migrations: cache migrations and add note (#522)
* Cache migrations instead of query each time
* Add a note of migration execute order 

Co-authored-by: Anbraten <anton@ju60.de>
2021-11-23 05:06:46 +01:00
6543
51617e7f86
Rename struct field and add new types into server/model's (#523)
Resolve some todos in server/model:
 * Move persistent queue into its own package
 * Create Types: StatusValue, SCMKind, RepoVisibly
 * Rename struct Repo fields: SCMKind, IsSCMPrivate
2021-11-22 12:55:13 +01:00
6543
07b32fa0fe
Update database in one transaction on syncing user repositorys (#513)
* userSyncer Sync commit changes in one transaction
* cover new cases with unit tests
2021-11-18 15:42:18 +01:00
6543
4891f104f0
Update only relevant cols on repo sync (#512)
close #511
2021-11-15 00:55:37 +01:00
6543
2524c6900c
Fix reggression, save not activated repos to database on sync again (#510)
* Fix #508
* Add testcase

Unrelated nits:
* Rm one index (unique will already create an index)
* Do not drop  on UpdateUser
2021-11-14 22:13:59 +01:00
6543
ca8e215cfa
Migrate to Xorm (#474)
close #234

* Migrate store
* Migrate tests
* Rewrite migrations
* Init fresh DB in on step
* Rm old stuff (meddler, sql files, dead code, ...)
2021-11-13 20:18:06 +01:00
6543
aca5fddcf3
Use Goblin Assert as intended (#501)
this allow for better debugging if an error occur
2021-11-04 14:42:25 +01:00
6543
965235e421
Remove some wrapper and make code more redable (#478)
* meaningful var names
* no context wrapper, use store directly
* retrieve store from context once
* rm store.GetRepoOwnerName
2021-10-28 11:12:58 +02:00
Anbraten
3c5827f08a
Fix repo access (#476)
* fix repo access

* fix permission syncing
2021-10-26 21:29:30 +02:00
6543
5990d32fd3
More logging and refactor (#457)
* only use "context"
* enable 'h2' support at server
* trace log  remote and database config
* log loglevel on start
2021-10-19 11:44:49 +02:00
John Olheiser
4276a04f0c
Move entirely to zerolog (#426)
Completely switch to zerolog

(Remove usage of logrus and std logger)

Signed-off-by: jolheiser <john.olheiser@gmail.com>
Co-authored-by: 6543 <6543@obermui.de>
2021-10-12 09:25:13 +02:00
luzpaz
9b687a923e
Fix various typos (#416)
Fix various typos found via `codespell -q 3 -S vendor -L pullrequest,pullrequests`
2021-10-08 18:35:56 +02:00
Anbraten
bb37836600
Replace drone strings with woodpecker (#391)
Co-authored-by: 6543 <6543@obermui.de>
2021-10-02 10:59:34 +02:00
Marian Steinbach
17b8867b96
Clean up config environment variables for server and agent (#218)
The goal here is to make consistent use of configuration environment variables prefixed `WOODPECKER_`. Where several variants existed, this PR aims to remove all but one option, leaving the most explicit.

This PR only changes server and agent code, but not documentation, in order to keep the PR digestible. Once we have consensus that this is correct, I'll change docs accordingly.

User (rather: admin) facing changes in this PR:

- In general, support for all server and agent config environment variables (env vars) starting with `DRONE_` is removed. The according `WOODPECKER_*` variables must be used instead.
- The env var `WOODPECKER_HOST` replaces `DRONE_HOST`, and `DRONE_SERVER_HOST`.
- The env var `WOODPECKER_AGENT_SECRET` is used to configure the shared secret which agents use to authenticate against the server. It replaces `WOODPECKER_SECRET`, `DRONE_SECRET`, `WOODPECKER_PASSWORD`, `DRONE_PASSWORD`, and `DRONE_AGENT_SECRET`.
- The env var `WOODPECKER_DATABASE_DRIVER` replaces `DRONE_DATABASE_DRIVER` and `DATABASE_DRIVER`.
- The env var `WOODPECKER_DATABASE_DATASOURCE` replaces `DRONE_DATABASE_DATASOURCE` and `DATABASE_CONFIG`.
2021-09-28 15:43:44 +02:00
6543
a94807efeb
Refactor: move model/ to server/model/ (#366) 2021-09-27 19:51:55 +02:00
6543
0bd10fa507
Cleanup Code (#348)
* Fix "Empty slice declaration using a literal"
* Fix "collides with imported package name"
* Remove unused code in pipeline
* Remove unused oauth2.providerAuthHeaderWorks()
* Add TODOs
* Format Code
* Cleanup doublestar import
* Migrate deprecated functions

Co-authored-by: Anbraten <anton@ju60.de>
2021-09-24 16:29:26 +02:00
Jacob Floyd
5fe712cca6
Move package store/ to server/store/ (#341)
* Refactor: move store/ to server/store/

* fix pipeline for moved tests

Co-authored-by: 6543 <6543@obermui.de>
2021-09-23 13:33:59 +02:00