Commit graph

73 commits

Author SHA1 Message Date
qwerty287
3033abc3b4
Add own workflow model (#1784)
Closes #1287

---------

Co-authored-by: 6543 <6543@obermui.de>
2023-06-27 18:01:18 +02:00
Anbraten
ff01a9ff1d
Access repos by their ids (#1691)
closes #1295 
closes #648

# TODO
- [x] add new routes with `:repoID`
- [x] load repo in middleware using `:repoID` if present
- [x] update UI routes `:owner/:name` to `:repoID`
- [x] load repos using id in UI
- [x] add lookup endpoint `:owner/:name` to `:repoID`
- [x] redirect `:owner/:name` to `:repoID` in UI
- [x] use badge with `:repoID` route in UI
- [x] update `woodpecker-go`
- [x] check cli
- [x] add migrations / deprecation notes
- [x] check if #648 got solved directly
- [x] Test
  - [x] create repo
  - [x] repo pages
  - [x] ui redirects
  - [x] forge status links
2023-06-13 01:07:52 +02:00
qwerty287
5d74174bc3
Drop old columns (#1838) 2023-06-07 19:22:44 +02:00
Anbraten
556607b525
Rework log streaming and related functions (#1802)
closes #1801
closes #1815 
closes #1144
closes  #983
closes  #557
closes #1827
regression of #1791

# TODO
- [x] adjust log model
- [x] add migration for logs
- [x] send log line via grpc using step-id
- [x] save log-line to db
- [x] stream log-lines to UI
- [x] use less structs for log-data
- [x] make web UI work
  - [x] display logs loaded from db
  - [x] display streaming logs
- [ ] ~~make migration work~~ -> dedicated pull (#1828)

# TESTED
- [x] new logs are stored in database
- [x] log retrieval via cli (of new logs) works
- [x] log streaming works (tested via curl & webui)
- [x] log retrieval via web (of new logs) works

---------

Co-authored-by: 6543 <6543@obermui.de>
2023-06-06 09:52:08 +02:00
Martin W. Kirst
14177635b6
Update swagger API specification (#1782)
# Summary

This PR drops the outdated former swagger.yaml/json and introduced
automatic API document generation from Go code.
The generated code is also used to generate documentation/markdown for
the community page,
as well as enable the Woodpecker server to serve a Swagger Web UI for
manual tinkering.

I did opt-in for gin-swagger, a middleware for the Gin framework, to
ease implementation and have a sophisticated output.
This middleware only produces Swagger v2 specs. AFAIK the newer OpenApi
3x tooling is not yet that mature,
so I guess that's fine for now.

## Implemenation notes

- former swagger.json files removed
- former // swagger godocs removed
- introduced new dependency gin-swagger, which uses godoc annotations on
top of Gin Handler functions.
- reworked Makefile to automatically generate Go code for the server
- introduce new dependency go-swagger, to generate Markdown for
documentation purposes
- add a Swagger Web UI, incl. capabilities for manual API exploration
- consider relative root paths in the implementation
- write documentation for all exposed API endpoints
- incl. API docs in the community website (auto-generated)
- provide developer documentation, for the Woodpecker authors
- no other existing logic/code was intentionally changed

---------

close #292

---------

Co-authored-by: qwerty287 <80460567+qwerty287@users.noreply.github.com>
Co-authored-by: 6543 <6543@obermui.de>
2023-06-03 21:38:36 +02:00
Anbraten
c464f857ae
Remove unused file system api (#1791)
Co-authored-by: 6543 <6543@obermui.de>
2023-05-31 18:03:03 +02:00
qwerty287
6d2240b2e6
Identify users using their remote ID (#1732) 2023-05-11 05:19:35 +02:00
6543
f3074ddaf9
Some small code refactorings (#1727)
Refactorings taken from pull requests #1722 and #1725
2023-04-30 17:02:47 +02:00
qwerty287
54d4ec04c0
Drop Gogs private mode (#1726)
You should use `WOODPECKER_AUTHENTICATE_PUBLIC_REPOS` instead if repos
sent from forge as public should be authenticated
2023-04-30 14:01:11 +02:00
qwerty287
0f9188597e
Initiate Pagination Implementation for API and Infinite Scroll in UI (#1651)
- Add pagination support to the API endpoints that return lists of items
- Adjust UI to enable infinite scrolling via pagination
2023-04-30 03:40:13 +02:00
Julien Palard
6af9371011
Docs: The WOODPECKER_ADMIN environment variable is singular. (#1699) 2023-04-08 01:34:36 +02:00
qwerty287
0970f35df5
Do not store inactive repos (#1658)
Do not sync repos with forge if the repo is not necessary in DB.

In the DB, only repos that were active once or repos that are currently
active are stored. When trying to enable new repos, the repos list is
fetched from the forge instead and displayed directly. In addition to
this, the forge func `Perm` was removed and is now merged with `Repo`.

Solves a TODO on RepoBatch.

---------

Co-authored-by: Anbraten <anton@ju60.de>
2023-03-21 23:01:59 +01:00
Anbraten
f13ffc2c8f
Save agent-id for tasks and add endpoint to get agent tasks (#1631)
Save which agent is running a task. This is now visible in the admin UI
in the queue and in the agent details screen.

# changes
- [x] save id of agent executing a task
- [x] add endpoint to get tasks of an agent for #999 
- [x] show assigned agent-id in queue
- [x] (offtopic) use same colors for queue stats and icons (similar to
the ones used by pipelines)
- [x] (offtopic) use badges for queue labels & dependencies


![image](https://user-images.githubusercontent.com/6918444/226541271-23f3b7b2-7a08-45c2-a2e6-1c7fc31b6f1d.png)
2023-03-21 15:10:43 +02:00
6543
e28b43ab19
Only inject netrc to trusted clone plugins (#1352)
Co-authored-by: qwerty287 <ndev@web.de>
Co-authored-by: 6543 <6543@obermui.de>
2023-03-20 21:17:49 +01:00
Anbraten
2337f1854a
Add queue details UI for admins (#1632)
# Changes
- Adds an admin view to see the whole work-queue of the server. 
- The admin can also pause / resume the queue. 
- The view is reloading data every 5 seconds automatically.
- The task model from queue got removed in favor of the one from models.
2023-03-20 04:50:56 +01:00
qwerty287
42a115e19e
Add PR pipeline list (#1641)
Instead of viewing PR pipelines in the branches lists, add a separate
list for them. The API endpoint for PRs supports pagination (thus I
added a lot of pagination-related stuff), the UI doesn't yet though.


![wp](https://user-images.githubusercontent.com/80460567/226099133-bb4935d6-c357-4d73-8465-9157e25c0d82.png)

Closes #1619 

Extends this part of #1640

---------

Co-authored-by: Anbraten <anton@ju60.de>
2023-03-19 10:43:57 +01:00
Anbraten
277a839157
Add users UI for admins (#1634)
Co-authored-by: Lauris BH <lauris@nix.lv>
2023-03-18 21:21:20 +01:00
Lukas
1b43b0bf20
Add pull request labels as environment variable (#1321)
Closes #1308 

Co-authored-by: Anbraten <anton@ju60.de>
2023-03-17 03:43:04 +01:00
Lukasz
dc90f97524
Persist DepStatus of tasks (#1610) 2023-03-11 01:58:28 +01:00
6543
18d3139e9e
Use modern error handling and enforce it via lint (#1327)
Co-authored-by: Anbraten <anton@ju60.de>
2023-02-02 00:08:02 +01:00
Anbraten
71d6c03ca7
Add agent no-schedule flag (#1567)
This flag allows an agent owner / admin to stop the agent from taking
new workflows / pipelines.
2023-01-30 20:18:48 +01:00
Anbraten
d96032349a
Store an agents list and add agent heartbeats (#1189)
Co-authored-by: 6543 <6543@obermui.de>
2023-01-28 14:13:04 +01:00
antomy-gc
6516a28cdd
Secrets encryption in database (#1475)
closes #101

Added secrets encryption in database

- Google TINK or simple AES as encryption mechanisms
- Keys rotation support on TINK
- Existing SecretService is wrapped by encryption layer
- Encryption can be enabled and disabled at any time

Co-authored-by: Kuzmin Ilya <ilia.kuzmin@indrive.com>
Co-authored-by: 6543 <6543@obermui.de>
2023-01-12 20:59:07 +01:00
Joaquin
482678daf5
Fix pipeline feed not updated by new events (#1424)
Co-authored-by: Anbraten <anton@ju60.de>
2022-11-18 10:59:24 +01:00
Anbraten
de8ea95dd3
Rename forge-id to forge-remote-id (#1418) 2022-11-15 15:01:23 +01:00
Josh Soref
023d03dd61
Spelling (#1405)
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2022-11-09 08:12:17 +01:00
qwerty287
3372d1a87c
Rename remote to forge (#1357)
As of #745

Co-authored-by: Anbraten <anton@ju60.de>
2022-11-05 00:35:06 +01:00
Joonhyeok Ahn (Joon)
aed3a80287
Migrate sql querys to xorm query builder (#1356)
Co-authored-by: 6543 <6543@obermui.de>
2022-10-31 16:08:57 +01:00
Harikesh00
36e42914fa
Renamed procs/jobs to steps in code (#1331)
Renamed `procs` to `steps` in code for the issue #1288

Co-authored-by: Harikesh Prajapati <harikesh.prajapati@druva.com>
Co-authored-by: qwerty287 <ndev@web.de>
Co-authored-by: qwerty287 <80460567+qwerty287@users.noreply.github.com>
Co-authored-by: 6543 <6543@obermui.de>
2022-10-28 17:38:53 +02:00
qwerty287
e568c42e84
Support plugin-only secrets (#1344)
Closes #1071
2022-10-27 04:21:07 +02:00
qwerty287
f88c70b55e
Rename to pipeline in DB and JSONs (#1296)
Closes #1282 

Follow-up to #1224, addresses #745

- changes JSON fields
- adds migration to rename columns
- fixes some comments
2022-10-22 15:54:43 +02:00
qwerty287
849e05bb8b
Rename build to pipeline in code (#1224)
Ref:  #745

Co-authored-by: Anbraten <anton@ju60.de>
Co-authored-by: 6543 <6543@obermui.de>
2022-10-18 03:24:12 +02:00
Lauris BH
86bc8b46ed
Change build message column type to text (#1252)
Fixes #1115
Fixes #1194
2022-10-13 00:47:11 +02:00
6543
612eee8269
Sort secret lists and events (#1223)
close  #1176
2022-09-29 22:46:20 +02:00
6543
5d7fdbc71c
fix: create secret for manual build (#1222) 2022-09-29 07:30:05 +02:00
[X]
b4d89a1cce
Add ability to trigger manual builds (#1156)
closes #83 
closes #240 

Co-authored-by: Anbraten <anton@ju60.de>
Co-authored-by: qwerty287 <80460567+qwerty287@users.noreply.github.com>
Co-authored-by: 6543 <6543@obermui.de>
2022-09-27 11:05:00 +02:00
qwerty287
6433dfea23
Use package's structs and fix missing clone (#1172)
Closes #1169

Replaces structs that were added inline in hook structs with structs of
the corresponding SDKs. This makes it more readable and error-proof.
2022-09-07 18:16:40 +01:00
想出网名啦
eaf05dbae8
On Repo update, keep old "Clone" if update would empty it (#1170)
fixed #1169 

after debugging, I found that:
if `Clone` in database is empty, before
https://github.com/woodpecker-ci/woodpecker/blob/master/server/api/hook.go#L128,
the `repo.Clone` be always unset, so clone is failed.

This PR makes the empty `Clone` can't overwrite the value in database.
2022-09-07 17:34:08 +01:00
qwerty287
52d3652f2e
Fetch repositories with remote ID if possible (#1078)
Use IDs of the forge to fetch repositories instead of their names and owner names. This improves handling of renamed and transferred repos.

TODO

- [ ] try to support as many forges as possible
    - [x] Gogs (no API)
    - [ ] Bitbucket Server
    - [x] Coding (no API?)
- [x] update repo every time it is fetched or received from the forge
- [x] if repo remote IDs are not available, use owner / name to get it
- [x] handle redirections (redirect a renamed repo to its new path)
- [x] ~~pull all repos once during migration to update ID (?)~~ issue fixed by on-demand loading of remote IDs
- [x] handle redirections in web UI
- [ ] improve handling of hooks after a repo was renamed (currently it checks for a redirection to the repo)
- [x] tests
- [x] `UNIQUE` constraint for remote IDs after migration shouldn't work (all repos have an empty string as remote ID)

close #854
close #648 partial
close https://codeberg.org/Codeberg-CI/feedback/issues/46

Possible follow-up PRs
- apply the same scheme on everything fetched from the remote (currently only users)

Co-authored-by: 6543 <6543@obermui.de>
2022-09-05 17:08:51 +02:00
6543
383f273392
Add cron feature (#934)
https://woodpecker-ci.org/docs/usage/cron

Co-authored-by: Anbraten <anton@ju60.de>
Co-authored-by: qwerty287 <80460567+qwerty287@users.noreply.github.com>
2022-09-01 00:36:32 +02:00
Lauris BH
1ac2c42652
Add global and organization secrets (#1027)
* Implement database changes and store methods for global and organization secrets

* Add tests for new store methods
* Add organization secret API and UI
* Add global secrets API and UI

* Add suggestions

* Update warning style

* Apply suggestions from code review

Co-authored-by: Anbraten <anton@ju60.de>

* Fix lint warning

Co-authored-by: Anbraten <anton@ju60.de>
2022-08-14 13:48:53 +02:00
Simon Vieille
6351684070
Update type of 'log_data' from blob to longblob (#1050)
* Update type of 'log_data' from blob to longblob

* Add migration task to update log_data type
2022-07-29 14:57:18 +02:00
Lauris BH
19dfc331f4
Add method to check organization membership (#1037)
* Add remote method to check organization membership
* Use named return parameters in interface
* Add membership check service
* Update Gitea SDK
2022-07-25 03:09:35 +02:00
6543
0ab5182402
Document package architecture (#972)
to make it easier for devs to find the right place for code

close #655

Co-authored-by: Anbraten <anton@ju60.de>
Co-authored-by: qwerty287 <80460567+qwerty287@users.noreply.github.com>
2022-06-16 15:35:58 +02:00
Anbraten
cc30db44ac
Use asym key to sign webhooks (#916)
* use async key pair for webhooks

* fix tests

* fix linter

* improve code

* add key pair to database

* undo some changes

* more undo

* improve docs

* add api-endpoint

* add signaturne api endpoint

* fix error

* fix linting and test

* fix lint

* add test

* migration 006

* no need for migration

* replace httsign lib

* fix lint

Co-authored-by: 6543 <6543@obermui.de>
2022-06-01 20:06:27 +02:00
Anbraten
0b2b776ed8
Remove senders model (#923)
* remove senders
* drop table
2022-05-17 21:27:44 +02:00
6543
c350af645b
Move Badge & CCMenue into own packages (#908)
* move badge generation into own package

* move ccmenue away from models

* fix misspell
2022-05-14 14:30:09 +02:00
Zav Shotan
7313de2b1d
Add support for superseding runs (#831)
closes #11

Added support:
1. Environment variable `WOODPECKER_DELETE_MULTIPLE_RUNS_ON_EVENTS` (Default pull_request, push)
2. Builds will be marked as killed when they "override" another build
2022-05-09 11:26:09 +02:00
6543
2a5159f7fe
Fix pipeline timestamps (#730)
* only calculate time on running builds

* Add updated timestamp into database and use it in frontend

* add more trace logging

* refactor (move grpc unrelated func into related package)

* fix xorm schema

* add todo
2022-01-31 15:38:39 +01:00
Anbraten
5e595978fa
Send decline events back to UI (#680)
and remove unused code
2022-01-09 03:47:31 +01:00