Commit graph

107 commits

Author SHA1 Message Date
Anbraten
e0ed1b3e49
Change master to main (#2044) 2023-07-31 05:47:23 +02:00
6543
bc2e2c7a1b
let generic datastore error include func name (#2041)
error logs like:
```
{"level":"warn","error":"sql: no rows in result set","time":"2023-07-25T21:29:56Z"}
```

have to less context to be helpfull.
this will cange it as the message now looks like:

```
{"level":"warn","error":"GetPipelineLast: sql: no rows in result set", "time":"2023-07-27T02:54:25+02:00"}
```
2023-07-27 21:22:24 +02:00
Anbraten
e5d5ec8b47
Use id to access orgs (#1873)
closes #1743 

fixes: setting secrets for own user namespace

- create org in database
- use orgID for org related APIs

Co-authored-by: 6543 <6543@obermui.de>
2023-07-21 19:45:32 +02:00
Robert Kaussow
443c23c58a
Use global logger for xorm logs and add options (#1997)
Please let me know if this goes in the right direction. Needs some tests and docs.
2023-07-15 01:15:13 +02:00
6543
d9991e67e5
let HookParse func explicit ignore events (#1942)
for now it's not clear defined, what to do on an unsupported event.
e.g. gitea webhook panel shows 500 error and no message.

now we have a successful webhook and a message to show an info
2023-07-14 02:03:54 +02:00
6543
b54f6ebad6
Let pipeline-compiler export step types (#1958) 2023-07-11 15:53:05 +02:00
6543
7c99c8b843
Adjust model database type mapping (#1949)
adjust definitions according to feedback from real world usage
(ci.codeberg.org)

close  #1892
close  #1865
2023-07-08 20:09:53 +02:00
6543
2961aaf72b
Harden datastore (#1940)
- only explicit when filters
 - write all more complex filter with builder
 - reuse builder
2023-07-07 13:10:16 +02:00
qwerty287
cd982fcbd1
Fix pipeline status calculation/reporting (#1898)
Closes #1895 
Regression of #1784
2023-06-29 14:56:01 +02:00
6543
e1c31df6c6
Refactor AgentFindByToken (#1891)
followup of #1890
2023-06-28 06:59:58 +02:00
Anbraten
b57b6f27f6
Fix empty agent token (#1890)
Using an empty token for an agent was returning the first agent from the
database as the orm is not adding where clauses for empty strings of a
model when querying.

# Huge thanks for reporting and explaining the issue ❤️ 

- Dominik Heidler
- Timo Tomasini
2023-06-28 01:22:19 +02:00
qwerty287
3033abc3b4
Add own workflow model (#1784)
Closes #1287

---------

Co-authored-by: 6543 <6543@obermui.de>
2023-06-27 18:01:18 +02:00
6543
6be61e6c9a
logs table migration: skip on corrupted json and fail later (#1856)
enhance  #1828
2023-06-13 12:50:11 +03:00
Anbraten
ff01a9ff1d
Access repos by their ids (#1691)
closes #1295 
closes #648

# TODO
- [x] add new routes with `:repoID`
- [x] load repo in middleware using `:repoID` if present
- [x] update UI routes `:owner/:name` to `:repoID`
- [x] load repos using id in UI
- [x] add lookup endpoint `:owner/:name` to `:repoID`
- [x] redirect `:owner/:name` to `:repoID` in UI
- [x] use badge with `:repoID` route in UI
- [x] update `woodpecker-go`
- [x] check cli
- [x] add migrations / deprecation notes
- [x] check if #648 got solved directly
- [x] Test
  - [x] create repo
  - [x] repo pages
  - [x] ui redirects
  - [x] forge status links
2023-06-13 01:07:52 +02:00
6543
5139624cf8
Migrate old logs to new database schema (#1828)
address new models based on #1802
2023-06-12 22:43:14 +02:00
6543
b5b3b95721
Print ongoing alive sign for migrations after 20s (#1846) 2023-06-10 12:59:15 +03:00
qwerty287
5d74174bc3
Drop old columns (#1838) 2023-06-07 19:22:44 +02:00
Anbraten
556607b525
Rework log streaming and related functions (#1802)
closes #1801
closes #1815 
closes #1144
closes  #983
closes  #557
closes #1827
regression of #1791

# TODO
- [x] adjust log model
- [x] add migration for logs
- [x] send log line via grpc using step-id
- [x] save log-line to db
- [x] stream log-lines to UI
- [x] use less structs for log-data
- [x] make web UI work
  - [x] display logs loaded from db
  - [x] display streaming logs
- [ ] ~~make migration work~~ -> dedicated pull (#1828)

# TESTED
- [x] new logs are stored in database
- [x] log retrieval via cli (of new logs) works
- [x] log streaming works (tested via curl & webui)
- [x] log retrieval via web (of new logs) works

---------

Co-authored-by: 6543 <6543@obermui.de>
2023-06-06 09:52:08 +02:00
qwerty287
b82ed13586
Database migrations run in seperate sessions and commit on success right away (#1822)
This isolates single migration tasks from each other.
The migration itself is now not atomic anymore but each single migration
now on it's own.
This takes load away from databases, as new sessions have a committed
schema available.

We also disable xorm.cache, as the speed improvements are minor but
invalid cache caused by schema changes did happen already in the past.

---------
Reverts #1817
Closes #1821

---------

Co-authored-by: 6543 <6543@obermui.de>
2023-06-06 03:27:17 +02:00
qwerty287
b9731d8da9
Fix missing table name (#1817)
Bug from #1806 

We don't need the sync because the step_machine is there since the model
was added the first time so it mill *always* be in the table
2023-06-03 20:24:37 +02:00
6543
317bd24abb
Define Glossary (#1800)
based on
https://github.com/woodpecker-ci/woodpecker/issues/745#issuecomment-1313401427

close  #1724

---------

Co-authored-by: qwerty287 <80460567+qwerty287@users.noreply.github.com>
Co-authored-by: Anbraten <anton@ju60.de>
2023-06-03 16:39:56 +02:00
qwerty287
36870d1797
Drop step_machine column (#1806)
Bug 3 in #1712 

Was changed in f13ffc2c8f
2023-06-02 14:39:29 +02:00
qwerty287
f91ee5d23a
Remove MSSQL specific code and cleanups (#1796)
We don't have MSSQL support

---------

Co-authored-by: Anbraten <anton@ju60.de>
2023-05-31 21:27:57 +02:00
Anbraten
c464f857ae
Remove unused file system api (#1791)
Co-authored-by: 6543 <6543@obermui.de>
2023-05-31 18:03:03 +02:00
qwerty287
6d2240b2e6
Identify users using their remote ID (#1732) 2023-05-11 05:19:35 +02:00
qwerty287
0f9188597e
Initiate Pagination Implementation for API and Infinite Scroll in UI (#1651)
- Add pagination support to the API endpoints that return lists of items
- Adjust UI to enable infinite scrolling via pagination
2023-04-30 03:40:13 +02:00
qwerty287
9b32d021b3
Consistent status on delete (#1703)
Closes #1675
2023-04-15 15:22:39 +02:00
qwerty287
a06d3e1a61
Stable sort order for DB lists (#1702) 2023-04-11 09:33:27 +02:00
qwerty287
0970f35df5
Do not store inactive repos (#1658)
Do not sync repos with forge if the repo is not necessary in DB.

In the DB, only repos that were active once or repos that are currently
active are stored. When trying to enable new repos, the repos list is
fetched from the forge instead and displayed directly. In addition to
this, the forge func `Perm` was removed and is now merged with `Repo`.

Solves a TODO on RepoBatch.

---------

Co-authored-by: Anbraten <anton@ju60.de>
2023-03-21 23:01:59 +01:00
Anbraten
f13ffc2c8f
Save agent-id for tasks and add endpoint to get agent tasks (#1631)
Save which agent is running a task. This is now visible in the admin UI
in the queue and in the agent details screen.

# changes
- [x] save id of agent executing a task
- [x] add endpoint to get tasks of an agent for #999 
- [x] show assigned agent-id in queue
- [x] (offtopic) use same colors for queue stats and icons (similar to
the ones used by pipelines)
- [x] (offtopic) use badges for queue labels & dependencies


![image](https://user-images.githubusercontent.com/6918444/226541271-23f3b7b2-7a08-45c2-a2e6-1c7fc31b6f1d.png)
2023-03-21 15:10:43 +02:00
Anbraten
2337f1854a
Add queue details UI for admins (#1632)
# Changes
- Adds an admin view to see the whole work-queue of the server. 
- The admin can also pause / resume the queue. 
- The view is reloading data every 5 seconds automatically.
- The task model from queue got removed in favor of the one from models.
2023-03-20 04:50:56 +01:00
6543
92614dfb1e
Agent check gRPC version against server (#1653)
close #1114

As long as the `VersionResponse` type is not changed the check will
fail/pass gracefully

example output:
```
{"level":"error","error":"GRPC version mismatch","time":"2023-03-19T19:49:09+01:00","message":"Server version next-6923e7ab does report grpc version 2 but we only understand 1"}
GRPC version mismatch
```
2023-03-19 22:42:21 +01:00
qwerty287
f582ad3159
Various enhancements in configuration (#1645)
- backends: move to cli flags instead of os.Getenv
- ssh: support 2fa with key and password
- allow to set grpc jwt secret (solves todo)
- allow to set default and max timeout (solves todo)

Closes https://github.com/woodpecker-ci/woodpecker/issues/896
Closes https://github.com/woodpecker-ci/woodpecker/issues/1131
2023-03-19 20:24:43 +01:00
qwerty287
ade8e6d010
Various fixes and improvements (#1643)
- allow repo names to be case-insensitive
- improve backend error handling on DB get errors (record not found ->
404, else -> 500)
- replace magic numbers of http response codes
- unify the look and feel of cancel / save buttons on forms and view
them in one line

---------

Co-authored-by: Lauris BH <lauris@nix.lv>
2023-03-19 13:52:58 +01:00
qwerty287
42a115e19e
Add PR pipeline list (#1641)
Instead of viewing PR pipelines in the branches lists, add a separate
list for them. The API endpoint for PRs supports pagination (thus I
added a lot of pagination-related stuff), the UI doesn't yet though.


![wp](https://user-images.githubusercontent.com/80460567/226099133-bb4935d6-c357-4d73-8465-9157e25c0d82.png)

Closes #1619 

Extends this part of #1640

---------

Co-authored-by: Anbraten <anton@ju60.de>
2023-03-19 10:43:57 +01:00
Anbraten
277a839157
Add users UI for admins (#1634)
Co-authored-by: Lauris BH <lauris@nix.lv>
2023-03-18 21:21:20 +01:00
Anbraten
fa5b0fb96e
Fix linter (#1647) 2023-03-18 20:35:27 +01:00
Lukasz
dc90f97524
Persist DepStatus of tasks (#1610) 2023-03-11 01:58:28 +01:00
6543
18d3139e9e
Use modern error handling and enforce it via lint (#1327)
Co-authored-by: Anbraten <anton@ju60.de>
2023-02-02 00:08:02 +01:00
Anbraten
d96032349a
Store an agents list and add agent heartbeats (#1189)
Co-authored-by: 6543 <6543@obermui.de>
2023-01-28 14:13:04 +01:00
6543
c276bc8937
use same sql for set default statement on mysql/mariadb and postgres (#1540)
introduced by #1027 so it does not need a changelog or backport
2023-01-14 12:52:41 +02:00
antomy-gc
6516a28cdd
Secrets encryption in database (#1475)
closes #101

Added secrets encryption in database

- Google TINK or simple AES as encryption mechanisms
- Keys rotation support on TINK
- Existing SecretService is wrapped by encryption layer
- Encryption can be enabled and disabled at any time

Co-authored-by: Kuzmin Ilya <ilia.kuzmin@indrive.com>
Co-authored-by: 6543 <6543@obermui.de>
2023-01-12 20:59:07 +01:00
Joaquin
482678daf5
Fix pipeline feed not updated by new events (#1424)
Co-authored-by: Anbraten <anton@ju60.de>
2022-11-18 10:59:24 +01:00
Anbraten
de8ea95dd3
Rename forge-id to forge-remote-id (#1418) 2022-11-15 15:01:23 +01:00
Josh Soref
023d03dd61
Spelling (#1405)
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2022-11-09 08:12:17 +01:00
6543
18311d4360
Split and refactor (#1394)
Closes #974
2022-11-06 12:44:04 +01:00
qwerty287
3372d1a87c
Rename remote to forge (#1357)
As of #745

Co-authored-by: Anbraten <anton@ju60.de>
2022-11-05 00:35:06 +01:00
Joonhyeok Ahn (Joon)
aed3a80287
Migrate sql querys to xorm query builder (#1356)
Co-authored-by: 6543 <6543@obermui.de>
2022-10-31 16:08:57 +01:00
qwerty287
8f183c82a8
Support changed files for Gitea PRs (#1342)
- add tests to fetch changed files
- ignore error if gitea version is to low
- adjust docs accordingly

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lauris BH <lauris@nix.lv>
2022-10-28 19:17:30 +02:00
Harikesh00
36e42914fa
Renamed procs/jobs to steps in code (#1331)
Renamed `procs` to `steps` in code for the issue #1288

Co-authored-by: Harikesh Prajapati <harikesh.prajapati@druva.com>
Co-authored-by: qwerty287 <ndev@web.de>
Co-authored-by: qwerty287 <80460567+qwerty287@users.noreply.github.com>
Co-authored-by: 6543 <6543@obermui.de>
2022-10-28 17:38:53 +02:00