Commit graph

48 commits

Author SHA1 Message Date
hg
276b279b7f
Process workflow logs in batches (#4045) 2024-09-18 16:29:56 +02:00
6543
757f5a58e2
Gracefully shutdown server (#3896) 2024-07-14 01:46:01 +02:00
Lauris BH
28e982fffb
Global and organization registries (#1672)
Co-authored-by: Anbraten <6918444+anbraten@users.noreply.github.com>
2024-07-03 06:33:11 -07:00
6543
402938e216
Exclude mocks from release build (#3831) 2024-06-25 12:23:07 +02:00
Robert Kaussow
9972c24924
Add filter options to GetPipelines API (#3645)
Separate this change from
https://github.com/woodpecker-ci/woodpecker/pull/3506

I would like to get at least this change into v2.5.0 if possible.

---------

Co-authored-by: qwerty287 <80460567+qwerty287@users.noreply.github.com>
2024-04-25 09:37:42 +02:00
Anbraten
d494b6a959
Use forge from db (#1417)
This is the first step towards support for multiple forges (#138). It
inserts a forge using the currently existing env varaibles into db and
uses this forge from db later on in all places of the code.

closes #621

addresses #138 

# TODO
- [x] add forges table
- [x] add id of forge to repo
- [x] use forge of repo
- [x] add forge from env vars to db if not exists
- [x] migrate repo.ForgeID to the newly generated forge
- [x] support cache with forge from repo
- [x] maybe add forge loading cache? (use LRU cache for forges, I expect
users to have less than 10 forges normally)

---------

Co-authored-by: qwerty287 <80460567+qwerty287@users.noreply.github.com>
2024-04-16 08:04:55 +02:00
Robert Kaussow
781628816d
Fix make in_docker generate (#3616) 2024-04-15 17:46:44 +02:00
6543
e57a09a404
Update pipeline state on server as a whole on approval (#3504)
We can not just update some records for steps, as we want the pipeline
engine as single source of truth but not manage the state.
And the server should only manage the state but not how pipelines work.

We can match the pipeline but neither workflows or steps 1:1, so we
"update" them as a whole by deleting existing workflow and step data and
insert the new info from engine.

close   #3494
close  #3472

---------
*Sponsored by Kithara Software GmbH*

---------

Co-authored-by: Robert Kaussow <xoxys@rknet.org>
2024-03-18 20:07:45 +01:00
qwerty287
cb3efd2cd9
Simplify store interfaces (#3437)
Use `store.Store` interface if possible.
2024-02-25 10:37:10 +01:00
6543
7b29d1da49
Make PipelineConfig unique again (#3215)
fix https://github.com/woodpecker-ci/woodpecker/issues/3093

reverts https://github.com/woodpecker-ci/woodpecker/pull/3128
2024-01-18 23:50:29 +02:00
Robert Kaussow
6de5922408
Fix skipped pipelines model (#2923)
Fixes: https://github.com/woodpecker-ci/woodpecker/issues/2901

---------

Co-authored-by: Lauris BH <lauris@nix.lv>
2023-12-12 22:30:52 +02:00
runephilosof-karnovgroup
adb2c82790
Update go module path for major version 2 (#2905)
https://go.dev/doc/modules/release-workflow#breaking

Fixes https://github.com/woodpecker-ci/woodpecker/issues/2913 fixes
#2654
```
runephilosof@fedora:~/code/platform-woodpecker/woodpecker-repo-configurator (master)$ go get go.woodpecker-ci.org/woodpecker@v2.0.0
go: go.woodpecker-ci.org/woodpecker@v2.0.0: invalid version: module contains a go.mod file, so module path must match major version ("go.woodpecker-ci.org/woodpecker/v2")
```

---------

Co-authored-by: qwerty287 <80460567+qwerty287@users.noreply.github.com>
2023-12-08 08:15:08 +01:00
qwerty287
7bacbd5699
Migrate to Xormigrate (#2711)
Co-authored-by: Anbraten <anton@ju60.de>
2023-11-28 10:31:54 +01:00
6543
5a7b689e30
Switch to go vanity urls (#2706)
Co-authored-by: Anbraten <anton@ju60.de>
2023-11-07 08:04:33 +01:00
qwerty287
e74115027b
Add some tests (#2652)
and some cleanups
2023-10-28 13:37:54 +02:00
qwerty287
d04bb72e5e
Add repos list for admins (#2347) 2023-09-08 12:26:20 +02:00
qwerty287
479ced3b25
Add org list (#2338)
![Screenshot 2023-08-28 at 10-08-20
Woodpecker](https://github.com/woodpecker-ci/woodpecker/assets/80460567/e3248b05-7899-43ca-a0cf-4834eae078d8)
Closes #2307
2023-08-28 11:15:16 +02:00
Anbraten
e5d5ec8b47
Use id to access orgs (#1873)
closes #1743 

fixes: setting secrets for own user namespace

- create org in database
- use orgID for org related APIs

Co-authored-by: 6543 <6543@obermui.de>
2023-07-21 19:45:32 +02:00
qwerty287
3033abc3b4
Add own workflow model (#1784)
Closes #1287

---------

Co-authored-by: 6543 <6543@obermui.de>
2023-06-27 18:01:18 +02:00
Anbraten
556607b525
Rework log streaming and related functions (#1802)
closes #1801
closes #1815 
closes #1144
closes  #983
closes  #557
closes #1827
regression of #1791

# TODO
- [x] adjust log model
- [x] add migration for logs
- [x] send log line via grpc using step-id
- [x] save log-line to db
- [x] stream log-lines to UI
- [x] use less structs for log-data
- [x] make web UI work
  - [x] display logs loaded from db
  - [x] display streaming logs
- [ ] ~~make migration work~~ -> dedicated pull (#1828)

# TESTED
- [x] new logs are stored in database
- [x] log retrieval via cli (of new logs) works
- [x] log streaming works (tested via curl & webui)
- [x] log retrieval via web (of new logs) works

---------

Co-authored-by: 6543 <6543@obermui.de>
2023-06-06 09:52:08 +02:00
Anbraten
c464f857ae
Remove unused file system api (#1791)
Co-authored-by: 6543 <6543@obermui.de>
2023-05-31 18:03:03 +02:00
qwerty287
6d2240b2e6
Identify users using their remote ID (#1732) 2023-05-11 05:19:35 +02:00
qwerty287
0f9188597e
Initiate Pagination Implementation for API and Infinite Scroll in UI (#1651)
- Add pagination support to the API endpoints that return lists of items
- Adjust UI to enable infinite scrolling via pagination
2023-04-30 03:40:13 +02:00
qwerty287
0970f35df5
Do not store inactive repos (#1658)
Do not sync repos with forge if the repo is not necessary in DB.

In the DB, only repos that were active once or repos that are currently
active are stored. When trying to enable new repos, the repos list is
fetched from the forge instead and displayed directly. In addition to
this, the forge func `Perm` was removed and is now merged with `Repo`.

Solves a TODO on RepoBatch.

---------

Co-authored-by: Anbraten <anton@ju60.de>
2023-03-21 23:01:59 +01:00
Anbraten
d96032349a
Store an agents list and add agent heartbeats (#1189)
Co-authored-by: 6543 <6543@obermui.de>
2023-01-28 14:13:04 +01:00
antomy-gc
6516a28cdd
Secrets encryption in database (#1475)
closes #101

Added secrets encryption in database

- Google TINK or simple AES as encryption mechanisms
- Keys rotation support on TINK
- Existing SecretService is wrapped by encryption layer
- Encryption can be enabled and disabled at any time

Co-authored-by: Kuzmin Ilya <ilia.kuzmin@indrive.com>
Co-authored-by: 6543 <6543@obermui.de>
2023-01-12 20:59:07 +01:00
Anbraten
de8ea95dd3
Rename forge-id to forge-remote-id (#1418) 2022-11-15 15:01:23 +01:00
Josh Soref
023d03dd61
Spelling (#1405)
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2022-11-09 08:12:17 +01:00
qwerty287
3372d1a87c
Rename remote to forge (#1357)
As of #745

Co-authored-by: Anbraten <anton@ju60.de>
2022-11-05 00:35:06 +01:00
Harikesh00
36e42914fa
Renamed procs/jobs to steps in code (#1331)
Renamed `procs` to `steps` in code for the issue #1288

Co-authored-by: Harikesh Prajapati <harikesh.prajapati@druva.com>
Co-authored-by: qwerty287 <ndev@web.de>
Co-authored-by: qwerty287 <80460567+qwerty287@users.noreply.github.com>
Co-authored-by: 6543 <6543@obermui.de>
2022-10-28 17:38:53 +02:00
qwerty287
849e05bb8b
Rename build to pipeline in code (#1224)
Ref:  #745

Co-authored-by: Anbraten <anton@ju60.de>
Co-authored-by: 6543 <6543@obermui.de>
2022-10-18 03:24:12 +02:00
qwerty287
52d3652f2e
Fetch repositories with remote ID if possible (#1078)
Use IDs of the forge to fetch repositories instead of their names and owner names. This improves handling of renamed and transferred repos.

TODO

- [ ] try to support as many forges as possible
    - [x] Gogs (no API)
    - [ ] Bitbucket Server
    - [x] Coding (no API?)
- [x] update repo every time it is fetched or received from the forge
- [x] if repo remote IDs are not available, use owner / name to get it
- [x] handle redirections (redirect a renamed repo to its new path)
- [x] ~~pull all repos once during migration to update ID (?)~~ issue fixed by on-demand loading of remote IDs
- [x] handle redirections in web UI
- [ ] improve handling of hooks after a repo was renamed (currently it checks for a redirection to the repo)
- [x] tests
- [x] `UNIQUE` constraint for remote IDs after migration shouldn't work (all repos have an empty string as remote ID)

close #854
close #648 partial
close https://codeberg.org/Codeberg-CI/feedback/issues/46

Possible follow-up PRs
- apply the same scheme on everything fetched from the remote (currently only users)

Co-authored-by: 6543 <6543@obermui.de>
2022-09-05 17:08:51 +02:00
6543
18c1807f4f
Create mock for store and add unit tests for cron (#1146)
* pass remote on init as argument like store

* mock store

* TestCreateBuild
2022-09-01 01:19:49 +02:00
6543
383f273392
Add cron feature (#934)
https://woodpecker-ci.org/docs/usage/cron

Co-authored-by: Anbraten <anton@ju60.de>
Co-authored-by: qwerty287 <80460567+qwerty287@users.noreply.github.com>
2022-09-01 00:36:32 +02:00
Lauris BH
1ac2c42652
Add global and organization secrets (#1027)
* Implement database changes and store methods for global and organization secrets

* Add tests for new store methods
* Add organization secret API and UI
* Add global secrets API and UI

* Add suggestions

* Update warning style

* Apply suggestions from code review

Co-authored-by: Anbraten <anton@ju60.de>

* Fix lint warning

Co-authored-by: Anbraten <anton@ju60.de>
2022-08-14 13:48:53 +02:00
Anbraten
cc30db44ac
Use asym key to sign webhooks (#916)
* use async key pair for webhooks

* fix tests

* fix linter

* improve code

* add key pair to database

* undo some changes

* more undo

* improve docs

* add api-endpoint

* add signaturne api endpoint

* fix error

* fix linting and test

* fix lint

* add test

* migration 006

* no need for migration

* replace httsign lib

* fix lint

Co-authored-by: 6543 <6543@obermui.de>
2022-06-01 20:06:27 +02:00
Anbraten
0b2b776ed8
Remove senders model (#923)
* remove senders
* drop table
2022-05-17 21:27:44 +02:00
Zav Shotan
7313de2b1d
Add support for superseding runs (#831)
closes #11

Added support:
1. Environment variable `WOODPECKER_DELETE_MULTIPLE_RUNS_ON_EVENTS` (Default pull_request, push)
2. Builds will be marked as killed when they "override" another build
2022-05-09 11:26:09 +02:00
6543
acbde30c3b
Lint code with gofumpt (#664) 2022-01-06 07:44:14 +01:00
6543
ff8ad5bb83
Fix registry delete (#532)
- Make make it possible to delete `"§4fda`
- Reject non urls for registry-address
- Fix #520
2021-11-25 19:04:36 +01:00
6543
07b32fa0fe
Update database in one transaction on syncing user repositorys (#513)
* userSyncer Sync commit changes in one transaction
* cover new cases with unit tests
2021-11-18 15:42:18 +01:00
6543
ca8e215cfa
Migrate to Xorm (#474)
close #234

* Migrate store
* Migrate tests
* Rewrite migrations
* Init fresh DB in on step
* Rm old stuff (meddler, sql files, dead code, ...)
2021-11-13 20:18:06 +01:00
6543
965235e421
Remove some wrapper and make code more redable (#478)
* meaningful var names
* no context wrapper, use store directly
* retrieve store from context once
* rm store.GetRepoOwnerName
2021-10-28 11:12:58 +02:00
Anbraten
3c5827f08a
Fix repo access (#476)
* fix repo access

* fix permission syncing
2021-10-26 21:29:30 +02:00
6543
5990d32fd3
More logging and refactor (#457)
* only use "context"
* enable 'h2' support at server
* trace log  remote and database config
* log loglevel on start
2021-10-19 11:44:49 +02:00
John Olheiser
4276a04f0c
Move entirely to zerolog (#426)
Completely switch to zerolog

(Remove usage of logrus and std logger)

Signed-off-by: jolheiser <john.olheiser@gmail.com>
Co-authored-by: 6543 <6543@obermui.de>
2021-10-12 09:25:13 +02:00
6543
a94807efeb
Refactor: move model/ to server/model/ (#366) 2021-09-27 19:51:55 +02:00
Jacob Floyd
5fe712cca6
Move package store/ to server/store/ (#341)
* Refactor: move store/ to server/store/

* fix pipeline for moved tests

Co-authored-by: 6543 <6543@obermui.de>
2021-09-23 13:33:59 +02:00
Renamed from store/store.go (Browse further)