Commit graph

49 commits

Author SHA1 Message Date
qwerty287
d5e68efc95
Do not print log level on CLI (#2638)
if it's not in debug log level

closes https://github.com/woodpecker-ci/woodpecker/issues/1764
2023-10-24 12:23:42 +02:00
qwerty287
31240b5771
Remove SSH backend (#2635)
The SSH backend is, similar to Gogs and Coding for forges, completely
unmaintained and seems unused (it is likely broken but we didn't get any
reports).
Instead, you should directly run the agent on the SSH machine with the
`local` backend.
2023-10-24 01:55:30 +02:00
renovate[bot]
3d19d863d1
fix(deps): update module github.com/tevino/abool to v2 (#2460) 2023-09-14 07:34:36 +02:00
6543
d253f8cc30
Make sure we dont have hidden options for backend and pipeline compiler (#2123)
move options based on **os.Getenv** into flags

---------
*Sponsored by Kithara Software GmbH*
2023-08-07 21:13:26 +02:00
6543
3d4758578a
Add opt save global log output to file (#2115)
close  #1933

---------
*Sponsored by Kithara Software GmbH*
2023-08-07 20:47:30 +02:00
praneeth-ovckd
9f987155f3
Fix WOODPECKER_GRPC_VERIFY being ignored (#2077)
This fixes #2076
2023-08-01 09:06:27 +02:00
6543
3cd78c9409
Refactor agent (#2021)
- code cleanup
- init backend engine only once
- pass a taskUUID to the backend

---
*Sponsored by Kithara Software GmbH*
2023-07-20 20:39:20 +02:00
6543
3a85559763
Remove code to read agent-id.conf (#2009)
I want to have c805c87e90 published at
least for 2 days ...
... so the migration did happen
2023-07-19 22:09:06 +02:00
6543
c805c87e90
fix write agent config 2023-07-17 14:53:02 +02:00
Lauris BH
724a3bf56e
Fix agent config reading error handling (#2008)
Currently for non-existing config file it would report error instead of info message
2023-07-17 14:22:32 +02:00
6543
fa951a8e95
Add agent config file (#1971) 2023-07-12 18:51:40 +02:00
Timo Tomasini
eb5c48a85f
Store agent ID in config file (#1888) 2023-07-02 17:22:05 +02:00
Alexander Matyushentsev
049c26229d
Let agent retry to connecting to server (#1728)
Closes https://github.com/woodpecker-ci/woodpecker/issues/1721

PR introduces connection retries during agent startup.

Signed-off-by: Alexander Matyushentsev <AMatyushentsev@gmail.com>
2023-05-03 13:31:29 +02:00
6543
2dd1b4f777
Correct label argument parsing in agent code (#1717)
close  #1704
2023-04-28 20:41:05 +02:00
Anbraten
81998b7001
Add backend label to agents (#1692)
closes #708 

Allows users to filter for a specific backend type.
2023-04-03 12:30:52 +02:00
6543
92614dfb1e
Agent check gRPC version against server (#1653)
close #1114

As long as the `VersionResponse` type is not changed the check will
fail/pass gracefully

example output:
```
{"level":"error","error":"GRPC version mismatch","time":"2023-03-19T19:49:09+01:00","message":"Server version next-6923e7ab does report grpc version 2 but we only understand 1"}
GRPC version mismatch
```
2023-03-19 22:42:21 +01:00
qwerty287
f582ad3159
Various enhancements in configuration (#1645)
- backends: move to cli flags instead of os.Getenv
- ssh: support 2fa with key and password
- allow to set grpc jwt secret (solves todo)
- allow to set default and max timeout (solves todo)

Closes https://github.com/woodpecker-ci/woodpecker/issues/896
Closes https://github.com/woodpecker-ci/woodpecker/issues/1131
2023-03-19 20:24:43 +01:00
Anbraten
fa5b0fb96e
Fix linter (#1647) 2023-03-18 20:35:27 +01:00
Anbraten
d96032349a
Store an agents list and add agent heartbeats (#1189)
Co-authored-by: 6543 <6543@obermui.de>
2023-01-28 14:13:04 +01:00
Stephen Muth
1816f6c715
Allow adding additional labels/annotations to kubernetes worker pods (#1510)
Example agent environment configuration using the new value:
```yaml
  - env:
    - name: WOODPECKER_BACKEND
      value: kubernetes
    - name: WOODPECKER_BACKEND_K8S_NAMESPACE
      value: default
    - name: WOODPECKER_BACKEND_K8S_POD_LABELS
      value: '{"sidecar.istio.io/inject":"false"}'
```
2022-12-31 01:37:09 +01:00
velsinki
dda4998261
Change healtcheck port into address format, redo #1197 (#1423)
As discussed in the comments in PR #1197. Also add documenation
accordingly.

One thing I'm not sure about is the simple check in health.go if the
address is usable in the GET request or not. From reading
https://pkg.go.dev/net#Dial it seems that the only non-standard address
format that would work in the `net` package but not in a GET url would
likely only be `:port`, as the others listed here are actually also
valid urls:

`For TCP, UDP and IP networks, if the host is empty or a literal
unspecified IP address, as in ":80", "0.0.0.0:80" or "[::]:80" for TCP
and UDP, "", "0.0.0.0" or "::" for IP, the local system is assumed.`

One additional thing I noticed is that while `WOODPECKER_SERVER_ADDR`
and `WOODPECKER_SERVER_ADDR` use the default value format of `:PORT`,
`WOODPECKER_SERVER` actually uses `localhost:9000`. I guess it makes a
bit of sense, considering the server might not be local to the agent,
but it looks a bit inconsistent this way. I don't think it would hurt to
make the `WOODPECKER_HEALTHCHECK_ADDR` in this format too, but then it's
different from the server flags again... :-)
2022-11-19 13:06:51 +02:00
Harikesh00
36e42914fa
Renamed procs/jobs to steps in code (#1331)
Renamed `procs` to `steps` in code for the issue #1288

Co-authored-by: Harikesh Prajapati <harikesh.prajapati@druva.com>
Co-authored-by: qwerty287 <ndev@web.de>
Co-authored-by: qwerty287 <80460567+qwerty287@users.noreply.github.com>
Co-authored-by: 6543 <6543@obermui.de>
2022-10-28 17:38:53 +02:00
qwerty287
849e05bb8b
Rename build to pipeline in code (#1224)
Ref:  #745

Co-authored-by: Anbraten <anton@ju60.de>
Co-authored-by: 6543 <6543@obermui.de>
2022-10-18 03:24:12 +02:00
qwerty287
d28080a9eb
Make healthcheck port configurable (#1197)
Closes https://github.com/woodpecker-ci/woodpecker/issues/1193

Co-authored-by: 6543 <6543@obermui.de>
2022-09-19 00:07:37 +02:00
Anbraten
3b0263442a
Adding initial version of Kubernetes backend (#552)
Co-authored-by: laszlocph <laszlo@laszlo.cloud>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Rynoxx <rynoxx@grid-servers.net>
2022-09-05 06:01:14 +02:00
Anbraten
5ca7ede9e4
Improve logging (#1158)
* switch default log level to info add start message and cleanup server start
* refactor code
* fix agent debug / trace logging
2022-09-03 20:41:23 +02:00
vanous
b7957c53aa
Fix filter cli attribute name (#1103)
Co-authored-by: Anbraten <anton@ju60.de>
2022-08-14 09:48:52 +02:00
Anbraten
e79ad00826
Add agent tagging / filtering for pipelines (#902)
Officially support labels for pipelines and agents to improve pipeline picking. 

* add pipeline labels
* update, improve docs  and add migration
* update proto file

---
closes #304 & #860
2022-05-31 01:12:18 +02:00
Anbraten
e178b7b4b2
Improve agent backend loading and suppress expectable errors (#818)
* improve agent loop loading backend once

* supress container not found or stopped warnings
2022-03-08 16:21:43 +01:00
Lukas Bachschwell
09e6460f95
Allow loading sensitive flags from files (#815)
With systems like docker swarm or docker compose it is usually a little awkward to manage secrets. 
There is no way to directly inject them into the environment config. So you often have to write your secrets directly into the compose file

There are hacky workarounds such as overriding the entry-point of the container and loading a script which then fetches secrets from /run/secrets and replaces the environment variables, but this becomes very difficult once we are using docker images built from "scratch" (which is a really great practice otherwise) as there is no shell or standard tooling available

This adds a *_FILE variant of their Environment config values to work around this issue.

Signed-off-by: Lukas Bachschwell <lukas@lbsfilm.at>
2022-03-01 16:09:33 +01:00
6543
a3ac393264
Use shared func for registering Sigterm on a context (#799) 2022-02-28 09:27:31 +01:00
Anbraten
6af94d79e3
Remove unused flags / options (#693)
Some flags where unused and / or unnecessary as they are covered by alternatives implemented in PRs of milestone 0.15.0 and just complicated the setup.

closes #681
2022-01-31 15:38:00 +01:00
6543
b6e47a3f4a
Update deps (#724)
* update github.com/containerd/containerd v1.5.7 -> v1.5.9

* update github.com/lib/pq v1.10.3 -> v1.10.4

* update github.com/prometheus/client_golang v1.11.0 -> v1.12.0

* update github.com/rs/zerolog v1.25.0 -> v1.26.1

* update golang.org/x/crypto 2021-12-15 -> 2022-01-28
2022-01-29 16:04:50 +01:00
Anbraten
50570cba5c
Automatically determine platform of agent (#690)
* Automatically determine platform of agent

* add migration hint

* cleanup docs a bit
2022-01-17 15:19:30 +01:00
6543
2f91bdd4a0
gofumpt -w -l -extra . (#661) 2022-01-05 21:50:23 +01:00
6543
4cbdacb21c
Nits Collected over last month (#595)
- add coverage.out
- add context queue
- fix misspell
- sanitize config: WOODPECKER_GITEA_URL
- storage backend migration should have no xorm session within migration function
2021-12-11 13:15:04 +01:00
6543
4ea00f0035
Make cancel pipeline work again (#585)
* logger: on level debug or trace add caller to logs

* more logging

* fix cancel bug

* ignore error if occur for queue cancel
2021-12-08 23:40:00 +01:00
Anbraten
c1a8884d62
Add backend selection for agent (#463)
- add backend selection option
- by default it will auto-detect a backend
2021-11-26 03:34:48 +01:00
Lukas
116c310820
Add linter misspell (#530)
* Add linter misspell

* Fix spelling

Co-authored-by: Anbraten <anton@ju60.de>
2021-11-24 02:01:12 +01:00
6543
fe31fb1e06
Drop error only on purpose or else report back or log (#514)
- Remove Deadcode
- Simplify Code
- Drop error only on purpose
2021-11-23 15:36:52 +01:00
6543
798c2bc8b2
Upgrade urfave/cli to v2 (#483)
* migrate urfave/ci v1 -> v2
* refactor cli (format flag)
* log error if agent can not listen on port 3000

close #452
2021-10-27 21:03:14 +02:00
6543
5990d32fd3
More logging and refactor (#457)
* only use "context"
* enable 'h2' support at server
* trace log  remote and database config
* log loglevel on start
2021-10-19 11:44:49 +02:00
John Olheiser
03bb0e69d8
Replace debug with log-level flag (#440)
* Add logging level to CLI and API

Signed-off-by: jolheiser <john.olheiser@gmail.com>

# Conflicts:
#	server/router/router.go

* Apply suggestions from code review

Co-authored-by: Anbraten <anton@ju60.de>

* Refactor log level

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Remove API

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* Fix API imports

Signed-off-by: jolheiser <john.olheiser@gmail.com>

* do not relay on external default

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Anbraten <anton@ju60.de>
2021-10-16 17:41:36 -05:00
John Olheiser
4276a04f0c
Move entirely to zerolog (#426)
Completely switch to zerolog

(Remove usage of logrus and std logger)

Signed-off-by: jolheiser <john.olheiser@gmail.com>
Co-authored-by: 6543 <6543@obermui.de>
2021-10-12 09:25:13 +02:00
Masaya Watanabe
7913d6db95
Fix channel buffer used with signal.Notify (#421)
Co-authored-by: Anbraten <anton@ju60.de>
2021-10-09 03:14:25 +02:00
Marian Steinbach
17b8867b96
Clean up config environment variables for server and agent (#218)
The goal here is to make consistent use of configuration environment variables prefixed `WOODPECKER_`. Where several variants existed, this PR aims to remove all but one option, leaving the most explicit.

This PR only changes server and agent code, but not documentation, in order to keep the PR digestible. Once we have consensus that this is correct, I'll change docs accordingly.

User (rather: admin) facing changes in this PR:

- In general, support for all server and agent config environment variables (env vars) starting with `DRONE_` is removed. The according `WOODPECKER_*` variables must be used instead.
- The env var `WOODPECKER_HOST` replaces `DRONE_HOST`, and `DRONE_SERVER_HOST`.
- The env var `WOODPECKER_AGENT_SECRET` is used to configure the shared secret which agents use to authenticate against the server. It replaces `WOODPECKER_SECRET`, `DRONE_SECRET`, `WOODPECKER_PASSWORD`, `DRONE_PASSWORD`, and `DRONE_AGENT_SECRET`.
- The env var `WOODPECKER_DATABASE_DRIVER` replaces `DRONE_DATABASE_DRIVER` and `DATABASE_DRIVER`.
- The env var `WOODPECKER_DATABASE_DATASOURCE` replaces `DRONE_DATABASE_DATASOURCE` and `DATABASE_CONFIG`.
2021-09-28 15:43:44 +02:00
Jacob Floyd
e34daae0cf
Move cncd/pipeline/pipeline/ to pipeline/ (#347)
* Refactor: move cncd/pipeline/ to pipeline/

* Refactor: move pipeline/pipeline/ to pipeline/
2021-09-24 13:18:34 +02:00
Jacob Floyd
bfb9af121c
Create agent/ package for backend agnostic logic (#338)
* Refactor: Move agent stats structs to agent/

Based in part on changes by laszlocph (kube branch):
7f4a0c8c12 Factor into Runner
f7fe9edd0b Using Runner in server

* Refactor: Add Runner constructor with counter field

Based in part on changes by laszlocph (kube branch):
7f4a0c8c12 Factor into Runner
f7fe9edd0b Using Runner in server

* Refactor: Move Runner to agent/ package

Based in part on changes by laszlocph (kube branch):
7f4a0c8c12 Factor into Runner
f7fe9edd0b Using Runner in server

* Refactor: pass pipeline.backend.Engine into agent.Runner

Based in part on changes by laszlocph (kube branch):
7f4a0c8 Factor into Runner
f7fe9ed Using Runner in server

* Use well-known function signature for WriteTo

* Rename stats.go -> state.go

Co-authored-by: Anbraten <anton@ju60.de>
2021-09-23 16:58:12 +02:00
Anbraten
b3d40024a9
Rename cmd agent and server folders and binaries (#330)
Renamed `cmd/drone-agent` to `cmd/agent` and `cmd/drone-server` to `cmd/server` and binaries to get rid of the drone name.
2021-09-21 17:06:13 +02:00