Commit graph

53 commits

Author SHA1 Message Date
Andrew Melnick
b52b021acb
Implement registries for Kubernetes backend (#4092)
According to [the documentation](https://woodpecker-ci.org/docs/administration/backends/kubernetes#images-from-private-registries), per-organization and per-pipeline registries are currently unsupported for the Kubernetes backend.

This patch implements this missing functionality by creating and deleting a matching secret for each pod with a matched registry, using the same name, labels, and annotations as the pod, and appending it to its `imagePullSecrets` list.

This patch adds tests for the new functionality, and has been manually end-to-end-tested in KinD by using a private image hosted in the matching gitea instance.

This will require updating the matching helm charts to add the create/delete permissions to the agent role, which **is already done**.

close  #2987
2024-09-30 01:03:05 +01:00
Thomas Anderson
dc10fb95ad
Removed Kubernetes default image pull secret name (#4005)
Co-authored-by: qwerty287 <80460567+qwerty287@users.noreply.github.com>
2024-08-06 18:47:31 +02:00
Thomas Anderson
c5746ccb50
Fail on InvalidImageName (#4007) 2024-08-06 17:07:07 +02:00
6543
b2970dbf0d
Refactor docker backend and add more test coverage (#2700)
collection of some smal nit's and additions of tests
2024-07-21 21:28:10 +02:00
6543
cd5f6f71a2
Migrate to github.com/urfave/cli/v3 (#2951) 2024-07-18 01:26:35 +02:00
Thomas Anderson
7bc38a1d8b
K8s secrets reference from step (#3655) 2024-06-23 18:20:21 +02:00
Thomas Anderson
065eebd306
Agent-wide node selector (#3608)
Co-authored-by: qwerty287 <80460567+qwerty287@users.noreply.github.com>
2024-06-03 17:25:28 +02:00
6543
42f2734308
cspell lint go code (#3706) 2024-05-24 22:35:04 +02:00
Robert Kaussow
89e100cfd1
Add godot linter to harmonitze toplevel comments (#3650) 2024-05-13 22:58:21 +02:00
Thomas Anderson
ae72102503
Ability to set pod annotations and labels from step (#3609)
Co-authored-by: qwerty287 <80460567+qwerty287@users.noreply.github.com>
2024-05-11 11:45:29 +02:00
Elias
f211a780f3
Handle ImagePullBackOff pod status (#3580)
close: https://github.com/woodpecker-ci/woodpecker/issues/3555

Put the same logic from `waitStep` and call the function
`isImagePullBackOffState` in the `tailStep` function.

---------

Co-authored-by: elias.souza <elias.souza@quintoandar.com.br>
Co-authored-by: Anbraten <6918444+anbraten@users.noreply.github.com>
2024-04-15 09:08:13 +02:00
Robert Kaussow
a779eed3df
Enable golangci linter gomnd (#3171) 2024-03-15 18:00:25 +01:00
Anbraten
0b91317cde
Fix linter (#3354) 2024-02-08 22:49:07 +01:00
qwerty287
6892a9ca57
Parse backend options in backend (#3227)
Currently, backend options are parsed in the yaml parser.
This has some issues:
- backend specific code should be in the backend folders
- it is not possible to add backend options for backends added via
addons
2024-02-08 18:39:32 +01:00
qwerty287
f92f8b17a3
Make agent usable for external backends (#3270) 2024-02-08 16:33:22 +01:00
Fernando Barbosa
c7467b9828
fix: agent panic when node is terminated during step execution (#3331)
Fixes https://github.com/woodpecker-ci/woodpecker/issues/3330

This adds error handling on the agent's WaitStep function, on two
sections where it could encounter a `panic: runtime error: invalid
memory address or nil pointer dereference` in case it could no longer
access complete information about a specific pod.

This error was found to happen if the node in which the pod was running
was terminated during the step's execution.
spite active pipelines being executed on the node.

Now instead of a panic on the agent's logs and undefined behavior on the
UI it will display a more helpful error message on the UI.

### Additional context

We observed the bug first on v2.1.1, but tested the fix internally on
top of 2.3.0.


![image](https://github.com/woodpecker-ci/woodpecker/assets/7269710/dfbcf089-85f7-4b5d-8102-f21af95c5cda)
2024-02-05 22:46:14 +01:00
qwerty287
6925afd83b
Pin prettier version (#3260) 2024-01-22 21:38:47 +02:00
Elias
32a1199519
fix: bug annotations (#3255)
Fix Issue: https://github.com/woodpecker-ci/woodpecker/issues/3254

Co-authored-by: elias.souza <elias.souza@quintoandar.com.br>
2024-01-22 13:39:49 +01:00
Thomas Anderson
072fa29f4a
Fixed Pods creation of WP services (#3236)
Closes #3178
2024-01-21 03:56:37 +01:00
Thomas Anderson
10f2e209d6
Secured kubernetes backend configuration (#3204)
Follow up of #3165
2024-01-15 03:59:08 +01:00
qwerty287
b9f6f3f9fb
Replace goimports with gci (#3202)
`gci` seems to be much more strict.
2024-01-14 18:22:06 +01:00
Robert Kaussow
9bbba4441d
Enable golangci linter forcetypeassert (#3168)
Split out from https://github.com/woodpecker-ci/woodpecker/pull/2960
2024-01-12 02:01:02 +01:00
Robert Kaussow
f813badcf9
Enable golangci linter contextcheck (#3170)
Split out from https://github.com/woodpecker-ci/woodpecker/pull/2960
2024-01-11 22:15:15 +01:00
qwerty287
b0a2b1cf2d
Lowercase all log strings (#3173)
from #3161

---------

Co-authored-by: 6543 <6543@obermui.de>
2024-01-11 19:17:07 +01:00
Robert Kaussow
7756c60a33
Enable golangci linter stylecheck (#3167)
This PR only fixes error string formatting, log message strings are
still mixed upper/lowercase (see
https://github.com/woodpecker-ci/woodpecker/pull/3161#issuecomment-1885140649)
and I'm not aware of a linter to enforce it.
2024-01-10 22:56:42 +01:00
qwerty287
00df53e941
Clean up logging (#3161)
- use `Err` method instead of format strings
- use `Msg` if no format string is used
2024-01-10 20:57:12 +01:00
6543
31614d0e38
Use step type to detect services in Kubernetes backend (#3141)
and use the correct name for tail log

---------

Co-authored-by: Anbraten <anton@ju60.de>
2024-01-09 05:42:36 +01:00
Thomas Anderson
c0fc4828ff
Flexible image pull secret reference (#3016)
Co-authored-by: pat-s <patrick.schratz@gmail.com>
2024-01-05 08:33:56 +01:00
Thomas Anderson
253d702bc7
Fix IPv6 host aliases for kubernetes (#2992)
Closes #2991


[Tests](https://github.com/woodpecker-ci/woodpecker/pull/2993#issuecomment-1868048169)

---------

Co-authored-by: 6543 <6543@obermui.de>
2023-12-23 00:42:30 +01:00
Thomas Anderson
01a955ed0e
Kubernetes refactor (#2794)
Kubernetes backend refactoring and tests

---------
Co-authored-by: 6543 <6543@obermui.de>
2023-12-19 04:53:52 +01:00
qwerty287
ff1f51d6a9
Rename engine to backend (#2950)
rename based on https://woodpecker-ci.org/docs/usage/terminiology

---------

Co-authored-by: 6543 <6543@obermui.de>
2023-12-14 19:20:47 +01:00
runephilosof-karnovgroup
adb2c82790
Update go module path for major version 2 (#2905)
https://go.dev/doc/modules/release-workflow#breaking

Fixes https://github.com/woodpecker-ci/woodpecker/issues/2913 fixes
#2654
```
runephilosof@fedora:~/code/platform-woodpecker/woodpecker-repo-configurator (master)$ go get go.woodpecker-ci.org/woodpecker@v2.0.0
go: go.woodpecker-ci.org/woodpecker@v2.0.0: invalid version: module contains a go.mod file, so module path must match major version ("go.woodpecker-ci.org/woodpecker/v2")
```

---------

Co-authored-by: qwerty287 <80460567+qwerty287@users.noreply.github.com>
2023-12-08 08:15:08 +01:00
Thomas Anderson
3adb98b287
Simple security context options (Kubernetes) (#2550) 2023-11-26 08:46:06 +01:00
qwerty287
70711ed9db
Replace interface{} with any (#2807)
like golang:
2580d0e08d
2023-11-12 18:23:48 +01:00
6543
5a7b689e30
Switch to go vanity urls (#2706)
Co-authored-by: Anbraten <anton@ju60.de>
2023-11-07 08:04:33 +01:00
J-Ha
9af71dcc98
Use unique label selector for pod label for kubernetes services (#2723)
Co-authored-by: Julian Haseleu <julian.haseleu@dreamit.de>
Co-authored-by: qwerty287 <80460567+qwerty287@users.noreply.github.com>
2023-11-04 10:35:37 +01:00
Thomas Anderson
de53b906e8
Add ports into pipeline backend step model (#2656)
Closes #2655.


[Pipeline](https://woodpecker-ci.org/docs/next/usage/services#complete-pipeline-example):
```yaml
services:
  database:
    image: mysql
    environment:
      - MYSQL_DATABASE=test
      - MYSQL_ROOT_PASSWORD=example
    ports:
      - 3306

steps:
  get-version:
    image: ubuntu
    commands:
      - ( apt update && apt dist-upgrade -y && apt install -y mysql-client 2>&1 )> /dev/null
      - sleep 60s # need to wait for mysql-server init
      - echo 'SHOW VARIABLES LIKE "version"' | mysql -uroot -hdatabase test -pexample
```

Service:
```yaml
apiVersion: v1
kind: Service
metadata:
  name: wp-01hdq6gbkw1mn6k1655fs3rntf-0-services-0
  namespace: woodpecker-runtime
  ...
  selfLink: >-
    /api/v1/namespaces/woodpecker-runtime/services/wp-01hdq6gbkw1mn6k1655fs3rntf-0-services-0
status:
  loadBalancer: {}
spec:
  ports:
    - protocol: TCP
      port: 3306
      targetPort: 3306
  selector:
    step: database
  clusterIP: 10.43.180.120
  clusterIPs:
    - 10.43.180.120
  type: ClusterIP
  sessionAffinity: None
  ipFamilies:
    - IPv4
  ipFamilyPolicy: SingleStack
  internalTrafficPolicy: Cluster
```
2023-11-02 04:12:41 +01:00
Thomas Anderson
3620c84da4
Unregister stateless agents from server on termination (#2606)
Closes #2027

---------

Co-authored-by: 6543 <6543@obermui.de>
2023-11-02 01:53:47 +02:00
6543
ebe0307c6b
Let the backend engine report the current platform (#2688)
if you run woodpecker-agent on windows and connect it to an docker
daemon, there could be two different platforms possible, as you can
switch from linux to windows mode and visa versa


---
*Sponsored by Kithara Software GmbH*
2023-11-01 15:38:37 +01:00
Anbraten
4198c447fb
Destroy steps after they are done (#2681)
Co-authored-by: 6543 <m.huber@kithara.com>
2023-11-01 09:35:11 +01:00
qwerty287
8cdac56d8f
Check for correct license header (#2137) 2023-08-10 11:06:00 +02:00
6543
d253f8cc30
Make sure we dont have hidden options for backend and pipeline compiler (#2123)
move options based on **os.Getenv** into flags

---------
*Sponsored by Kithara Software GmbH*
2023-08-07 21:13:26 +02:00
6543
3cd78c9409
Refactor agent (#2021)
- code cleanup
- init backend engine only once
- pass a taskUUID to the backend

---
*Sponsored by Kithara Software GmbH*
2023-07-20 20:39:20 +02:00
Neil Hanlon
a95a5b43bf
fix(backend/kubernetes): Ensure valid naming of name field (#1661)
- Kubernetes v1.26 on VKE causes error when creating persistent volume
claim because of uppercase characters in name field

This patch is trivial just in order to get it working - happy to
implement differently.

The error in question:

```
The PersistentVolumeClaim "wp-01G1131R63FWBSPMA4ZAZTKLE-0-clone-0" is invalid: metadata.name: Invalid value: "wp-01G1131R63FWBSPMA4ZAZTKLE-0-clone-0": a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')
```
2023-03-21 20:00:45 +01:00
Lauris BH
46452fbd84
Update Go dependencies and minimal Go version to 1.20 (#1650)
Signed-off-by: 6543 <6543@obermui.de>
Co-authored-by: 6543 <6543@obermui.de>
2023-03-21 00:48:15 +01:00
qwerty287
f582ad3159
Various enhancements in configuration (#1645)
- backends: move to cli flags instead of os.Getenv
- ssh: support 2fa with key and password
- allow to set grpc jwt secret (solves todo)
- allow to set default and max timeout (solves todo)

Closes https://github.com/woodpecker-ci/woodpecker/issues/896
Closes https://github.com/woodpecker-ci/woodpecker/issues/1131
2023-03-19 20:24:43 +01:00
Stephen Muth
be4ce2e02d
Ensure the SharedInformerFactory closes eventually (#1585)
Otherwise it will continue to collect events in the background, consuming memory that can't be GC'd.
2023-02-16 00:54:33 +01:00
6543
5c617accd8
kubernetes ignore labels and annotations if not set (#1528)
close #1527
regression of #1510
2023-01-04 19:51:21 +02:00
Stephen Muth
1816f6c715
Allow adding additional labels/annotations to kubernetes worker pods (#1510)
Example agent environment configuration using the new value:
```yaml
  - env:
    - name: WOODPECKER_BACKEND
      value: kubernetes
    - name: WOODPECKER_BACKEND_K8S_NAMESPACE
      value: default
    - name: WOODPECKER_BACKEND_K8S_POD_LABELS
      value: '{"sidecar.istio.io/inject":"false"}'
```
2022-12-31 01:37:09 +01:00
Anbraten
3b0263442a
Adding initial version of Kubernetes backend (#552)
Co-authored-by: laszlocph <laszlo@laszlo.cloud>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Rynoxx <rynoxx@grid-servers.net>
2022-09-05 06:01:14 +02:00