mirrors/woodpecker
1
0
Fork 0
mirror of https://github.com/woodpecker-ci/woodpecker.git synced 2024-12-19 15:06:34 +00:00
Code Issues Projects Releases Wiki Activity
6220 commits 31 branches 77 tags 148 MiB
Commit graph

179 commits

Author SHA1 Message Date
Elias
32a1199519
fix: bug annotations (#3255) 
Fix Issue: https://github.com/woodpecker-ci/woodpecker/issues/3254

Co-authored-by: elias.souza <elias.souza@quintoandar.com.br>
 2024-01-22 13:39:49 +01:00
qwerty287
5e2f7d81b3
Clean up models (#3228) 2024-01-22 07:56:18 +01:00
Thomas Anderson
072fa29f4a
Fixed Pods creation of WP services (#3236) 
Closes #3178
 2024-01-21 03:56:37 +01:00
qwerty287
d1d2e9723d
Support custom steps entrypoint (#2985) 
Closes https://github.com/woodpecker-ci/woodpecker/issues/278

---------

Co-authored-by: Anbraten <anton@ju60.de>
Co-authored-by: 6543 <6543@obermui.de>
 2024-01-19 05:34:02 +01:00
Thomas Anderson
10f2e209d6
Secured kubernetes backend configuration (#3204) 
Follow up of #3165
 2024-01-15 03:59:08 +01:00
qwerty287
001b5639a6
Use assert for test (#3201) 
instead of `if`s
 2024-01-14 19:33:58 +01:00
qwerty287
b9f6f3f9fb
Replace goimports with gci (#3202) 
`gci` seems to be much more strict.
 2024-01-14 18:22:06 +01:00
Thomas Anderson
0611fa9b32
Added protocol in port configuration (#2993) 
Closes  #2727
 2024-01-12 23:57:24 +01:00
Thomas Anderson
9bbc446009
Kubernetes AppArmor and seccomp (#3123) 
Closes #2545

seccomp
https://kubernetes.io/docs/tutorials/security/seccomp/

https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/135-seccomp/README.md

AppArmor
https://kubernetes.io/docs/tutorials/security/apparmor/

fddcbb9cbf/keps/sig-node/24-apparmor/README.md
Went ahead and implemented API from KEP-24 above.
 2024-01-12 23:32:24 +01:00
Robert Kaussow
9bbba4441d
Enable golangci linter forcetypeassert (#3168) 
Split out from https://github.com/woodpecker-ci/woodpecker/pull/2960
 2024-01-12 02:01:02 +01:00
Robert Kaussow
f813badcf9
Enable golangci linter contextcheck (#3170) 
Split out from https://github.com/woodpecker-ci/woodpecker/pull/2960
 2024-01-11 22:15:15 +01:00
6543
f8fb28e651
More docker backend test remove more undocumented (#3156) 
remove Sysctls and IpcMode
 2024-01-11 19:30:13 +01:00
qwerty287
b0a2b1cf2d
Lowercase all log strings (#3173) 
from #3161

---------

Co-authored-by: 6543 <6543@obermui.de>
 2024-01-11 19:17:07 +01:00
6543
d1fe86b7be
Use UUID as podName and cleanup arguments for Kubernetes backend (#3135) 
to much args are just horrible to maintain. And we already have it nice
structured stored as step.
 2024-01-11 16:32:37 +01:00
Robert Kaussow
7756c60a33
Enable golangci linter stylecheck (#3167) 
This PR only fixes error string formatting, log message strings are
still mixed upper/lowercase (see
https://github.com/woodpecker-ci/woodpecker/pull/3161#issuecomment-1885140649)
and I'm not aware of a linter to enforce it.
 2024-01-10 22:56:42 +01:00
qwerty287
00df53e941
Clean up logging (#3161) 
- use `Err` method instead of format strings
- use `Msg` if no format string is used
 2024-01-10 20:57:12 +01:00
qwerty287
12c40eb957
Enable gocritic and don't ignore globally (#3159) 
Use `nolint` directives instead.

From #2960
 2024-01-10 15:34:44 +01:00
qwerty287
768fd71841
Enable some linters (#3129) 
Mostly those that did not require much work.

From #2960

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
 2024-01-09 21:35:37 +01:00
6543
c91c6fbe9e
TestToConfigSmall ignore sort order of env vars (#3154) 
so this https://ci.woodpecker-ci.org/repos/3780/pipeline/11803/30 can
not randomly happen
 2024-01-09 18:34:11 +01:00
6543
e3cf7b212c
fix also test of #3137 2024-01-09 16:36:17 +01:00
6543
06538504f8
fix regression: save env into configEnv 
introduced by #3137
 2024-01-09 16:28:10 +01:00
6543
cd59a85230
Use name in backend types instead of alias (#3142) 2024-01-09 15:22:59 +01:00
6543
12df59d0ec
Add step name as label to docker containers (#3137) 
and add a test
 2024-01-09 06:01:34 +01:00
6543
c64340cf8f
Use UUID instead of step name where possible (#3136) 
things I found while looking at  #3109
 2024-01-09 05:43:03 +01:00
6543
31614d0e38
Use step type to detect services in Kubernetes backend (#3141) 
and use the correct name for tail log

---------

Co-authored-by: Anbraten <anton@ju60.de>
 2024-01-09 05:42:36 +01:00
Thomas Anderson
c0fc4828ff
Flexible image pull secret reference (#3016) 
Co-authored-by: pat-s <patrick.schratz@gmail.com>
 2024-01-05 08:33:56 +01:00
qwerty287
e575ffe72d
Add some tests (#3030) 2023-12-27 10:36:49 +01:00
Thomas Anderson
253d702bc7
Fix IPv6 host aliases for kubernetes (#2992) 
Closes #2991


[Tests](https://github.com/woodpecker-ci/woodpecker/pull/2993#issuecomment-1868048169)

---------

Co-authored-by: 6543 <6543@obermui.de>
 2023-12-23 00:42:30 +01:00
6543
cd9d425a0d
Fix cli lint throwing error on warnings (#2995) 
split out from  #2771

---------

Co-authored-by: Anbraten <anton@ju60.de>
 2023-12-23 00:34:17 +02:00
Thomas Anderson
01a955ed0e
Kubernetes refactor (#2794) 
Kubernetes backend refactoring and tests

---------
Co-authored-by: 6543 <6543@obermui.de>
 2023-12-19 04:53:52 +01:00
Nikolai Rodionov
f7f78b2a3f
feat(k8s): Add a port name to service definition (#2933) 
It should cover this issue:  https://github.com/woodpecker-ci/woodpecker/issues/2931

To sum up, when several ports need to be specified, they must be named
 2023-12-19 02:38:18 +01:00
renovate[bot]
b66f6cb118
fix(deps): update golang (packages) (#2958) 
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
|
[github.com/caddyserver/certmagic](https://togithub.com/caddyserver/certmagic)
| require | minor | `v0.19.2` -> `v0.20.0` |
| [github.com/expr-lang/expr](https://togithub.com/expr-lang/expr) |
require | patch | `v1.15.6` -> `v1.15.7` |
| [github.com/google/uuid](https://togithub.com/google/uuid) | require |
minor | `v1.4.0` -> `v1.5.0` |
|
[github.com/jellydator/ttlcache/v3](https://togithub.com/jellydator/ttlcache)
| require | patch | `v3.1.0` -> `v3.1.1` |
| [github.com/mattn/go-sqlite3](https://togithub.com/mattn/go-sqlite3) |
require | patch | `v1.14.18` -> `v1.14.19` |
| [github.com/xanzy/go-gitlab](https://togithub.com/xanzy/go-gitlab) |
require | minor | `v0.94.0` -> `v0.95.2` |
| [google.golang.org/grpc](https://togithub.com/grpc/grpc-go) | require
| minor | `v1.59.0` -> `v1.60.0` |
| [k8s.io/api](https://togithub.com/kubernetes/api) | require | minor |
`v0.28.4` -> `v0.29.0` |
| [k8s.io/apimachinery](https://togithub.com/kubernetes/apimachinery) |
require | minor | `v0.28.4` -> `v0.29.0` |
| [k8s.io/client-go](https://togithub.com/kubernetes/client-go) |
require | minor | `v0.28.4` -> `v0.29.0` |

---

### Release Notes

<details>
<summary>caddyserver/certmagic
(github.com/caddyserver/certmagic)</summary>

###
[`v0.20.0`](https://togithub.com/caddyserver/certmagic/releases/tag/v0.20.0)

[Compare
Source](https://togithub.com/caddyserver/certmagic/compare/v0.19.2...v0.20.0)

This release vastly improves storage cleaning as well improving a few
smaller things. There is a minor breaking change as we get ever closer
to v1.0.

- ⚠️ The `DecisionFunc` for On-Demand TLS now takes a
`context.Context` value as its first argument. The context carries the
`ClientHelloInfo` value (keyed by `ClientHelloInfoCtxKey`) for logging
purposes.
- Storage cleaning is now synchronized across the cluster, including
process restarts. The state of cleaning expired certificates and OCSP
staples is written to storage, and distributed locking is used to ensure
that only 1 instance does it at a time. This greatly reduces costs for
expensive storage backends! Cleaning is also done less often when the
process is frequently restarted because the state is written to storage,
so it is not forgotten after shutting down.
-   `.home.arpa` is now considered an internal suffix.
-   Backoff timings have been tuned based on real-world experience.

#### What's Changed

- README: Add hint about NextProtos for certmagic.TLS by
[@&#8203;oliverpool](https://togithub.com/oliverpool) in
[https://github.com/caddyserver/certmagic/pull/251](https://togithub.com/caddyserver/certmagic/pull/251)
- Bump golang.org/x/net from 0.11.0 to 0.17.0 by
[@&#8203;dependabot](https://togithub.com/dependabot) in
[https://github.com/caddyserver/certmagic/pull/253](https://togithub.com/caddyserver/certmagic/pull/253)
- Optionally pass the context argument down to the OnDemand decision
func by [@&#8203;ankon](https://togithub.com/ankon) in
[https://github.com/caddyserver/certmagic/pull/255](https://togithub.com/caddyserver/certmagic/pull/255)
- Retain the error stack if `checkIfCertShouldBeObtained` returns an
error by [@&#8203;ankon](https://togithub.com/ankon) in
[https://github.com/caddyserver/certmagic/pull/256](https://togithub.com/caddyserver/certmagic/pull/256)
- Add OCSP stapling unit tests by
[@&#8203;kenjenkins](https://togithub.com/kenjenkins) in
[https://github.com/caddyserver/certmagic/pull/259](https://togithub.com/caddyserver/certmagic/pull/259)

#### New Contributors

- [@&#8203;oliverpool](https://togithub.com/oliverpool) made their first
contribution in
[https://github.com/caddyserver/certmagic/pull/251](https://togithub.com/caddyserver/certmagic/pull/251)

**Full Changelog**:
https://github.com/caddyserver/certmagic/compare/v0.19.2...v0.20.0

</details>

<details>
<summary>expr-lang/expr (github.com/expr-lang/expr)</summary>

###
[`v1.15.7`](https://togithub.com/expr-lang/expr/releases/tag/v1.15.7)

[Compare
Source](https://togithub.com/expr-lang/expr/compare/v1.15.6...v1.15.7)

**Expr** is a Go-centric expression language designed to deliver dynamic
configurations with unparalleled accuracy, safety, and speed.

##### In this release:

- Fixed commutative property for comparison between a value and a
pointer. ([#&#8203;490](https://togithub.com/expr-lang/expr/issues/490))
- Checker: forbid accessing built-ins and custom functions from `$env`.
([#&#8203;495](https://togithub.com/expr-lang/expr/issues/495))
- Enhanced the number parser to include support for parsing hexadecimal,
binary, and octal literals.
([#&#8203;483](https://togithub.com/expr-lang/expr/issues/483))
- Added `GetSource()` method to `vm.Program`.
([#&#8203;491](https://togithub.com/expr-lang/expr/issues/491))

</details>

<details>
<summary>google/uuid (github.com/google/uuid)</summary>

### [`v1.5.0`](https://togithub.com/google/uuid/releases/tag/v1.5.0)

[Compare
Source](https://togithub.com/google/uuid/compare/v1.4.0...v1.5.0)

##### Features

- Validate UUID without creating new UUID
([#&#8203;141](https://togithub.com/google/uuid/issues/141))
([9ee7366](9ee7366e66))

</details>

<details>
<summary>jellydator/ttlcache
(github.com/jellydator/ttlcache/v3)</summary>

###
[`v3.1.1`](https://togithub.com/jellydator/ttlcache/releases/tag/v3.1.1)

[Compare
Source](https://togithub.com/jellydator/ttlcache/compare/v3.1.0...v3.1.1)

Fix a bug in the `Range` method that causes a panic when the cache is
empty

</details>

<details>
<summary>mattn/go-sqlite3 (github.com/mattn/go-sqlite3)</summary>

###
[`v1.14.19`](https://togithub.com/mattn/go-sqlite3/compare/v1.14.18...v1.14.19)

[Compare
Source](https://togithub.com/mattn/go-sqlite3/compare/v1.14.18...v1.14.19)

</details>

<details>
<summary>xanzy/go-gitlab (github.com/xanzy/go-gitlab)</summary>

###
[`v0.95.2`](https://togithub.com/xanzy/go-gitlab/compare/v0.95.1...v0.95.2)

[Compare
Source](https://togithub.com/xanzy/go-gitlab/compare/v0.95.1...v0.95.2)

###
[`v0.95.1`](https://togithub.com/xanzy/go-gitlab/compare/v0.95.0...v0.95.1)

[Compare
Source](https://togithub.com/xanzy/go-gitlab/compare/v0.95.0...v0.95.1)

###
[`v0.95.0`](https://togithub.com/xanzy/go-gitlab/compare/v0.94.0...v0.95.0)

[Compare
Source](https://togithub.com/xanzy/go-gitlab/compare/v0.94.0...v0.95.0)

</details>

<details>
<summary>grpc/grpc-go (google.golang.org/grpc)</summary>

### [`v1.60.0`](https://togithub.com/grpc/grpc-go/releases/tag/v1.60.0):
Release 1.60.0

[Compare
Source](https://togithub.com/grpc/grpc-go/compare/v1.59.0...v1.60.0)

### Security

- credentials/tls: if not set, set TLS MinVersion to 1.2 and
CipherSuites according to supported suites not forbidden by RFC7540.
- This is a behavior change to bring us into better alignment with RFC
7540.

### API Changes

- resolver: remove deprecated and experimental
`ClientConn.NewServiceConfig`
([#&#8203;6784](https://togithub.com/grpc/grpc-go/issues/6784))
- client: remove deprecated `grpc.WithServiceConfig` `DialOption`
([#&#8203;6800](https://togithub.com/grpc/grpc-go/issues/6800))

### Bug Fixes

- client: fix race that could cause a deadlock while entering idle mode
and receiving a name resolver update
([#&#8203;6804](https://togithub.com/grpc/grpc-go/issues/6804))
- client: always enable TCP keepalives with OS defaults
([#&#8203;6834](https://togithub.com/grpc/grpc-go/issues/6834))
- credentials/alts: fix a bug preventing ALTS from connecting to the
metadata server if the default scheme is overridden
([#&#8203;6686](https://togithub.com/grpc/grpc-go/issues/6686))
- Special Thanks: [@&#8203;mjamaloney](https://togithub.com/mjamaloney)

### Behavior Changes

- server: Do not return from Stop() or GracefulStop() until all
resources are released
([#&#8203;6489](https://togithub.com/grpc/grpc-go/issues/6489))
    -   Special Thanks: [@&#8203;fho](https://togithub.com/fho)

### Documentation

- codes: clarify that only codes defined by this package are valid and
that users should not cast other values to `codes.Code`
([#&#8203;6701](https://togithub.com/grpc/grpc-go/issues/6701))

</details>

<details>
<summary>kubernetes/api (k8s.io/api)</summary>

###
[`v0.29.0`](https://togithub.com/kubernetes/api/compare/v0.28.4...v0.29.0)

[Compare
Source](https://togithub.com/kubernetes/api/compare/v0.28.4...v0.29.0)

</details>

<details>
<summary>kubernetes/apimachinery (k8s.io/apimachinery)</summary>

###
[`v0.29.0`](https://togithub.com/kubernetes/apimachinery/compare/v0.28.4...v0.29.0)

[Compare
Source](https://togithub.com/kubernetes/apimachinery/compare/v0.28.4...v0.29.0)

</details>

<details>
<summary>kubernetes/client-go (k8s.io/client-go)</summary>

###
[`v0.29.0`](https://togithub.com/kubernetes/client-go/compare/v0.28.4...v0.29.0)

[Compare
Source](https://togithub.com/kubernetes/client-go/compare/v0.28.4...v0.29.0)

</details>

---

### Configuration

📅 **Schedule**: Branch creation - "before 4am" (UTC), Automerge -
"before 4am" (UTC).

🚦 **Automerge**: Enabled.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

👻 **Immortal**: This PR will be recreated if closed unmerged. Get
[config help](https://togithub.com/renovatebot/renovate/discussions) if
that's undesired.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://developer.mend.io/github/woodpecker-ci/woodpecker).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy45My4xIiwidXBkYXRlZEluVmVyIjoiMzcuOTMuMSIsInRhcmdldEJyYW5jaCI6Im1haW4ifQ==-->

---------

Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
Co-authored-by: Robert Kaussow <mail@thegeeklab.de>
 2023-12-17 14:37:26 +01:00
6543
1f8b3b5e1b
Only update pipelineStatus in one place (#2952) 2023-12-15 10:03:05 +01:00
qwerty287
ff1f51d6a9
Rename engine to backend (#2950) 
rename based on https://woodpecker-ci.org/docs/usage/terminiology

---------

Co-authored-by: 6543 <6543@obermui.de>
 2023-12-14 19:20:47 +01:00
runephilosof-karnovgroup
adb2c82790
Update go module path for major version 2 (#2905) 
https://go.dev/doc/modules/release-workflow#breaking

Fixes https://github.com/woodpecker-ci/woodpecker/issues/2913 fixes
#2654
```
runephilosof@fedora:~/code/platform-woodpecker/woodpecker-repo-configurator (master)$ go get go.woodpecker-ci.org/woodpecker@v2.0.0
go: go.woodpecker-ci.org/woodpecker@v2.0.0: invalid version: module contains a go.mod file, so module path must match major version ("go.woodpecker-ci.org/woodpecker/v2")
```

---------

Co-authored-by: qwerty287 <80460567+qwerty287@users.noreply.github.com>
 2023-12-08 08:15:08 +01:00
Kaylyn Bogle
bc95443853
Fix podman agent container in v2 (#2897) 
Add additional string matching to determine when container is not found
or running when invoked via podman compatibility socket

Co-authored-by: qwerty287 <80460567+qwerty287@users.noreply.github.com>
 2023-12-02 20:22:23 +01:00
Thomas Anderson
3adb98b287
Simple security context options (Kubernetes) (#2550) 2023-11-26 08:46:06 +01:00
qwerty287
70711ed9db
Replace interface{} with any (#2807) 
like golang:
2580d0e08d
 2023-11-12 18:23:48 +01:00
6543
5a7b689e30
Switch to go vanity urls (#2706) 
Co-authored-by: Anbraten <anton@ju60.de>
 2023-11-07 08:04:33 +01:00
J-Ha
9af71dcc98
Use unique label selector for pod label for kubernetes services (#2723) 
Co-authored-by: Julian Haseleu <julian.haseleu@dreamit.de>
Co-authored-by: qwerty287 <80460567+qwerty287@users.noreply.github.com>
 2023-11-04 10:35:37 +01:00
6543
7bc40f20cb
Option to change temp dir for local backend (#2702) 
---
*Sponsored by Kithara Software GmbH*
 2023-11-02 15:45:18 +01:00
6543
c75068920c
save GOOS and GOARCH in local engine (#2701) 
make refactoring easyer
 2023-11-02 07:58:32 +01:00
Thomas Anderson
de53b906e8
Add ports into pipeline backend step model (#2656) 
Closes #2655.


[Pipeline](https://woodpecker-ci.org/docs/next/usage/services#complete-pipeline-example):
```yaml
services:
  database:
    image: mysql
    environment:
      - MYSQL_DATABASE=test
      - MYSQL_ROOT_PASSWORD=example
    ports:
      - 3306

steps:
  get-version:
    image: ubuntu
    commands:
      - ( apt update && apt dist-upgrade -y && apt install -y mysql-client 2>&1 )> /dev/null
      - sleep 60s # need to wait for mysql-server init
      - echo 'SHOW VARIABLES LIKE "version"' | mysql -uroot -hdatabase test -pexample
```

Service:
```yaml
apiVersion: v1
kind: Service
metadata:
  name: wp-01hdq6gbkw1mn6k1655fs3rntf-0-services-0
  namespace: woodpecker-runtime
  ...
  selfLink: >-
    /api/v1/namespaces/woodpecker-runtime/services/wp-01hdq6gbkw1mn6k1655fs3rntf-0-services-0
status:
  loadBalancer: {}
spec:
  ports:
    - protocol: TCP
      port: 3306
      targetPort: 3306
  selector:
    step: database
  clusterIP: 10.43.180.120
  clusterIPs:
    - 10.43.180.120
  type: ClusterIP
  sessionAffinity: None
  ipFamilies:
    - IPv4
  ipFamilyPolicy: SingleStack
  internalTrafficPolicy: Cluster
```
 2023-11-02 04:12:41 +01:00
Thomas Anderson
3620c84da4
Unregister stateless agents from server on termination (#2606) 
Closes #2027

---------

Co-authored-by: 6543 <6543@obermui.de>
 2023-11-02 01:53:47 +02:00
6543
ebe0307c6b
Let the backend engine report the current platform (#2688) 
if you run woodpecker-agent on windows and connect it to an docker
daemon, there could be two different platforms possible, as you can
switch from linux to windows mode and visa versa


---
*Sponsored by Kithara Software GmbH*
 2023-11-01 15:38:37 +01:00
6543
48e4eceb0a
local backend make cmd log output similar to other shells (#2678) 
the cmd currently shows the full prompt and drop the exact error level.

this set the prompt to be hidden and let cmd exit with error level
reported by the command

---
*Sponsored by Kithara Software GmbH*
 2023-11-01 11:49:36 +01:00
Anbraten
4198c447fb
Destroy steps after they are done (#2681) 
Co-authored-by: 6543 <m.huber@kithara.com>
 2023-11-01 09:35:11 +01:00
6543
5b3bba726d
local backend ignore errors in commands inbetween (#2636) 
for normal posix shells we have to add the `-e ` option ... but as there
are more shells out there we have to handle this edgecases on base per
base case.

create a switch case statement in woodpecker should be fine as there is
only a finite number of shells, used in production.

also close  #2612

---
*Sponsored by Kithara Software GmbH*
 2023-10-28 13:38:47 +02:00
qwerty287
e74115027b
Add some tests (#2652) 
and some cleanups
 2023-10-28 13:37:54 +02:00
qwerty287
31240b5771
Remove SSH backend (#2635) 
The SSH backend is, similar to Gogs and Coding for forges, completely
unmaintained and seems unused (it is likely broken but we didn't get any
reports).
Instead, you should directly run the agent on the SSH machine with the
`local` backend.
 2023-10-24 01:55:30 +02:00
qwerty287
efbde332a3
Fix CI_WORKSPACE in local backend (#2627) 
closes https://github.com/woodpecker-ci/woodpecker/issues/2610
 2023-10-23 12:54:10 +02:00
Patrick Schratz
1b98e58014
Update to docker 23 (#2577) 
fix #1807
 2023-10-14 12:39:45 +02:00
qwerty287
46273e54d8
Require Go 1.21 (#2553) 
Main change are the new `maps` and `slices` stdlib packages so we can
replace `golang.org/x/exp`.
 2023-10-09 09:11:08 +02:00
6543
8629a418f8
local backend set home variable windows-equivalent too (#2323) 2023-10-01 14:56:49 +02:00
Kleber Rocha
61b5672051
Add option to configure tolerations in kubernetes backend (#2249) 
This code add a feature to support tolerations in Kubernetes Backend

---------

Signed-off-by: Kleber Rocha <klinux@gmail.com>
 2023-08-22 22:34:59 +02:00
6543
3954d85a5b
Add "plugin" support to local backend (#2239) 
and a nit to improve the clone


---
*Sponsored by Kithara Software GmbH*
 2023-08-22 22:00:32 +02:00
6543
0c282e86e8
fix regression and add trace in clone for local backend (#2241) 
regression of #2017 
close  #2211
 2023-08-18 14:26:04 +02:00
qwerty287
8cdac56d8f
Check for correct license header (#2137) 2023-08-10 11:06:00 +02:00
6543
d253f8cc30
Make sure we dont have hidden options for backend and pipeline compiler (#2123) 
move options based on **os.Getenv** into flags

---------
*Sponsored by Kithara Software GmbH*
 2023-08-07 21:13:26 +02:00
6543
10b1cfcd3b
Enhance local backend (#2017) 
make local backend able to clone from private

---------
*Sponsored by Kithara Software GmbH*
Co-authored-by: Bruno BELANYI <bruno@belanyi.fr>
 2023-08-07 15:39:58 +02:00
6543
a83b0fcfd3
Drop deprecated built-in environment variables (#2048) 
Drop:
 - `CI_BUILD_*`
 - `CI_PREV_BUILD_*`
 - `CI_JOB_*`
 - `*_LINK`
 - `CI_SYSTEM_ARCH`
 - `CI_REPO_REMOTE`
 2023-07-28 14:58:20 +02:00
6543
3cd78c9409
Refactor agent (#2021) 
- code cleanup
- init backend engine only once
- pass a taskUUID to the backend

---
*Sponsored by Kithara Software GmbH*
 2023-07-20 20:39:20 +02:00
6543
b54f6ebad6
Let pipeline-compiler export step types (#1958) 2023-07-11 15:53:05 +02:00
6543
7b97e27fea
docker: use uuid instead of name as identifyer (#1967) 
close #1959
 2023-07-10 17:59:25 +02:00
6543
5393aa5d3b
Move docker specific volume & network settings into backend code (#1956) 
... if we want to make them be changed ... it should be an
agent-backend-option
 2023-07-09 21:03:19 +02:00
Thomas Anderson
1875e2a416
Fix platform to node-selector mapping for k8s (#1963) 
Closes #1955
 2023-07-09 20:22:50 +03:00
Thomas Anderson
cef135eba5
Do not set Pod's Image pull policy if not explicitly set (#1914) 2023-07-07 20:02:13 +02:00
Thomas Anderson
d46b91a94c
Refactored K8s Pod node selector (#1909) 
- using existing map instead of an array
- applying var name convention
- using arch constant
 2023-07-07 07:46:48 +02:00
Ovidiu Calbajos
609ba481b5
Support for k8s serviceAccount and nodeSelector (#1842) 
Add the possiblity to specify the Kubernetes serviceAccount and/or
nodeSelector to be used on individual steps for Kubernetes executor
 2023-06-12 16:00:59 +02:00
Ben Cordero
3158980d3e
Get workflow id from clone steps (#1839) 
When in local mode, `getWorkflowIDFromStep` can handle normal steps with
a name like `wp_01h2a6qggwz68zekrkbwqq9rny_0_step_0`.

However, it will fail on clone (unless `skip_clone: true`) with an
`invalid step name` error.

```
invalid step name wp_01h2a2ebppp43bwjdfdsyj1m6m_0_clone
```

This patch handles either `_stage_` or `_clone` as the separator that
the local backend can use to extract the workflowID.
 2023-06-08 03:33:23 +02:00
Anbraten
556607b525
Rework log streaming and related functions (#1802) 
closes #1801
closes #1815 
closes #1144
closes  #983
closes  #557
closes #1827
regression of #1791

# TODO
- [x] adjust log model
- [x] add migration for logs
- [x] send log line via grpc using step-id
- [x] save log-line to db
- [x] stream log-lines to UI
- [x] use less structs for log-data
- [x] make web UI work
  - [x] display logs loaded from db
  - [x] display streaming logs
- [ ] ~~make migration work~~ -> dedicated pull (#1828)

# TESTED
- [x] new logs are stored in database
- [x] log retrieval via cli (of new logs) works
- [x] log streaming works (tested via curl & webui)
- [x] log retrieval via web (of new logs) works

---------

Co-authored-by: 6543 <6543@obermui.de>
 2023-06-06 09:52:08 +02:00
6543
f81b447638
remove undocumented docker specific option (#1825) 2023-06-05 17:30:32 +02:00
Stephen Muth
2941e508b3
Allow setting resources for kubernetes on a per-step basis (#1767) 
This add a simple implementation of requests/limits for individual
steps. There is no validation of what the resource actually is beyond
checking that it can successfully be converted to a Quantity, so it can
be used for things other than just memory/CPU.

close #1809
 2023-06-04 00:50:08 +02:00
Patrick Schratz
3d0338315f
Revert to docker 20.10.x for API 1.41 compatibility (#1792) 
fix #1786

---------

Co-authored-by: 6543 <6543@obermui.de>
 2023-06-02 15:09:38 +02:00
Patrick Schratz
d13a8744ce
Add nodeSelector using platform for kubernetes (#1737) 
Co-authored-by: Anbraten <anton@ju60.de>
 2023-05-18 11:21:20 +02:00
Anbraten
435f5ae207
Improve local backend (#1762) 
Co-authored-by: 6543 <6543@obermui.de>
 2023-05-17 14:53:23 +02:00
Anbraten
188123ea74
Update dependencies (#1744) 2023-05-11 00:28:02 +02:00
6543
80d5d233b3
Local Backend should create SubDirs (#1706) 
close #1705
 2023-04-20 00:56:03 +02:00
Bruno BELANYI
ea95d5aa76
Set 'HOME' during local pipeline step (#1686) 
close #1685

---------

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lauris BH <lauris@nix.lv>
 2023-04-02 16:47:22 +02:00
Neil Hanlon
a95a5b43bf
fix(backend/kubernetes): Ensure valid naming of name field (#1661) 
- Kubernetes v1.26 on VKE causes error when creating persistent volume
claim because of uppercase characters in name field

This patch is trivial just in order to get it working - happy to
implement differently.

The error in question:

```
The PersistentVolumeClaim "wp-01G1131R63FWBSPMA4ZAZTKLE-0-clone-0" is invalid: metadata.name: Invalid value: "wp-01G1131R63FWBSPMA4ZAZTKLE-0-clone-0": a lowercase RFC 1123 subdomain must consist of lower case alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character (e.g. 'example.com', regex used for validation is '[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*')
```
 2023-03-21 20:00:45 +01:00
Lauris BH
46452fbd84
Update Go dependencies and minimal Go version to 1.20 (#1650) 
Signed-off-by: 6543 <6543@obermui.de>
Co-authored-by: 6543 <6543@obermui.de>
 2023-03-21 00:48:15 +01:00
qwerty287
f582ad3159
Various enhancements in configuration (#1645) 
- backends: move to cli flags instead of os.Getenv
- ssh: support 2fa with key and password
- allow to set grpc jwt secret (solves todo)
- allow to set default and max timeout (solves todo)

Closes https://github.com/woodpecker-ci/woodpecker/issues/896
Closes https://github.com/woodpecker-ci/woodpecker/issues/1131
 2023-03-19 20:24:43 +01:00
Anbraten
fa5b0fb96e
Fix linter (#1647) 2023-03-18 20:35:27 +01:00
Alexander Matyushentsev
ee969979c6
fix: backend auto-detection should be consistent (#1618) 
Closes https://github.com/woodpecker-ci/woodpecker/issues/1617

The `woodpecker exec` auto-detects the backend by iterating over a map
of backends. However, since Go 1 the runtime randomizes map iteration
order, so a random backend might be chosen during each execution.

PR changes to auto-detection to iterate over the backends list with
predefined priority: `docker`, `local`, `ssh`, `kubernetes`.

---------

Signed-off-by: Alexander Matyushentsev <AMatyushentsev@gmail.com>
 2023-03-13 09:07:41 +02:00
Alexander Matyushentsev
5e1171d7a7
fix: docker backend should not close 'engine.Tail' result (#1616) 
Closes https://github.com/woodpecker-ci/woodpecker/issues/1615

The error described in
https://github.com/woodpecker-ci/woodpecker/issues/1615 is happening
because `Tail` method of the docker backend closes the instance of
`io.ReadCloser` it returns in `defer` function. As a result anything
that try to read data returned by `Tail` method eventually will attempt
to read from closes reader and get an error:


2171212c5a/pipeline/backend/docker/docker.go (L229)

The fix is just don't close returned reader and let the consumer of
`Tail` method do it. Good thing is that `Tail` is used only in one place
and reader is correctly closed:


2171212c5a/pipeline/pipeline.go (L231-L237)

Example of `woodpecker exec` output using pipeline from
https://github.com/woodpecker-ci/woodpecker/issues/1615 with the fix:

```
woodpecker exec .woodpecker.yaml
[step1:L0:0s] + echo step1
[step1:L1:0s] step1
[step2:L0:0s] + echo step2
[step2:L1:0s] step2
```

Signed-off-by: Alexander Matyushentsev <AMatyushentsev@gmail.com>
Co-authored-by: Lauris BH <lauris@nix.lv>
 2023-03-12 16:01:09 +01:00
Stephen Muth
be4ce2e02d
Ensure the SharedInformerFactory closes eventually (#1585) 
Otherwise it will continue to collect events in the background, consuming memory that can't be GC'd.
 2023-02-16 00:54:33 +01:00
6543
18d3139e9e
Use modern error handling and enforce it via lint (#1327) 
Co-authored-by: Anbraten <anton@ju60.de>
 2023-02-02 00:08:02 +01:00
Lauris BH
f26a87acce
Deduplicate step docker container volumes (#1571) 
Try to fix #1495

It's very hard to reproduce it and only way to fix when it gets in this
state is woodpecker agent restart.

This anyway fixes problem if step mounts and
`WOODPECKER_BACKEND_DOCKER_VOLUMES` conflict
 2023-01-31 21:33:40 +01:00
6543
5c617accd8
kubernetes ignore labels and annotations if not set (#1528) 
close #1527
regression of #1510
 2023-01-04 19:51:21 +02:00
Stephen Muth
1816f6c715
Allow adding additional labels/annotations to kubernetes worker pods (#1510) 
Example agent environment configuration using the new value:
```yaml
  - env:
    - name: WOODPECKER_BACKEND
      value: kubernetes
    - name: WOODPECKER_BACKEND_K8S_NAMESPACE
      value: default
    - name: WOODPECKER_BACKEND_K8S_POD_LABELS
      value: '{"sidecar.istio.io/inject":"false"}'
```
 2022-12-31 01:37:09 +01:00
Sergio Fenoll
f0e518a5a2
Add option to ignore failures on steps (#1219) 
closes #1181
closes #834 

Adds `ignore_failure` to pipeline steps. When it's set to true,
if the step fails the following steps continue to execute as if no failure had occurred.

---

failure enums idea:
* fail (default) = if other steps run in parallel, wait for them and
then let workflow fail
* cancel = if other steps run in parallel, kill them
* ignore = we mark the step as failed but it wont have any impact
 2022-11-15 19:47:27 +01:00
Josh Soref
023d03dd61
Spelling (#1405) 
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
 2022-11-09 08:12:17 +01:00
6543
e8490a757f
GenerateScript should not return encoded script (#1397) 
followup to #1395
 2022-11-06 13:36:34 +01:00
qwerty287
e901f605b1
Fix local and ssh backends (#1395) 
Base64-encoded string was not decoded.
 2022-11-05 14:44:33 +02:00
6543
b15ca52a63
Move constrain to only have a single command in backend to run to dedicated backends (#1032) 
at the moment we compile a script that we can pipe in as single command
this is because of the constrains the docker backend gives us.

so we move it into the docker backend and eventually get rid of it altogether
 2022-10-31 00:26:49 +01:00
Harikesh00
36e42914fa
Renamed procs/jobs to steps in code (#1331) 
Renamed `procs` to `steps` in code for the issue #1288

Co-authored-by: Harikesh Prajapati <harikesh.prajapati@druva.com>
Co-authored-by: qwerty287 <ndev@web.de>
Co-authored-by: qwerty287 <80460567+qwerty287@users.noreply.github.com>
Co-authored-by: 6543 <6543@obermui.de>
 2022-10-28 17:38:53 +02:00
qwerty287
849e05bb8b
Rename build to pipeline in code (#1224) 
Ref:  #745

Co-authored-by: Anbraten <anton@ju60.de>
Co-authored-by: 6543 <6543@obermui.de>
 2022-10-18 03:24:12 +02:00
Anbraten
70114ed1fd
Fix handling of empty strings for default docker volumes (#1209) 2022-09-27 11:43:35 +02:00
Lauris BH
2e08dd2333
Add option to set default volumes for docker backend (#1203) 2022-09-26 16:59:26 +02:00
6543
9c99406a06
Use archive.org to preserve code comment 2022-09-26 14:52:28 +02:00