diff --git a/.drone.sh b/.drone.sh index f82021cdd..c38654db6 100755 --- a/.drone.sh +++ b/.drone.sh @@ -1,27 +1,8 @@ #!/bin/sh -# only execute this script as part of the pipeline. -# [ -z "$CI" ] && echo "missing ci environment variable" && exit 2 - -# only execute the script when github token exists. -# [ -z "$SSH_KEY" ] && echo "missing ssh key" && exit 3 - -# write the ssh key. -# mkdir /root/.ssh -# echo -n "$SSH_KEY" > /root/.ssh/id_rsa -# chmod 600 /root/.ssh/id_rsa - -# add github.com to our known hosts. -# touch /root/.ssh/known_hosts -# chmod 600 /root/.ssh/known_hosts -# ssh-keyscan -H github.com > /etc/ssh/ssh_known_hosts 2> /dev/null - -# clone the extras project. set -e set -x -# git clone git@github.com:drone/drone-enterprise.git extras -# build a static binary with the build number and extra features. go build -ldflags '-extldflags "-static" -X github.com/laszlocph/drone-oss-08/version.VersionDev=build.'${DRONE_BUILD_NUMBER} -o release/drone-server github.com/laszlocph/drone-oss-08/cmd/drone-server GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -ldflags '-X github.com/laszlocph/drone-oss-08/version.VersionDev=build.'${DRONE_BUILD_NUMBER} -o release/drone-agent github.com/laszlocph/drone-oss-08/cmd/drone-agent GOOS=linux GOARCH=arm64 CGO_ENABLED=0 go build -ldflags '-X github.com/laszlocph/drone-oss-08/version.VersionDev=build.'${DRONE_BUILD_NUMBER} -o release/linux/arm64/drone-agent github.com/laszlocph/drone-oss-08/cmd/drone-agent diff --git a/LICENSE b/LICENSE index 04b10d8b6..fed1248b2 100644 --- a/LICENSE +++ b/LICENSE @@ -1,80 +1,3 @@ -The Drone Community Edition (the "Community Edition") is licensed under the -Apache License, Version 2.0 (the "Apache License"). You may obtain a copy of -the Apache License at +Drone-OSS-08 is Apache 2.0 licensed with the source files in this repository having a header indicating which license they are under and what copyrights apply. - http://www.apache.org/licenses/LICENSE-2.0 - -The Drone Enterprise Edition (the "Enterprise Edition") is licensed under -the Drone Enterprise License, Version 1.1 (the "Enterprise License"). A copy -of the Enterprise License is provided below. - -The source files in this repository have a header indicating which license -they are under. The BUILDING file provides instructions for creating the -Community Edition distribution subject to the terms of the Apache License. - ------------------------------------------------------------------------------ - -Licensor: Drone.IO, Inc -Licensed Work: Drone Enterprise Edition - -Additional Use Grant: Usage of the software is free for entities with both: - (a) annual gross revenue under (USD) $1 million - (according to GAAP, or the equivalent in its country - of domicile); and (b) less than (USD) $5 million in - aggregate debt and equity funding. - -Change Date: 2022-01-01 - -Change License: Apache-2.0 - -Notice - -The Drone Enterprise License (this document, or the "License") is not an Open -Source license. However, the Licensed Work will eventually be made available -under an Open Source License, as stated in this License. - ------------------------------------------------------------------------------ - -Drone Enterprise License 1.1 - -Terms - -The Licensor hereby grants you the right to copy, modify, create derivative -works, redistribute, and make non-production use of the Licensed Work. The -Licensor may make an Additional Use Grant, above, permitting limited -production use. - -Effective on the Change Date, or the fourth anniversary of the first publicly -available distribution of a specific version of the Licensed Work under this -License, whichever comes first, the Licensor hereby grants you rights under -the terms of the Change License, and the rights granted in the paragraph -above terminate. - -If your use of the Licensed Work does not comply with the requirements -currently in effect as described in this License, you must purchase a -commercial license from the Licensor, its affiliated entities, or authorized -resellers, or you must refrain from using the Licensed Work. - -All copies of the original and modified Licensed Work, and derivative works -of the Licensed Work, are subject to this License. This License applies -separately for each version of the Licensed Work and the Change Date may vary -for each version of the Licensed Work released by Licensor. - -You must conspicuously display this License on each original or modified copy -of the Licensed Work. If you receive the Licensed Work in original or -modified form from a third party, the terms and conditions set forth in this -License apply to your use of that work. - -Any use of the Licensed Work in violation of this License will automatically -terminate your rights under this License for the current and all other -versions of the Licensed Work. - -This License does not grant you any right in any trademark or logo of -Licensor or its affiliates (provided that you may use a trademark or logo of -Licensor as expressly required by this License). - -TO THE EXTENT PERMITTED BY APPLICABLE LAW, THE LICENSED WORK IS PROVIDED ON -AN "AS IS" BASIS. LICENSOR HEREBY DISCLAIMS ALL WARRANTIES AND CONDITIONS, -EXPRESS OR IMPLIED, INCLUDING (WITHOUT LIMITATION) WARRANTIES OF -MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, NON-INFRINGEMENT, AND -TITLE. +Files under the `docs/` folder is licensed under Creative Commons Attribution-ShareAlike 4.0 International Public License. It is a derivative work of the https://github.com/drone/docs git repository. diff --git a/README.md b/README.md index 3581240c0..281e707b7 100644 --- a/README.md +++ b/README.md @@ -1,31 +1,326 @@ -## Yes, it's a fork +# Drone-OSS-08 -This repository is a hard fork of the Drone CI system. +An opinionated fork of the Drone CI system. -Forked at the `0.8.9` version https://github.com/drone/drone/commit/768ed784bd74b0e0c2d8d49c4c8b6dca99b25e96 +- Based on the v0.8 code tree +- Focused on developer experience. -## Why fork? +[![Build Status](https://cloud.drone.io/api/badges/laszlocph/drone-oss-08/status.svg)](https://cloud.drone.io/laszlocph/drone-oss-08) [![Go Report Card](https://goreportcard.com/badge/github.com/laszlocph/drone-oss-08)](https://goreportcard.com/report/github.com/laszlocph/drone-oss-08) [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) -Drone has been an open-core project since many prior versions. With each source file indicating whether it is part of the Apache 2.0 licensed or the propritary enterprise license. In the 0.8 line the enterprise features were limited to features like autoscaling and secret vaults. +![Drone-OSS-08](docs/drone.png) -However in the 1.0 line, databases other than SQLite, TLS support and agent based horizontal scaling were also moved under the enterprise license. Limiting the open source version to single node, hobbyist deployments. +## Table of contents -The above feature reductions and the lack of clear communication of what is part of the open-source version led to this fork. +- [About this fork](#about-this-fork) + - [Motivation](#motivation) + - [The focus of this fork](#the-focus-of-this-fork) + - [Who uses this fork](#who-uses-this-fork) +- [Pipelines](#pipelines) + - [Getting started](#getting-started) + - [Pipeline documentation](#pipeline-documentation) +- [Plugins](#plugins) + - [Custom plugins](#custom-plugins) +- [Server setup](#server-setup) + - [Quickstart](#quickstart) + - [Authentication](#authentication) + - [Database](#database) + - [SSL](#ssl) + - [Metrics](#metrics) + - [Behind a proxy](#behind-a-proxy) +- [Contributing](#contributing) +- [License](#license) -## The focus of this fork +## About this fork -The focus of this fork is +#### Motivation -- Github -- Kubernetes and VM based backends -- Linux/amd64 -- Some really good features that Drone 1.0 introduced: multiple pipelines, cron triggers +Why fork? See my [motivation](docs/motivation.md) -## Why should you use this fork? +#### The focus of this fork -you shouldn't necessarily. Paying for Drone 1.0 is a fine choice. +This fork is not meant to compete with Drone or reimplement its enterprise features in the open. -Check the issues and releases of this project if you are evaluating this project. -Also you can check the devlog to get the nuances: https://laszlo.cloud/drone-oss-08-devlog-1 +Instead, I'm taking a proven CI system - that Drone 0.8 is - and applying a distinct set of product ideas focusing on: -The project is currently used by one user, with 50+ repos and 500+ builds a week. \ No newline at end of file +- UI experience +- the developer feedback loop +- documentation and best practices +- tighter Github integration +- Kubernetes backend + +with less focus on: + +- niche git systems like gitea, gogs +- computing architectures like arm64 +- new pipeline formats like jsonnet + +#### Who uses this fork + +Currently I know of one organization using this fork. With 50+ users, 130+ repos and more than 300 builds a week. + +## Pipelines + +#### Getting started + +Place this snippet into a file called `.drone.yml` + +```yaml +pipeline: + build: + image: debian:stable-slim + commands: + - echo "This is the build step" + a-test-step: + image: debian:stable-slim + commands: + - echo "Testing.." +``` + +The pipeline runs on the Drone CI server and typically triggered by webhooks. One benefit of the container architecture is that it runs on your laptop too: + +```sh +$ drone exec --local +stable-slim: Pulling from library/debian +a94641239323: Pull complete +Digest: sha256:d846d80f98c8aca7d3db0fadd14a0a4c51a2ce1eb2e9e14a550b3bd0c45ba941 +Status: Downloaded newer image for debian:stable-slim +[build:L0:0s] + echo "This is the build step" +[build:L1:0s] This is the build step +[a-test-step:L0:0s] + echo "Testing.." +[a-test-step:L1:0s] Testing.. +``` + +Pipeline steps are commands running in container images. +These containers are wired together and they share a volume with the source code on it. + +#### Pipeline documentation + +See all [pipeline features](docs/usage/pipeline.md). + +## Plugins + +Plugins are Docker containers that perform pre-defined tasks and are configured as steps in your pipeline. Plugins can be used to deploy code, publish artifacts, send notification, and more. + +Example pipeline using the Docker and Slack plugins: + +```yaml +pipeline: + backend: + image: golang + commands: + - go get + - go build + - go test + + docker: + image: plugins/docker + username: kevinbacon + password: pa55word + repo: foo/bar + tags: latest + + notify: + image: plugins/slack + channel: developers + username: drone +``` + +#### Custom plugins + +Plugins are Docker containers with their entrypoint set to a predefined script. + +[See how an example plugin can be implemented in a bash script](docs/usage/bash_plugin.md). + +## Server setup + +#### Quickstart + +The below [docker-compose](https://docs.docker.com/compose/) configuration can be used to start the Drone server with a single agent. It relies on a number of environment variables that you must set before running `docker-compose up`. The variables are described below. + +Each agent is able to process one build by default. If you have 4 agents installed and connected to the Drone server, your system will process 4 builds in parallel. You can add more agents to increase the number of parallel builds or set the agent's `DRONE_MAX_PROCS=1` environment variable to increase the number of parallel builds for that agent. + +```yaml +version: '2' + +services: + drone-server: + image: drone/drone:{{% version %}} + ports: + - 80:8000 + - 9000 + volumes: + - drone-server-data:/var/lib/drone/ + restart: always + environment: + - DRONE_OPEN=true + - DRONE_HOST=${DRONE_HOST} + - DRONE_GITHUB=true + - DRONE_GITHUB_CLIENT=${DRONE_GITHUB_CLIENT} + - DRONE_GITHUB_SECRET=${DRONE_GITHUB_SECRET} + - DRONE_SECRET=${DRONE_SECRET} + + drone-agent: + image: drone/agent:{{% version %}} + command: agent + restart: always + depends_on: + - drone-server + volumes: + - /var/run/docker.sock:/var/run/docker.sock + environment: + - DRONE_SERVER=drone-server:9000 + - DRONE_SECRET=${DRONE_SECRET} + +volumes: + drone-server-data: +``` + +Drone needs to know its own address. You must therefore provide the address in `://` format. Please omit trailing slashes. + +```diff +services: + drone-server: + image: drone/drone:{{% version %}} + environment: + - DRONE_OPEN=true ++ - DRONE_HOST=${DRONE_HOST} + - DRONE_GITHUB=true + - DRONE_GITHUB_CLIENT=${DRONE_GITHUB_CLIENT} + - DRONE_GITHUB_SECRET=${DRONE_GITHUB_SECRET} + - DRONE_SECRET=${DRONE_SECRET} +``` + +Drone agents require access to the host machine Docker daemon. + +```diff +services: + drone-agent: + image: drone/agent:{{% version %}} + command: agent + restart: always + depends_on: [ drone-server ] ++ volumes: ++ - /var/run/docker.sock:/var/run/docker.sock +``` + +Drone agents require the server address for agent-to-server communication. + +```diff +services: + drone-agent: + image: drone/agent:{{% version %}} + command: agent + restart: always + depends_on: [ drone-server ] + volumes: + - /var/run/docker.sock:/var/run/docker.sock + environment: ++ - DRONE_SERVER=drone-server:9000 + - DRONE_SECRET=${DRONE_SECRET} +``` + +Drone server and agents use a shared secret to authenticate communication. This should be a random string of your choosing and should be kept private. You can generate such string with `openssl rand -hex 32`. + +```diff +services: + drone-server: + image: drone/drone:{{% version %}} + environment: + - DRONE_OPEN=true + - DRONE_HOST=${DRONE_HOST} + - DRONE_GITHUB=true + - DRONE_GITHUB_CLIENT=${DRONE_GITHUB_CLIENT} + - DRONE_GITHUB_SECRET=${DRONE_GITHUB_SECRET} ++ - DRONE_SECRET=${DRONE_SECRET} + drone-agent: + image: drone/agent:{{% version %}} + environment: + - DRONE_SERVER=drone-server:9000 + - DRONE_DEBUG=true ++ - DRONE_SECRET=${DRONE_SECRET} +``` + +Drone registration is closed by default. This example enables open registration for users that are members of approved GitHub organizations. + +```diff +services: + drone-server: + image: drone/drone:{{% version %}} + environment: ++ - DRONE_OPEN=true ++ - DRONE_ORGS=dolores,dogpatch + - DRONE_HOST=${DRONE_HOST} + - DRONE_GITHUB=true + - DRONE_GITHUB_CLIENT=${DRONE_GITHUB_CLIENT} + - DRONE_GITHUB_SECRET=${DRONE_GITHUB_SECRET} + - DRONE_SECRET=${DRONE_SECRET} +``` + +Drone administrators should also be enumerated in your configuration. + +```diff +services: + drone-server: + image: drone/drone:{{% version %}} + environment: + - DRONE_OPEN=true + - DRONE_ORGS=dolores,dogpatch ++ - DRONE_ADMIN=johnsmith,janedoe + - DRONE_HOST=${DRONE_HOST} + - DRONE_GITHUB=true + - DRONE_GITHUB_CLIENT=${DRONE_GITHUB_CLIENT} + - DRONE_GITHUB_SECRET=${DRONE_GITHUB_SECRET} + - DRONE_SECRET=${DRONE_SECRET} +``` + +#### Authentication + +Authentication is done using OAuth and is delegated to one of multiple version control providers, configured using environment variables. The example above demonstrates basic GitHub integration. + +See the complete reference for [Github](docs/administration/github.md), [Bitbucket Cloud](docs/administration/bitbucket.md), [Bitbucket Server](docs/administration/bitbucket_server.md) and [Gitlab](docs/administration/gitlab.md). + +#### Database + +Drone mounts a [data volume](https://docs.docker.com/storage/volumes/#create-and-manage-volumes) to persist the sqlite database. + +See the [database settings](docs/administration/database.md) page to configure Postgresql or MySQL as database. + +```diff +services: + drone-server: + image: drone/drone:{{% version %}} + ports: + - 80:8000 + - 9000 ++ volumes: ++ - drone-server-data:/var/lib/drone/ + restart: always +``` + +#### SSL + +Drone supports ssl configuration by mounting certificates into your container. + +See the [SSL guide](docs/administration/ssl.md). + +Automated [Lets Encrypt](docs/administration/lets_encrypt.md) is also supported. + +#### Metrics + +A [Prometheus endpoint](docs/administration/lets_encrypt.md) is exposed. + +#### Behind a proxy + +See the [proxy guide](docs/administration/proxy.md) if you want to see a setup behind Apache, Nginx, Caddy or ngrok. + +## Contributing + +Drone-OSS-08 is Apache 2.0 licensed and accepts contributions via GitHub pull requests. + +[How to build the project]() + +## License + +Drone-OSS-08 is Apache 2.0 licensed with the source files in this repository having a header indicating which license they are under and what copyrights apply. + +Files under the `docs/` folder is licensed under Creative Commons Attribution-ShareAlike 4.0 International Public License. It is a derivative work of the https://github.com/drone/docs git repository. diff --git a/docs/LICENSE b/docs/LICENSE new file mode 100644 index 000000000..53c093f79 --- /dev/null +++ b/docs/LICENSE @@ -0,0 +1 @@ +Files in this folder are licensed under Creative Commons Attribution-ShareAlike 4.0 International Public License. It is a derivative work of the https://github.com/drone/docs git repository. diff --git a/docs/administration/bitbucket.md b/docs/administration/bitbucket.md new file mode 100644 index 000000000..020e1686d --- /dev/null +++ b/docs/administration/bitbucket.md @@ -0,0 +1,70 @@ +Drone comes with built-in support for Bitbucket Cloud. To enable Bitbucket Cloud you should configure the Drone container using the following environment variables: + +```diff +version: '2' + +services: + drone-server: + image: drone/drone:{{% version %}} + ports: + - 80:8000 + - 9000 + volumes: + - /var/lib/drone:/var/lib/drone/ + restart: always + environment: + - DRONE_OPEN=true + - DRONE_HOST=${DRONE_HOST} ++ - DRONE_BITBUCKET=true ++ - DRONE_BITBUCKET_CLIENT=95c0282573633eb25e82 ++ - DRONE_BITBUCKET_SECRET=30f5064039e6b359e075 + - DRONE_SECRET=${DRONE_SECRET} + + drone-agent: + image: drone/agent:{{% version %}} + restart: always + depends_on: + - drone-server + volumes: + - /var/run/docker.sock:/var/run/docker.sock + environment: + - DRONE_SERVER=drone-server:9000 + - DRONE_SECRET=${DRONE_SECRET} +``` + +# Configuration + +This is a full list of configuration options. Please note that many of these options use default configuration values that should work for the majority of installations. + +DRONE_BITBUCKET=true +: Set to true to enable the Bitbucket driver. + +DRONE_BITBUCKET_CLIENT +: Bitbucket oauth2 client id + +DRONE_BITBUCKET_SECRET +: Bitbucket oauth2 client secret + +# Registration + +You must register your application with Bitbucket in order to generate a client and secret. Navigate to your account settings and choose OAuth from the menu, and click Add Consumer. + +Please use the Authorization callback URL: + +```nohighlight +http://drone.mycompany.com/authorize +``` + +Please also be sure to check the following permissions: + +```nohighlight +Account:Email +Account:Read +Team Membership:Read +Repositories:Read +Webhooks:Read and Write +``` + +# Missing Features + +Merge requests are not currently supported. We are interested in patches to include this functionality. If you are interested in contributing to Drone and submitting a patch please [contact us](https://discourse.drone.io). diff --git a/docs/administration/bitbucket_server.md b/docs/administration/bitbucket_server.md new file mode 100644 index 000000000..46595df25 --- /dev/null +++ b/docs/administration/bitbucket_server.md @@ -0,0 +1,133 @@ +Drone comes with experimental support for Bitbucket Server, formerly known as Atlassian Stash. To enable Bitbucket Server you should configure the Drone container using the following environment variables: + +```diff +version: '2' + +services: + drone-server: + image: drone/drone:{{% version %}} + ports: + - 80:8000 + - 9000 + volumes: + - /var/lib/drone:/var/lib/drone/ + restart: always + environment: + - DRONE_OPEN=true + - DRONE_HOST=${DRONE_HOST} ++ - DRONE_STASH=true ++ - DRONE_STASH_GIT_USERNAME=foo ++ - DRONE_STASH_GIT_PASSWORD=bar ++ - DRONE_STASH_CONSUMER_KEY=95c0282573633eb25e82 ++ - DRONE_STASH_CONSUMER_RSA=/etc/bitbucket/key.pem ++ - DRONE_STASH_URL=http://stash.mycompany.com + - DRONE_SECRET=${DRONE_SECRET} + volumes: ++ - /path/to/key.pem:/path/to/key.pem + + drone-agent: + image: drone/agent:{{% version %}} + restart: always + depends_on: + - drone-server + volumes: + - /var/run/docker.sock:/var/run/docker.sock + environment: + - DRONE_SERVER=drone-server:9000 + - DRONE_SECRET=${DRONE_SECRET} +``` + +# Private Key File + +The OAuth process in Bitbucket server requires a private and a public RSA certificate. This is how you create the private RSA certificate. + +```nohighlight +openssl genrsa -out /etc/bitbucket/key.pem 1024 +``` + +This stores the private RSA certificate in `key.pem`. The next command generates the public RSA certificate and stores it in `key.pub`. + +```nohighlight +openssl rsa -in /etc/bitbucket/key.pem -pubout >> /etc/bitbucket/key.pub +``` + +Please note that the private key file can be mounted into your Drone conatiner at runtime or as an environment variable + +Private key file mounted into your Drone container at runtime as a volume. + +```diff +version: '2' + +services: + drone-server: + image: drone/drone:{{% version %}} + environment: + - DRONE_OPEN=true + - DRONE_HOST=${DRONE_HOST} + - DRONE_STASH=true + - DRONE_STASH_GIT_USERNAME=foo + - DRONE_STASH_GIT_PASSWORD=bar + - DRONE_STASH_CONSUMER_KEY=95c0282573633eb25e82 ++ - DRONE_STASH_CONSUMER_RSA=/etc/bitbucket/key.pem + - DRONE_STASH_URL=http://stash.mycompany.com + - DRONE_SECRET=${DRONE_SECRET} ++ volumes: ++ - /etc/bitbucket/key.pem:/etc/bitbucket/key.pem +``` + +Private key as environment variable + +```diff +version: '2' + +services: + drone-server: + image: drone/drone:{{% version %}} + environment: + - DRONE_OPEN=true + - DRONE_HOST=${DRONE_HOST} + - DRONE_STASH=true + - DRONE_STASH_GIT_USERNAME=foo + - DRONE_STASH_GIT_PASSWORD=bar + - DRONE_STASH_CONSUMER_KEY=95c0282573633eb25e82 ++ - DRONE_STASH_CONSUMER_RSA_STRING=contentOfPemKeyAsString + - DRONE_STASH_URL=http://stash.mycompany.com + - DRONE_SECRET=${DRONE_SECRET} +``` + +# Service Account + +Drone uses `git+https` to clone repositories, however, Bitbucket Server does not currently support cloning repositories with oauth token. To work around this limitation, you must create a service account and provide the username and password to Drone. This service account will be used to authenticate and clone private repositories. + +# Registration + +You must register your application with Bitbucket Server in order to generate a consumer key. Navigate to your account settings and choose Applications from the menu, and click Register new application. Now copy & paste the text value from `/etc/bitbucket/key.pub` into the `Public Key` in the incoming link part of the application registration. + +Please use http://drone.mycompany.com/authorize as the Authorization callback URL. + + +# Configuration + +This is a full list of configuration options. Please note that many of these options use default configuration values that should work for the majority of installations. + + +DRONE_STASH=true +: Set to true to enable the Bitbucket Server (Stash) driver. + +DRONE_STASH_URL +: Bitbucket Server address. + +DRONE_STASH_CONSUMER_KEY +: Bitbucket Server oauth1 consumer key + +DRONE_STASH_CONSUMER_RSA +: Bitbucket Server oauth1 private key file + +DRONE_STASH_CONSUMER_RSA_STRING +: Bibucket Server oauth1 private key as a string + +DRONE_STASH_GIT_USERNAME +: Machine account username used to clone repositories. + +DRONE_STASH_GIT_PASSWORD +: Machine account password used to clone repositories. diff --git a/docs/administration/database.md b/docs/administration/database.md new file mode 100644 index 000000000..0bc1d9a21 --- /dev/null +++ b/docs/administration/database.md @@ -0,0 +1,48 @@ + +This guide provides instructions for using alternate storage engines. Please note this is optional. The default storage engine is an embedded SQLite database which requires zero installation or configuration. + +# Configure MySQL + +The below example demonstrates mysql database configuration. See the official driver [documentation](https://github.com/go-sql-driver/mysql#dsn-data-source-name) for configuration options and examples. + +```diff +version: '2' + +services: + drone-server: + image: drone/drone:{{% version %}} + environment: ++ DRONE_DATABASE_DRIVER: mysql ++ DRONE_DATABASE_DATASOURCE: root:password@tcp(1.2.3.4:3306)/drone?parseTime=true +``` + +# Configure Postgres + +The below example demonstrates postgres database configuration. See the official driver [documentation](https://www.postgresql.org/docs/current/static/libpq-connect.html#LIBPQ-CONNSTRING) for configuration options and examples. + +```diff +version: '2' + +services: + drone-server: + image: drone/drone:{{% version %}} + environment: ++ DRONE_DATABASE_DRIVER: postgres ++ DRONE_DATABASE_DATASOURCE: postgres://root:password@1.2.3.4:5432/postgres?sslmode=disable +``` + +# Database Creation + +Drone does not create your database automatically. If you are using the mysql or postgres driver you will need to manually create your database using `CREATE DATABASE` + +# Database Migration + +Drone automatically handles database migration, including the initial creation of tables and indexes. New versions of Drone will automatically upgrade the database unless otherwise specified in the release notes. + +# Database Backups + +Drone does not perform database backups. This should be handled by separate third party tools provided by your database vendor of choice. + +# Database Archiving + +Drone does not perform data archival; it considered out-of-scope for the project. Drone is rather conservative with the amount of data it stores, however, you should expect the database logs to grow the size of your database considerably. diff --git a/docs/administration/github.md b/docs/administration/github.md new file mode 100644 index 000000000..929f1131d --- /dev/null +++ b/docs/administration/github.md @@ -0,0 +1,78 @@ +Drone comes with built-in support for GitHub and GitHub Enterprise. To enable GitHub you should configure the Drone container using the following environment variables: + +```diff +version: '2' + +services: + drone-server: + image: drone/drone:{{% version %}} + ports: + - 80:8000 + - 9000 + volumes: + - /var/lib/drone:/var/lib/drone/ + restart: always + environment: + - DRONE_OPEN=true + - DRONE_HOST=${DRONE_HOST} ++ - DRONE_GITHUB=true ++ - DRONE_GITHUB_CLIENT=${DRONE_GITHUB_CLIENT} ++ - DRONE_GITHUB_SECRET=${DRONE_GITHUB_SECRET} + - DRONE_SECRET=${DRONE_SECRET} + + drone-agent: + image: drone/agent:{{% version %}} + restart: always + depends_on: + - drone-server + volumes: + - /var/run/docker.sock:/var/run/docker.sock + environment: + - DRONE_SERVER=drone-server:9000 + - DRONE_SECRET=${DRONE_SECRET} +``` + +# Registration + +Register your application with GitHub to create your client id and secret. It is very import the authorization callback URL matches your http(s) scheme and hostname exactly with `:///authorize` as the path. + +Please use this screenshot for reference: + +![github oauth setup](github_oauth.png) + +# Configuration + +This is a full list of configuration options. Please note that many of these options use default configuration values that should work for the majority of installations. + +DRONE_GITHUB=true +: Set to true to enable the GitHub driver. + +DRONE_GITHUB_URL=`https://github.com` +: GitHub server address. + +DRONE_GITHUB_CLIENT +: Github oauth2 client id. + +DRONE_GITHUB_SECRET +: Github oauth2 client secret. + +DRONE_GITHUB_SCOPE=repo,repo:status,user:email,read:org +: Comma-separated Github oauth scope. + +DRONE_GITHUB_GIT_USERNAME +: Optional. Use a single machine account username to clone all repositories. + +DRONE_GITHUB_GIT_PASSWORD +: Optional. Use a single machine account password to clone all repositories. + +DRONE_GITHUB_PRIVATE_MODE=false +: Set to true if Github is running in private mode. + +DRONE_GITHUB_MERGE_REF=true +: Set to true to use the `refs/pulls/%d/merge` vs `refs/pulls/%d/head` + +DRONE_GITHUB_CONTEXT=continuous-integration/drone +: Customize the GitHub status message context + +DRONE_GITHUB_SKIP_VERIFY=false +: Set to true to disable SSL verification. diff --git a/docs/administration/github_oauth.png b/docs/administration/github_oauth.png new file mode 100644 index 000000000..f37b19a74 Binary files /dev/null and b/docs/administration/github_oauth.png differ diff --git a/docs/administration/gitlab.md b/docs/administration/gitlab.md new file mode 100644 index 000000000..553c08f98 --- /dev/null +++ b/docs/administration/gitlab.md @@ -0,0 +1,66 @@ +Drone comes with built-in support for the GitLab version 8.2 and higher. To enable GitLab you should configure the Drone container using the following environment variables: + +```diff +version: '2' + +services: + drone-server: + image: drone/drone:{{% version %}} + ports: + - 80:8000 + - 9000 + volumes: + - /var/lib/drone:/var/lib/drone/ + restart: always + environment: ++ - DRONE_GITLAB=true ++ - DRONE_GITLAB_CLIENT=95c0282573633eb25e82 ++ - DRONE_GITLAB_SECRET=30f5064039e6b359e075 ++ - DRONE_GITLAB_URL=http://gitlab.mycompany.com + - DRONE_SECRET=${DRONE_SECRET} + + drone-agent: + image: drone/agent:{{% version %}} + restart: always + depends_on: + - drone-server + volumes: + - /var/run/docker.sock:/var/run/docker.sock + environment: + - DRONE_SERVER=drone-server:9000 + - DRONE_SECRET=${DRONE_SECRET} +``` + +# Configuration + +This is a full list of configuration options. Please note that many of these options use default configuration values that should work for the majority of installations. + +DRONE_GITLAB=true +: Set to true to enable the GitLab driver. + +DRONE_GITLAB_URL=`https://gitlab.com` +: GitLab Server address. + +DRONE_GITLAB_CLIENT +: GitLab oauth2 client id. + +DRONE_GITLAB_SECRET +: GitLab oauth2 client secret. + +DRONE_GITLAB_GIT_USERNAME +: Optional. Use a single machine account username to clone all repositories. + +DRONE_GITLAB_GIT_PASSWORD +: Optional. Use a single machine account password to clone all repositories. + +DRONE_GITLAB_SKIP_VERIFY=false +: Set to true to disable SSL verification. + +DRONE_GITLAB_PRIVATE_MODE=false +: Set to true if GitLab is running in private mode. + +# Registration + +You must register your application with GitLab in order to generate a Client and Secret. Navigate to your account settings and choose Applications from the menu, and click New Application. + +Please use `http://drone.mycompany.com/authorize` as the Authorization callback URL. Grant `api` scope to the application. diff --git a/docs/administration/lets_encrypt.md b/docs/administration/lets_encrypt.md new file mode 100644 index 000000000..f608e5f8b --- /dev/null +++ b/docs/administration/lets_encrypt.md @@ -0,0 +1,38 @@ +Drone supports automated ssl configuration and updates using let's encrypt. You can enable let's encrypt by making the following modifications to your server configuration: + +```diff +services: + drone-server: + image: drone/drone:{{% version %}} + ports: ++ - 80:80 ++ - 443:443 + - 9000:9000 + volumes: + - /var/lib/drone:/var/lib/drone/ + restart: always + environment: + - DRONE_OPEN=true + - DRONE_HOST=${DRONE_HOST} + - DRONE_GITHUB=true + - DRONE_GITHUB_CLIENT=${DRONE_GITHUB_CLIENT} + - DRONE_GITHUB_SECRET=${DRONE_GITHUB_SECRET} + - DRONE_SECRET=${DRONE_SECRET} ++ - DRONE_LETS_ENCRYPT=true +``` + +Note that Drone uses the hostname from the `DRONE_HOST` environment variable when requesting certificates. For example, if `DRONE_HOST=https://foo.com` the certificate is requested for `foo.com`. + +>Once enabled you can visit your website at both the http and the https address + +# Certificate Cache + +Drone writes the certificates to the below directory: + +``` +/var/lib/drone/golang-autocert +``` + +# Certificate Updates + +Drone uses the official Go acme library which will handle certificate upgrades. There should be no addition configuration or management required. diff --git a/docs/administration/prometheus.md b/docs/administration/prometheus.md new file mode 100644 index 000000000..4e2c2cc46 --- /dev/null +++ b/docs/administration/prometheus.md @@ -0,0 +1,162 @@ + +Drone is compatible with Prometheus and exposes a `/metrics` endpoint. Please note that access to the metrics endpoint is restricted and requires an authorization token with administrative privileges. + +```nohighlight +global: + scrape_interval: 60s + +scrape_configs: + - job_name: 'drone' + bearer_token: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9... + + static_configs: + - targets: ['drone.domain.com'] +``` + +# Authorization + +An administrator will need to generate a user api token and configure in the prometheus configuration file as a bearer token. Please see the following example: + +```diff +global: + scrape_interval: 60s + +scrape_configs: + - job_name: 'drone' ++ bearer_token: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9... + + static_configs: + - targets: ['drone.domain.com'] +``` + +# Metric Reference + +List of prometheus metrics specific to Drone: + +``` +# HELP drone_build_count Total number of builds. +# TYPE drone_build_count gauge +drone_build_count 7275 +# HELP drone_pending_jobs Total number of pending build processes. +# TYPE drone_pending_jobs gauge +drone_pending_jobs 0 +# HELP drone_repo_count Total number of registered repositories. +# TYPE drone_repo_count gauge +drone_repo_count 133 +# HELP drone_running_jobs Total number of running build processes. +# TYPE drone_running_jobs gauge +drone_running_jobs 0 +# HELP drone_user_count Total number of active users. +# TYPE drone_user_count gauge +drone_user_count 15 +``` + +List of prometheus metrics for server resource usage: + +``` +# HELP go_gc_duration_seconds A summary of the GC invocation durations. +# TYPE go_gc_duration_seconds summary +go_gc_duration_seconds{quantile="0"} 0.000189189 +go_gc_duration_seconds{quantile="0.25"} 0.000391444 +go_gc_duration_seconds{quantile="0.5"} 0.001895967 +go_gc_duration_seconds{quantile="0.75"} 0.003075854 +go_gc_duration_seconds{quantile="1"} 0.004224575 +go_gc_duration_seconds_sum 0.019922696 +go_gc_duration_seconds_count 10 +# HELP go_goroutines Number of goroutines that currently exist. +# TYPE go_goroutines gauge +go_goroutines 24 +# HELP go_memstats_alloc_bytes Number of bytes allocated and still in use. +# TYPE go_memstats_alloc_bytes gauge +go_memstats_alloc_bytes 2.556344e+06 +# HELP go_memstats_alloc_bytes_total Total number of bytes allocated, even if freed. +# TYPE go_memstats_alloc_bytes_total counter +go_memstats_alloc_bytes_total 2.0479656e+07 +# HELP go_memstats_buck_hash_sys_bytes Number of bytes used by the profiling bucket hash table. +# TYPE go_memstats_buck_hash_sys_bytes gauge +go_memstats_buck_hash_sys_bytes 1.45144e+06 +# HELP go_memstats_frees_total Total number of frees. +# TYPE go_memstats_frees_total counter +go_memstats_frees_total 200332 +# HELP go_memstats_gc_cpu_fraction The fraction of this program's available CPU time used by the GC since the program started. +# TYPE go_memstats_gc_cpu_fraction gauge +go_memstats_gc_cpu_fraction 8.821705133777562e-05 +# HELP go_memstats_gc_sys_bytes Number of bytes used for garbage collection system metadata. +# TYPE go_memstats_gc_sys_bytes gauge +go_memstats_gc_sys_bytes 557056 +# HELP go_memstats_heap_alloc_bytes Number of heap bytes allocated and still in use. +# TYPE go_memstats_heap_alloc_bytes gauge +go_memstats_heap_alloc_bytes 2.556344e+06 +# HELP go_memstats_heap_idle_bytes Number of heap bytes waiting to be used. +# TYPE go_memstats_heap_idle_bytes gauge +go_memstats_heap_idle_bytes 3.842048e+06 +# HELP go_memstats_heap_inuse_bytes Number of heap bytes that are in use. +# TYPE go_memstats_heap_inuse_bytes gauge +go_memstats_heap_inuse_bytes 4.972544e+06 +# HELP go_memstats_heap_objects Number of allocated objects. +# TYPE go_memstats_heap_objects gauge +go_memstats_heap_objects 19986 +# HELP go_memstats_heap_released_bytes Number of heap bytes released to OS. +# TYPE go_memstats_heap_released_bytes gauge +go_memstats_heap_released_bytes 0 +# HELP go_memstats_heap_sys_bytes Number of heap bytes obtained from system. +# TYPE go_memstats_heap_sys_bytes gauge +go_memstats_heap_sys_bytes 8.814592e+06 +# HELP go_memstats_last_gc_time_seconds Number of seconds since 1970 of last garbage collection. +# TYPE go_memstats_last_gc_time_seconds gauge +go_memstats_last_gc_time_seconds 1.4941783810383117e+09 +# HELP go_memstats_lookups_total Total number of pointer lookups. +# TYPE go_memstats_lookups_total counter +go_memstats_lookups_total 325 +# HELP go_memstats_mallocs_total Total number of mallocs. +# TYPE go_memstats_mallocs_total counter +go_memstats_mallocs_total 220318 +# HELP go_memstats_mcache_inuse_bytes Number of bytes in use by mcache structures. +# TYPE go_memstats_mcache_inuse_bytes gauge +go_memstats_mcache_inuse_bytes 2400 +# HELP go_memstats_mcache_sys_bytes Number of bytes used for mcache structures obtained from system. +# TYPE go_memstats_mcache_sys_bytes gauge +go_memstats_mcache_sys_bytes 16384 +# HELP go_memstats_mspan_inuse_bytes Number of bytes in use by mspan structures. +# TYPE go_memstats_mspan_inuse_bytes gauge +go_memstats_mspan_inuse_bytes 81016 +# HELP go_memstats_mspan_sys_bytes Number of bytes used for mspan structures obtained from system. +# TYPE go_memstats_mspan_sys_bytes gauge +go_memstats_mspan_sys_bytes 98304 +# HELP go_memstats_next_gc_bytes Number of heap bytes when next garbage collection will take place. +# TYPE go_memstats_next_gc_bytes gauge +go_memstats_next_gc_bytes 4.819216e+06 +# HELP go_memstats_other_sys_bytes Number of bytes used for other system allocations. +# TYPE go_memstats_other_sys_bytes gauge +go_memstats_other_sys_bytes 672584 +# HELP go_memstats_stack_inuse_bytes Number of bytes in use by the stack allocator. +# TYPE go_memstats_stack_inuse_bytes gauge +go_memstats_stack_inuse_bytes 622592 +# HELP go_memstats_stack_sys_bytes Number of bytes obtained from system for stack allocator. +# TYPE go_memstats_stack_sys_bytes gauge +go_memstats_stack_sys_bytes 622592 +# HELP go_memstats_sys_bytes Number of bytes obtained from system. +# TYPE go_memstats_sys_bytes gauge +go_memstats_sys_bytes 1.2232952e+07 +# HELP go_threads Number of OS threads created +# TYPE go_threads gauge +go_threads 9 +# HELP process_cpu_seconds_total Total user and system CPU time spent in seconds. +# TYPE process_cpu_seconds_total counter +process_cpu_seconds_total 0.9 +# HELP process_max_fds Maximum number of open file descriptors. +# TYPE process_max_fds gauge +process_max_fds 524288 +# HELP process_open_fds Number of open file descriptors. +# TYPE process_open_fds gauge +process_open_fds 17 +# HELP process_resident_memory_bytes Resident memory size in bytes. +# TYPE process_resident_memory_bytes gauge +process_resident_memory_bytes 2.5296896e+07 +# HELP process_start_time_seconds Start time of the process since unix epoch in seconds. +# TYPE process_start_time_seconds gauge +process_start_time_seconds 1.494177893e+09 +# HELP process_virtual_memory_bytes Virtual memory size in bytes. +# TYPE process_virtual_memory_bytes gauge +process_virtual_memory_bytes 4.23243776e+08 +``` diff --git a/docs/administration/proxy.md b/docs/administration/proxy.md new file mode 100644 index 000000000..968aa7c26 --- /dev/null +++ b/docs/administration/proxy.md @@ -0,0 +1,148 @@ +**Table of Contents** +- [Apache](#apache) +- [Nginx](#nginx) +- [Caddy](#caddy) +- [Ngrok](#ngrok) + +# Apache +This guide provides a brief overview for installing Drone server behind the Apache2 webserver. This is an example configuration: + +```nohighlight +ProxyPreserveHost On + +RequestHeader set X-Forwarded-Proto "https" + +ProxyPass / http://127.0.0.1:8000/ +ProxyPassReverse / http://127.0.0.1:8000/ +``` + +You must have the below Apache modules installed. + +```nohighlight +a2enmod proxy +a2enmod proxy_http +``` + +You must configure Apache to set `X-Forwarded-Proto` when using https. + +```diff +ProxyPreserveHost On + ++RequestHeader set X-Forwarded-Proto "https" + +ProxyPass / http://127.0.0.1:8000/ +ProxyPassReverse / http://127.0.0.1:8000/ +``` + +# Nginx + +This guide provides a basic overview for installing Drone server behind the nginx webserver. For more advanced configuration options please consult the official nginx [documentation](https://www.nginx.com/resources/admin-guide/). + +Example configuration: + +```nginx +server { + listen 80; + server_name drone.example.com; + + location / { + proxy_set_header X-Forwarded-For $remote_addr; + proxy_set_header X-Forwarded-Proto $scheme; + proxy_set_header Host $http_host; + + proxy_pass http://127.0.0.1:8000; + proxy_redirect off; + proxy_http_version 1.1; + proxy_buffering off; + + chunked_transfer_encoding off; + } +} +``` + +You must configure the proxy to set `X-Forwarded` proxy headers: + +```diff +server { + listen 80; + server_name drone.example.com; + + location / { ++ proxy_set_header X-Forwarded-For $remote_addr; ++ proxy_set_header X-Forwarded-Proto $scheme; + + proxy_pass http://127.0.0.1:8000; + proxy_redirect off; + proxy_http_version 1.1; + proxy_buffering off; + + chunked_transfer_encoding off; + } +} +``` + +# Caddy + +This guide provides a brief overview for installing Drone server behind the [Caddy webserver](https://caddyserver.com/). This is an example caddyfile proxy configuration: + +```nohighlight +drone.mycompany.com { + gzip { + not /stream/ + } + proxy / localhost:8000 { + websocket + transparent + } +} +``` +You must disable gzip compression for streamed data otherwise the live updates won't be instant: + +```diff +drone.mycompany.com { ++ gzip { ++ not /stream/ ++ } + proxy / localhost:8000 { + websocket + transparent + } +} +``` + +You must configure the proxy to enable websocket upgrades: + +```diff +drone.mycompany.com { + gzip { + not /stream/ + } + proxy / localhost:8000 { ++ websocket + transparent + } +} +``` + +You must configure the proxy to include `X-Forwarded` headers using the `transparent` directive: + +```diff +drone.mycompany.com { + gzip { + not /stream/ + } + proxy / localhost:8000 { + websocket ++ transparent + } +} +``` + +# Ngrok +After installing [ngrok](https://ngrok.com/), open a new console and run: + +``` +ngrok http 80 +``` + +Set `DRONE_HOST` (for example in `docker-compose.yml`) to the ngrok url (usually xxx.ngrok.io) and start the server. diff --git a/docs/administration/ssl.md b/docs/administration/ssl.md new file mode 100644 index 000000000..bdb7c0b73 --- /dev/null +++ b/docs/administration/ssl.md @@ -0,0 +1,77 @@ +Drone supports ssl configuration by mounting certificates into your container. + +```diff +services: + drone-server: + image: drone/drone:{{% version %}} + ports: ++ - 80:80 ++ - 443:443 + - 9000:9000 + volumes: + - /var/lib/drone:/var/lib/drone/ ++ - /etc/certs/drone.foo.com/server.crt:/etc/certs/drone.foo.com/server.crt ++ - /etc/certs/drone.foo.com/server.key:/etc/certs/drone.foo.com/server.key + restart: always + environment: ++ - DRONE_SERVER_CERT=/etc/certs/drone.foo.com/server.crt ++ - DRONE_SERVER_KEY=/etc/certs/drone.foo.com/server.key +``` + +Update your configuration to expose the following ports: + +```diff +services: + drone-server: + image: drone/drone:{{% version %}} + ports: ++ - 80:80 ++ - 443:443 + - 9000:9000 +``` + +Update your configuration to mount your certificate and key: + +```diff +services: + drone-server: + image: drone/drone:{{% version %}} + ports: + - 80:80 + - 443:443 + - 9000:9000 + volumes: + - /var/lib/drone:/var/lib/drone/ ++ - /etc/certs/drone.foo.com/server.crt:/etc/certs/drone.foo.com/server.crt ++ - /etc/certs/drone.foo.com/server.key:/etc/certs/drone.foo.com/server.key +``` + +Update your configuration to provide the paths of your certificate and key: + +```diff +services: + drone-server: + image: drone/drone:{{% version %}} + ports: + - 80:80 + - 443:443 + - 9000:9000 + volumes: + - /var/lib/drone:/var/lib/drone/ + - /etc/certs/drone.foo.com/server.crt:/etc/certs/drone.foo.com/server.crt + - /etc/certs/drone.foo.com/server.key:/etc/certs/drone.foo.com/server.key + restart: always + environment: ++ - DRONE_SERVER_CERT=/etc/certs/drone.foo.com/server.crt ++ - DRONE_SERVER_KEY=/etc/certs/drone.foo.com/server.key +``` + +# Certificate Chain + +The most common problem encountered is providing a certificate file without the intermediate chain. + +> LoadX509KeyPair reads and parses a public/private key pair from a pair of files. The files must contain PEM encoded data. The certificate file may contain intermediate certificates following the leaf certificate to form a certificate chain. + +# Certificate Errors + +SSL support is provided using the [ListenAndServeTLS](https://golang.org/pkg/net/http/#ListenAndServeTLS) function from the Go standard library. If you receive certificate errors or warnings please examine your configuration more closely. Please do not create issues claiming SSL is broken. diff --git a/docs/drone.png b/docs/drone.png new file mode 100644 index 000000000..28a162463 Binary files /dev/null and b/docs/drone.png differ diff --git a/docs/motivation.md b/docs/motivation.md new file mode 100644 index 000000000..aa3441756 --- /dev/null +++ b/docs/motivation.md @@ -0,0 +1,18 @@ +# Motivation + +I was using Drone for two years with great satisfaction. The container architecture, the speedy backend and UI, the simple plugin system made it a flexible and simple platform. Kudos for the author, Brad to make it such a joy to use. + +It wasn't without flaws +- inconsistencies in variables and CLI features +- lack of documentation +- lack of published best practices +- UI/UX issues +- stuck builds + +Things that could be circumvented by reading the codebase. Over time however these started to annoy me, also PRs that tried to address these were not merged. Instead the development of Drone headed towards a 1.0 release with features less interesting to me. + +1.0 landed and it came with a licence change. Drone has been an open-core project since many prior versions, but the enterprise features were limited to features like autoscaling and secret vaults. + +In the 1.0 line however, Postgresql, Mysql and TLS support along with agent based horizontal scaling were also moved under the enterprise license. Limiting the open source version to single node, hobbyist deployments. + +These feature reductions and my long time UX annoyance and general dissatisfaction of the CI space lead to this fork. diff --git a/docs/usage/bash_plugin.md b/docs/usage/bash_plugin.md new file mode 100644 index 000000000..635854d64 --- /dev/null +++ b/docs/usage/bash_plugin.md @@ -0,0 +1,49 @@ +This provides a brief tutorial for creating a Drone webhook plugin, using simple shell scripting, to make an http requests during the build pipeline. The below example demonstrates how we might configure a webhook plugin in the Yaml file: + +```yaml +pipeline: + webhook: + image: foo/webhook + url: http://foo.com + method: post + body: | + hello world +``` + +Create a simple shell script that invokes curl using the Yaml configuration parameters, which are passed to the script as environment variables in uppercase and prefixed with `PLUGIN_`. + +```bash +#!/bin/sh + +curl \ + -X ${PLUGIN_METHOD} \ + -d ${PLUGIN_BODY} \ + ${PLUGIN_URL} + +``` +Create a Dockerfile that adds your shell script to the image, and configures the image to execute your shell script as the main entrypoint. + +```dockerfile +FROM alpine +ADD script.sh /bin/ +RUN chmod +x /bin/script.sh +RUN apk -Uuv add curl ca-certificates +ENTRYPOINT /bin/script.sh +``` + +Build and publish your plugin to the Docker registry. Once published your plugin can be shared with the broader Drone community. + +```nohighlight +docker build -t foo/webhook . +docker push foo/webhook +``` + +Execute your plugin locally from the command line to verify it is working: + +```nohighlight +docker run --rm \ + -e PLUGIN_METHOD=post \ + -e PLUGIN_URL=http://foo.com \ + -e PLUGIN_BODY="hello world" \ + foo/webhook +``` diff --git a/docs/usage/pipeline.md b/docs/usage/pipeline.md new file mode 100644 index 000000000..e2f7c0f51 --- /dev/null +++ b/docs/usage/pipeline.md @@ -0,0 +1,1400 @@ + + + +**Table of Contents** _generated with [DocToc](https://github.com/thlorenz/doctoc)_ + +- [Pipeline basics](#pipeline-basics) + - [Activation](#activation) + - [Configuration](#configuration) + - [Execution](#execution) +- [Pipelines](#pipelines) + - [Build Steps](#build-steps) + - [Images](#images) + - [Images from private registries](#images-from-private-registries) + - [GCR Registry Support](#gcr-registry-support) + - [Parallel Execution](#parallel-execution) + - [Conditional Pipeline Execution](#conditional-pipeline-execution) + - [Conditional Step Execution](#conditional-step-execution) + - [Failure Execution](#failure-execution) +- [Services](#services) + - [Configuration](#configuration-1) + - [Detachment](#detachment) + - [Initialization](#initialization) +- [Plugins](#plugins) + - [Plugin Isolation](#plugin-isolation) + - [Plugin Marketplace](#plugin-marketplace) +- [Environment variables](#environment-variables) + - [Built-in environment variables](#built-in-environment-variables) + - [String Substitution](#string-substitution) + - [String Operations](#string-operations) +- [Secrets](#secrets) + - [Adding Secrets](#adding-secrets) + - [Alternate Names](#alternate-names) + - [Pull Requests](#pull-requests) + - [Examples](#examples) +- [Volumes](#volumes) +- [Webhooks](#webhooks) + - [Required Permissions](#required-permissions) + - [Skip Commits](#skip-commits) + - [Skip Branches](#skip-branches) +- [Workspace](#workspace) +- [Cloning](#cloning) + - [Git Submodules](#git-submodules) +- [Privileged mode](#privileged-mode) +- [Promoting](#promoting) + - [Triggering Deployments](#triggering-deployments) +- [Matrix builds](#matrix-builds) + - [Interpolation](#interpolation) + - [Examples](#examples-1) +- [Multi-pipeline builds](#multi-pipeline-builds) + - [Example multi-pipeline definition](#example-multi-pipeline-definition) + - [Flow control](#flow-control) + - [Status lines](#status-lines) + - [Rational](#rational) +- [Badges](#badges) + + + +This document explains the process for activating and configuring a continuous delivery pipeline. + +# Pipeline basics + +## Activation + +To activate your project navigate to your account settings. You will see a list of repositories which can be activated with a simple toggle. When you activate your repository, Drone automatically adds webhooks to your version control system (e.g. GitHub). + +Webhooks are used to trigger pipeline executions. When you push code to your repository, open a pull request, or create a tag, your version control system will automatically send a webhook to Drone which will in turn trigger pipeline execution. + +![repository list](`repo_list.png) + +## Configuration + +To configure you pipeline you should place a `.drone.yml` file in the root of your repository. The .drone.yml file is used to define your pipeline steps. It is a superset of the widely used docker-compose file format. + +Example pipeline configuration: + +```yaml +pipeline: + build: + image: golang + commands: + - go get + - go build + - go test + +services: + postgres: + image: postgres:9.4.5 + environment: + - POSTGRES_USER=myapp +``` + +Example pipeline configuration with multiple, serial steps: + +```yaml +pipeline: + backend: + image: golang + commands: + - go get + - go build + - go test + + frontend: + image: node:6 + commands: + - npm install + - npm test + + notify: + image: plugins/slack + channel: developers + username: drone +``` + +## Execution + +To trigger your first pipeline execution you can push code to your repository, open a pull request, or push a tag. Any of these events triggers a webhook from your version control system and execute your pipeline. + +# Pipelines + +The pipeline section defines a list of steps to build, test and deploy your code. Pipeline steps are executed serially, in the order in which they are defined. If a step returns a non-zero exit code, the pipeline immediately aborts and returns a failure status. + +Example pipeline: + +```yaml +pipeline: + backend: + image: golang + commands: + - go build + - go test + frontend: + image: node + commands: + - npm install + - npm run test + - npm run build +``` + +In the above example we define two pipeline steps, `frontend` and `backend`. The names of these steps are completely arbitrary. + +## Build Steps + +Build steps are steps in your pipeline that execute arbitrary commands inside the specified docker container. The commands are executed using the workspace as the working directory. + +```diff +pipeline: + backend: + image: golang + commands: ++ - go build ++ - go test +``` + +There is no magic here. The above commands are converted to a simple shell script. The commands in the above example are roughly converted to the below script: + +```diff +#!/bin/sh +set -e + +go build +go test +``` + +The above shell script is then executed as the docker entrypoint. The below docker command is an (incomplete) example of how the script is executed: + +``` +docker run --entrypoint=build.sh golang +``` + +> Please note that only build steps can define commands. You cannot use commands with plugins or services. + +## Images + +Drone uses Docker images for the build environment, for plugins and for service containers. The image field is exposed in the container blocks in the Yaml: + +```diff +pipeline: + build: ++ image: golang:1.6 + commands: + - go build + - go test + + publish: ++ image: plugins/docker + repo: foo/bar + +services: + database: ++ image: mysql +``` + +Drone supports any valid Docker image from any Docker registry: + +```text +image: golang +image: golang:1.7 +image: library/golang:1.7 +image: index.docker.io/library/golang +image: index.docker.io/library/golang:1.7 +``` + +Drone does not automatically upgrade docker images. Example configuration to always pull the latest image when updates are available: + +```diff +pipeline: + build: + image: golang:latest ++ pull: true +``` + +#### Images from private registries + +You must provide registry credentials on the UI in order to pull private pipeline images defined in your Yaml configuration file. + +These credentials are never exposed to your pipeline, which means they cannot be used to push, and are safe to use with pull requests, for example. Pushing to a registry still require setting credentials for the appropriate plugin. + +Example configuration using a private image: + +```diff +pipeline: + build: ++ image: gcr.io/custom/golang + commands: + - go build + - go test +``` + +Drone matches the registry hostname to each image in your yaml. If the hostnames match, the registry credentials are used to authenticate to your registry and pull the image. Note that registry credentials are used by the Drone agent and are never exposed to your build containers. + +Example registry hostnames: + +- Image `gcr.io/foo/bar` has hostname `gcr.io` +- Image `foo/bar` has hostname `docker.io` +- Image `qux.com:8000/foo/bar` has hostname `qux.com:8000` + +Example registry hostname matching logic: + +- Hostname `gcr.io` matches image `gcr.io/foo/bar` +- Hostname `docker.io` matches `golang` +- Hostname `docker.io` matches `library/golang` +- Hostname `docker.io` matches `bradyrydzewski/golang` +- Hostname `docker.io` matches `bradyrydzewski/golang:latest` + +#### GCR Registry Support + +For specific details on configuring access to Google Container Registry, please view the docs [here](https://cloud.google.com/container-registry/docs/advanced-authentication#using_a_json_key_file). + +## Parallel Execution + +Drone supports parallel step execution for same-machine fan-in and fan-out. Parallel steps are configured using the `group` attribute. This instructs the pipeline runner to execute the named group in parallel. + +Example parallel configuration: + +```diff +pipeline: + backend: ++ group: build + image: golang + commands: + - go build + - go test + frontend: ++ group: build + image: node + commands: + - npm install + - npm run test + - npm run build + publish: + image: plugins/docker + repo: octocat/hello-world +``` + +In the above example, the `frontend` and `backend` steps are executed in parallel. The pipeline runner will not execute the `publish` step until the group completes. + +## Conditional Pipeline Execution + +Drone supports defining conditional pipelines to skip commits based on the target branch. If the branch matches the `branches:` block the pipeline is executed, otherwise it is skipped. + +Example skipping a commit when the target branch is not master: + +```diff +pipeline: + build: + image: golang + commands: + - go build + - go test + ++branches: master +``` + +Example matching multiple target branches: + +```diff +pipeline: + build: + image: golang + commands: + - go build + - go test + ++branches: [ master, develop ] +``` + +Example uses glob matching: + +```diff +pipeline: + build: + image: golang + commands: + - go build + - go test + ++branches: [ master, feature/* ] +``` + +Example includes branches: + +```diff +pipeline: + build: + image: golang + commands: + - go build + - go test + ++branches: ++ include: [ master, feature/* ] +``` + +Example excludes branches: + +```diff +pipeline: + build: + image: golang + commands: + - go build + - go test + ++branches: ++ exclude: [ develop, feature/* ] +``` + +## Conditional Step Execution + +Drone supports defining conditional pipeline steps in the `when` block. If all conditions in the `when` block evaluate to true the step is executed, otherwise it is skipped. + +Example conditional execution by branch: + +```diff +pipeline: + slack: + image: plugins/slack + channel: dev ++ when: ++ branch: master +``` + +> The step now triggers on master, but also if the target branch of a pull request is `master`. Add an event condition to limit it further to pushes on master only. + +Execute a step if the branch is `master` or `develop`: + +```diff +when: + branch: [master, develop] +``` + +Execute a step if the branch starts with `prefix/*`: + +```diff +when: + branch: prefix/* +``` + +Execute a step using custom include and exclude logic: + +```diff +when: + branch: + include: [ master, release/* ] + exclude: [ release/1.0.0, release/1.1.* ] +``` + +Execute a step if the build event is a `tag`: + +```diff +when: + event: tag +``` + +Execute a step if the build event is a `tag` created from the specified branch: + +```diff +when: + event: tag ++ branch: master +``` + +Execute a step for all non-pull request events: + +```diff +when: + event: [push, tag, deployment] +``` + +Execute a step for all build events: + +```diff +when: + event: [push, pull_request, tag, deployment] +``` + +Execute a step if the tag name starts with `release`: + +```diff +when: + tag: release* +``` + +Execute a step when the build status changes: + +```diff +when: + status: changed +``` + +Execute a step when the build is passing or failing: + +```diff +when: + status: [ failure, success ] +``` + +Execute a step for a specific platform: + +```diff +when: + platform: linux/amd64 +``` + +Execute a step for a specific platform using wildcards: + +```diff +when: + platform: [ linux/*, windows/amd64 ] +``` + +Execute a step for deployment events matching the target deployment environment: + +```diff +when: + environment: production + event: deployment +``` + +Execute a step for a single matrix permutation: + +```diff +when: + matrix: + GO_VERSION: 1.5 + REDIS_VERSION: 2.8 +``` + +Execute a step only on a certain Drone instance: + +```diff +when: + instance: stage.drone.company.com +``` + +#### Failure Execution + +Drone uses the container exit code to determine the success or failure status of a build. Non-zero exit codes fail the build and cause the pipeline to immediately exit. + +There are use cases for executing pipeline steps on failure, such as sending notifications for failed builds. Use the status constraint to override the default behavior and execute steps even when the build status is failure: + +```diff +pipeline: + slack: + image: plugins/slack + channel: dev ++ when: ++ status: [ success, failure ] +``` + +# Services + +Drone provides a services section in the Yaml file used for defining service containers. The below configuration composes database and cache containers. + +```diff +pipeline: + build: + image: golang + commands: + - go build + - go test + +services: + database: + image: mysql + + cache: + image: redis +``` + +Services are accessed using custom hostnames. In the above example the mysql service is assigned the hostname `database` and is available at `database:3306`. + +## Configuration + +Service containers generally expose environment variables to customize service startup such as default usernames, passwords and ports. Please see the official image documentation to learn more. + +```diff +services: + database: + image: mysql ++ environment: ++ - MYSQL_DATABASE=test ++ - MYSQL_ALLOW_EMPTY_PASSWORD=yes + + cache: + image: redis +``` + +## Detachment + +Service and long running containers can also be included in the pipeline section of the configuration using the detach parameter without blocking other steps. This should be used when explicit control over startup order is required. + +```diff +pipeline: + build: + image: golang + commands: + - go build + - go test + + database: + image: redis ++ detach: true + + test: + image: golang + commands: + - go test +``` + +Containers from detached steps will terminate when the pipeline ends. + +## Initialization + +Service containers require time to initialize and begin to accept connections. If you are unable to connect to a service you may need to wait a few seconds or implement a backoff. + +```diff +pipeline: + test: + image: golang + commands: ++ - sleep 15 + - go get + - go test + +services: + database: + image: mysql +``` + +# Plugins + +Plugins are Docker containers that perform pre-defined tasks and are configured as steps in your pipeline. Plugins can be used to deploy code, publish artifacts, send notification, and more. + +Example pipeline using the Docker and Slack plugins: + +```yaml +pipeline: + build: + image: golang + commands: + - go build + - go test + + publish: + image: plugins/docker + repo: foo/bar + tags: latest + + notify: + image: plugins/slack + channel: dev +``` + +## Plugin Isolation + +Plugins are executed in Docker containers and are isolated from the other steps in your build pipeline. Plugins do share the build workspace, mounted as a volume, and therefore have access to your source tree. + +## Plugin Marketplace + +Plugins are packaged and distributed as Docker containers. They are conceptually similar to software libraries (think npm) and can be published and shared with the community. You can find a list of available plugins at [http://plugins.drone.io](http://plugins.drone.io). + +# Environment variables + +Drone provides the ability to define environment variables scoped to individual build steps. Example pipeline step with custom environment variables: + +```diff +pipeline: + build: + image: golang ++ environment: ++ - CGO=0 ++ - GOOS=linux ++ - GOARCH=amd64 + commands: + - go build + - go test +``` + +Please note that the environment section is not able to expand environment variables. If you need to expand variables they should be exported in the commands section. + +```diff +pipeline: + build: + image: golang +- environment: +- - PATH=$PATH:/go + commands: ++ - export PATH=$PATH:/go + - go build + - go test +``` + +Please be warned that `${variable}` expressions are subject to pre-processing. If you do not want the pre-processor to evaluate your expression it must be escaped: + +```diff +pipeline: + build: + image: golang + commands: +- - export PATH=${PATH}:/go ++ - export PATH=$${PATH}:/go + - go build + - go test +``` + +## Built-in environment variables + +This is the reference list of all environment variables available to your build environment. These are injected into your build and plugins containers, at runtime. + +| NAME | DESC | +| ---------------------------- | -------------------------------------- | +| `CI=drone` | environment is drone | +| `DRONE=true` | environment is drone | +| `DRONE_ARCH` | environment architecture (linux/amd64) | +| `DRONE_REPO` | repository full name | +| `DRONE_REPO_OWNER` | repository owner | +| `DRONE_REPO_NAME` | repository name | +| `DRONE_REPO_SCM` | repository scm (git) | +| `DRONE_REPO_LINK` | repository link | +| `DRONE_REPO_AVATAR` | repository avatar | +| `DRONE_REPO_BRANCH` | repository default branch (master) | +| `DRONE_REPO_PRIVATE` | repository is private | +| `DRONE_REPO_TRUSTED` | repository is trusted | +| `DRONE_REMOTE_URL` | repository clone url | +| `DRONE_COMMIT_SHA` | commit sha | +| `DRONE_COMMIT_REF` | commit ref | +| `DRONE_COMMIT_BRANCH` | commit branch | +| `DRONE_COMMIT_LINK` | commit link in remote | +| `DRONE_COMMIT_MESSAGE` | commit message | +| `DRONE_COMMIT_AUTHOR` | commit author username | +| `DRONE_COMMIT_AUTHOR_EMAIL` | commit author email address | +| `DRONE_COMMIT_AUTHOR_AVATAR` | commit author avatar | +| `DRONE_BUILD_NUMBER` | build number | +| `DRONE_BUILD_EVENT` | build event (push, pull_request, tag) | +| `DRONE_BUILD_STATUS` | build status (success, failure) | +| `DRONE_BUILD_LINK` | build result link | +| `DRONE_BUILD_CREATED` | build created unix timestamp | +| `DRONE_BUILD_STARTED` | build started unix timestamp | +| `DRONE_BUILD_FINISHED` | build finished unix timestamp | +| `DRONE_PREV_BUILD_STATUS` | prior build status | +| `DRONE_PREV_BUILD_NUMBER` | prior build number | +| `DRONE_PREV_COMMIT_SHA` | prior build commit sha | +| `DRONE_JOB_NUMBER` | job number | +| `DRONE_JOB_STATUS` | job status | +| `DRONE_JOB_STARTED` | job started | +| `DRONE_JOB_FINISHED` | job finished | +| `DRONE_BRANCH` | commit branch | +| `DRONE_COMMIT` | commit sha | +| `DRONE_TAG` | commit tag | +| `DRONE_PULL_REQUEST` | pull request number | +| `DRONE_DEPLOY_TO` | deployment target (ie production) | + +## String Substitution + +Drone provides the ability to substitute environment variables at runtime. This gives us the ability to use dynamic build or commit details in our pipeline configuration. + +Example commit substitution: + +```diff +pipeline: + docker: + image: plugins/docker ++ tags: ${DRONE_COMMIT_SHA} +``` + +Example tag substitution: + +```diff +pipeline: + docker: + image: plugins/docker ++ tags: ${DRONE_TAG} +``` + +## String Operations + +Drone also emulates bash string operations. This gives us the ability to manipulate the strings prior to substitution. Example use cases might include substring and stripping prefix or suffix values. + +| OPERATION | DESC | +| ------------------ | ------------------------------------------------ | +| `${param}` | parameter substitution | +| `${param,}` | parameter substitution with lowercase first char | +| `${param,,}` | parameter substitution with lowercase | +| `${param^}` | parameter substitution with uppercase first char | +| `${param^^}` | parameter substitution with uppercase | +| `${param:pos}` | parameter substitution with substring | +| `${param:pos:len}` | parameter substitution with substring and length | +| `${param=default}` | parameter substitution with default | +| `${param##prefix}` | parameter substitution with prefix removal | +| `${param%%suffix}` | parameter substitution with suffix removal | +| `${param/old/new}` | parameter substitution with find and replace | + +Example variable substitution with substring: + +```diff +pipeline: + docker: + image: plugins/docker ++ tags: ${DRONE_COMMIT_SHA:0:8} +``` + +Example variable substitution strips `v` prefix from `v.1.0.0`: + +```diff +pipeline: + docker: + image: plugins/docker ++ tags: ${DRONE_TAG##v} +``` + +# Secrets + +Drone provides the ability to store named parameters external to the Yaml configuration file, in a central secret store. Individual steps in the yaml can request access to these named parameters at runtime. + +Secrets are exposed to your pipeline steps and plugins as uppercase environment variables and can therefore be referenced in the commands section of your pipeline. + +```diff +pipeline: + docker: + image: docker + commands: ++ - echo $DOCKER_USERNAME ++ - echo $DOCKER_PASSWORD + secrets: [ docker_username, docker_password ] +``` + +Please note parameter expressions are subject to pre-processing. When using secrets in parameter expressions they should be escaped. + +```diff +pipeline: + docker: + image: docker + commands: +- - echo ${DOCKER_USERNAME} +- - echo ${DOCKER_PASSWORD} ++ - echo $${DOCKER_USERNAME} ++ - echo $${DOCKER_PASSWORD} + secrets: [ docker_username, docker_password ] +``` + +## Adding Secrets + +Secrets are added to the Drone secret store on the UI or with the CLI. + +## Alternate Names + +There may be scenarios where you are required to store secrets using alternate names. You can map the alternate secret name to the expected name using the below syntax: + +```diff +pipeline: + docker: + image: plugins/docker + repo: octocat/hello-world + tags: latest ++ secrets: ++ - source: docker_prod_password ++ target: docker_password +``` + +## Pull Requests + +Secrets are not exposed to pull requests by default. You can override this behavior by creating the secret and enabling the `pull_request` event type. + +```diff +drone secret add \ + -repository octocat/hello-world \ + -image plugins/docker \ ++ -event pull_request \ ++ -event push \ ++ -event tag \ + -name docker_username \ + -value +``` + +Please be careful when exposing secrets to pull requests. If your repository is open source and accepts pull requests your secrets are not safe. A bad actor can submit a malicious pull request that exposes your secrets. + +## Examples + +Create the secret using default settings. The secret will be available to all images in your pipeline, and will be available to all push, tag, and deployment events (not pull request events). + +```diff +drone secret add \ + -repository octocat/hello-world \ + -name aws_access_key_id \ + -value +``` + +Create the secret and limit to a single image: + +```diff +drone secret add \ + -repository octocat/hello-world \ ++ -image plugins/s3 \ + -name aws_access_key_id \ + -value +``` + +Create the secrets and limit to a set of images: + +```diff +drone secret add \ + -repository octocat/hello-world \ ++ -image plugins/s3 \ ++ -image peloton/drone-ecs \ + -name aws_access_key_id \ + -value +``` + +Create the secret and enable for multiple hook events: + +```diff +drone secret add \ + -repository octocat/hello-world \ + -image plugins/s3 \ ++ -event pull_request \ ++ -event push \ ++ -event tag \ + -name aws_access_key_id \ + -value +``` + +Loading secrets from file using curl `@` syntax. This is the recommended approach for loading secrets from file to preserve newlines: + +```diff +drone secret add \ + -repository octocat/hello-world \ + -name ssh_key \ ++ -value @/root/ssh/id_rsa +``` + +# Volumes + +Drone gives the ability to define Docker volumes in the Yaml. You can use this parameter to mount files or folders on the host machine into your containers. + +> Volumes are only available to trusted repositories and for security reasons should only be used in private environments. + +```diff +pipeline: + build: + image: docker + commands: + - docker build --rm -t octocat/hello-world . + - docker run --rm octocat/hello-world --test + - docker push octocat/hello-world + - docker rmi octocat/hello-world + volumes: ++ - /var/run/docker.sock:/var/run/docker.sock +``` + +Please note that Drone mounts volumes on the host machine. This means you must use absolute paths when you configure volumes. Attempting to use relative paths will result in an error. + +```diff +- volumes: [ ./certs:/etc/ssl/certs ] ++ volumes: [ /etc/ssl/certs:/etc/ssl/certs ] +``` + +# Webhooks + +When you activate your repository Drone automatically add webhooks to your version control system (e.g. GitHub). There is no manual configuration required. + +Webhooks are used to trigger pipeline executions. When you push code to your repository, open a pull request, or create a tag, your version control system will automatically send a webhook to Drone which will in turn trigger pipeline execution. + +## Required Permissions + +The user who enables a repo in Drone must have `Admin` rights on that repo, so that Drone can add the webhook. + +Note that manually creating webhooks yourself is not possible. This is because webhooks are signed using a per-repository secret key which is not exposed to end users. + +## Skip Commits + +Drone gives the ability to skip individual commits by adding `[CI SKIP]` to the commit message. Note this is case-insensitive. + +```diff +git commit -m "updated README [CI SKIP]" +``` + +## Skip Branches + +Drone gives the ability to skip commits based on the target branch. The below example will skip a commit when the target branch is not master. + +```diff +pipeline: + build: + image: golang + commands: + - go build + - go test + ++branches: master +``` + +Please see the pipeline conditions [documentation]({{< ref "usage/config/pipeline-conditions.md" >}}) for more options and details. + +# Workspace + +The workspace defines the shared volume and working directory shared by all pipeline steps. The default workspace matches the below pattern, based on your repository url. + +``` +/drone/src/github.com/octocat/hello-world +``` + +The workspace can be customized using the workspace block in the Yaml file: + +```diff ++workspace: ++ base: /go ++ path: src/github.com/octocat/hello-world + +pipeline: + build: + image: golang:latest + commands: + - go get + - go test +``` + +The base attribute defines a shared base volume available to all pipeline steps. This ensures your source code, dependencies and compiled binaries are persisted and shared between steps. + +```diff +workspace: ++ base: /go + path: src/github.com/octocat/hello-world + +pipeline: + deps: + image: golang:latest + commands: + - go get + - go test + build: + image: node:latest + commands: + - go build +``` + +This would be equivalent to the following docker commands: + +``` +docker volume create my-named-volume + +docker run --volume=my-named-volume:/go golang:latest +docker run --volume=my-named-volume:/go node:latest +``` + +The path attribute defines the working directory of your build. This is where your code is cloned and will be the default working directory of every step in your build process. The path must be relative and is combined with your base path. + +```diff +workspace: + base: /go ++ path: src/github.com/octocat/hello-world +``` + +```text +git clone https://github.com/octocat/hello-world \ + /go/src/github.com/octocat/hello-world +``` + +# Cloning + +Drone automatically configures a default clone step if not explicitly defined. You can manually configure the clone step in your pipeline for customization: + +```diff ++clone: ++ git: ++ image: plugins/git + +pipeline: + build: + image: golang + commands: + - go build + - go test +``` + +Example configuration to override depth: + +```diff +clone: + git: + image: plugins/git ++ depth: 50 +``` + +Example configuration to use a custom clone plugin: + +```diff +clone: + git: ++ image: octocat/custom-git-plugin +``` + +Example configuration to clone Mercurial repository: + +```diff +clone: + hg: ++ image: plugins/hg ++ path: bitbucket.org/foo/bar +``` + +## Git Submodules + +To use the credentials that cloned the repository to clone it's submodules, update `.gitmodules` to use `https` instead of `git`: + +```diff +[submodule "my-module"] + path = my-module +- url = git@github.com:octocat/my-module.git ++ url = https://github.com/octocat/my-module.git +``` + +To use the ssh git url in `.gitmodules` for users cloning with ssh, and also use the https url in drone, add `submodule_override`: + +```diff +clone: + git: + image: plugins/git + recursive: true ++ submodule_override: ++ my-module: https://github.com/octocat/my-module.git + +pipeline: + ... +``` + +# Privileged mode + +Drone gives the ability to configure privileged mode in the Yaml. You can use this parameter to launch containers with escalated capabilities. + +> Privileged mode is only available to trusted repositories and for security reasons should only be used in private environments. + +```diff +pipeline: + build: + image: docker + environment: + - DOCKER_HOST=tcp://docker:2375 + commands: + - docker --tls=false ps + +services: + docker: + image: docker:dind + command: [ "--storage-driver=vfs", "--tls=false" ] ++ privileged: true +``` + +# Promoting + +Drone provides the ability to promote individual commits or tags (e.g. promote to production). When you promote a commit or tag it triggers a new pipeline execution with event type `deployment`. You can use the event type and target environment to limit step execution. + +```diff +pipeline: + build: + image: golang + commands: + - go build + - go test + + publish: + image: plugins/docker + registry: registry.heroku.com + repo: registry.heroku.com/my-staging-app/web + when: ++ event: deployment ++ environment: staging + + publish_to_prod: + image: plugins/docker + registry: registry.heroku.com + repo: registry.heroku.com/my-production-app/web + when: ++ event: deployment ++ environment: production +``` + +The above example demonstrates how we can configure pipeline steps to only execute when the deployment matches a specific target environment. + +## Triggering Deployments + +Deployments are triggered from the command line utility. They are triggered from an existing build. This is conceptually similar to promoting builds. + +```text +drone deploy +``` + +Promote the specified build number to your staging environment: + +```text +drone deploy octocat/hello-world 24 staging +``` + +Promote the specified build number to your production environment: + +```text +drone deploy octocat/hello-world 24 production +``` + +# Matrix builds + +Drone has integrated support for matrix builds. Drone executes a separate build task for each combination in the matrix, allowing you to build and test a single commit against multiple configurations. + +Example matrix definition: + +```yaml +matrix: + GO_VERSION: + - 1.4 + - 1.3 + REDIS_VERSION: + - 2.6 + - 2.8 + - 3.0 +``` + +Example matrix definition containing only specific combinations: + +```yaml +matrix: + include: + - GO_VERSION: 1.4 + REDIS_VERSION: 2.8 + - GO_VERSION: 1.5 + REDIS_VERSION: 2.8 + - GO_VERSION: 1.6 + REDIS_VERSION: 3.0 +``` + +## Interpolation + +Matrix variables are interpolated in the yaml using the `${VARIABLE}` syntax, before the yaml is parsed. This is an example yaml file before interpolating matrix parameters: + +```yaml +pipeline: + build: + image: golang:${GO_VERSION} + commands: + - go get + - go build + - go test + +services: + database: + image: ${DATABASE} + +matrix: + GO_VERSION: + - 1.4 + - 1.3 + DATABASE: + - mysql:5.5 + - mysql:6.5 + - mariadb:10.1 +``` + +Example Yaml file after injecting the matrix parameters: + +```diff +pipeline: + build: +- image: golang:${GO_VERSION} ++ image: golang:1.4 + commands: + - go get + - go build + - go test ++ environment: ++ - GO_VERSION=1.4 ++ - DATABASE=mysql:5.5 + +services: + database: +- image: ${DATABASE} ++ image: mysql:5.5 +``` + +## Examples + +Example matrix build based on Docker image tag: + +```yaml +pipeline: + build: + image: golang:${TAG} + commands: + - go build + - go test + +matrix: + TAG: + - 1.7 + - 1.8 + - latest +``` + +Example matrix build based on Docker image: + +```yaml +pipeline: + build: + image: ${IMAGE} + commands: + - go build + - go test + +matrix: + IMAGE: + - golang:1.7 + - golang:1.8 + - golang:latest +``` + +# Multi-pipeline builds + +By default, Drone looks for the pipeline definition in `.drone.yml` in the project root. + +The Multi-Pipeline feature allows the pipeline to be splitted to several files and placed in the `.drone/` folder + +## Example multi-pipeline definition + +```bash +.drone +├── .build.yml +├── .deploy.yml +├── .lint.yml +└── .test.yml +``` + +.drone/.build.yml + +```yaml +pipeline: + build: + image: debian:stable-slim + commands: + - echo building + - sleep 5 +``` + +.drone/.deploy.yml + +```yaml +pipeline: + deploy: + image: debian:stable-slim + commands: + - echo deploying + +depends_on: + - lint + - build + - test +``` + +.drone/.test.yml + +```yaml +pipeline: + test: + image: debian:stable-slim + commands: + - echo testing + - sleep 5 + +depends_on: + - build +``` + +.drone/.lint.yml + +```yaml +pipeline: + lint: + image: debian:stable-slim + commands: + - echo linting + - sleep 5 +``` + +## Flow control + +The pipelines run in parallel on a separate agents and share nothing. + +Dependencies between pipelines can be set with the `depends_on` element. A pipeline doesn't execute until its dependencies did not complete succesfully. + +```diff +pipeline: + deploy: + image: debian:stable-slim + commands: + - echo deploying + ++depends_on: ++ - lint ++ - build ++ - test +``` + +Pipelines that need to run even on failures should set the `run_on` tag. + +```diff +pipeline: + notify: + image: debian:stable-slim + commands: + - echo notifying + +depends_on: + - deploy + ++run_on: [ success, failure ] +``` + +Some pipelines don't need the source code, set the `skip_clone` tag to skip cloning: + +```diff + +pipeline: + notify: + image: debian:stable-slim + commands: + - echo notifying + +depends_on: + - deploy + +run_on: [ success, failure ] ++skip_clone: true +``` + +## Status lines + +Each pipeline has its own status line on Github. + +## Rational + +- faster lint/test feedback, the pipeline doesn't have to run fully to have a lint status pushed to the the remote +- better organization of the pipeline along various concerns: testing, linting, feature apps +- utilizaing more agents to speed up build + +# Badges + +Drone has integrated support for repository status badges. These badges can be added to your website or project readme file to display the status of your code. + +Badge endpoint: + +```text +:///api/badges///status.svg +``` + +The status badge displays the status for the latest build to your default branch (e.g. master). You can customize the branch by adding the `branch` query parameter. + +```diff +-:///api/badges///status.svg ++:///api/badges///status.svg?branch= +``` + +Please note status badges do not include pull request results, since the status of a pull request does not provide an accurate representation of your repository state. diff --git a/docs/usage/repo_list.png b/docs/usage/repo_list.png new file mode 100644 index 000000000..368c961ea Binary files /dev/null and b/docs/usage/repo_list.png differ