From 2252341af823ec9cf9aedfa90245879ad7b4c107 Mon Sep 17 00:00:00 2001 From: Brad Rydzewski Date: Mon, 25 May 2015 15:27:33 -0700 Subject: [PATCH] updated .drone.yml to archive packages in s3 for download --- .drone.yml | 12 +++++++ doc/install-source.md | 44 ----------------------- doc/setup-agents.md | 34 ++++++++++++++++++ doc/setup-docker.md | 83 +++++++++++++++++++++++++++++++++++++++++++ doc/setup-nginx.md | 2 ++ 5 files changed, 131 insertions(+), 44 deletions(-) delete mode 100644 doc/install-source.md create mode 100644 doc/setup-agents.md create mode 100644 doc/setup-docker.md diff --git a/.drone.yml b/.drone.yml index 8420f27a9..1b6049bf6 100644 --- a/.drone.yml +++ b/.drone.yml @@ -16,6 +16,18 @@ notify: recipients: - brad@drone.io +publish: + s3: + acl: public-read + region: us-east-1 + bucket: downloads.drone.io + access_key: $$AWS_KEY + secret_key: $$AWS_SECRET + source: dist/drone.deb + target: $DRONE_BRANCH/ + when: + owner: drone + # new .drone.yml syntax clone: diff --git a/doc/install-source.md b/doc/install-source.md deleted file mode 100644 index 1da0b9a10..000000000 --- a/doc/install-source.md +++ /dev/null @@ -1,44 +0,0 @@ -This document provides a brief overview of Drone's build process, so that you can build and run Drone directly from source. For more detail, please see the `.drone.yml` and `Makefile`. - -### Requirements - -* Make -* Go 1.4+ -* Libsqlite3 - -### Build and Test - -We use `make` to build and package Drone. You can execute the following commands to compile the Drone binary: - -```bash -make deps -make -make test -``` - -The `all` directive compiles binary files to the `bin/` directory and embeds all static content (ie html, javascript, css files) directly into the binary for simplified distribution. - -Note: if you experience slow compile times you can `go install` the `go-sqlite3` library from the vendored dependencies. This will prevent Drone from re-compiling on every build: - -```bash -go install github.com/drone/drone/Godeps/_workspace/src/github.com/mattn/go-sqlite3 -``` - -### Run - -To run Drone you can invoke `make run`. This will start Drone with the `--debug` flag which instructs Drone to server static content from disk. This is helpful if you are doing local development and editing the static JavaScript or CSS files. - - -### Distribute - -To generate a debian package: - -```bash -make dist -``` - -To generate a Docker container: - -```bash -docker build --rm=true -t drone/drone . -``` \ No newline at end of file diff --git a/doc/setup-agents.md b/doc/setup-agents.md new file mode 100644 index 000000000..fdff9b1f7 --- /dev/null +++ b/doc/setup-agents.md @@ -0,0 +1,34 @@ +> **NOTE** this is an advanced feature and should not be required for most installs + +If you are running a large build cluster (25+ servers) we recommend using build agents. A build agent is a daemon that is installed on each build server that polls the central Drone server for work. You can add and remove build agents at any time, without having to configure and re-start the central Drone server. This is a great option if you need to frequently scale your build infrastrucutre up or down by adding or removing servers. + +You will need to configure the `/etc/drone/drone.toml` to enable build agents. This includes specifying a secret token (ie password) that will allow agents to authenticate to the central Drone server to poll for builds: + +```ini +[agents] +secret = "c0aaff74c060ff4a950d" +``` + +And then on each build server, pull the Drone agent image: + +```bash +docker pull drone/drone-agent +``` + +And start the Drone agent: + +```bash +docker run -d + -v /var/run/docker.sock:/var/run/docker.sock + -p 1991:1991 drone/drone-agent \ + --addr=http://localhost:8000 \ + --token=c0aaff74c060ff4a950d \ +``` + +The Drone agent is started with port `1991` exposed allowing the central Drone server to communicate directly with the agent. We also mount the Docker socket, `/var/run/docker.sock`, allowing the agent to spawn build containers. + +The Drone agent also requires the following command line arguments: + +* **addr** address of the running Drone server +* **token** token used to authorize requestests to the Drone server + diff --git a/doc/setup-docker.md b/doc/setup-docker.md new file mode 100644 index 000000000..4f1476cf2 --- /dev/null +++ b/doc/setup-docker.md @@ -0,0 +1,83 @@ +Drone is configured to connect to the local Docker daemon. Drone will attempt to use the `DOCKER_HOST` environment variable to determine the daemon URL. If not set, Drone will attempt to use the default socket connection `unix:///var/run/docker.sock`. + +You can modify the Docker daemon URL in the Drone configuration file: + +```ini +[docker] +nodes=[ + "unix:///var/run/docker.sock", + "unix:///var/run/docker.sock" +] +``` + +### Concurrency + +Each node is capable of processing a single build. Therefore, the below configuration will only execute one build at a time: + +```ini +[docker] +nodes=[ + "unix:///var/run/docker.sock" +] +``` + +In order to increase concurrency you can increase the number of nodes. The below configuration is capable of processing four builds at a time, all using the local Docker daemon: + +```ini +[docker] +nodes=[ + "unix:///var/run/docker.sock", + "unix:///var/run/docker.sock", + "unix:///var/run/docker.sock", + "unix:///var/run/docker.sock" +] +``` + +### Distribution + +As your installation grows you may need to distribute your builds across multiple servers. Since Docker exposes a REST API we can easily configure Drone to communicate with remote servers. First we'll need to generate an SSL certificate in order to secure communication across nodes. + +We recommend using this Gist to generate keys: +https://gist.github.com/bradrydzewski/a6090115b3fecfc25280 + +This will generate the following files: + +* ca.pem +* cert.pem +* key.pem +* server-cert.pem +* server-key.pem + +Update your Drone configuration to use the `cert.pem` and `key.pem` files and remote daemon URLs: + +```ini +[docker] +cert="/path/to/cert.pem" +key="/path/to/key.pem" +nodes = [ + "tcp://172.17.42.1:2376", + "tcp://172.17.42.2:2376", + "tcp://172.17.42.3:2376", + "tcp://172.17.42.4:2376" +] +``` + +> Remember that you can add the same URL multiple times to increase concurrency! + +Finally, you need to place the server key, certificate and ca on each remote server. You'll need to update the Docker daemon configuration on each remote server (in `/etc/init/drone-dart.conf`) and restart Docker: + +```bash +# Use DOCKER_OPTS to modify the daemon startup options. +DOCKER_OPTS="--tlsverify --tlscacert=/etc/ssl/docker/ca.pem --tlscert=/etc/ssl/docker/server-cert.pem --tlskey=/etc/ssl/docker/server-key.pem -H=0.0.0.0:2376 -H unix:///var/run/docker.sock" +``` + +Lastly, we can verify that everything is configured correctly. We can try to connect to a remote Docker server from our Drone server using the following command: + +```bash +sudo docker \ + --tls \ + --tlscacert=/path/to/ca.pem \ + --tlscert=/path/to/cert.pem \ + --tlskey=/path/to/key.pem \ + -H=1.2.3.4:2376 version +``` diff --git a/doc/setup-nginx.md b/doc/setup-nginx.md index 7a98b3946..1e1a63a64 100644 --- a/doc/setup-nginx.md +++ b/doc/setup-nginx.md @@ -13,6 +13,8 @@ location / { } ``` +It is very important to set the `X-Forwarded-For` and `X-Forwarded-Proto` header variables. Drone needs to know its own URL so that it can configure a repository's post-commit hooks properly. + You may also want to change Drone’s default port when proxying traffic. You can change the port in the `/etc/drone/drone.toml` configuration file: ```toml