In this tutorial, we'll detail how to deploy self-hosted GitHub Actions runners to DigitalOcean with Docker. We'll also see how to scale the runners both vertically (via Docker Compose) and horizontally (via Docker Swarm).
Dependencies:
- Docker v19.03.8
- Docker-Compose v1.29.2
- Docker-Machine v0.16.2
Contents
GitHub Actions
GitHub Actions is a continuous integration and delivery (CI/CD) solution, fully integrated with GitHub. Jobs from a GitHub Actions workflow are run on applications called runners. You can either use GitHub-hosted runners or run your own self-hosted runners on your own infrastructure.
Runners can be added either to an individual repository or to an organization. We'll go with the latter approach so that the runners can process jobs from multiple repositories in the same GitHub organization.
Before beginning, you'll need to create a personal access token. Within your Developer Settings, click "Personal access tokens". Then, click "Generate new token". Provide a descriptive note and select the repo
, workflow
, and admin:org
scopes.
DigitalOcean Setup
First, sign up for a DigitalOcean account if you don't already have one, and then generate an access token so you can access the DigitalOcean API.
Add the token to your environment:
$ export DIGITAL_OCEAN_ACCESS_TOKEN=[your_digital_ocean_token]
Install Docker Machine if you don't already have it on your local machine.
Spin up a single droplet called runner-node
:
$ docker-machine create \
--driver digitalocean \
--digitalocean-access-token $DIGITAL_OCEAN_ACCESS_TOKEN \
--digitalocean-region "nyc1" \
--digitalocean-image "debian-10-x64" \
--digitalocean-size "s-4vcpu-8gb" \
--engine-install-url "https://releases.rancher.com/install-docker/19.03.9.sh" \
runner-node;
Docker Deployment
SSH into the droplet:
$ docker-machine ssh runner-node
Add the following Dockerfile, taking note of the comments:
# base
FROM ubuntu:18.04
# set the github runner version
ARG RUNNER_VERSION="2.283.3"
# update the base packages and add a non-sudo user
RUN apt-get update -y && apt-get upgrade -y && useradd -m docker
# install python and the packages the your code depends on along with jq so we can parse JSON
# add additional packages as necessary
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
curl jq build-essential libssl-dev libffi-dev python3 python3-venv python3-dev python3-pip
# cd into the user directory, download and unzip the github actions runner
RUN cd /home/docker && mkdir actions-runner && cd actions-runner \
&& curl -O -L https://github.com/actions/runner/releases/download/v${RUNNER_VERSION}/actions-runner-linux-x64-${RUNNER_VERSION}.tar.gz \
&& tar xzf ./actions-runner-linux-x64-${RUNNER_VERSION}.tar.gz
# install some additional dependencies
RUN chown -R docker ~docker && /home/docker/actions-runner/bin/installdependencies.sh
# copy over the start.sh script
COPY start.sh start.sh
# make the script executable
RUN chmod +x start.sh
# since the config and run script for actions are not allowed to be run by root,
# set the user to "docker" so all subsequent commands are run as the docker user
USER docker
# set the entrypoint to the start.sh script
ENTRYPOINT ["./start.sh"]
Update the
RUNNER_VERSION
variable with the latest version of the runner, which can be found here.
Add the start.sh file as well:
#!/bin/bash
ORGANIZATION=$ORGANIZATION
ACCESS_TOKEN=$ACCESS_TOKEN
REG_TOKEN=$(curl -sX POST -H "Authorization: token ${ACCESS_TOKEN}" https://api.github.com/orgs/${ORGANIZATION}/actions/runners/registration-token | jq .token --raw-output)
cd /home/docker/actions-runner
./config.sh --url https://github.com/${ORGANIZATION} --token ${REG_TOKEN}
cleanup() {
echo "Removing runner..."
./config.sh remove --unattended --token ${REG_TOKEN}
}
trap 'cleanup; exit 130' INT
trap 'cleanup; exit 143' TERM
./run.sh & wait $!
The ORGANIZATION
and ACCESS_TOKEN
(GitHub personal access token) environment variables are used for requesting a runner registration token.
For more, review the Create a registration token for an organization section from the documentation.
Take note of:
cleanup() {
echo "Removing runner..."
./config.sh remove --unattended --token ${REG_TOKEN}
}
trap 'cleanup; exit 130' INT
trap 'cleanup; exit 143' TERM
./run.sh & wait $!
Essentially, the cleanup logic, for removing the runner when the container is brought down, will execute when the container is stopped.
For more on shell signal handling along with
wait
, review this Stack Exchange answer.
Build the image and spin up the container in detached mode:
$ docker build --tag runner-image .
$ docker run \
--detach \
--env ORGANIZATION=<YOUR-GITHUB-ORGANIZATION> \
--env ACCESS_TOKEN=<YOUR-GITHUB-ACCESS-TOKEN> \
--name runner \
runner-image
Make sure to replace
<YOUR-GITHUB-ORGANIZATION>
and<YOUR-GITHUB-ACCESS-TOKEN>
with your organization and personal access token, respectively.
Take a quick look at the container logs:
$ docker logs runner -f
You should see something similar to:
--------------------------------------------------------------------------------
| ____ _ _ _ _ _ _ _ _ |
| / ___(_) |_| | | |_ _| |__ / \ ___| |_(_) ___ _ __ ___ |
| | | _| | __| |_| | | | | '_ \ / _ \ / __| __| |/ _ \| '_ \/ __| |
| | |_| | | |_| _ | |_| | |_) | / ___ \ (__| |_| | (_) | | | \__ \ |
| \____|_|\__|_| |_|\__,_|_.__/ /_/ \_\___|\__|_|\___/|_| |_|___/ |
| |
| Self-hosted runner registration |
| |
--------------------------------------------------------------------------------
# Authentication
√ Connected to GitHub
# Runner Registration
Enter the name of the runner group to add this runner to: [press Enter for Default]
Enter the name of runner: [press Enter for 332d0614b5e9]
This runner will have the following labels: 'self-hosted', 'Linux', 'X64'
Enter any additional labels (ex. label-1,label-2): [press Enter to skip]
√ Runner successfully added
√ Runner connection is good
# Runner settings
Enter name of work folder: [press Enter for _work]
√ Settings Saved.
√ Connected to GitHub
2021-10-23 22:36:01Z: Listening for Jobs
Then, under your GitHub organization name, click "Settings". In the left sidebar, click "Actions" and then "Runners". You should see a registered runner:
To test, add runs-on: [self-hosted] to a repository's workflow YAML file.
For example:
name: Sample Python
on: [push]
jobs:
build:
runs-on: [self-hosted]
steps:
- uses: actions/checkout@v2
- name: Install dependencies
run: |
python3 -m pip install --upgrade pip
pip3 install pytest
- name: Test with pytest
run: |
python3 -m pytest
Then, run a new build. Back in your terminal, within the Docker logs, you should see the status of the job:
2021-10-24 00:46:26Z: Running job: build
2021-10-24 00:46:34Z: Job build completed with result: Succeeded
If the repository that you're trying to run the job on is public, you'll have to update the default runner group to allow public repositories. For more, review Self-hosted runner security with public repositories.
Bring down the container once done:
$ docker stop runner
View the logs again. You should see:
Removing runner...
# Runner removal
√ Runner removed successfully
√ Removed .credentials
√ Removed .runner
The runner should no longer be available within your GitHub organization's Actions settings:
Remove the container:
$ docker rm runner
Vertical Scaling with Docker Compose
Want to spin up more than one runner on a single droplet?
Start by adding the following docker-compose.yml file to the box:
version: '3'
services:
runner:
build: .
environment:
- ORGANIZATION=<YOUR-GITHUB-ORGANIZATION>
- ACCESS_TOKEN=<YOUR-GITHUB-ACCESS-TOKEN>
Make sure to replace <YOUR-GITHUB-ORGANIZATION>
and <YOUR-GITHUB-ACCESS-TOKEN>
with your organization and personal access token, respectively.
Follow the official installation guide to download and install Docker Compose on the droplet, and then build the image:
$ docker-compose build
Spin up two container instances:
$ docker-compose up --scale runner=2 -d
You should see two runners on GitHub:
Kick off two builds. Open the Compose logs:
$ docker-compose logs -f
You should see something like:
runner_2 | 2021-10-24 00:52:56Z: Running job: build
runner_1 | 2021-10-24 00:52:58Z: Running job: build
runner_2 | 2021-10-24 00:53:04Z: Job build completed with result: Succeeded
runner_1 | 2021-10-24 00:53:11Z: Job build completed with result: Succeeded
You can scale down like so:
$ docker-compose up --scale runner=1 -d
Exit from the SSH session and destroy the Machine/droplet:
$ docker-machine rm runner-node -y
$ eval $(docker-machine env -u)
Horizontal Scaling with Docker Swarm
Want to scale horizontally across multiple DigitalOcean droplets?
Configure Droplets
Add the DigitalOcean access token to your environment:
$ export DIGITAL_OCEAN_ACCESS_TOKEN=[your_digital_ocean_token]
Spin up three new DigitalOcean droplets:
$ for i in 1 2 3; do
docker-machine create \
--driver digitalocean \
--digitalocean-access-token $DIGITAL_OCEAN_ACCESS_TOKEN \
--digitalocean-region "nyc1" \
--digitalocean-image "debian-10-x64" \
--digitalocean-size "s-4vcpu-8gb" \
--engine-install-url "https://releases.rancher.com/install-docker/19.03.9.sh" \
runner-node-$i;
done
Initialize Swarm mode on the first node, runner-node-1
:
$ docker-machine ssh runner-node-1 -- docker swarm init --advertise-addr $(docker-machine ip runner-node-1)
Use the join token from the output of the previous command to add the remaining two nodes to the Swarm as workers:
$ for i in 2 3; do
docker-machine ssh runner-node-$i -- docker swarm join --token YOUR_JOIN_TOKEN HOST:PORT;
done
For example:
$ for i in 2 3; do
docker-machine ssh runner-node-$i -- docker swarm join --token SWMTKN-1-4a341wv2n8c2c0cn3f9d0nwxndpohwuyr58vtal63wx90spfoo-09vdgcfarp6oqxnncgfjyrh0i 161.35.12.185:2377;
done
You should see:
This node joined a swarm as a worker.
This node joined a swarm as a worker.
Build Docker Image
This time let's build the image locally and push it up to the Docker Hub image registry.
Dockerfile:
# base
FROM ubuntu:18.04
# set the github runner version
ARG RUNNER_VERSION="2.283.3"
# update the base packages and add a non-sudo user
RUN apt-get update -y && apt-get upgrade -y && useradd -m docker
# install python and the packages the your code depends on along with jq so we can parse JSON
# add additional packages as necessary
RUN DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
curl jq build-essential libssl-dev libffi-dev python3 python3-venv python3-dev python3-pip
# cd into the user directory, download and unzip the github actions runner
RUN cd /home/docker && mkdir actions-runner && cd actions-runner \
&& curl -O -L https://github.com/actions/runner/releases/download/v${RUNNER_VERSION}/actions-runner-linux-x64-${RUNNER_VERSION}.tar.gz \
&& tar xzf ./actions-runner-linux-x64-${RUNNER_VERSION}.tar.gz
# install some additional dependencies
RUN chown -R docker ~docker && /home/docker/actions-runner/bin/installdependencies.sh
# copy over the start.sh script
COPY start.sh start.sh
# make the script executable
RUN chmod +x start.sh
# since the config and run script for actions are not allowed to be run by root,
# set the user to "docker" so all subsequent commands are run as the docker user
USER docker
# set the entrypoint to the start.sh script
ENTRYPOINT ["./start.sh"]
start.sh:
#!/bin/bash
ORGANIZATION=$ORGANIZATION
ACCESS_TOKEN=$ACCESS_TOKEN
REG_TOKEN=$(curl -sX POST -H "Authorization: token ${ACCESS_TOKEN}" https://api.github.com/orgs/${ORGANIZATION}/actions/runners/registration-token | jq .token --raw-output)
cd /home/docker/actions-runner
./config.sh --url https://github.com/${ORGANIZATION} --token ${REG_TOKEN}
cleanup() {
echo "Removing runner..."
./config.sh remove --unattended --token ${REG_TOKEN}
}
trap 'cleanup; exit 130' INT
trap 'cleanup; exit 143' TERM
./run.sh & wait $!
Build the image:
$ docker build --tag <your-docker-hub-username>/actions-image:latest .
Make sure to replace <your-docker-hub-username>
with your Docker Hub username. Then, push the image to the registry:
$ docker push <your-docker-hub-username>/actions-image:latest
Deploy
To deploy the stack, first add a Docker Compose file:
version: '3'
services:
runner:
image: <your-docker-hub-username>/actions-image:latest
deploy:
mode: replicated
replicas: 1
placement:
constraints:
- node.role == worker
environment:
- ORGANIZATION=<YOUR-GITHUB-ORGANIZATION>
- ACCESS_TOKEN=<YOUR-GITHUB-ACCESS-TOKEN>
Again, update <your-docker-hub-username>
as well as the environment variables.
Point the Docker daemon at runner-node-1
and deploy the stack:
$ eval $(docker-machine env runner-node-1)
$ docker stack deploy --compose-file=docker-compose.yml actions
List out the services in the stack:
$ docker stack ps -f "desired-state=running" actions
You should see something similar to:
ID NAME IMAGE NODE DESIRED STATE
xhh3r8rfhh46 actions_runner.1 mjhea0/actions-image:latest runner-node-2 Running
Make sure the runner is up on GitHub:
Let's add two more nodes:
$ docker service scale actions_runner=3
actions_runner scaled to 3
overall progress: 3 out of 3 tasks
1/3: running
2/3: running
3/3: running
verify: Service converged
Verify:
Kick off a few jobs and ensure they successfully finish.
Scale back down to a single runner:
$ docker service scale actions_runner=1
actions_runner scaled to 1
overall progress: 1 out of 1 tasks
1/1: running
verify: Service converged
Verify:
Bring down the Machines/droplets once done:
$ docker-machine rm runner-node-1 runner-node-2 runner-node-3 -y