vault and consul

Deploying Vault and Consul

Posted by Michael Herman on Aug 6, 2018

Let’s look at how to deploy Hashicorp’s Vault and Consul to DigitalOcean with Docker Swarm.

This tutorial assumes that you have a basic working knowledge of using Vault and Consul to manage secrets. Please refer to the Managing Secrets with Vault and Consul blog post for more info.

Upon completion, you will be able to:

  1. Provision hosts on Digital Ocean with Docker Machine
  2. Configure a Docker Swarm cluster to run on Digital Ocean
  3. Run Vault and Consul on Docker Swarm

Contents

Main dependencies:

  • Docker v18.06.0-ce
  • Docker-Compose v1.22.0
  • Docker-Machine v0.15.0
  • Vault v0.10.3
  • Consul v1.2.1

Consul

Create a new project directory:

$ mkdir vault-consul-swarm && cd vault-consul-swarm

Then, add a docker-compose.yml file to the project root:

version: '3.6'

services:

  server-bootstrap:
    image: consul:1.2.1
    ports:
      - 8500:8500
    command: "agent -server -bootstrap-expect 3 -ui -client 0.0.0.0 -bind '{{ GetInterfaceIP \"eth0\" }}'"

  server:
    image: consul:1.2.1
    command: "agent -server -retry-join server-bootstrap -client 0.0.0.0 -bind '{{ GetInterfaceIP \"eth0\" }}'"
    deploy:
      replicas: 2
    depends_on:
      - server-bootstrap

  client:
    image: consul:1.2.1
    command: "agent -retry-join server-bootstrap -client 0.0.0.0 -bind '{{ GetInterfaceIP \"eth0\" }}'"
    deploy:
      replicas: 2
    depends_on:
      - server-bootstrap

networks:
  default:
    external:
      name: core

This configuration should look familiar.

  1. Refer to the Compose File section of the Running Flask on Docker Swarm blog post for more info on using a compose file for Docker Swarm mode.
  2. Review the Consul with Containers guide for info on the above Consul config.

Docker Swarm

Sign up for a DigitalOcean account (if you don’t already have one), and then generate an access token so you can access the DigitalOcean API.

Add the token to your environment:

$ export DIGITAL_OCEAN_ACCESS_TOKEN=[your_digital_ocean_token]

Spin up three droplets:

$ for i in 1 2 3; do
    docker-machine create \
      --digitalocean-region "nyc1" \
      --driver digitalocean \
      --digitalocean-size "8gb" \
      --digitalocean-access-token $DIGITAL_OCEAN_ACCESS_TOKEN \
      node-$i;
done

Initialize Swarm mode on the first node - node-1:

$ docker-machine ssh node-1 -- docker swarm init --advertise-addr $(docker-machine ip node-1)

Use the join token from the output of the previous command to add the remaining two nodes to the Swarm as workers:

$ for i in 2 3; do
    docker-machine ssh node-$i -- docker swarm join --token YOUR_JOIN_TOKEN;
done

You should see:

This node joined a swarm as a worker.
This node joined a swarm as a worker.

Point the Docker daemon at node-1, create an attachable overlay network (called core), and deploy the stack:

$ eval $(docker-machine env node-1)
$ docker network create -d overlay --attachable core
$ docker stack deploy --compose-file=docker-compose.yml secrets

List out the services in the stack:

$ docker stack ps -f "desired-state=running" secrets

You should see something similar to:

ID            NAME                        IMAGE         NODE    DESIRED STATE   CURRENT STATE
2y4onn0vq6nc  secrets_server.1            consul:1.2.1  node-1  Running         Running 50 seconds ago
xl9nca1y0ki8  secrets_server-bootstrap.1  consul:1.2.1  node-3  Running         Running 51 seconds ago
q5t58fwy2chc  secrets_client.1            consul:1.2.1  node-2  Running         Running 50 seconds ago
u60ff2em64lr  secrets_server.2            consul:1.2.1  node-2  Running         Running 50 seconds ago
yueouxid900k  secrets_client.2            consul:1.2.1  node-1  Running         Running 50 seconds ago

Grab the IP associated with node-1:

$ docker-machine ip node-1

Then, test out the Consul UI in your browser at http://YOUR_MACHINE_IP:8500/ui. There should be three running services and five nodes.

consul ui services

consul ui nodes

Vault

Add the vault service to docker-compose.yml:

  vault:
    image: vault:0.10.3
    deploy:
      replicas: 1
    ports:
      - 8200:8200
    environment:
      - VAULT_ADDR=http://127.0.0.1:8200
      - VAULT_LOCAL_CONFIG={"backend":{"consul":{"address":"http://server-bootstrap:8500","path":"vault/"}},"listener":{"tcp":{"address":"0.0.0.0:8200","tls_disable":1}},"ui":true, "disable_mlock":true}
    command: server
    depends_on:
      - consul

Take note of the VAULT_LOCAL_CONFIG environment variable:

{
  "backend": {
    "consul": {
      "address": "http://server-bootstrap:8500",
      "path": "vault/"
    }
  },
  "listener": {
    "tcp": {
      "address": "0.0.0.0:8200",
      "tls_disable": 1
    }
  },
  "ui": true,
  "disable_mlock": true
}

Review the Consul Backend section from the Managing Secrets with Vault and Consul blog post for more info. Also, setting disable_mlock to true is not recommended for production environments; however, it must be enabled since --cap-add is not available in Docker Swarm mode. See the following GitHub issues for details:

  1. –cap-add=IPC_LOCK unavailable in docker swarm
  2. Missing from Swarmmode –cap-add

Test

Re-deploy the stack:

$ docker stack deploy --compose-file=docker-compose.yml secrets

Wait a few seconds for the services to spin up, then check the status:

$ docker stack ps -f "desired-state=running" secrets

Again, you should see something similar to:

ID             NAME                        IMAGE          NODE    DESIRED STATE  CURRENT STATE
8lpsi3x3ic2n   secrets_vault.1             vault:0.10.3   node-3  Running        Running 2 minutes ago
2y4onn0vq6nc   secrets_server.1            consul:1.2.1   node-1  Running        Running 9 minutes ago
xl9nca1y0ki8   secrets_server-bootstrap.1  consul:1.2.1   node-3  Running        Running 9 minutes ago
q5t58fwy2chc   secrets_client.1            consul:1.2.1   node-2  Running        Running 9 minutes ago
u60ff2em64lr   secrets_server.2            consul:1.2.1   node-2  Running        Running 9 minutes ago
yueouxid900k   secrets_client.2            consul:1.2.1   node-1  Running        Running 9 minutes ago

Next, ensure Vault is listed on the “Services” section of the Consul UI:

consul ui services

You should now be able to interact with Vault via the CLI, HTTP API, and the UI. We’ll be using the UI in this tutorial to unseal, log in, and interact with the secrets backend, but feel free to test things out on your own via the CLI or HTTP API.

Init, unseal, and log in:

vault init

Create a new secret:

vault add secret

Remove the nodes once done:

$ docker-machine rm node-1 node-2 node-3 -y

Automation Script

Finally, let’s create a quick script to automate the deployment process:

  1. Provision three DigitalOcean droplets with Docker Machine
  2. Configure Docker Swarm mode
  3. Add nodes to the Swarm
  4. Deploy the stack

Add a new file called deploy.sh to the project root:

#!/bin/bash


echo "Spinning up three droplets..."

for i in 1 2 3; do
  docker-machine create \
    --digitalocean-region "nyc1" \
    --driver digitalocean \
    --digitalocean-size "8gb" \
    --digitalocean-access-token $DIGITAL_OCEAN_ACCESS_TOKEN \
    node-$i;
done


echo "Initializing Swarm mode..."

docker-machine ssh node-1 -- docker swarm init --advertise-addr $(docker-machine ip node-1)


echo "Adding the nodes to the Swarm..."

TOKEN=`docker-machine ssh node-1 docker swarm join-token worker | grep token | awk '{ print $5 }'`

for i in 2 3; do
  docker-machine ssh node-$i \
    -- docker swarm join --token ${TOKEN} $(docker-machine ip node-1):2377;
done


echo "Creating networking..."

eval $(docker-machine env node-1)
docker network create -d overlay --attachable core


echo "Deploying the stack..."

docker stack deploy --compose-file=docker-compose.yml secrets

Try it out!

$ sh deploy.sh

Bring down the droplets once done:

$ docker-machine rm node-1 node-2 node-3 -y

The code can be found in the vault-consul-swarm repo. Cheers!


Microservices with Docker, Flask, and React

Get the full course. Learn how to build, test, and deploy microservices powered by Docker, Flask, and React!


Table of Contents


Join our mailing list to be notified about course updates and new tutorials.