Swarm Clusters on Digital Ocean

9 minute read Updated

How to set-up a two-node Swarm cluster on Digital Ocean using Docker Machine.
Table of Contents

Lately I’ve been learning more about cloud architecture and related tooling. Stuff like Lambda, Serverless, AWS CLI and – now that it’s a part of Docker Machine – container orchestration with Docker Swarm clusters.

As an AWS user I’m particularly geeked about the Docker Private Beta, which makes it possible to experiment with Swarm using Amazon Web Services. But rather than waiting for a private beta we’re going to experiment with Docker Swarm using one of my favorite prototyping tools apart from the RPi: Digital Ocean.

In this tutorial you will learn how to use the Docker Machine CLI to create a Swarm cluster on Digital Ocean directly from the command line. Upon completion you will have a better understanding of container orchestration, Docker Swarm Mode, Docker Machine and how to use Machine provision a multi-node Swarm cluster on Digital Ocean.

Getting started

Before creating a swarm cluster you’ll need a Digital Ocean an account with a Personal Access Token generated, Docker installed alongside Docker Machine and your SSH keys set-up on your development machine.

Profit: If you don’t already have an account with Digital Ocean you may use my referral code and receive $10 in free credit toward usage fees.

Use your Digital Ocean account to generate a Personal Access Token for use with Docker Swarm and save it somewhere for later retrieval.

Then download and install Docker for Mac or Windows depending on your platform. Docker Machine is included with the basic installation so there should not be anything else to download in order to use Machine.

If you have Docker installed already, update to v1.12 or later for use of Swarm from the Docker Machine CLI tool. Use docker -v for version.

Finally, follow the instructions to Set up SSH keys on your machine.

Create swarm cluster

It’s time to create a swarm cluster. We’ll accomplish this using the legacy token-based discovery method which uses something called a Swarm token.

The Swarm token allows clustered nodes to communicate with one another. Generate one while you create your cluster using the following command:

docker run swarm create

Resulting in output like:

Unable to find image 'swarm:latest' locally
latest: Pulling from library/swarm
d85c18077b82: Pull complete
1e6bb16f8cb1: Pull complete
85bac13497d7: Pull complete
Digest: sha256:406022f04a3d0c5ce4dbdb60422f24052c20ab7e6d41ebe5723aa649c3833975
Status: Downloaded newer image for swarm:latest
efa10e961263e7370fe719f47931708b

The last line of which is the Swarm token. Set it to an environment variable called SWARM_CLUSTER_TOKEN for use in the next section as shown here:

export SWARM_CLUSTER_TOKEN=efa10e961263e7370fe719f47931708b
Note: If you run into any friction here, see the legacy Provision a Swarm cluster with Docker Machine page on the Docker Docs website for help.

Add nodes to swarm

Now use Machine CLI to create and add two nodes to the cluster. One node will be the leader of the swarm — the other a slave. Each node will be created using a Digital Ocean droplet which we will remove in a later step.

Note: See Pricing on Digital Ocean for the latest prices and availability.

Start by setting the Personal Access Token you generated while Getting started to an environment variable named DIGITAL_OCEAN_TOKEN as shown here:

export DIGITAL_OCEAN_TOKEN=51d1e4532d3943dfd033a053a506bf15489596db035b3374e87d80fc4cc01505

And use it along with the Swarm token to provision the first Swarm node. This will be the master, or swarm leader. Create it using Machine CLI:

docker-machine create \
--driver digitalocean \
--digitalocean-access-token ${DIGITAL_OCEAN_TOKEN} \
--digitalocean-region nyc3 \
--digitalocean-size 512mb \
--swarm --swarm-master \
--swarm-discovery token://${SWARM_CLUSTER_TOKEN} \
swarm-manager

Resulting in output like:

swarm-manager
Running pre-create checks...
Creating machine...
(swarm-manager) Creating SSH key...
(swarm-manager) Creating Digital Ocean droplet...
(swarm-manager) Waiting for IP address to be assigned to the Droplet...
Waiting for machine to be running, this may take a few minutes...
Detecting operating system of created instance...
Waiting for SSH to be available...
Detecting the provisioner...
Provisioning with ubuntu(systemd)...
Installing Docker...
Copying certs to the local machine directory...
Copying certs to the remote machine...
Setting Docker configuration on the remote daemon...
Configuring swarm...
Checking connection to Docker...
Docker is up and running!

Repeat the process, replacing swarm-manager with swarm-worker and dropping --swarm-master, to create a second, or swarm slave, as shown here:

docker-machine create \
--driver digitalocean \
--digitalocean-access-token=${DIGITAL_OCEAN_TOKEN} \
--digitalocean-region=nyc3 \
--digitalocean-size=512mb \
--swarm \
--swarm-discovery token://${SWARM_CLUSTER_TOKEN} \
swarm-worker
Note: In early 2018 Digital Ocean announced new droplet plans and the 512mb option was retired and replaced with with a s-1vcpu-1gb option.

If everything worked as expected you now have two Digital Ocean droplets each hosting a swarm node, with one designated as the master. Let’s verify that.

Verify node creation

Verify host nodes were created as expected using the Machine CLI. Run the following to to get a list of nodes:

docker-machine ls

You should see output like:

NAME            ACTIVE   DRIVER         STATE     URL                          SWARM                    DOCKER    ERRORS
swarm-worker    -        digitalocean   Running   tcp://104.131.163.130:2376   swarm-manager            v1.12.0
swarm-manager   -        digitalocean   Running   tcp://104.236.207.117:2376   swarm-manager (master)   v1.12.0

If you don’t see the nodes you expected, you can use the docker-machine rm command to remove nodes by NAME or go back and try and recreate them.

Note: Removing machines listed with the digitalocean driver will attempt to destroy the host droplet or assume the droplet was already destroyed.

Once both machines are up and running on Digital Ocean you’re ready to move on.

Connect the nodes

With the two nodes created and verified it’s time to connect them. To do so we will initialize swarm on swarm-manager then ask swarm-worker to join them.

Start by initializing swarm on the manager:

eval $(docker-machine env swarm-manager) && \
docker swarm init --advertise-addr $(docker-machine ip swarm-manager)

The above command will attach a shell to the manager, and use it to initialize a swarm and begin advertising its address. Once run you should see output like:

Swarm initialized: current node (8uaq2r536t4mgs6ci95yqmf5m) is now a manager.

To add a worker to this swarm, run the following command:
    docker swarm join \
    --token SWMTKN-1-1zm9fvpvk03vwxasrxjk6y7lueor77789ip2xfo0czmny3tvui-evnoe2a0x0ygex88vl3io9n1f \
    104.236.207.117:2377

To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.

Now switch over to the worker and ask it to join the swarm:

eval $(docker-machine env swarm-worker) && \
docker swarm join \
--token SWMTKN-1-1zm9fvpvk03vwxasrxjk6y7lueor77789ip2xfo0czmny3tvui-evnoe2a0x0ygex88vl3io9n1f \
104.236.207.117:2377

Be sure to use the expected token. When the worker joins you will see:

This node joined a swarm as a worker.

And now the two nodes are swarming (clustered).

Verify the connection

Check with the manager to verify the worker actually joined their swarm:

eval $(docker-machine env --swarm swarm-manager) && \
docker info | grep worker -A 8

The --swarm flag is used to get info about the swarm. Grep to show more information about the worker. You should see output like:

swarm-worker: 104.131.163.130:2376
 └ ID: DKKE:XMMP:Y274:4N5H:CW6T:Z5CD:5I3H:KI35:RL77:DY3J:3D3F:ALDX
 └ Status: Healthy
 └ Containers: 1 (1 Running, 0 Paused, 0 Stopped)
 └ Reserved CPUs: 0 / 1
 └ Reserved Memory: 0 B / 512.9 MiB
 └ Labels: kernelversion=4.4.0-31-generic, operatingsystem=Ubuntu 16.04.1 LTS, provider=digitalocean, storagedriver=aufs
 └ UpdatedAt: 2016-08-11T01:49:27Z
 └ ServerVersion: 1.12.0

With the worker and manager swarming you can give them something to do.

Create a service

To make use of our swarm we’re going to create a Swarm service that consists of six replicated Nginx containers, spread equally across both nodes.

First shell into manager and create the service:

eval $(docker-machine env swarm-manager) && \
docker service create -p 8000:80 \
--replicas 6 --name web nginx

The above command uses Engine to create a new service called web using the official build of nginx replicated six times across your two swarm host nodes.

You should see output indicating progress:

swly782eykcinix09jns3t7xy
overall progress: 2 out of 6 tasks
1/6: starting
2/6: running
3/6: running
4/6: starting
5/6: starting
6/6: starting

When tasks are stable a verify: Service converged message will appear.

If you run into any errors...
Expand for debugging information

Attempt to create service on the worker:
Error response from daemon: This node is not a swarm manager. Worker nodes can't be used to view or modify cluster state. Please run this command on a manager node or promote the current node to a manager.

Attempt to use reuse active ingress port:
Error response from daemon: rpc error: code = InvalidArgument desc = port '8000' is already in use by service 'web' (xim9nv43t3otpg57soauhnu70) as an ingress port

Attempt to use an existing service name:
Error response from daemon: rpc error: code = Unknown desc = name conflicts with an existing object

Attempt to remove nonexistant service:
Error: No such service: web

Once the service converges, verify it’s working as expected.

Verify service

Run the docker service ps command:

docker service ps web

You should see six replicas spread evenly across both of your swarm nodes:

ID                         NAME   IMAGE  NODE           DESIRED STATE  CURRENT STATE               ERROR
c9m65b4efoz1026o6e3k8yvpv  web.1  nginx  swarm-worker   Running        Running about a minute ago
66y7fwei2i3k863lyvdf2e202  web.2  nginx  swarm-manager  Running        Running about a minute ago
4hqsepx7cgkdlgby7wtjrntx8  web.3  nginx  swarm-manager  Running        Running about a minute ago
3inowh5wxm5zymqva4kv7ksf9  web.4  nginx  swarm-worker   Running        Running about a minute ago
72mkdmm0l36702m9g24xnkaa7  web.5  nginx  swarm-manager  Running        Running about a minute ago
2dz8zmaf9ijqmhcnwq9c3mtfm  web.6  nginx  swarm-worker   Running        Running about a minute ago
Note: Swarm has a built-in load balancer to distribute workload event across replicas using round-robin. Learn about it inside and out.

Verify both nodes are serving content:

wget -qO - http://$(docker-machine ip swarm-worker):8000 | grep title && \
wget -qO - open http://$(docker-machine ip swarm-manager):8000 | grep title

If you see the following everything worked:

<title>Welcome to nginx!</title>
<title>Welcome to nginx!</title>

You’ve now set-up a basic two-node cluster on Digital Ocean using Docker Swarm.

Wrapping up

In this post you learned how to use the command line to create a Swarm cluster with Machine CLI and use it to create basic two-node Swarm service running 6 nginx replicas spread across two swarm host nodes on Digital Ocean.

If you like You can now learn how to scale the service. But don’t forget to take down your service clean-up remove your nodes when you’re finished:

docker service rm web && \
docker-machine rm swarm-worker && \
docker-machine rm swarm-manager

The second two commands will remove the droplets created on Digital Ocean so they don’t run up your bills in case you forget about them.

You can confirm the local reference and remote instance of the nodes were removed by inspecting your droplets on Digital Ocean Cloud.

Welcome to the swarm!

Further Reading

This post wouldn’t have be possible without some initial inspiration and guidance from following resources, so please check them out:

Advertisement
Ledger Nano X - The secure hardware wallet