Deployment of Swarm

What is Docker

Containers are a powerful tool. They simplify what used to be a cumbersome deployment process.

In the era of containerization, developers only need to package their applications into a container and deploy that container to any container-supported environment to run their applications.

Therefore, the process of running software is perfectly divided into two domains:

  • Developing a high-quality container (secure, stable, fast, stateless, reproducible)
  • Operating a container platform (high availability, scalability, automation, monitoring, logging, networking, alerting)

Docker is an excellent container platform. It provides a complete container solution, including container building, deployment, and execution.

file

This blog post assumes that you have a complete grasp of Docker's basic concepts. If you are not yet familiar with Docker, please read Deploying a Single Docker Application on a Server first.

Understanding Docker Compose

Unlike Docker, which can isolate the runtime environment of a single process, Docker Compose focuses on using a declarative syntax to define the runtime environments of multiple containers.

The core concept of Docker Compose is the docker-compose.yml file. This file defines the runtime environments for multiple containers. For example:

version: '3'
networks:
  my-net:
    driver: overlay

services:
  web:
    build: .
    ports:
      - "5000:5000"
    networks:
      - my-net
    volumes:
      - .:/code
    depends_on:
      - redis

  redis:
    image: "redis:alpine"
    networks:
      - my-net

In the example above, we defined two services: web and redis.

  • The web service uses a custom image, mounting the local . directory into the container's /code directory.
  • The redis service uses the official redis:alpine image.

When we start the stack using docker-compose up, Docker Compose automatically creates a network and connects both the web and redis containers to it. This allows the web container to access the redis service using the redis service name.

file

Docker Compose perfectly addresses deployment workflows for complex container compositions. It features simple, easy-to-understand syntax and is user-friendly, making it an excellent container orchestration tool.

However, Docker Compose also has some limitations. For example, it does not support cluster mode or dynamic scaling. These features require more complex container orchestration tools.

Understanding Docker Swarm Mode

Docker Swarm Mode, which I will refer to as Docker Swarm for brevity, is a container cluster orchestration tool provided officially by Docker. Its goal is to provide a simple and easy-to-use container orchestration solution.

Docker Swarm can use the same definitions as local Docker Composer to deploy application orchestration plans.

Docker Swarm Mode makes it very easy for you to achieve:

  • Reproducibility: Deploy applications using the same syntax as locally and get the same runtime behavior.
  • Clustering: Use multiple machines to deploy applications, achieving high availability and load balancing.
  • Dynamic scaling: Dynamically scale applications up or down based on load conditions.
  • Simplicity: No additional configuration or installation is required, and there is no need to learn new concepts.

If your computer already has Docker installed, you already have Docker Swarm installed. There is no need to install any additional software.

Outdated?

Docker Swarm is not very popular because there are many more powerful container orchestration tools in the community:

  • Kubernetes
  • Mesos
  • Nomad
  • ...

file

However, all of the above tools require you to spend days learning them. They introduce a whole new set of concepts, requiring you to master new configurations, syntax, file formats, commands, and so on.

In contrast, Docker Swarm can be used directly as long as you know the syntax of Docker Compose. It requires almost no additional learning. This is a significant advantage.

For a home HomeLab, a small company with fewer than 200 employees and fewer than 2,000 physical servers, or for someone looking to work on startup projects, side projects, or hackathons, I would recommend Docker Swarm over other container orchestration tools.

file

Of course, if you are Google, Amazon, or Microsoft, then Kubernetes is a better choice.

Similarly, if you have read this article and decided to learn Kubernetes, I would be happy for you. Because Kubernetes is an excellent container orchestration tool.

Building Your Small Cluster from the Ground Up

Prerequisite knowledge you need to have:

  • Linux
  • Docker
  • Server setup

Create 3 Servers

First, you need to prepare 3 servers. Name them: master, worker1, and worker2.

These 3 servers can be physical servers or virtual machines. You can buy small hosts from Taobao, purchase VPS instances from cloud service providers, or create them yourself using software such as Hyper-V, ESXi, PVE, or KVM.

file

Your 3 servers can run any operating system. However, I strongly recommend using Ubuntu 22.04. In my example, all servers are running Ubuntu 22.04.

Prepare the Network

Here, I recommend that all three of your servers be placed within the same dedicated local area network. This way, you won't need to configure additional firewall or router settings.

Most cloud providers offer virtual switch functionality, which you can use to create a dedicated local area network. Of course, software such as HyperV, ESXI, PVE, and KVM also provides similar capabilities.

I assume the internal IP addresses of your three servers are as follows:

  • master: 10.0.0.100/8
  • worker1: 10.0.0.101/8
  • worker2: 10.0.0.102/8

Use the following commands to set the Hostname for each of the three servers:

# Server 1
sudo hostnamectl set-hostname master
# Server 2
sudo hostnamectl set-hostname worker1
# Server 3
sudo hostnamectl set-hostname worker2

Here, you need to ensure that the network is connected between the three servers. You can use the ping command to test this.

Configure Server

This way, the Hostnames of your three servers are respectively:

  • master
  • worker1
  • worker2

Here, master will be used to manage the cluster, and all three servers will be able to run containers.

Install the necessary updates for them:

# Run on all 3 servers!
sudo apt update
sudo apt upgrade -y

Install Docker on three separate servers:

# Run on all 3 servers!
curl -fsSL get.docker.com -o get-docker.sh
CHANNEL=stable sh get-docker.sh
rm get-docker.sh

file

Create New Cluster

Log in to your master server here, and then run the following command:

sudo docker swarm init --advertise-addr 10.0.0.100

For a Vsphere data center, the Swarm cluster you build may not be able to use Overlay Network. I struggled with this error for a long time before finally finding a solution:

sudo docker swarm init --advertise-addr=10.0.0.100 --listen-addr=0.0.0.0 --data-path-port=7779 --force-new-cluster=true

Here, 10.0.0.100 is the internal IP address of your master server.

Soon, you will see an output guiding you on how to add worker1 and worker2 to the cluster.

file

docker swarm join --token SWMTKN-1-XXXXXXXXX 10.0.0.100

You need to run this command on the worker1 and worker2 servers, and they will join the cluster:

file

Now check your cluster! Go to the master server and run the following command:

sudo docker node ls

You will see the following successful output:

file

Of course, in the method described above, the configured cluster consists of one Manager and two Workers. This setup presents a potential issue: if the Manager goes down, you will lose management capabilities. To improve availability, you can also choose to designate all three nodes as Managers.

# 在 Manager 上运行
sudo docker swarm join-token manager
# 在 Worker 上运行
docker swarm join --token SWMTKN-1-YYYYYYYYY 10.64.50.159:2377

Cluster configuration complete

It's that simple. You now have a distributed Docker Swarm cluster.

Next, we will explain in detail how to deploy applications to this cluster.

Deploy a GUI Manager

Clearly, looking at these command-line interfaces can feel quite unintuitive. What does my cluster actually look like?

Currently, the most popular graphical interface manager for Docker Swarm is Swarmpit. It is an open-source project that provides a very beautiful graphical interface for managing your Docker Swarm cluster.

file

Run the following command on the master node:

sudo docker run -it --rm \
  --name swarmpit-installer \
  --volume /var/run/docker.sock:/var/run/docker.sock \
swarmpit/install:1.9

This will deploy Swarmpit to your cluster.

The command above may take some time to complete. You can use the command below to check the deployment status:

sudo docker service ls

If the number of replicas for a service remains at 0/1, the deployment may have failed. You can first check its process status:

sudo docker service ps <app-name, may be 'swarmpit_app'>

You can use the command below to view the deployment logs:

sudo docker service logs <app-name, may be 'swarmpit_app'>

file

If you see output similar to the above, it confirms that all Services in your Stack have been deployed successfully. You can now browse and access them normally!

You can visit http://master:888 in your browser to view the Swarmpit graphical interface.

On your first visit, you can create an administrator user.

file

Of course, if you accidentally messed something up and want to redo this step, simply run the command below:

sudo docker stack rm swarmpit

Learning Basic Concepts

The order of this blog post may seem odd. It is only at this point that I begin to explain the relationships between the basic concepts.

I believe you already understand the concept of a Docker Stack. A Stack is essentially the same as a docker-compose.yml file. Both are used to define the runtime environment for multiple containers.

Each docker-compose.yml file can be viewed as a template for a Stack.

A Service is the fundamental building block of a Stack. It represents a single service that the Stack needs to run. Note: A Service is not a single container, but rather a group of containers that share the same configuration, network, storage, etc., to collectively accomplish a task.

Each Service can contain multiple Tasks. A Task is the fundamental building block of a Service. It represents a single actual container that the Service needs to run.

Tasks map one-to-one with containers. One Task is one container. Therefore, for a given Service, the number of Tasks is equal to the number of containers. You can also use docker ps to check the status of Tasks.

file

These Tasks require actual hardware resources to run. These resources are Nodes. A Node is the fundamental building block of a Docker Swarm cluster. It represents a real Linux server.

A Node can be either a manager or a worker. Both types can run containers.

In addition to a series of Services, a Stack can also contain a series of Volumes.

A Volume is used for storing data. It can be shared by multiple containers. It can be used to store databases, files, logs, and so on. A Service can explicitly specify which paths should use which Volumes.

In addition to Volumes, another important concept is the Network. A Network is used to connect containers. It can be shared by multiple containers. It can be used to connect databases, caches, logs, and so on.

Practice Managing These Basic Concepts

In the Swarmpit graphical interface, you can view all Services, Tasks, Nodes, Volumes, and Networks.

Of course, we are still using the command line here, but you can also combine it with the Swarmpit graphical interface to practice managing these basic concepts.

file

For managing Stacks:

# 创建一个 Stack
sudo docker stack deploy -c <stack.yml> <stack-name>

# 查看所有的 Stack
sudo docker stack ls

# 查看一个 Stack 里的所有 Service
sudo docker stack services <stack-name>

# 删除一个 Stack
sudo docker stack rm <stack-name>

file

Managing Services:

# 查看所有的 Service
sudo docker service ls

# 查看一个 Service 里的所有 Task
sudo docker service ps <service-name>

# 查看一个 Service 的详细信息
sudo docker service inspect <service-name>

# 查看一个 Service 的日志
sudo docker service logs <service-name>

# 删除一个 Service
sudo docker service rm <service-name>

file

For Node management:

# 查看添加 Node 的命令
sudo docker swarm join-token worker
sudo docker swarm join-token manager

# 查看所有的 Node
sudo docker node ls

# 查看一个 Node 的详细信息
sudo docker node inspect <node-name>

# 查看一个 Node 上运行的所有 Task
sudo docker node ps <node-name>

# 将一个 Node 下线维护
sudo docker node update --availability drain <node-name>

# 将一个 Node 上线
sudo docker node update --availability active <node-name>

# 踢出一个 Node
sudo docker node rm <node-name>

file

Sometimes, we need to run some debugging commands within a specific container. You can:

# 先通过 Service 定位其是哪个 Task
sudo docker service ps <service-name>
<task-name> <node-name>

# SSH 到这个 Node
ssh user@<node-name>

# 找到其对应的容器 ID
sudo docker ps
<container-id> <image-name>

# 进入容器
sudo docker exec -it <container-id> sh

file

Deploying a Real Business

Now that you understand the core concepts and have deployed Swarmpit for management, it's time to deploy your actual business application.

When deploying for real, you only need to write the yml file. If the business logic changes, simply modify the yml file and run the deployment script again.

For example, if you need to deploy moongladepure:

version: '3.3'

services:
  moongladepure:
    image: hub.aiursoft.cn/aiursoft/moongladepure:latest
    volumes:
     - moonglade-data:/data
    deploy:
      replicas: 5
      endpoint_mode: vip

    ports:
      - 8000:5000

volumes:
  moonglade-data:
    driver: local

Then run:

sudo docker stack deploy -c moongladepure.yaml moon

Soon, accessing port 8000 on any Node will serve your deployed service.

Of course, exposing port 8000 as described above is not a best practice.

You can deploy caddy, expose ports 80 and 443, and thereby enable automated HTTPS.

Endless possibilities await you!

Setting Up Shared Storage

# Assuming using /dev/sdb is your new empty disk
# Assuming 10.0.0.* is your private network

# Install and configure GlusterFS. (Run on all nodes)
apt-get install glusterfs-server -y
systemctl start glusterd
systemctl enable glusterd

# Format the disk and mount it (Run on all nodes)
mkfs.xfs /dev/sdb
echo '/dev/sdb /var/no-direct-write-here/gluster-bricks xfs defaults 0 0' >> /etc/fstab
mkdir -p /var/no-direct-write-here/gluster-bricks
mount /var/no-direct-write-here/gluster-bricks

# Add the peers (Run on node1)
gluster peer probe node2
gluster peer probe node3
gluster peer status
gluster pool list

# Create the volume (Run on node1)
gluster volume create swarm-vol replica 3 \
  node1:/var/no-direct-write-here/gluster-bricks/swarm-vol \
  node2:/var/no-direct-write-here/gluster-bricks/swarm-vol \
  node3:/var/no-direct-write-here/gluster-bricks/swarm-vol
gluster volume set swarm-vol auth.allow 10.64.50.*
gluster volume start swarm-vol
gluster volume status
gluster volume info swarm-vol

# Mount the volume (Run on all nodes) 
echo 'localhost:/swarm-vol /swarm-vol glusterfs defaults,_netdev,noauto,x-systemd.automount 0 0' >> /etc/fstab
mkdir -p /swarm-vol
mount /swarm-vol
chown -R root:docker /swarm-vol

# Final check
cd /swarm-vol/
df -Th .

# READ: https://docs.gluster.org/en/v3/Administrator%20Guide/Managing%20Volumes/

Bring Nodes Back Online After Restart

echo "[Unit]
Description=Update Docker node availability
After=docker.service
Requires=docker.service

[Service]
Type=oneshot
ExecStart=/usr/bin/bash -c 'sleep 10 && sudo docker node update --availability active $(hostname)'

[Install]
WantedBy=multi-user.target" | sudo tee /etc/systemd/system/update-docker-node-availability.service