blog-ahmza-com/content/posts/building-a-docker-swarm-with-pis.md at a1138a52cf13c123eb2a950dc4254bd967f14533

ahmza/blog-ahmza-com

Fork 0

Files

T

AHMZA 95cdcc860a added back pi-docker-swam branch

2021-12-17 19:15:08 +00:00

8.8 KiB

Raw Blame History

title, date, author, draft, weight, tags, description, canonicalURL, cover, showtoc, searchHidden

title

date

author

draft

weight

Intro

The Raspberry Pi from it's first iteration has been a great hit. Adopted by many as the computer of choice for projects, and for very good reason. The Pi is a fully fledged computer that fits in your pocket and the most recent edition has blown fans away with the flagship model boasting an ARM64 CPU and 8GB of RAM.

Now one Raspberry Pi 4 8GB is more than powerful enough for most workloads and in some cases totally unnecessary, so you may even get away with a single 4GB model. However taking into consideration their relatively low price and power consumption why not go all out and build a cluster.

You can never have too many Raspberry Pis, just not enough Ideas

My excuse to build a Pi cluster is, I would like to migrate the services currently running on a bare metal server that sucks a lot of power to a low power solution. What better way to do that than a Pi cluster.

My Setup

4 x Raspberry Pi 4 8GB
4 x 32GB SD cards
4 x PoE hats
1 x 5 port PoE Switch

Insert Image of cluster here

Point of this post

Hopefully after following this post you will have setup Docker swarm on a cluster of Pis - stay even if you currently only have one - and understand how to bend it to your will 🦾.

Why Docker Swarm 🧐

While we can manage each docker node - Raspberry Pi - individually , Docker swarm enables us to manage all the devices from a single main node making managing the cluster a lot easier.

Think of Docker swarm as a set of 🤖 hands which will run tasks on each individual node for us. These tasks can consist of:

Cluster Management
Deployment and Scaling across the cluster
Multi-host networking
load balancing
etc...

Well why not Kubernetes

Although Kubernetes is awesome, it adds a bunch of complexity that is simply not needed in a home lab environment where we only have a small cluster. As well as that, from my experience creating docker-compose files is a lot easier than writing manifests. Docker swarm can do everything I want it to and requires less effort which is great if you're like me 🥱 😂

Required Stuff

The links below are reccomendations/stuff I used

Raspberry Pi 4B (Reccomend 4-8 @ 8GB)
Raspberry Pi PoE Hats (If you can find the PoE+ version get them)
Cluster Case (Get any case you want)
PoE switch (Get one that has enough ports for the nuber of pis you want)
RJ45 Ethernet Cable (Reccomend short cables as it will look neater)
Micro SD cards (Get enough cards for the number of Pis, Get the biggest cards you can afford)

This is without a doubt an expensive project, however in my eyes it's the cost of another server 😂 and the best part about this project is you can add to the cluster over time spreding the cost which is how I managed to purchase all the hardware. A suggestion would be to buy equipment that allows you room to grow so an 8 port PoE switch instead of a 4 port if you know you will scale to 8 nodes

If you build this then you can safely say another server is not needed. This cluster should be more than capable to serve all your home needs even if you only have 2 nodes. Put it this way, if you have 4 8GB nodes then you have a combined total of:

32GB of RAM
16 physical CPU cores @ 1.5GHz

Money Saving Tips

Just get 1-2 nodes to start
cut PoE and use normal power cables
Use Raspberry Pis with less memmory eg 4GB
Grab a really cheap case or make one using these
Use smaller SD Cards try not to go below 32GB (Reccomended 64GB If money saved elsewhere)

BUILD TIME !!

Pull yourself up by your bootstraps were jumping into this one runnig 💨

Image the SD Cards

So interesting fact the CPUs used in the raspberry pi 4 use an ARM64 archetecture. When looking for images of Raspbian you will only find 32bit variants which will work however will not make use of all the hardware which is just a waste. 64bit images exist however are well hidden, Dont fear 😨 you can download the latest images from the raspberrypi.org index

To write the cards I use the Raspberry Pi imager. once you have rpi-imager installed and you have the 64bit raspbian image, open sudo rpi-imager and on the "CHOOSE OS" section go to the bottom and select use custom image.

We dont need to unzip the file as the tool is clever enough to handle that for us

Before removing the SD card, mount the boot partition - usually the first partition, look for a file called cmdline.txt - and create an empty file called ssh sudo touch ssh this basically enables SSH on the Pis first boot allowing us to access the Pis in a totally headless manner.

Assemble the cluster

So assembly is very dependant on the kit you bought, and more often than not the stuff needed here comes with instructions or is self explanitory. So go ahead and assemble the cluster 🛠

There is an order of operations that I reccomend:

Install PoE hats onto the Pis
Install the SD Cards into the Pis
Assemble the case and install Pis
plug in ethernet cables

Now allow the Pis to boot for a few minutes so it can resize the file system

Sort out the Network bits

This is the easy part, there are two methods you can use to set a static IP address, one is on your DHCP server and the other is on the device itself. Personally I prefer to set static IPs on the device itself so I will cover that here.

To find the current ip of your Pis you can go to your DHCP server and look for the DHCP leases for the Pis.

there are naming conventions for naming primary and secondary things but this is your cluster so be creative with it, I have my primary node named Goob and the secondary nodes named Doris1 Doris2 and Doris3 reference

SSH to the Pi then open and edit the following files

/etc/hosts
- edit this line with your hostname: 127.0.1.1 goob
/etc/hostname
- edit this file with your hostname: goob

/etc/dhcpcd.conf

knowing we will be using only static IPs here I delete everything in the file add the following

interface eth0
static ip_address=10.10.10.20/24
static routers=10.10.10.1
static domain_name_servers=10.10.10.1

Now reboot the pi sudo shutdown -r now

Once the Pis are back up log into each one and change the default passwords you may also want to create a new user for yourself but thats totally optional.

Provision with Ansible

you want to create an Ansible control node and that is as simple as installing a package onto your machine. Ansible is built using python which makes it easy to install as we can use the python pip package manger. To check you have it installed do pip --version if it's not installed do the following

curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py

python get-pip.py --user

once you have pip installed proceed to installing Ansible

Docs

pip install ansible

Now that ansible is installed we can create a runbok that defines the steps ansible needs to take on the remote machine to correctly provision it. This playbook is written in yaml so get ready to make sure your indents are correct 😬

Creating the playbook now you can copy what I have and it should work - after changing a thing or two - which is fine however I encourage you to copy what I have, read the Docs and make any changes you feel are necessary.

8.8 KiB Raw Blame History