220 lines
60 KiB
HTML
220 lines
60 KiB
HTML
<!doctype html><html lang=en dir=auto><head><meta charset=utf-8><meta http-equiv=X-UA-Compatible content="IE=edge"><meta name=viewport content="width=device-width,initial-scale=1,shrink-to-fit=no"><meta name=robots content="index, follow"><title>Building a Docker Swarm With Pi's 👾 👾 👾 👾 | ahmza's ramblings</title>
|
||
<meta name=keywords content="Homelab,Docker,Docker Swarm,Raspberry Pi,Pi4"><meta name=description content="Pis are very capable computers, when banded together they're almost unstoppable"><meta name=author content="ahmza"><link rel=canonical href=https://blog.ahmza.com/Page%28/posts/building-a-docker-swarm-with-pis.md%29><link crossorigin=anonymous href=/assets/css/stylesheet.49e35863ac4b53ab71c7e3249dd5e722c701ee07f8d0cb006a4f4e954364b6fe.css integrity="sha256-SeNYY6xLU6txx+MkndXnIscB7gf40MsAak9OlUNktv4=" rel="preload stylesheet" as=style><script defer crossorigin=anonymous src=/assets/js/highlight.f413e19d0714851f6474e7ee9632408e58ac146fbdbe62747134bea2fa3415e0.js integrity="sha256-9BPhnQcUhR9kdOfuljJAjlisFG+9vmJ0cTS+ovo0FeA=" onload=hljs.initHighlightingOnLoad()></script><link rel=icon href=https://blog.ahmza.com/img/favicon.png><link rel=icon type=image/png sizes=16x16 href=https://blog.ahmza.com/img/favicon.png><link rel=icon type=image/png sizes=32x32 href=https://blog.ahmza.com/img/favicon.png><link rel=apple-touch-icon href=https://blog.ahmza.com/img/favicon.png><link rel=mask-icon href=https://blog.ahmza.com/img/favicon.png><meta name=theme-color content="#2e2e33"><meta name=msapplication-TileColor content="#2e2e33"><noscript><style>#theme-toggle,.top-link{display:none}</style></noscript><script async src="https://www.googletagmanager.com/gtag/js?id=G-HSJHLHP8FZ"></script><script>var doNotTrack=!1;if(!doNotTrack){window.dataLayer=window.dataLayer||[];function gtag(){dataLayer.push(arguments)}gtag("js",new Date),gtag("config","G-HSJHLHP8FZ",{anonymize_ip:!1})}</script><meta property="og:title" content="Building a Docker Swarm With Pi's 👾 👾 👾 👾"><meta property="og:description" content="Pis are very capable computers, when banded together they're almost unstoppable"><meta property="og:type" content="article"><meta property="og:url" content="https://blog.ahmza.com/posts/building-a-docker-swarm-with-pis/"><meta property="og:image" content="https://blog.ahmza.com/webp/posts/pi-docker-swarm/feature-anim.webp"><meta property="article:section" content="posts"><meta property="article:published_time" content="2021-12-12T17:23:32+00:00"><meta property="article:modified_time" content="2021-12-12T17:23:32+00:00"><meta property="og:site_name" content="ahmza's ramblings"><meta name=twitter:card content="summary_large_image"><meta name=twitter:image content="https://blog.ahmza.com/webp/posts/pi-docker-swarm/feature-anim.webp"><meta name=twitter:title content="Building a Docker Swarm With Pi's 👾 👾 👾 👾"><meta name=twitter:description content="Pis are very capable computers, when banded together they're almost unstoppable"><script type=application/ld+json>{"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":2,"name":"Posts","item":"https://blog.ahmza.com/posts/"},{"@type":"ListItem","position":3,"name":"Building a Docker Swarm With Pi's 👾 👾 👾 👾","item":"https://blog.ahmza.com/posts/building-a-docker-swarm-with-pis/"}]}</script><script type=application/ld+json>{"@context":"https://schema.org","@type":"BlogPosting","headline":"Building a Docker Swarm With Pi's 👾 👾 👾 👾","name":"Building a Docker Swarm With Pi\u0027s 👾 👾 👾 👾","description":"Pis are very capable computers, when banded together they're almost unstoppable","keywords":["Homelab","Docker","Docker Swarm","Raspberry Pi","Pi4"],"articleBody":"Intro The Raspberry Pi from it’s first iteration has been a great hit. Adopted by many as the computer of choice for projects, and for very good reason. The Pi is a fully fledged computer that fits in your pocket and the most recent edition has blown fans away with the flagship model boasting an ARM64 CPU and 8GB of RAM.\nNow one Raspberry Pi 4 8GB is more than powerful enough for most workloads and in some cases totally unnecessary, so you may even get away with a single 4GB model. However taking into consideration their relatively low price and power consumption why not go all out and build a cluster.\nYou can never have too many Raspberry Pis, just not enough Ideas\nMy excuse to build a Pi cluster is, I would like to migrate the services currently running on a bare metal server that sucks a lot of power to a low power solution. What better way to do that than a Pi cluster.\nMy Setup 4 x Raspberry Pi 4 8GB 4 x 32GB SD cards 4 x PoE hats 1 x 5 port PoE Switch Point of this post Hopefully after following this post you will have setup Docker swarm on a cluster of Pis - stay even if you currently only have one - and understand how to bend it to your will 🦾.\nWhy Docker Swarm 🧐 While we can manage each docker node - Raspberry Pi - individually , Docker swarm enables us to manage all the devices from a single main node making managing the cluster a lot easier.\nThink of Docker swarm as a set of 🤖 hands which will run tasks on each individual node for us. These tasks can consist of:\nCluster Management Deployment and Scaling across the cluster Multi-host networking load balancing etc… Well why not Kubernetes Although Kubernetes is awesome, it adds a bunch of complexity that is simply not needed in a home lab environment where we only have a small cluster. As well as that, from my experience creating docker-compose files is a lot easier than writing manifests. Docker swarm can do everything I want it to and requires less effort which is great if you’re like me 🥱 😂\nRequired Stuff The links below are reccomendations/stuff I used\nRaspberry Pi 4B (Reccomend 4-8 @ 8GB) Raspberry Pi PoE Hats (If you can find the PoE+ version get them) Cluster Case (Get any case you want) PoE switch (Get one that has enough ports for the nuber of pis you want) RJ45 Ethernet Cable (Reccomend short cables as it will look neater) Micro SD cards (Get enough cards for the number of Pis, Get the biggest cards you can afford) This is without a doubt an expensive project, however in my eyes it’s the cost of another server 😂 and the best part about this project is you can add to the cluster over time spreding the cost which is how I managed to purchase all the hardware. A suggestion would be to buy equipment that allows you room to grow so an 8 port PoE switch instead of a 4 port if you know you will scale to 8 nodes\nIf you build this then you can safely say another server is not needed. This cluster should be more than capable to serve all your home needs even if you only have 2 nodes. Put it this way, if you have 4 8GB nodes then you have a combined total of:\n32GB of RAM 16 physical CPU cores @ 1.5GHz Money Saving Tips Just get 1-2 nodes to start cut PoE and use normal power cables Use Raspberry Pis with less memmory eg 4GB Grab a really cheap case or make one using these Use smaller SD Cards try not to go below 32GB (Reccomended 64GB If money saved elsewhere) BUILD TIME !! Pull yourself up by your bootstraps were jumping into this one runnig 💨\nImage the SD Cards So interesting fact the CPUs used in the raspberry pi 4 use an ARM64 archetecture. When looking for images of Raspbian you will only find 32bit variants which will work however will not make use of all the hardware which is just a waste. 64bit images exist however are well hidden, Dont fear 😨 you can download the latest images from the raspberrypi.org index To write the cards I use the Raspberry Pi imager . once you have rpi-imager installed and you have the 64bit raspbian image, open sudo rpi-imager and on the “CHOOSE OS” section go to the bottom and select use custom image.\nWe dont need to unzip the file as the tool is clever enough to handle that for us\nBefore removing the SD card, mount the boot partition - usually the first partition, look for a file called cmdline.txt - and create an empty file called ssh sudo touch ssh this basically enables SSH on the Pis first boot allowing us to access the Pis in a totally headless manner.\nAssemble the cluster So assembly is very dependant on the kit you bought, and more often than not the stuff needed here comes with instructions or is self explanitory. So go ahead and assemble the cluster 🛠\nThere is an order of operations that I reccomend:\nInstall PoE hats onto the Pis Install the SD Cards into the Pis Assemble the case and install Pis plug in ethernet cables **Now allow the Pis to boot for a few minutes so it can resize the\nSort out the Network bits This is the easy part, there are two methods you can use to set a static IP address, one is on your DHCP server and the other is on the device itself. Personally I prefer to set static IPs on the device itself so I will cover that here.\nTo find the current ip of your Pis you can go to your DHCP server and look for the DHCP leases for the Pis.\nthere are naming conventions for naming primary and secondary things but this is your cluster so be creative with it, I have my primary node named Goob and the secondary nodes named Doris1 Doris2 and Doris3 reference SSH to the Pi then open and edit the following files\n/etc/hosts edit this line with your hostname: 127.0.1.1 goob /etc/hostname edit this file with your hostname: goob /etc/dhcpcd.conf knowing we will be using only static IPs here I delete everything in the file add the following interface eth0 static ip_address=10.10.10.20/24 static routers=10.10.10.1 static domain_name_servers=10.10.10.1 Now reboot the pi sudo shutdown -r now\nOnce the Pis are back up log into each one and change the default passwords you may also want to create a new user for yourself but thats totally optional.\nProvision with Ansible you want to create an Ansible control node and that is as simple as installing a package onto your machine. Ansible is built using python which makes it easy to install as we can use the python pip package manger. To check you have it installed do pip --version if it’s not installed do the following\ncurl https://bootstrap.pypa.io/get-pip.py -o get-pip.py\npython get-pip.py --user\nonce you have pip installed proceed to installing Ansible\nDocs pip install ansible\nNow that Ansible is installed we can create a runbok that defines the steps Ansible needs to take on the remote machine to correctly provision it. This playbook is written in yaml so get ready to make sure your indents are correct 😬\nCreating the playbook now you can copy what I have and it should work - after changing a thing or two - which is fine however I encourage you to copy what I have, read the Docs and make any changes you feel are necessary.\nCheck the git repo for the most up to date yaml file im using.\nThis script assumes you are using SSH keys, which you should but if not you can skip the whole “Configure SSH” block, I RECCOMEND YOU USE SSH KEYS - read this Create the deploy.yml\n--- - hosts: piCluster become: yes tasks: - name: Configure SSH block: - name: Auth local SSH Keys ansible.posix.authorized_key: user: state: present key: \"{{ item }}\" with_file: - /home//.ssh/id_rsa.pub - ./pi_rsa.pub - name: Copy SSH Pi Intercoms Private Key copy: src: ./pi_rsa dest: /home//.ssh/id_rsa mode: '0600' owner: group: - name: Copy SSH Pi Intercoms Public Key copy: src: ./pi_rsa.pub dest: /home//.ssh/id_rsa.pub mode: '0644' owner: group: - name: Check for any updates and install them apt: update_cache: true upgrade: dist - name: Install Required build dependencies apt: update_cache: true pkg: - python3 - python3-pip - vim - curl - name: Check Docker is installed shell: \"docker --version\" register: docker_version - name: Install Docker block: - name: Download Docker script command: curl -fsSL https://get.docker.com -o get-docker.sh - name: Run Docker install script command: sudo sh get-docker.sh - name: Enable docker service service: name: \"{{ item }}\" enabled: true state: started loop: - docker - containerd - name: Add user to docker group user: name: groups: docker append: yes when: docker_version.stdout.find('Docker version') - name: check Docker compose is installed shell: \"docker-compose --version\" register: dc_version - name: Install docker compose command: sudo pip3 -v install docker-compose when: dc_version.stdout.find('docker-compose version') - name: wait for network on boot command: raspi-config nonint do_boot_wait 0 - name: Reboot reboot: The steps this playbook takes are as follows:\nCopy your SSH keys to the pi authorized_keys Copy same Pi ssh key between the Pis for intercommunication Check for and Install updates Install any build dependancies Check if Docker is Installed Download the Docker Script Install Docker from script Enable the Docker service Add user to Docker group Check if docker-compose is installed Install docker-compose (This will take a while on each pi so be patient, grab a coffee) Change raspi config to wait for network before boot Reboot take some time to read through the playbook, read the docs and try to understand whats going on. Change the usernames to your username.\nMake the hosts file in the same directory as the deploy.yml\n[piCluster] 10.10.10.20 10.10.10.21 10.10.10.22 [piCluster] - specifies a group of hosts 10.10.10.xx - specifies the Pis IP create the Pi Intercommunication keys\nssh-keygen Enter file in which to save the key (/home/ahmza/.ssh/id_rsa): .pi_rsa save the keys in the same directory the deploy.yml file is heres the command to run the playbook.\nansible-playbook -i hosts -kK deploy.yml -c paramiko\nansible-playbook - the command to run the playbook -i hosts - specifies the host/inventory file -kK - asks you for password for the SSH connection(-k) and the sudo password(-K) deploy.yml - the Ansible playbook -c paramiko - Use the paramiko connection method, I found this works best with ssh keys You can do so much with Ansible, for example mount network shares etc, if you can do it in the cli you can do it in Ansible !!\nNow that the Ansible script is done we can go ahead and actually configure all the Pis into the cluster.\nCreating/Joining the swarm Creating and joining the cluster is easy - It is manual and tedious so this is something you could impliment into the ansible playbook using multiple host groups.\nFollow these steps to create the swarm:\nssh to the main Pi, this will be the manager node. run docker swarm init This will return a command and token. Make sure to copy them exit the main Pi ssh to one of the worker nodes paste and run the copied command do the same on the remaining worker nodes You should now have a correctly configured cluster !!\nOptional Step - Portainer Portainer is a webgui that will allow us to manage the cluster, it will make deploying and destroying containers across the cluster a breeze. I do understand that all of what portainer does can simply be accomplished in the command line, however portainer just makes it less hastle to manage.\nInstall Portainer\nFirstly we want to download the yaml file that will be used to deploy the portainer contiainer on our cluster.\ncurl -L https://downloads.portainer.io/portainer-agent-stack.yml -o portainer-agent-stack.yml\nNow we can actually deploy portainer to the cluster using the following command:\ndocker stack deploy -c portainer-agent-stack.yml portainer\nOnce the stack is up and running navigate to:\nhttps://raspberry-ip:9443\nwhere raspberry-ip is the IP address of the manager node in the swarm.\nOptional Step - Private Registry If you intend to build any of your own docker images then it may be useful to create a private repository to host any images that you need to run across the cluster.\nto create this private repository create a file called docker-compose.yml in a folder called repository or whatever and enter the following.\nversion: '3.9' services: registry: image: registry:2 ports: - 5000:5000 volumes: - /home//docker/registry/data:/var/lib/registry Be sure to change pi-user to your pi user\ncreate a file /etc/docker/daemon.json and eter the following\n{ \"insecure-registries\": [\"localhost:5000\", \":5000\"] } where pi-address is the ip address of the main swarm node.\nrestart the docker service with sudo systemctl restart docker\nnow we need to deploy the registry to the stack like we did with portainer\ndocker stack deploy -c docker-compose.yml registry\nand we should have a private registry setup that we can use to push our custom docker images to.\nConclusion Well done, you have successfully setup a docker swarm cluster on a bunch of Pis and along the way you managed to learn a bit about SSH, Ansible, docker-compose, yaml and networking on Pis. The next step would be to setup a few containers but I’ll leave that to you, dont be affraid to play around and even ruin the cluster, you can always rebuild it so have fun 😊\n","wordCount":"2290","inLanguage":"en","image":"https://blog.ahmza.com/webp/posts/pi-docker-swarm/feature-anim.webp","datePublished":"2021-12-12T17:23:32Z","dateModified":"2021-12-12T17:23:32Z","author":{"@type":"Person","name":"ahmza"},"mainEntityOfPage":{"@type":"WebPage","@id":"https://blog.ahmza.com/posts/building-a-docker-swarm-with-pis/"},"publisher":{"@type":"Organization","name":"ahmza's ramblings","logo":{"@type":"ImageObject","url":"https://blog.ahmza.com/img/favicon.png"}}}</script></head><body class=dark id=top><script>localStorage.getItem("pref-theme")==="light"&&document.body.classList.remove("dark")</script><header class=header><nav class=nav><div class=logo><a href=https://blog.ahmza.com accesskey=h title="ahmza's ramblings (Alt + H)">ahmza's ramblings</a><div class=logo-switches><button id=theme-toggle accesskey=t title="(Alt + T)"><svg id="moon" xmlns="http://www.w3.org/2000/svg" width="24" height="18" viewBox="0 0 24 24" fill="none" stroke="currentcolor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M21 12.79A9 9 0 1111.21 3 7 7 0 0021 12.79z"/></svg><svg id="sun" xmlns="http://www.w3.org/2000/svg" width="24" height="18" viewBox="0 0 24 24" fill="none" stroke="currentcolor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><circle cx="12" cy="12" r="5"/><line x1="12" y1="1" x2="12" y2="3"/><line x1="12" y1="21" x2="12" y2="23"/><line x1="4.22" y1="4.22" x2="5.64" y2="5.64"/><line x1="18.36" y1="18.36" x2="19.78" y2="19.78"/><line x1="1" y1="12" x2="3" y2="12"/><line x1="21" y1="12" x2="23" y2="12"/><line x1="4.22" y1="19.78" x2="5.64" y2="18.36"/><line x1="18.36" y1="5.64" x2="19.78" y2="4.22"/></svg></button></div></div><ul id=menu><li><a href=https://blog.ahmza.com/posts/ title=posts><span>posts</span></a></li><li><a href=https://blog.ahmza.com/archives/ title=archive><span>archive</span></a></li><li><a href=https://blog.ahmza.com/tags/ title=tags><span>tags</span></a></li><li><a href=https://blog.ahmza.com/contact title=contact><span>contact</span></a></li><li><a href=https://blog.ahmza.com/search/ title="search (Alt + /)" accesskey=/><span>search</span></a></li></ul></nav></header><main class=main><article class=post-single><header class=post-header><div class=breadcrumbs><a href=https://blog.ahmza.com>Home</a> » <a href=https://blog.ahmza.com/posts/>Posts</a></div><h1 class=post-title>Building a Docker Swarm With Pi's 👾 👾 👾 👾</h1><div class=post-description>Pis are very capable computers, when banded together they're almost unstoppable</div><div class=post-meta><span title='2021-12-12 17:23:32 +0000 UTC'>December 12, 2021</span>&nbsp;·&nbsp;11 min&nbsp;·&nbsp;2290 words&nbsp;·&nbsp;ahmza | <a href=https://gitlab.com/AHMZA/blog/-/blob/main/content/posts/building-a-docker-swarm-with-pis.md rel="noopener noreferrer" target=_blank>Suggest Changes</a></div></header><figure class=entry-cover><img loading=lazy src=https://blog.ahmza.com/webp/posts/pi-docker-swarm/feature-anim.webp alt="Pi Docker Swarm"><p></p></figure><div class=toc><details><summary accesskey=c title="(Alt + C)"><span class=details>Table of Contents</span></summary><div class=inner><ul><li><a href=#intro aria-label=Intro><strong>Intro</strong></a><ul><li><a href=#my-setup aria-label="My Setup"><strong>My Setup</strong></a></li><li><a href=#point-of-this-post aria-label="Point of this post"><strong>Point of this post</strong></a></li><li><a href=#why-docker-swarm- aria-label="Why Docker Swarm 🧐"><strong>Why</strong> <em><strong>Docker Swarm</strong></em> 🧐</a></li><li><a href=#well-why-not-kubernetes aria-label="Well why not Kubernetes"><strong>Well why not</strong> <em><strong>Kubernetes</strong></em></a></li><li><a href=#required-stuff aria-label="Required Stuff"><strong>Required Stuff</strong></a><ul><li><a href=#money-saving-tips aria-label="Money Saving Tips"><strong>Money Saving Tips</strong></a></li></ul></li></ul></li><li><a href=#build-time- aria-label="BUILD TIME !!"><strong>BUILD TIME !!</strong></a><ul><li><a href=#image-the-sd-cards aria-label="Image the SD Cards"><strong>Image the SD Cards</strong></a></li><li><a href=#assemble-the-cluster aria-label="Assemble the cluster">Assemble the cluster</a></li><li><a href=#sort-out-the-network-bits aria-label="Sort out the Network bits"><strong>Sort out the Network bits</strong></a></li><li><a href=#provision-with-ansible aria-label="Provision with Ansible"><strong>Provision with</strong> <em><strong>Ansible</strong></em></a></li><li><a href=#creatingjoining-the-swarm aria-label="Creating/Joining the swarm"><strong>Creating/Joining the swarm</strong></a></li><li><a href=#optional-step---portainer aria-label="Optional Step - Portainer"><strong>Optional Step - Portainer</strong></a></li><li><a href=#optional-step---private-registry aria-label="Optional Step - Private Registry"><strong>Optional Step - Private Registry</strong></a></li></ul></li><li><a href=#conclusion aria-label=Conclusion><strong>Conclusion</strong></a></li></ul></div></details></div><div class=post-content><h2 id=intro><strong>Intro</strong><a hidden class=anchor aria-hidden=true href=#intro>#</a></h2><p>The <em>Raspberry Pi</em> from it’s first iteration has been a great hit.
|
||
Adopted by many as the computer of choice for projects, and for
|
||
very good reason. The Pi is a fully fledged computer that fits in
|
||
your pocket and the most recent edition has blown fans away with
|
||
the flagship model boasting an ARM64 CPU and 8GB of RAM.</p><p>Now one <em>Raspberry Pi 4 8GB</em> is more than powerful enough for most
|
||
workloads and in some cases totally unnecessary, <strong>so you may even
|
||
get away with a single 4GB model</strong>. However taking into consideration
|
||
their relatively low price and power consumption why not go all out
|
||
and build a cluster.</p><blockquote><p><strong>You can never have too many <em>Raspberry Pis</em>, just
|
||
not enough Ideas</strong></p></blockquote><p>My excuse to build a Pi cluster is, I would like to migrate the
|
||
services currently running on a <strong>bare metal server that sucks
|
||
a lot of power to a low power solution.</strong> What better way to do that
|
||
than a Pi cluster.</p><h3 id=my-setup><strong>My Setup</strong><a hidden class=anchor aria-hidden=true href=#my-setup>#</a></h3><ul><li>4 x <em>Raspberry Pi 4 8GB</em></li><li>4 x 32GB SD cards</li><li>4 x PoE hats</li><li>1 x 5 port PoE Switch</li></ul><p><img loading=lazy src=/img/pi-docker-swarm/cluster.webp alt=Cluster></p><h3 id=point-of-this-post><strong>Point of this post</strong><a hidden class=anchor aria-hidden=true href=#point-of-this-post>#</a></h3><p>Hopefully after following this post you will have setup <em>Docker
|
||
swarm</em> on a cluster of Pis <strong>- stay even if you currently only have
|
||
one -</strong> and understand how to bend it to your will 🦾.</p><h3 id=why-docker-swarm-><strong>Why</strong> <em><strong>Docker Swarm</strong></em> 🧐<a hidden class=anchor aria-hidden=true href=#why-docker-swarm->#</a></h3><p>While we can manage each docker node - <em>Raspberry Pi</em> - individually
|
||
, <strong><em>Docker swarm</em> enables us to manage all the devices from a single
|
||
main node</strong> making managing the cluster a lot easier.</p><p>Think of <em>Docker swarm</em> as a set of 🤖 hands which will run tasks on
|
||
each individual node for us. These tasks can consist of:</p><ul><li>Cluster Management</li><li>Deployment and Scaling across the cluster</li><li>Multi-host networking</li><li>load balancing</li><li>etc…</li></ul><h3 id=well-why-not-kubernetes><strong>Well why not</strong> <em><strong>Kubernetes</strong></em><a hidden class=anchor aria-hidden=true href=#well-why-not-kubernetes>#</a></h3><p>Although <em>Kubernetes</em> is awesome, <strong>it adds a bunch of complexity that
|
||
is simply not needed in a home lab environment</strong> where we only have
|
||
a small cluster. As well as that, from my experience creating
|
||
<code>docker-compose</code> files is a lot easier than writing manifests.
|
||
<strong><em>Docker swarm</em> can do everything I want it to and requires less
|
||
effort</strong> which is great if you’re like me 🥱 😂</p><h3 id=required-stuff><strong>Required Stuff</strong><a hidden class=anchor aria-hidden=true href=#required-stuff>#</a></h3><blockquote><p>The links below are reccomendations/stuff I used</p></blockquote><ul><li><a href="https://thepihut.com/products/raspberry-pi-4-model-b?variant=31994565689406" target=_blank><em>Raspberry Pi</em> 4B</a>
|
||
(Reccomend 4-8 @ 8GB)</li><li><a href=https://thepihut.com/products/power-over-ethernet-poe-hat-d-for-raspberry-pi target=_blank><em>Raspberry Pi</em> PoE Hats</a>
|
||
(If you can find the <a href=https://thepihut.com/products/raspberry-pi-poe-plus-hat target=_blank>PoE+</a>
|
||
version get them)</li><li><a href=https://thepihut.com/products/cluster-case-for-raspberry-pi target=_blank>Cluster Case</a>
|
||
(Get any case you want)</li><li><a href=https://www.amazon.co.uk/gp/product/B076982FVC/ target=_blank>PoE switch</a>
|
||
(Get one that has enough ports for the nuber of pis you want)</li><li><a href=https://www.amazon.co.uk/gp/product/B00C4U030G target=_blank>RJ45 Ethernet Cable</a>
|
||
(Reccomend short cables as it will look neater)</li><li><a href=https://www.amazon.co.uk/gp/product/B08J4HJ98L target=_blank>Micro SD cards</a>
|
||
(Get enough cards for the number of Pis, Get the biggest cards you can afford)</li></ul><p><strong>This is without a doubt an expensive project</strong>, however in my eyes
|
||
it’s the cost of another server 😂 and the best part about this
|
||
project is <strong>you can add to the cluster over time</strong> spreding the cost
|
||
which is how I managed to purchase all the hardware. A suggestion
|
||
would be to <strong>buy equipment that allows you room to grow</strong> so an 8 port
|
||
PoE switch instead of a 4 port if you know you will scale to 8
|
||
nodes</p><p>If you build this then you can safely say another server is not
|
||
needed. This cluster should be more than capable to serve all your
|
||
home needs even if you only have 2 nodes. Put it this way, if you
|
||
have 4 8GB nodes then you have a combined total of:</p><ul><li>32GB of RAM</li><li>16 physical CPU cores @ 1.5GHz</li></ul><h4 id=money-saving-tips><strong>Money Saving Tips</strong><a hidden class=anchor aria-hidden=true href=#money-saving-tips>#</a></h4><ul><li>Just get 1-2 nodes to start</li><li>cut PoE and use normal power cables</li><li>Use Raspberry Pis with less memmory eg 4GB</li><li>Grab a really cheap case or make one using <a href=https://www.amazon.co.uk/dp/B091KJNZLB/ target=_blank>these</a></li><li>Use smaller SD Cards try not to go below 32GB (Reccomended 64GB If money saved elsewhere)</li></ul><h2 id=build-time-><strong>BUILD TIME !!</strong><a hidden class=anchor aria-hidden=true href=#build-time->#</a></h2><p>Pull yourself up by your bootstraps were jumping into this one
|
||
runnig 💨</p><h3 id=image-the-sd-cards><strong>Image the SD Cards</strong><a hidden class=anchor aria-hidden=true href=#image-the-sd-cards>#</a></h3><p>So interesting fact the CPUs used in the raspberry pi 4 use an ARM64
|
||
archetecture. When looking for images of Raspbian you will only find
|
||
32bit variants which will work however will not make use of all the
|
||
hardware which is just a waste. 64bit images exist however are well
|
||
hidden, Dont fear 😨 <strong>you can download the latest images from the
|
||
<a href=https://downloads.raspberrypi.org/raspios_arm64/images/ target=_blank>raspberrypi.org index</a></strong></p><p>To write the cards I use the <a href=https://www.raspberrypi.com/software/ target=_blank><strong>Raspberry Pi imager</strong></a>
|
||
.
|
||
once you have <strong>rpi-imager</strong> installed and you have the 64bit raspbian
|
||
image, open <code>sudo rpi-imager</code> and on the “CHOOSE OS” section go to
|
||
the bottom and select use custom image.</p><blockquote><p>We dont need to unzip the file as the tool is clever enough to
|
||
handle that for us</p></blockquote><p><strong>Before removing the SD card, mount the boot partition - usually the
|
||
first partition, look for a file called cmdline.txt - and create an
|
||
empty file called ssh</strong> <code>sudo touch ssh</code> this basically enables
|
||
SSH on the Pis first boot allowing us to access the Pis in a totally
|
||
headless manner.</p><h3 id=assemble-the-cluster>Assemble the cluster<a hidden class=anchor aria-hidden=true href=#assemble-the-cluster>#</a></h3><p>So assembly is very dependant on the kit you bought, and more
|
||
often than not the stuff needed here comes with instructions or is
|
||
self explanitory. So go ahead and assemble the cluster 🛠</p><p><strong>There is an order of operations that I reccomend:</strong></p><ul><li>Install PoE hats onto the Pis</li><li>Install the SD Cards into the Pis</li><li>Assemble the case and install Pis</li><li>plug in ethernet cables</li></ul><p>**Now allow the Pis to boot for a few minutes so it can resize the</p><h3 id=sort-out-the-network-bits><strong>Sort out the Network bits</strong><a hidden class=anchor aria-hidden=true href=#sort-out-the-network-bits>#</a></h3><p>This is the easy part, there are two methods you can use to set a
|
||
static IP address, one is on your DHCP server and the other is on
|
||
the device itself. Personally I prefer to set static IPs on the
|
||
device itself so I will cover that here.</p><p>To find the current ip of your Pis you can go to your DHCP server
|
||
and look for the DHCP leases for the Pis.</p><blockquote><p>there are naming conventions for naming primary and secondary
|
||
things but this is your cluster so be creative with it, I have my
|
||
primary node named <code>Goob</code> and the secondary nodes named <code>Doris1</code>
|
||
<code>Doris2</code> and <code>Doris3</code>
|
||
<a href="https://www.youtube.com/watch?v=oFb7FVe8sdQ" target=_blank>reference</a></p></blockquote><p><strong>SSH to the Pi</strong> then open and edit the following files</p><ul><li>/etc/hosts<ul><li>edit this line with your hostname: <code>127.0.1.1 goob</code></li></ul></li><li>/etc/hostname<ul><li>edit this file with your hostname: <code>goob</code></li></ul></li><li>/etc/dhcpcd.conf<ul><li>knowing we will be using only static IPs here I delete
|
||
everything in the file add the following</li><li><div class=highlight><pre tabindex=0 class=chroma><code class=language-fallback data-lang=fallback><span class=line><span class=cl>interface eth0
|
||
</span></span><span class=line><span class=cl>static ip_address=10.10.10.20/24
|
||
</span></span><span class=line><span class=cl>static routers=10.10.10.1
|
||
</span></span><span class=line><span class=cl>static domain_name_servers=10.10.10.1
|
||
</span></span></code></pre></div></li></ul></li></ul><p><strong>Now reboot the pi</strong> <code>sudo shutdown -r now</code></p><p><strong>Once the Pis are back up log into each one and change the default
|
||
passwords</strong> you may also want to create a new user for yourself but
|
||
thats totally optional.</p><h3 id=provision-with-ansible><strong>Provision with</strong> <em><strong>Ansible</strong></em><a hidden class=anchor aria-hidden=true href=#provision-with-ansible>#</a></h3><p>you want to <strong>create an <em>Ansible</em> control node</strong> and that is as simple as
|
||
installing a package onto your machine. <em>Ansible</em> is built using python
|
||
which makes it easy to install as <strong>we can use the python pip package
|
||
manger.</strong> To check you have it installed do <code>pip --version</code> if
|
||
it’s not installed do the following</p><p><code>curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py</code></p><p><code>python get-pip.py --user</code></p><p><strong>once you have pip installed proceed to installing <em>Ansible</em></strong></p><p><a href=https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html target=_blank>Docs</a></p><p><code>pip install ansible</code></p><p>Now that <em>Ansible</em> is installed we can create a runbok that defines the
|
||
steps <em>Ansible</em> needs to take on the remote machine to correctly
|
||
provision it. This playbook is written in yaml so get ready to make
|
||
sure your indents are correct 😬</p><p><strong>Creating the playbook</strong>
|
||
now you can copy what I have and it should work - after changing a
|
||
thing or two - which is fine however I encourage you to copy what I
|
||
have, read the <a href=https://docs.ansible.com/ansible/latest/user_guide/index.html target=_blank>Docs</a>
|
||
and make any changes you feel are necessary.</p><blockquote><p>Check the <a href=https://gitlab.com/AHMZA/pi-docker-swarm-ansible-deploy/-/tree/main target=_blank>git repo</a>
|
||
for the most up to date yaml file im using.</p></blockquote><p><strong>This script assumes you are using SSH keys, which you should but if
|
||
not you can skip the whole “Configure SSH” block,</strong>
|
||
<em><strong>I RECCOMEND YOU USE SSH KEYS</strong></em> - <a href=https://blog.ahmza.com/posts/securing-ssh/ target=_blank>read this</a></p><p><strong>Create the deploy.yml</strong></p><div class=highlight><pre tabindex=0 class=chroma><code class=language-yaml data-lang=yaml><span class=line><span class=cl><span class=nn>---</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w></span>- <span class=nt>hosts</span><span class=p>:</span><span class=w> </span><span class=l>piCluster</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>become</span><span class=p>:</span><span class=w> </span><span class=kc>yes</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>tasks</span><span class=p>:</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span>- <span class=nt>name</span><span class=p>:</span><span class=w> </span><span class=l>Configure SSH</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>block</span><span class=p>:</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span>- <span class=nt>name</span><span class=p>:</span><span class=w> </span><span class=l>Auth local SSH Keys</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>ansible.posix.authorized_key</span><span class=p>:</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>user</span><span class=p>:</span><span class=w> </span><span class=l><pi_user></span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>state</span><span class=p>:</span><span class=w> </span><span class=l>present</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>key</span><span class=p>:</span><span class=w> </span><span class=s2>"{{ item }}"</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>with_file</span><span class=p>:</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span>- <span class=l>/home/<local_user>/.ssh/id_rsa.pub</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span>- <span class=l>./pi_rsa.pub</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span>- <span class=nt>name</span><span class=p>:</span><span class=w> </span><span class=l>Copy SSH Pi Intercoms Private Key</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>copy</span><span class=p>:</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>src</span><span class=p>:</span><span class=w> </span><span class=l>./pi_rsa</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>dest</span><span class=p>:</span><span class=w> </span><span class=l>/home/<pi_user>/.ssh/id_rsa</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>mode</span><span class=p>:</span><span class=w> </span><span class=s1>'0600'</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>owner</span><span class=p>:</span><span class=w> </span><span class=l><pi_user></span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>group</span><span class=p>:</span><span class=w> </span><span class=l><pi_user></span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span>- <span class=nt>name</span><span class=p>:</span><span class=w> </span><span class=l>Copy SSH Pi Intercoms Public Key</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>copy</span><span class=p>:</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>src</span><span class=p>:</span><span class=w> </span><span class=l>./pi_rsa.pub</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>dest</span><span class=p>:</span><span class=w> </span><span class=l>/home/<pi_user>/.ssh/id_rsa.pub</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>mode</span><span class=p>:</span><span class=w> </span><span class=s1>'0644'</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>owner</span><span class=p>:</span><span class=w> </span><span class=l><pi_user></span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>group</span><span class=p>:</span><span class=w> </span><span class=l><pi_user></span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span>- <span class=nt>name</span><span class=p>:</span><span class=w> </span><span class=l>Check for any updates and install them</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>apt</span><span class=p>:</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>update_cache</span><span class=p>:</span><span class=w> </span><span class=kc>true</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>upgrade</span><span class=p>:</span><span class=w> </span><span class=l>dist</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span>- <span class=nt>name</span><span class=p>:</span><span class=w> </span><span class=l>Install Required build dependencies</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>apt</span><span class=p>:</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>update_cache</span><span class=p>:</span><span class=w> </span><span class=kc>true</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>pkg</span><span class=p>:</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span>- <span class=l>python3</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span>- <span class=l>python3-pip</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span>- <span class=l>vim</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span>- <span class=l>curl</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span>- <span class=nt>name</span><span class=p>:</span><span class=w> </span><span class=l>Check Docker is installed</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>shell</span><span class=p>:</span><span class=w> </span><span class=s2>"docker --version"</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>register</span><span class=p>:</span><span class=w> </span><span class=l>docker_version</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span>- <span class=nt>name</span><span class=p>:</span><span class=w> </span><span class=l>Install Docker</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>block</span><span class=p>:</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span>- <span class=nt>name</span><span class=p>:</span><span class=w> </span><span class=l>Download Docker script</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>command</span><span class=p>:</span><span class=w> </span><span class=l>curl -fsSL https://get.docker.com -o get-docker.sh</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span>- <span class=nt>name</span><span class=p>:</span><span class=w> </span><span class=l>Run Docker install script</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>command</span><span class=p>:</span><span class=w> </span><span class=l>sudo sh get-docker.sh</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span>- <span class=nt>name</span><span class=p>:</span><span class=w> </span><span class=l>Enable docker service</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>service</span><span class=p>:</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>name</span><span class=p>:</span><span class=w> </span><span class=s2>"{{ item }}"</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>enabled</span><span class=p>:</span><span class=w> </span><span class=kc>true</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>state</span><span class=p>:</span><span class=w> </span><span class=l>started</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>loop</span><span class=p>:</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span>- <span class=l>docker</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span>- <span class=l>containerd</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span>- <span class=nt>name</span><span class=p>:</span><span class=w> </span><span class=l>Add user to docker group</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>user</span><span class=p>:</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>name</span><span class=p>:</span><span class=w> </span><span class=l><pi_user></span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>groups</span><span class=p>:</span><span class=w> </span><span class=l>docker</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>append</span><span class=p>:</span><span class=w> </span><span class=kc>yes</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>when</span><span class=p>:</span><span class=w> </span><span class=l>docker_version.stdout.find('Docker version')</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span>- <span class=nt>name</span><span class=p>:</span><span class=w> </span><span class=l>check Docker compose is installed</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>shell</span><span class=p>:</span><span class=w> </span><span class=s2>"docker-compose --version"</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>register</span><span class=p>:</span><span class=w> </span><span class=l>dc_version</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span>- <span class=nt>name</span><span class=p>:</span><span class=w> </span><span class=l>Install docker compose</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>command</span><span class=p>:</span><span class=w> </span><span class=l>sudo pip3 -v install docker-compose</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>when</span><span class=p>:</span><span class=w> </span><span class=l>dc_version.stdout.find('docker-compose version')</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span>- <span class=nt>name</span><span class=p>:</span><span class=w> </span><span class=l>wait for network on boot</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>command</span><span class=p>:</span><span class=w> </span><span class=l>raspi-config nonint do_boot_wait 0</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span>- <span class=nt>name</span><span class=p>:</span><span class=w> </span><span class=l>Reboot</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>reboot</span><span class=p>:</span><span class=w>
|
||
</span></span></span></code></pre></div><p>The steps this playbook takes are as follows:</p><ol><li>Copy your SSH keys to the pi authorized_keys</li><li>Copy same Pi ssh key between the Pis for intercommunication</li><li>Check for and Install updates</li><li>Install any build dependancies</li><li>Check if Docker is Installed<ul><li>Download the Docker Script</li><li>Install Docker from script</li><li>Enable the Docker service</li><li>Add user to Docker group</li></ul></li><li>Check if docker-compose is installed<ul><li>Install docker-compose (This will take a while on each pi so be patient, grab a coffee)</li></ul></li><li>Change raspi config to wait for network before boot</li><li>Reboot</li></ol><p>take some time to read through the playbook, read the docs and try to
|
||
understand whats going on. Change the usernames to your username.</p><p><strong>Make the hosts file in the same directory as the deploy.yml</strong></p><div class=highlight><pre tabindex=0 class=chroma><code class=language-yaml data-lang=yaml><span class=line><span class=cl><span class=p>[</span><span class=l>piCluster]</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w></span><span class=m>10.10.10.20</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w></span><span class=m>10.10.10.21</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w></span><span class=m>10.10.10.22</span><span class=w>
|
||
</span></span></span></code></pre></div><ul><li><code>[piCluster]</code> - specifies a group of hosts</li><li><code>10.10.10.xx</code> - specifies the Pis IP</li></ul><p><strong>create the Pi Intercommunication keys</strong></p><div class=highlight><pre tabindex=0 class=chroma><code class=language-bash data-lang=bash><span class=line><span class=cl>ssh-keygen
|
||
</span></span><span class=line><span class=cl>Enter file in which to save the key <span class=o>(</span>/home/ahmza/.ssh/id_rsa<span class=o>)</span>: .pi_rsa
|
||
</span></span></code></pre></div><ul><li>save the keys in the same directory the deploy.yml file is</li></ul><p><strong>heres the command to run the playbook.</strong></p><p><code>ansible-playbook -i hosts -kK deploy.yml -c paramiko</code></p><ul><li><code>ansible-playbook</code> - the command to run the playbook</li><li><code>-i hosts</code> - specifies the host/inventory file</li><li><code>-kK</code> - asks you for password for the SSH connection(-k) and the sudo password(-K)</li><li><code>deploy.yml</code> - the <em>Ansible</em> playbook</li><li><code>-c paramiko</code> - Use the paramiko connection method, I found this works best with ssh keys</li></ul><p>You can do so much with <em>Ansible</em>, for example mount network shares etc,
|
||
if you can do it in the cli you can do it in <em>Ansible</em> !!</p><p>Now that the <em>Ansible</em> script is done we can go ahead and actually
|
||
configure all the Pis into the cluster.</p><h3 id=creatingjoining-the-swarm><strong>Creating/Joining the swarm</strong><a hidden class=anchor aria-hidden=true href=#creatingjoining-the-swarm>#</a></h3><p>Creating and joining the cluster is easy - It is manual and tedious so
|
||
this is something you could impliment into the ansible playbook using
|
||
multiple host groups.</p><p><strong>Follow these steps to create the swarm:</strong></p><ol><li>ssh to the main Pi, this will be the manager node.</li><li>run <code>docker swarm init</code><ul><li>This will return a command and token. Make sure to copy them</li></ul></li><li>exit the main Pi</li><li>ssh to one of the worker nodes</li><li>paste and run the copied command</li><li>do the same on the remaining worker nodes</li></ol><p>You should now have a correctly configured cluster !!</p><h3 id=optional-step---portainer><strong>Optional Step - Portainer</strong><a hidden class=anchor aria-hidden=true href=#optional-step---portainer>#</a></h3><p>Portainer is a webgui that will allow us to manage the cluster, it will
|
||
make deploying and destroying containers across the cluster a breeze.
|
||
I do understand that all of what portainer does can simply be accomplished
|
||
in the command line, however portainer just makes it less hastle to manage.</p><p><strong>Install Portainer</strong></p><p>Firstly we want to download the yaml file that will be used to deploy
|
||
the portainer contiainer on our cluster.</p><p><code>curl -L https://downloads.portainer.io/portainer-agent-stack.yml -o portainer-agent-stack.yml</code></p><p>Now we can actually deploy portainer to the cluster using the following
|
||
command:</p><p><code>docker stack deploy -c portainer-agent-stack.yml portainer</code></p><p>Once the stack is up and running navigate to:</p><p><code>https://raspberry-ip:9443</code></p><p>where raspberry-ip is the IP address of the manager node in the swarm.</p><h3 id=optional-step---private-registry><strong>Optional Step - Private Registry</strong><a hidden class=anchor aria-hidden=true href=#optional-step---private-registry>#</a></h3><p>If you intend to build any of your own docker images then it may be useful
|
||
to create a private repository to host any images that you need to run
|
||
across the cluster.</p><p>to create this private repository create a file called docker-compose.yml
|
||
in a folder called repository or whatever and enter the following.</p><div class=highlight><pre tabindex=0 class=chroma><code class=language-yaml data-lang=yaml><span class=line><span class=cl><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w></span><span class=nt>version</span><span class=p>:</span><span class=w> </span><span class=s1>'3.9'</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w></span><span class=nt>services</span><span class=p>:</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>registry</span><span class=p>:</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>image</span><span class=p>:</span><span class=w> </span><span class=l>registry:2</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>ports</span><span class=p>:</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span>- <span class=m>5000</span><span class=p>:</span><span class=m>5000</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span><span class=nt>volumes</span><span class=p>:</span><span class=w>
|
||
</span></span></span><span class=line><span class=cl><span class=w> </span>- <span class=l>/home/<pi-user>/docker/registry/data:/var/lib/registry</span><span class=w>
|
||
</span></span></span></code></pre></div><p><strong>Be sure to change pi-user to your pi user</strong></p><p><strong>create a file /etc/docker/daemon.json and eter the following</strong></p><div class=highlight><pre tabindex=0 class=chroma><code class=language-json data-lang=json><span class=line><span class=cl><span class=p>{</span>
|
||
</span></span><span class=line><span class=cl> <span class=nt>"insecure-registries"</span><span class=p>:</span> <span class=p>[</span><span class=s2>"localhost:5000"</span><span class=p>,</span> <span class=s2>"<pi-address>:5000"</span><span class=p>]</span>
|
||
</span></span><span class=line><span class=cl><span class=p>}</span>
|
||
</span></span></code></pre></div><p>where <code>pi-address</code> is the ip address of the main swarm node.</p><p><strong>restart the docker service with <code>sudo systemctl restart docker</code></strong></p><p>now we need to deploy the registry to the stack like we did with portainer</p><p><code>docker stack deploy -c docker-compose.yml registry</code></p><p>and we should have a private registry setup that we can use to push our
|
||
custom docker images to.</p><h2 id=conclusion><strong>Conclusion</strong><a hidden class=anchor aria-hidden=true href=#conclusion>#</a></h2><p>Well done, you have successfully setup a docker swarm cluster on a bunch
|
||
of Pis and along the way you managed to learn a bit about SSH, Ansible,
|
||
docker-compose, yaml and networking on Pis. The next step would be to
|
||
setup a few containers but I’ll leave that to you, dont be affraid to
|
||
play around and even ruin the cluster, you can always rebuild it so have
|
||
fun 😊</p></div><footer class=post-footer><ul class=post-tags><li><a href=https://blog.ahmza.com/tags/homelab/>Homelab</a></li><li><a href=https://blog.ahmza.com/tags/docker/>Docker</a></li><li><a href=https://blog.ahmza.com/tags/docker-swarm/>Docker Swarm</a></li><li><a href=https://blog.ahmza.com/tags/raspberry-pi/>Raspberry Pi</a></li><li><a href=https://blog.ahmza.com/tags/pi4/>Pi4</a></li></ul></footer></article></main><footer class=footer><span>© 2023 <a href=https://blog.ahmza.com>ahmza's ramblings</a></span>
|
||
<span>Powered by
|
||
<a href=https://gohugo.io/ rel="noopener noreferrer" target=_blank>Hugo</a> &
|
||
<a href=https://github.com/adityatelange/hugo-PaperMod/ rel=noopener target=_blank>PaperMod</a></span></footer><a href=#top aria-label="go to top" title="Go to Top (Alt + G)" class=top-link id=top-link accesskey=g><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 12 6" fill="currentcolor"><path d="M12 6H0l6-6z"/></svg></a><script src=https://storage.ko-fi.com/cdn/scripts/overlay-widget.js></script><script>kofiWidgetOverlay.draw("ahmza",{type:"floating-chat","floating-chat.donateButton.text":"Tip","floating-chat.donateButton.background-color":"rgb(145, 70, 255)","floating-chat.donateButton.text-color":"#fff"})</script><script>let menu=document.getElementById("menu");menu&&(menu.scrollLeft=localStorage.getItem("menu-scroll-position"),menu.onscroll=function(){localStorage.setItem("menu-scroll-position",menu.scrollLeft)}),document.querySelectorAll('a[href^="#"]').forEach(e=>{e.addEventListener("click",function(e){e.preventDefault();var t=this.getAttribute("href").substr(1);window.matchMedia("(prefers-reduced-motion: reduce)").matches?document.querySelector(`[id='${decodeURIComponent(t)}']`).scrollIntoView():document.querySelector(`[id='${decodeURIComponent(t)}']`).scrollIntoView({behavior:"smooth"}),t==="top"?history.replaceState(null,null," "):history.pushState(null,null,`#${t}`)})})</script><script>var mybutton=document.getElementById("top-link");window.onscroll=function(){document.body.scrollTop>800||document.documentElement.scrollTop>800?(mybutton.style.visibility="visible",mybutton.style.opacity="1"):(mybutton.style.visibility="hidden",mybutton.style.opacity="0")}</script><script>document.getElementById("theme-toggle").addEventListener("click",()=>{document.body.className.includes("dark")?(document.body.classList.remove("dark"),localStorage.setItem("pref-theme","light")):(document.body.classList.add("dark"),localStorage.setItem("pref-theme","dark"))})</script><script>document.querySelectorAll("pre > code").forEach(e=>{const n=e.parentNode.parentNode,t=document.createElement("button");t.classList.add("copy-code"),t.innerHTML="copy";function s(){t.innerHTML="copied!",setTimeout(()=>{t.innerHTML="copy"},2e3)}t.addEventListener("click",t=>{if("clipboard"in navigator){navigator.clipboard.writeText(e.textContent),s();return}const n=document.createRange();n.selectNodeContents(e);const o=window.getSelection();o.removeAllRanges(),o.addRange(n);try{document.execCommand("copy"),s()}catch{}o.removeRange(n)}),n.classList.contains("highlight")?n.appendChild(t):n.parentNode.firstChild==n||(e.parentNode.parentNode.parentNode.parentNode.parentNode.nodeName=="TABLE"?e.parentNode.parentNode.parentNode.parentNode.parentNode.appendChild(t):e.parentNode.appendChild(t))})</script></body></html> |