Merge branch 'pi-docker-swarm' into 'main'
Pi docker swarm See merge request AHMZA/blog!5
This commit is contained in:
Hamza Ahmed
@@ -25,6 +25,7 @@ Build:
|
||||
only:
|
||||
- main
|
||||
|
||||
|
||||
Dockerize:
|
||||
stage: dockerize
|
||||
image:
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
title: Building a Docker Swarm With Pi's
|
||||
date: 2021-12-12T17:23:32Z
|
||||
author: "ahmza"
|
||||
draft: true
|
||||
draft: false
|
||||
weight:
|
||||
|
||||
tags: ["Homelab","Docker","Docker Swarm","Raspberry Pi","Pi4"]
|
||||
@@ -145,7 +145,7 @@ self explanitory. So go ahead and assemble the cluster 🛠
|
||||
file system**
|
||||
|
||||
### **Sort out the Network bits**
|
||||
This is the easy part, there are two methods we can use to set a
|
||||
This is the easy part, there are two methods you can use to set a
|
||||
static IP address, one is on your DHCP server and the other is on
|
||||
the device itself. Personally I prefer to set static IPs on the
|
||||
device itself so I will cover that here.
|
||||
@@ -153,11 +153,17 @@ device itself so I will cover that here.
|
||||
To find the current ip of your Pis you can go to your DHCP server
|
||||
and look for the DHCP leases for the Pis.
|
||||
|
||||
> there are naming conventions for naming primary and secondary
|
||||
things but this is your cluster so be creative with it, I have my
|
||||
primary node named ```Goob``` and the secondary nodes named ```Doris1```
|
||||
```Doris2``` and ```Doris3```
|
||||
[reference](https://www.youtube.com/watch?v=oFb7FVe8sdQ)
|
||||
|
||||
**SSH to the Pi** then open and edit the following files
|
||||
* /etc/hosts
|
||||
* edit this line with your hostname: ```127.0.1.1 mainpi```
|
||||
* edit this line with your hostname: ```127.0.1.1 goob```
|
||||
* /etc/hostname
|
||||
* edit this file with your hostname: ```mainpi```
|
||||
* edit this file with your hostname: ```goob```
|
||||
* /etc/dhcpcd.conf
|
||||
* knowing we will be using only static IPs here I delete
|
||||
everything in the file add the following
|
||||
@@ -169,4 +175,271 @@ and look for the DHCP leases for the Pis.
|
||||
```
|
||||
**Now reboot the pi** ```sudo shutdown -r now```
|
||||
|
||||
**Once the Pis are back up log into each one and change the default
|
||||
passwords** you may also want to create a new user for yourself but
|
||||
thats totally optional.
|
||||
|
||||
|
||||
### **Provision with** ***Ansible***
|
||||
you want to **create an _Ansible_ control node** and that is as simple as
|
||||
installing a package onto your machine. _Ansible_ is built using python
|
||||
which makes it easy to install as **we can use the python pip package
|
||||
manger.** To check you have it installed do ```pip --version``` if
|
||||
it's not installed do the following
|
||||
|
||||
|
||||
```curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py```
|
||||
|
||||
```python get-pip.py --user```
|
||||
|
||||
**once you have pip installed proceed to installing _Ansible_**
|
||||
|
||||
[Docs](https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html)
|
||||
|
||||
```pip install ansible```
|
||||
|
||||
Now that _Ansible_ is installed we can create a runbok that defines the
|
||||
steps _Ansible_ needs to take on the remote machine to correctly
|
||||
provision it. This playbook is written in yaml so get ready to make
|
||||
sure your indents are correct 😬
|
||||
|
||||
**Creating the playbook**
|
||||
now you can copy what I have and it should work - after changing a
|
||||
thing or two - which is fine however I encourage you to copy what I
|
||||
have, read the [Docs](https://docs.ansible.com/ansible/latest/user_guide/index.html)
|
||||
and make any changes you feel are necessary.
|
||||
|
||||
> Check the [git repo](https://gitlab.com/AHMZA/pi-docker-swarm-ansible-deploy/-/tree/main) for the most up to date yaml file im using.
|
||||
|
||||
**This script assumes you are using SSH keys, which you should but if
|
||||
not you can skip the whole "Configure SSH" block,**
|
||||
***I RECCOMEND YOU USE SSH KEYS*** - [read this](https://blog.ahmza.com/posts/securing-ssh/)
|
||||
|
||||
**Create the deploy.yml**
|
||||
|
||||
```yaml
|
||||
---
|
||||
- hosts: piCluster
|
||||
become: yes
|
||||
tasks:
|
||||
|
||||
- name: Configure SSH
|
||||
block:
|
||||
|
||||
- name: Auth local SSH Keys
|
||||
ansible.posix.authorized_key:
|
||||
user: <pi_user>
|
||||
state: present
|
||||
key: "{{ item }}"
|
||||
with_file:
|
||||
- /home/<local_user>/.ssh/id_rsa.pub
|
||||
- ./pi_rsa.pub
|
||||
|
||||
- name: Copy SSH Pi Intercoms Private Key
|
||||
copy:
|
||||
src: ./pi_rsa
|
||||
dest: /home/<pi_user>/.ssh/id_rsa
|
||||
mode: '0600'
|
||||
owner: <pi_user>
|
||||
group: <pi_user>
|
||||
|
||||
- name: Copy SSH Pi Intercoms Public Key
|
||||
copy:
|
||||
src: ./pi_rsa.pub
|
||||
dest: /home/<pi_user>/.ssh/id_rsa.pub
|
||||
mode: '0644'
|
||||
owner: <pi_user>
|
||||
group: <pi_user>
|
||||
|
||||
|
||||
- name: Check for any updates and install them
|
||||
apt:
|
||||
update_cache: true
|
||||
upgrade: dist
|
||||
|
||||
- name: Install Required build dependencies
|
||||
apt:
|
||||
update_cache: true
|
||||
pkg:
|
||||
- python3
|
||||
- python3-pip
|
||||
- vim
|
||||
- curl
|
||||
|
||||
|
||||
- name: Check Docker is installed
|
||||
shell: "docker --version"
|
||||
register: docker_version
|
||||
|
||||
- name: Install Docker
|
||||
block:
|
||||
|
||||
- name: Download Docker script
|
||||
command: curl -fsSL https://get.docker.com -o get-docker.sh
|
||||
|
||||
- name: Run Docker install script
|
||||
command: sudo sh get-docker.sh
|
||||
|
||||
- name: Enable docker service
|
||||
service:
|
||||
name: "{{ item }}"
|
||||
enabled: true
|
||||
state: started
|
||||
loop:
|
||||
- docker
|
||||
- containerd
|
||||
|
||||
- name: Add user to docker group
|
||||
user:
|
||||
name: <pi_user>
|
||||
groups: docker
|
||||
append: yes
|
||||
|
||||
when: docker_version.stdout.find('Docker version')
|
||||
|
||||
|
||||
- name: check Docker compose is installed
|
||||
shell: "docker-compose --version"
|
||||
register: dc_version
|
||||
|
||||
- name: Install docker compose
|
||||
command: sudo pip3 -v install docker-compose
|
||||
when: dc_version.stdout.find('docker-compose version')
|
||||
|
||||
|
||||
- name: wait for network on boot
|
||||
command: raspi-config nonint do_boot_wait 0
|
||||
|
||||
- name: Reboot
|
||||
reboot:
|
||||
```
|
||||
|
||||
The steps this playbook takes are as follows:
|
||||
1. Copy your SSH keys to the pi authorized_keys
|
||||
2. Copy same Pi ssh key between the Pis for intercommunication
|
||||
3. Check for and Install updates
|
||||
4. Install any build dependancies
|
||||
5. Check if Docker is Installed
|
||||
* Download the Docker Script
|
||||
* Install Docker from script
|
||||
* Enable the Docker service
|
||||
* Add user to Docker group
|
||||
6. Check if docker-compose is installed
|
||||
* Install docker-compose (This will take a while on each pi so be patient, grab a coffee)
|
||||
9. Change raspi config to wait for network before boot
|
||||
10. Reboot
|
||||
|
||||
take some time to read through the playbook, read the docs and try to
|
||||
understand whats going on. Change the usernames to your username.
|
||||
|
||||
**Make the hosts file in the same directory as the deploy.yml**
|
||||
``` yaml
|
||||
[piCluster]
|
||||
10.10.10.20
|
||||
10.10.10.21
|
||||
10.10.10.22
|
||||
```
|
||||
* `[piCluster]` - specifies a group of hosts
|
||||
* `10.10.10.xx` - specifies the Pis IP
|
||||
|
||||
**create the Pi Intercommunication keys**
|
||||
``` bash
|
||||
ssh-keygen
|
||||
Enter file in which to save the key (/home/ahmza/.ssh/id_rsa): .pi_rsa
|
||||
```
|
||||
* save the keys in the same directory the deploy.yml file is
|
||||
|
||||
**heres the command to run the playbook.**
|
||||
|
||||
`ansible-playbook -i hosts -kK deploy.yml -c paramiko`
|
||||
|
||||
* `ansible-playbook` - the command to run the playbook
|
||||
* `-i hosts` - specifies the host/inventory file
|
||||
* `-kK` - asks you for password for the SSH connection(-k) and the sudo password(-K)
|
||||
* `deploy.yml` - the _Ansible_ playbook
|
||||
* `-c paramiko` - Use the paramiko connection method, I found this works best with ssh keys
|
||||
|
||||
You can do so much with _Ansible_, for example mount network shares etc,
|
||||
if you can do it in the cli you can do it in _Ansible_ !!
|
||||
|
||||
Now that the _Ansible_ script is done we can go ahead and actually
|
||||
configure all the Pis into the cluster.
|
||||
|
||||
### **Creating/Joining the swarm**
|
||||
Creating and joining the cluster is easy - It is manual and tedious so
|
||||
this is something you could impliment into the ansible playbook using
|
||||
multiple host groups.
|
||||
|
||||
**Follow these steps to create the swarm:**
|
||||
1. ssh to the main Pi, this will be the manager node.
|
||||
2. run `docker swarm init`
|
||||
* This will return a command and token. Make sure to copy them
|
||||
3. exit the main Pi
|
||||
4. ssh to one of the worker nodes
|
||||
5. paste and run the copied command
|
||||
6. do the same on the remaining worker nodes
|
||||
|
||||
You should now have a correctly configured cluster !!
|
||||
|
||||
### **Optional Step - Portainer**
|
||||
Portainer is a webgui that will allow us to manage the cluster, it will
|
||||
make deploying and destroying containers across the cluster a breeze.
|
||||
I do understand that all of what portainer does can simply be accomplished
|
||||
in the command line, however portainer just makes it less hastle to manage.
|
||||
|
||||
**Install Portainer**
|
||||
|
||||
Firstly we want to download the yaml file that will be used to deploy
|
||||
the portainer contiainer on our cluster.
|
||||
|
||||
`curl -L https://downloads.portainer.io/portainer-agent-stack.yml -o portainer-agent-stack.yml`
|
||||
|
||||
Now we can actually deploy portainer to the cluster using the following
|
||||
command:
|
||||
|
||||
`docker stack deploy -c portainer-agent-stack.yml portainer`
|
||||
|
||||
Once the stack is up and running navigate to:
|
||||
|
||||
`https://raspberry-ip:9443`
|
||||
|
||||
where raspberry-ip is the IP address of the manager node in the swarm.
|
||||
|
||||
### **Optional Step - Private Registry**
|
||||
|
||||
If you intend to build any of your own docker images then it may be useful
|
||||
to create a private repository to host any images that you need to run
|
||||
across the cluster.
|
||||
|
||||
to create this private repository create a file called docker-compose.yml
|
||||
in a folder called repository or whatever and enter the following.
|
||||
``` yaml
|
||||
|
||||
version: '3.9'
|
||||
services:
|
||||
registry:
|
||||
image: registry:2
|
||||
ports:
|
||||
- 5000:5000
|
||||
volumes:
|
||||
- /home/<pi-user>/docker/registry/data:/var/lib/registry
|
||||
```
|
||||
|
||||
**Be sure to change pi-user to your pi user**
|
||||
|
||||
**create a file /etc/docker/daemon.json and eter the following**
|
||||
``` json
|
||||
{
|
||||
"insecure-registries": ["localhost:5000", "<pi-address>:5000"]
|
||||
}
|
||||
```
|
||||
where `pi-address` is the ip address of the main swarm node.
|
||||
|
||||
**restart the docker service with `sudo systemctl restart docker`**
|
||||
|
||||
now we need to deploy the registry to the stack like we did with portainer
|
||||
|
||||
`docker stack deploy -c docker-compose.yml registry`
|
||||
|
||||
and we should have a private registry setup that we can use to push our
|
||||
custom docker images to.
|
||||
|
||||
Reference in New Issue
Block a user