diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 1bbb74d..c6e84ff 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -25,6 +25,7 @@ Build: only: - main + Dockerize: stage: dockerize image: diff --git a/content/posts/building-a-docker-swarm-with-pis.md b/content/posts/building-a-docker-swarm-with-pis.md index 72cb47b..4efedbc 100644 --- a/content/posts/building-a-docker-swarm-with-pis.md +++ b/content/posts/building-a-docker-swarm-with-pis.md @@ -2,7 +2,7 @@ title: Building a Docker Swarm With Pi's date: 2021-12-12T17:23:32Z author: "ahmza" -draft: true +draft: false weight: tags: ["Homelab","Docker","Docker Swarm","Raspberry Pi","Pi4"] @@ -145,7 +145,7 @@ self explanitory. So go ahead and assemble the cluster 🛠 file system** ### **Sort out the Network bits** -This is the easy part, there are two methods we can use to set a +This is the easy part, there are two methods you can use to set a static IP address, one is on your DHCP server and the other is on the device itself. Personally I prefer to set static IPs on the device itself so I will cover that here. @@ -153,11 +153,17 @@ device itself so I will cover that here. To find the current ip of your Pis you can go to your DHCP server and look for the DHCP leases for the Pis. +> there are naming conventions for naming primary and secondary +things but this is your cluster so be creative with it, I have my +primary node named ```Goob``` and the secondary nodes named ```Doris1``` +```Doris2``` and ```Doris3``` +[reference](https://www.youtube.com/watch?v=oFb7FVe8sdQ) + **SSH to the Pi** then open and edit the following files * /etc/hosts - * edit this line with your hostname: ```127.0.1.1 mainpi``` + * edit this line with your hostname: ```127.0.1.1 goob``` * /etc/hostname - * edit this file with your hostname: ```mainpi``` + * edit this file with your hostname: ```goob``` * /etc/dhcpcd.conf * knowing we will be using only static IPs here I delete everything in the file add the following @@ -169,4 +175,271 @@ and look for the DHCP leases for the Pis. ``` **Now reboot the pi** ```sudo shutdown -r now``` +**Once the Pis are back up log into each one and change the default +passwords** you may also want to create a new user for yourself but +thats totally optional. + +### **Provision with** ***Ansible*** +you want to **create an _Ansible_ control node** and that is as simple as +installing a package onto your machine. _Ansible_ is built using python +which makes it easy to install as **we can use the python pip package +manger.** To check you have it installed do ```pip --version``` if +it's not installed do the following + + +```curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py``` + +```python get-pip.py --user``` + +**once you have pip installed proceed to installing _Ansible_** + +[Docs](https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html) + +```pip install ansible``` + +Now that _Ansible_ is installed we can create a runbok that defines the +steps _Ansible_ needs to take on the remote machine to correctly +provision it. This playbook is written in yaml so get ready to make +sure your indents are correct 😬 + +**Creating the playbook** +now you can copy what I have and it should work - after changing a +thing or two - which is fine however I encourage you to copy what I +have, read the [Docs](https://docs.ansible.com/ansible/latest/user_guide/index.html) +and make any changes you feel are necessary. + +> Check the [git repo](https://gitlab.com/AHMZA/pi-docker-swarm-ansible-deploy/-/tree/main) for the most up to date yaml file im using. + +**This script assumes you are using SSH keys, which you should but if +not you can skip the whole "Configure SSH" block,** +***I RECCOMEND YOU USE SSH KEYS*** - [read this](https://blog.ahmza.com/posts/securing-ssh/) + +**Create the deploy.yml** + +```yaml +--- +- hosts: piCluster + become: yes + tasks: + + - name: Configure SSH + block: + + - name: Auth local SSH Keys + ansible.posix.authorized_key: + user: + state: present + key: "{{ item }}" + with_file: + - /home//.ssh/id_rsa.pub + - ./pi_rsa.pub + + - name: Copy SSH Pi Intercoms Private Key + copy: + src: ./pi_rsa + dest: /home//.ssh/id_rsa + mode: '0600' + owner: + group: + + - name: Copy SSH Pi Intercoms Public Key + copy: + src: ./pi_rsa.pub + dest: /home//.ssh/id_rsa.pub + mode: '0644' + owner: + group: + + + - name: Check for any updates and install them + apt: + update_cache: true + upgrade: dist + + - name: Install Required build dependencies + apt: + update_cache: true + pkg: + - python3 + - python3-pip + - vim + - curl + + + - name: Check Docker is installed + shell: "docker --version" + register: docker_version + + - name: Install Docker + block: + + - name: Download Docker script + command: curl -fsSL https://get.docker.com -o get-docker.sh + + - name: Run Docker install script + command: sudo sh get-docker.sh + + - name: Enable docker service + service: + name: "{{ item }}" + enabled: true + state: started + loop: + - docker + - containerd + + - name: Add user to docker group + user: + name: + groups: docker + append: yes + + when: docker_version.stdout.find('Docker version') + + + - name: check Docker compose is installed + shell: "docker-compose --version" + register: dc_version + + - name: Install docker compose + command: sudo pip3 -v install docker-compose + when: dc_version.stdout.find('docker-compose version') + + + - name: wait for network on boot + command: raspi-config nonint do_boot_wait 0 + + - name: Reboot + reboot: +``` + +The steps this playbook takes are as follows: +1. Copy your SSH keys to the pi authorized_keys +2. Copy same Pi ssh key between the Pis for intercommunication +3. Check for and Install updates +4. Install any build dependancies +5. Check if Docker is Installed + * Download the Docker Script + * Install Docker from script + * Enable the Docker service + * Add user to Docker group +6. Check if docker-compose is installed + * Install docker-compose (This will take a while on each pi so be patient, grab a coffee) +9. Change raspi config to wait for network before boot +10. Reboot + +take some time to read through the playbook, read the docs and try to +understand whats going on. Change the usernames to your username. + +**Make the hosts file in the same directory as the deploy.yml** +``` yaml +[piCluster] +10.10.10.20 +10.10.10.21 +10.10.10.22 +``` +* `[piCluster]` - specifies a group of hosts +* `10.10.10.xx` - specifies the Pis IP + +**create the Pi Intercommunication keys** +``` bash +ssh-keygen +Enter file in which to save the key (/home/ahmza/.ssh/id_rsa): .pi_rsa +``` +* save the keys in the same directory the deploy.yml file is + +**heres the command to run the playbook.** + +`ansible-playbook -i hosts -kK deploy.yml -c paramiko` + +* `ansible-playbook` - the command to run the playbook +* `-i hosts` - specifies the host/inventory file +* `-kK` - asks you for password for the SSH connection(-k) and the sudo password(-K) +* `deploy.yml` - the _Ansible_ playbook +* `-c paramiko` - Use the paramiko connection method, I found this works best with ssh keys + +You can do so much with _Ansible_, for example mount network shares etc, +if you can do it in the cli you can do it in _Ansible_ !! + +Now that the _Ansible_ script is done we can go ahead and actually +configure all the Pis into the cluster. + +### **Creating/Joining the swarm** +Creating and joining the cluster is easy - It is manual and tedious so +this is something you could impliment into the ansible playbook using +multiple host groups. + +**Follow these steps to create the swarm:** +1. ssh to the main Pi, this will be the manager node. +2. run `docker swarm init` + * This will return a command and token. Make sure to copy them +3. exit the main Pi +4. ssh to one of the worker nodes +5. paste and run the copied command +6. do the same on the remaining worker nodes + +You should now have a correctly configured cluster !! + +### **Optional Step - Portainer** +Portainer is a webgui that will allow us to manage the cluster, it will +make deploying and destroying containers across the cluster a breeze. +I do understand that all of what portainer does can simply be accomplished +in the command line, however portainer just makes it less hastle to manage. + +**Install Portainer** + +Firstly we want to download the yaml file that will be used to deploy +the portainer contiainer on our cluster. + +`curl -L https://downloads.portainer.io/portainer-agent-stack.yml -o portainer-agent-stack.yml` + +Now we can actually deploy portainer to the cluster using the following +command: + +`docker stack deploy -c portainer-agent-stack.yml portainer` + +Once the stack is up and running navigate to: + +`https://raspberry-ip:9443` + +where raspberry-ip is the IP address of the manager node in the swarm. + +### **Optional Step - Private Registry** + +If you intend to build any of your own docker images then it may be useful +to create a private repository to host any images that you need to run +across the cluster. + +to create this private repository create a file called docker-compose.yml +in a folder called repository or whatever and enter the following. +``` yaml + +version: '3.9' +services: + registry: + image: registry:2 + ports: + - 5000:5000 + volumes: + - /home//docker/registry/data:/var/lib/registry +``` + +**Be sure to change pi-user to your pi user** + +**create a file /etc/docker/daemon.json and eter the following** +``` json +{ + "insecure-registries": ["localhost:5000", ":5000"] +} +``` +where `pi-address` is the ip address of the main swarm node. + +**restart the docker service with `sudo systemctl restart docker`** + +now we need to deploy the registry to the stack like we did with portainer + +`docker stack deploy -c docker-compose.yml registry` + +and we should have a private registry setup that we can use to push our +custom docker images to.