Move to a three-node cluster with a Talos VIP; add Longhorn
Every node is now both control plane and worker, giving real tolerance for losing any single machine (etcd quorum needs 3, not 1). A Talos VIP gives the cluster one stable address regardless of which node answers. Also pre-installs Longhorn for replicated storage, backed by a Talos UserVolume on each node's second disk (>=900GB, non-system) rather than the default path — mirroring the pattern from the homescale repo. Docs renumbered to make room for 06-longhorn.md; checklist, glossary, network map, and CLAUDE.md's guided-bootstrap instructions updated to match, including asking the user for node IPs, VIP, and disk info before generating any Talos config. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Sonnet 5
parent
92c3ad3191
commit
110bd97cca
+56
-21
@@ -5,31 +5,62 @@ PC, or an amd64 VM). If your hardware is arm64 (Raspberry Pi, Apple Silicon
|
||||
VM), the tools below all publish arm64 builds too; swap the architecture in
|
||||
whichever install command you use and the rest of this guide is unchanged.
|
||||
|
||||
## 1. A target machine
|
||||
## 1. Three target machines
|
||||
|
||||
You need one machine to become your cluster node — a spare physical PC, a
|
||||
mini PC, or a VM (VirtualBox, UTM, Proxmox, or any hypervisor that can boot
|
||||
from an ISO). Minimum realistic spec: 2 CPU cores, 4GB RAM, 20GB disk.
|
||||
This repo builds a **three-node cluster**, with every node acting as both
|
||||
control plane (the brain: API server, scheduler, the cluster's own
|
||||
database) and worker (runs your actual apps) at once. Three, not one,
|
||||
because it's the smallest number where losing any single machine doesn't
|
||||
take the whole cluster down — see
|
||||
[`docs/00-what-is-all-this.md`](00-what-is-all-this.md) for why three
|
||||
specifically, not two or four.
|
||||
|
||||
It needs to be reachable over the network from your everyday machine (the
|
||||
one you'll run `talosctl` and `kubectl` from), and it needs a stable IP
|
||||
address — see the note about DHCP reservations in
|
||||
[`talos/README.md`](../talos/README.md).
|
||||
You need three machines — spare physical PCs, mini PCs, or VMs (VirtualBox,
|
||||
UTM, Proxmox, or any hypervisor that can boot from an ISO). Minimum
|
||||
realistic spec per machine: 2 CPU cores, 4GB RAM, plus **two disks each**:
|
||||
one for Talos itself (20GB+ is plenty), and a second, separate disk of
|
||||
**at least 900GB** that will be handed entirely to Longhorn
|
||||
(see [`docs/06-longhorn.md`](06-longhorn.md)) for storage. If your spare
|
||||
hardware doesn't have a second disk that large, that's a real constraint —
|
||||
ask Claude about adjusting the size threshold in
|
||||
[`talos/patch-longhorn.yaml`](../talos/patch-longhorn.yaml) to fit what you
|
||||
actually have.
|
||||
|
||||
You do **not** need to install an OS on it yet. That's what the next doc
|
||||
covers — Talos gets installed by booting a special ISO and sending it
|
||||
configuration over the network, not through a normal installer.
|
||||
|
||||
Download the Talos amd64 ISO from the
|
||||
[Talos releases page](https://github.com/siderolabs/talos/releases) (look
|
||||
for `metal-amd64.iso` under the latest release) and boot your target machine
|
||||
from it. Leave it sitting at the boot screen — you'll configure it in
|
||||
Each machine needs to be reachable over the network from your everyday
|
||||
machine (the one you'll run `talosctl` and `kubectl` from), and each needs
|
||||
its own stable IP address — see the note about DHCP reservations in
|
||||
[`talos/README.md`](../talos/README.md). You'll also need one *additional*
|
||||
unused address for the cluster's shared VIP (Virtual IP) — more on that in
|
||||
[`docs/02-talos-bootstrap.md`](02-talos-bootstrap.md).
|
||||
|
||||
> **Ask Claude:** if you're not sure whether to use a VM or spare physical
|
||||
You do **not** need to install an OS on any of them yet. That's what the
|
||||
next doc covers — Talos gets installed by booting a special ISO and
|
||||
sending it configuration over the network, not through a normal installer.
|
||||
|
||||
### Building your Talos image
|
||||
|
||||
Stock Talos doesn't include everything Longhorn needs — specifically, the
|
||||
`iscsi-tools` and `util-linux-tools` **system extensions** (optional
|
||||
add-ons baked into the boot image itself, since Talos has no package
|
||||
manager to install them onto a running node). You build a custom image,
|
||||
called a **schematic**, once, and use it for all three nodes:
|
||||
|
||||
1. Go to [factory.talos.dev](https://factory.talos.dev).
|
||||
2. Under system extensions, select `siderolabs/iscsi-tools` and
|
||||
`siderolabs/util-linux-tools`.
|
||||
3. Pick the current stable Talos version and `metal-amd64` as the platform.
|
||||
4. Download the resulting ISO (the site calls this a "schematic" —
|
||||
your own custom Talos image, identified by a schematic ID).
|
||||
|
||||
Boot **all three** machines from this same ISO, and leave each sitting at
|
||||
its boot screen — you'll configure them in
|
||||
[`docs/02-talos-bootstrap.md`](02-talos-bootstrap.md).
|
||||
|
||||
> **Ask Claude:** if you're not sure whether to use VMs or spare physical
|
||||
> hardware, ask — the tradeoffs (snapshots and easy resets vs. more
|
||||
> realistic networking) are worth five minutes of discussion before you
|
||||
> commit.
|
||||
> realistic networking, and whether three VMs on one physical host really
|
||||
> gives you the resilience three nodes are supposed to buy you) are worth
|
||||
> five minutes of discussion before you commit.
|
||||
|
||||
## 2. talosctl
|
||||
|
||||
@@ -101,8 +132,12 @@ that references a repo URL means your fork's URL.
|
||||
|
||||
## Checklist
|
||||
|
||||
- [ ] Target machine chosen, network-reachable, IP address noted
|
||||
- [ ] Talos ISO downloaded, machine booted from it and idle at the boot screen
|
||||
- [ ] Three target machines chosen, each network-reachable, each with a
|
||||
stable IP address noted, each with a second disk ≥900GB for Longhorn
|
||||
- [ ] One additional unused IP address set aside for the cluster's VIP
|
||||
- [ ] Custom Talos schematic built at factory.talos.dev with `iscsi-tools`
|
||||
and `util-linux-tools`, ISO downloaded
|
||||
- [ ] All three machines booted from that ISO, idle at the boot screen
|
||||
- [ ] `talosctl version --client` works
|
||||
- [ ] `kubectl version --client` works
|
||||
- [ ] `argocd version --client` works
|
||||
|
||||
Reference in New Issue
Block a user