Move to a three-node cluster with a Talos VIP; add Longhorn

Every node is now both control plane and worker, giving real tolerance
for losing any single machine (etcd quorum needs 3, not 1). A Talos VIP
gives the cluster one stable address regardless of which node answers.

Also pre-installs Longhorn for replicated storage, backed by a Talos
UserVolume on each node's second disk (>=900GB, non-system) rather than
the default path — mirroring the pattern from the homescale repo.

Docs renumbered to make room for 06-longhorn.md; checklist, glossary,
network map, and CLAUDE.md's guided-bootstrap instructions updated to
match, including asking the user for node IPs, VIP, and disk info before
generating any Talos config.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

CLAUDE.md

@@ -59,7 +59,16 @@ end to end" — drive [`docs/checklist.md`](docs/checklist.md) interactively:
 6. Periodically remind them of `docs/glossary.md` and the "Ask Claude"
    checkpoints along the way — this mode shouldn't feel like a script being
    read at them.
-7. **Phase 5 (MetalLB) needs real information from the user, not
+7. **The Talos phase (docs/02) needs real information too, before any
+   config is generated.** Ask: what are the three nodes' IP addresses (or
+   DHCP reservations), and — separately — what unused address should the
+   cluster's VIP use? The VIP is a fourth address, distinct from all three
+   node IPs and from the MetalLB pool (below); it needs the same "ask about
+   their LAN and DHCP range" treatment as MetalLB. Also confirm each node
+   actually has a second disk ≥900GB before they generate config — if not,
+   that's a real constraint on `talos/patch-longhorn.yaml`'s `minSize`, not
+   something to paper over with a placeholder.
+8. **The MetalLB phase (docs/05) needs real information from the user, not
    placeholders.** Before editing `apps/metallb/templates/ipaddresspool.yaml`
    or `apps/demo/service.yaml`, actually ask: what does their LAN's subnet
    look like, and what IP range does their router's DHCP server already
@@ -80,10 +89,12 @@ useful if it's actually current, so treat updating it as part of the
 change itself, not an afterthought:
 
 - Whenever you (Claude) assign, learn, or change a real IP address for
-  something — walking the user through MetalLB's pool
+  something — the three node addresses and VIP in
-  ([`docs/05-metallb.md`](docs/05-metallb.md)), patching `argocd-server`,
+  [`docs/02-talos-bootstrap.md`](docs/02-talos-bootstrap.md), walking the
-  helping add a new app with a `LoadBalancer` Service — update the table in
+  user through MetalLB's pool ([`docs/05-metallb.md`](docs/05-metallb.md)),
-  `docs/network.md` in the same breath. Don't wait to be asked.
+  patching `argocd-server`, helping add a new app with a `LoadBalancer`
+  Service — update the table in `docs/network.md` in the same breath.
+  Don't wait to be asked.
 - Every row needs a full, correct URL — scheme, IP, and port — not just a
   bare address. Include the port number unless it's the protocol's default
   (80 for `http://`, 443 for `https://`). If you're not sure what port a
@@ -101,7 +112,7 @@ Point new users here first if they seem unsure where to start:
 
 1. [`README.md`](README.md) — orientation
 2. [`docs/00-what-is-all-this.md`](docs/00-what-is-all-this.md) — mental model, read before touching a terminal
-3. [`docs/01-prerequisites.md`](docs/01-prerequisites.md) → [`08-growing-up.md`](docs/08-growing-up.md) — in order
+3. [`docs/01-prerequisites.md`](docs/01-prerequisites.md) → [`docs/09-growing-up.md`](docs/09-growing-up.md) — in order
 4. [`docs/checklist.md`](docs/checklist.md) — condensed step-by-step reference once concepts are understood, or to drive Guided Bootstrap Mode
 5. [`docs/glossary.md`](docs/glossary.md) — reference throughout, not read start to finish
 
@@ -116,17 +127,18 @@ If asked to change manifests or docs rather than explain them:
   `argocd/`.
 - Keep the moving parts minimal. Ingress, cert-manager, and secrets
   management are intentionally **documented, not shipped** (see
-  [`docs/08-growing-up.md`](docs/08-growing-up.md)) — don't add them to
+  [`docs/09-growing-up.md`](docs/09-growing-up.md)) — don't add them to
-  make the repo "more complete" without being asked. MetalLB is the one
+  make the repo "more complete" without being asked. MetalLB and Longhorn
-  exception, shipped deliberately because bare metal has no other way to
+  are the two exceptions, shipped deliberately because bare metal has no
-  give a Service a real IP address — don't take that as license to add
+  other way to give a Service a real IP address or give an app real
-  other addons the same way without being asked.
+  persistent storage — don't take that as license to add other addons the
+  same way without being asked.
 - This repo is **public**. Never write a plaintext secret, credential, or
   private key into any file here. If a change would need one, stop and
   point to the Sealed Secrets / SOPS guidance in
-  `docs/08-growing-up.md` instead of improvising.
+  `docs/09-growing-up.md` instead of improvising.
 - New apps follow the existing `apps/<name>.yaml` + `apps/<name>/` pattern
-  — see [`docs/07-adding-new-apps.md`](docs/07-adding-new-apps.md) before
+  — see [`docs/08-adding-new-apps.md`](docs/08-adding-new-apps.md) before
   adding one. `apps/metallb/` is the reference example for a Helm-based
   one.
 - Every `apps/*.yaml` Application manifest needs its own `repoURL` fixed to

README.md

@@ -1,7 +1,7 @@
 # GitOps Starter Pack
 
-A minimal, real, working GitOps setup for a single Kubernetes cluster —
+A minimal, real, working GitOps setup for a three-node Kubernetes cluster
-built to teach, not just to run. If you've never touched Kubernetes,
+— built to teach, not just to run. If you've never touched Kubernetes,
 GitOps, Talos, or Argo CD before, this is meant to get you from a bare
 machine to a working cluster you actually understand, not just one you
 copy-pasted into existence.
@@ -35,6 +35,9 @@ shell history. This repo takes a different approach:
 5. **MetalLB** gives apps in this cluster real IP addresses on your own
    network — something a cloud provider would normally do for you, and
    bare metal doesn't have on its own.
+6. **Longhorn** does the same for storage — turning a spare disk on each
+   node into one pool of storage that's replicated across all three
+   nodes, so losing a machine doesn't mean losing its data.
 
 Git becomes the one place that's actually true. If you want the full
 picture with diagrams before doing anything, start with
@@ -46,13 +49,14 @@ picture with diagrams before doing anything, start with
 talos/    → Talos machine config + bootstrap guide (the OS layer)
 argocd/   → Argo CD install + the one manifest you apply by hand
 apps/     → Everything Argo CD manages from here on — demo app, MetalLB,
-            and wherever you add new apps
+            Longhorn, and wherever you add new apps
 docs/     → The numbered, progressive guide you should actually read
 ```
 
 ## Prerequisites
 
-- One machine (physical or VM) to become your cluster node
+- Three machines (physical or VM), each with a second disk ≥900GB, to
+  become your cluster's three nodes
 - `talosctl`, `kubectl`, and the `argocd` CLI installed locally
 - A GitHub account, and your own fork of this repo
 
@@ -68,11 +72,12 @@ Read these in order. Each is short — one concept per file.
 4. [`docs/03-installing-argocd.md`](docs/03-installing-argocd.md) — install Argo CD into the cluster
 5. [`docs/04-app-of-apps.md`](docs/04-app-of-apps.md) — apply the one manifest that hands control to Git
 6. [`docs/05-metallb.md`](docs/05-metallb.md) — give apps real IP addresses on your network
-7. [`docs/06-your-first-change.md`](docs/06-your-first-change.md) — edit a file, push, watch it deploy itself
+7. [`docs/06-longhorn.md`](docs/06-longhorn.md) — give apps real, replicated storage
-8. [`docs/07-adding-new-apps.md`](docs/07-adding-new-apps.md) — add something of your own
+8. [`docs/07-your-first-change.md`](docs/07-your-first-change.md) — edit a file, push, watch it deploy itself
-9. [`docs/08-growing-up.md`](docs/08-growing-up.md) — next steps: more nodes, ingress, secrets management
+9. [`docs/08-adding-new-apps.md`](docs/08-adding-new-apps.md) — add something of your own
-10. [`docs/glossary.md`](docs/glossary.md) — every term, plain English, for reference throughout
+10. [`docs/09-growing-up.md`](docs/09-growing-up.md) — next steps: more nodes, ingress, secrets management
-11. [`docs/network.md`](docs/network.md) — the running log of every real IP address in your cluster, kept current as you go
+11. [`docs/glossary.md`](docs/glossary.md) — every term, plain English, for reference throughout
+12. [`docs/network.md`](docs/network.md) — the running log of every real IP address in your cluster, kept current as you go
 
 Prefer a single condensed list you can work through step by step with
 Claude, checking items off as you go? Use
@@ -101,7 +106,7 @@ moving on.
 
 This repo deliberately does **not** ship ingress, TLS/cert-manager, or
 secrets management in v1 — they're real next steps, documented in
-[`docs/08-growing-up.md`](docs/08-growing-up.md), left out so the core
+[`docs/09-growing-up.md`](docs/09-growing-up.md), left out so the core
 loop stays small enough to fully understand first. This repo is also
 **public** — never commit plaintext secrets, credentials, or private keys
 to it; see that same doc for how to handle secrets properly once you need

apps/demo.yaml

@@ -16,7 +16,7 @@
 # (e.g. a database or ingress controller from a public chart repo). Argo CD
 # detects a Helm chart automatically when it finds a Chart.yaml at the
 # source path, resolves its dependencies, and syncs the rendered output just
-# like any other app. See docs/07-adding-new-apps.md for a worked example —
+# like any other app. See docs/08-adding-new-apps.md for a worked example —
 # the demo app deliberately stays plain manifests so the GitOps loop is easy
 # to read end to end with no templating in the way.
 apiVersion: argoproj.io/v1alpha1

apps/demo/configmap.yaml

@@ -4,7 +4,7 @@
 # baking the page into a custom container image.
 #
 # Why this matters for the GitOps story: this is the file you'll edit in
-# docs/06-your-first-change.md. Change the text here, commit, push, and
+# docs/07-your-first-change.md. Change the text here, commit, push, and
 # watch Argo CD notice the difference in Git and push it into the cluster
 # for you — no `kubectl` command involved. That's the whole loop, proven
 # with one line of HTML.

apps/demo/deployment.yaml

@@ -2,7 +2,7 @@
 # serving this ConfigMap's index.html") and hands it to Kubernetes, whose
 # job is to continuously make reality match that description. If the Pod
 # crashes, Kubernetes restarts it. If the node dies, Kubernetes (given more
-# than one node — see docs/08-growing-up.md) reschedules it elsewhere.
+# than one node — see docs/09-growing-up.md) reschedules it elsewhere.
 # You never tell Kubernetes "start a container" directly; you describe the
 # end state and it does the work. This is what "declarative" means.
 apiVersion: apps/v1

apps/longhorn.yaml

@@ -0,0 +1,36 @@
+# Longhorn — this repo's pre-installed storage layer. Same pattern as
+# apps/metallb.yaml: an Application pointing at a Helm chart
+# (apps/longhorn/) that pulls in the official upstream chart. Once this
+# syncs, any app in this cluster can request persistent storage (a
+# PersistentVolumeClaim) and actually get real, replicated storage backed
+# by the dedicated disk each node prepared in
+# talos/patch-longhorn.yaml — see docs/06-longhorn.md.
+#
+# Longhorn needs privileged Pods (it manages block devices directly) and
+# the `open-iscsi`/`util-linux-tools` Talos system extensions baked into
+# your node image — both covered in docs/01-prerequisites.md and
+# docs/02-talos-bootstrap.md. Skipping either means Longhorn's Pods will
+# sit unhealthy no matter how correctly this Application syncs.
+apiVersion: argoproj.io/v1alpha1
+kind: Application
+metadata:
+  name: longhorn
+  namespace: argocd
+spec:
+  project: default
+
+  source:
+    repoURL: https://github.com/YOUR_GITHUB_USERNAME/k8s-starter-pack.git
+    targetRevision: main
+    path: apps/longhorn
+
+  destination:
+    server: https://kubernetes.default.svc
+    namespace: longhorn-system
+
+  syncPolicy:
+    automated:
+      prune: true
+      selfHeal: true
+    syncOptions:
+      - CreateNamespace=true

apps/longhorn/Chart.yaml

@@ -0,0 +1,18 @@
+# An umbrella chart, same pattern as apps/metallb/Chart.yaml: it declares a
+# dependency on the upstream Longhorn chart and carries no templates of its
+# own — everything Longhorn needs (its CRDs, controllers, UI, default
+# StorageClass) comes from the dependency. See
+# docs/08-adding-new-apps.md for more on this pattern.
+apiVersion: v2
+name: longhorn-app
+description: Installs Longhorn, this repo's pre-installed distributed block storage
+version: 0.1.0
+dependencies:
+  - name: longhorn
+    # A version range rather than an exact pin — see the same note in
+    # apps/metallb/Chart.yaml. Check
+    # https://artifacthub.io/packages/helm/longhorn/longhorn for the
+    # current release and pin an exact version for a fully reproducible
+    # install.
+    version: ">=1.7.0"
+    repository: https://charts.longhorn.io

apps/longhorn/values.yaml

@@ -0,0 +1,22 @@
+# Values for the upstream Longhorn chart, nested under `longhorn:` (the
+# dependency's name in Chart.yaml).
+longhorn:
+  defaultSettings:
+    # Must match the mount path from the UserVolumeConfig in
+    # talos/patch-longhorn.yaml (/var/mnt/<name>) — this is what points
+    # Longhorn at the dedicated data disk instead of its own default path
+    # on the root filesystem, which on Talos is ephemeral/unsuited for
+    # real data anyway.
+    defaultDataPath: /var/mnt/longhorn
+    # Longhorn's own headroom settings, kept conservative: stop scheduling
+    # new replicas onto a disk once free space drops below these
+    # thresholds, so the cluster degrades gracefully instead of actually
+    # filling a disk.
+    storageMinimalAvailablePercentage: 25
+    storageReservedPercentageForDefaultDisk: 30
+  preUpgradeChecker:
+    # Longhorn's own pre-upgrade check runs as a Job that expects to reach
+    # an upgrade-responder service on the internet by default. Harmless to
+    # leave on, but pointless for a learning cluster and one less thing to
+    # explain — off by default here.
+    jobEnabled: false

apps/metallb/Chart.yaml

@@ -5,7 +5,7 @@
 # exists at this path, resolves the dependency, and syncs the combined
 # rendered output as one Application — exactly like any other app in
 # apps/. This is the real, working example of the Helm-based app pattern
-# mentioned in docs/07-adding-new-apps.md: use it as a template if you want
+# mentioned in docs/08-adding-new-apps.md: use it as a template if you want
 # to bring in another third-party chart later.
 apiVersion: v2
 name: metallb-app

argocd/README.md

@@ -105,8 +105,8 @@ argocd app get app-of-apps
 argocd app list
 ```
 
-You should see `app-of-apps` sync, then `demo` and `metallb` Applications
+You should see `app-of-apps` sync, then `demo`, `metallb`, and `longhorn`
-appear underneath it, then sync themselves. See
+Applications appear underneath it, then sync themselves. See
 [`docs/04-app-of-apps.md`](../docs/04-app-of-apps.md) for what's happening
 here and [`docs/checklist.md`](../docs/checklist.md) for the full guided
 path.

docs/00-what-is-all-this.md

@@ -19,17 +19,20 @@ the real cluster match it. You stop making changes by hand. You edit a file,
 push it, and a robot applies it for you — the same way every time, with a
 record of every change in `git log`.
 
-## The five pieces, in plain terms
+## The six pieces, in plain terms
 
 **Kubernetes** is software that runs on one or more machines and manages
 containers for you: starts them, restarts them if they crash, and moves
-them to a different machine if one dies (once you have more than one — see
+them to a different machine if one dies. You describe the end state you
-[`docs/08-growing-up.md`](08-growing-up.md)). You describe the end state you
+want ("run one copy of this"); Kubernetes' job is to make that true and
-want ("run one copy of this"); Kubernetes' job is to make that true and keep
+keep it true. This is what people mean by **declarative**: you state the
-it true. This is what people mean by **declarative**: you state the
+destination, not the steps to get there. This repo runs three machines
-destination, not the steps to get there.
+from the start, specifically so that "moves them to a different machine
+if one dies" is something you actually get, not just a fact about
+Kubernetes in general — see
+[`docs/02-talos-bootstrap.md`](02-talos-bootstrap.md) for why three.
 
-**Talos Linux** is the operating system your machine runs. It's built for
+**Talos Linux** is the operating system each machine runs. It's built for
 exactly one purpose — running Kubernetes — and nothing else. There's no SSH,
 no shell, no package manager to fiddle with by hand. You configure it by
 sending it a config file over the network (a "machine config"), the same
@@ -47,15 +50,23 @@ manages. Rather than one Application resource pointing at everything, one
 Application — the app-of-apps — watches a folder (`apps/`) full of *other*
 Application definitions, each pointing at its own small piece. Add a new
 app by adding a new file — see
-[`docs/07-adding-new-apps.md`](07-adding-new-apps.md).
+[`docs/08-adding-new-apps.md`](08-adding-new-apps.md).
 
 **MetalLB** solves a specific problem: on a cloud provider, asking for a
 public IP address for a service is one API call away; on your own bare
-metal machine, nothing hands those out automatically. MetalLB runs inside
+metal machines, nothing hands those out automatically. MetalLB runs inside
 the cluster and fills that gap, so apps here can get a real, dedicated IP
 address on your network instead of an awkward fixed port. It's pre-installed
 in this repo, not a "next step" — see [`docs/05-metallb.md`](05-metallb.md).
 
+**Longhorn** solves the equivalent problem for storage: a cloud provider
+gives you durable, replicated disks on demand; bare metal just gives you
+whatever's physically bolted into each machine. Longhorn turns a plain
+disk on each of your three nodes into one pool of replicated storage any
+app can request a slice of — and because it's replicated *across* nodes,
+losing one machine doesn't lose the data either. Also pre-installed, not a
+"next step" — see [`docs/06-longhorn.md`](06-longhorn.md).
+
 ## Architecture — the shape of what you'll build
 
 ```
@@ -64,9 +75,11 @@ in this repo, not a "next step" — see [`docs/05-metallb.md`](05-metallb.md).
 │   repo, on GitHub)      │
 │                         │
 │   apps/demo.yaml        │
-│   apps/demo/*.yaml      │──┐
+│   apps/demo/*.yaml      │
-│   apps/metallb.yaml     │  │  watched by
+│   apps/metallb.yaml     │──┐
-│   apps/metallb/*        │  │
+│   apps/metallb/*        │  │  watched by
+│   apps/longhorn.yaml    │  │
+│   apps/longhorn/*       │  │
 └─────────────────────────┘  │
                               ▼
                     ┌───────────────────┐
@@ -76,29 +89,29 @@ in this repo, not a "next step" — see [`docs/05-metallb.md`](05-metallb.md).
                     └─────────┬──────────┘
                               │ applies manifests
                               ▼
-                 ┌─────────────────────────────┐
+        ┌───────────────────────────────────────────────┐
-                 │   Talos Linux node           │
+        │      Three Talos Linux nodes (VIP: <VIP_IP>)    │
-                 │   (control plane + worker,   │
+        │      each: control plane + worker + etcd peer   │
-                 │    single node)               │
+        │                                                 │
-                 │                               │
+        │  demo namespace:        metallb-system:         │
-                 │  ┌────────────┐ ┌──────────┐  │
+        │  nginx Pod ⇄ Service    hands out a real IP      │
-                 │  │demo         │ metallb-   │  │
+        │  (LoadBalancer) ◄───────address to that Service  │
-                 │  │namespace:   │ system:    │  │
+        │                                                 │
-                 │  │nginx Pod ⇄  │ hands out  │  │
+        │  longhorn-system:       each node's 2nd disk    │
-                 │  │Service      │ a real IP  │  │
+        │  replicates data        (≥900GB, UserVolume)    │
-                 │  │(LoadBalancer)◄─address to │  │
+        │  across all 3 nodes  ◄──mounted at              │
-                 │  │             │ that Service│  │
+        │                         /var/mnt/longhorn        │
-                 │  └────────────┘ └──────────┘  │
+        └───────────────────────────────────────────────┘
-                 └─────────────────────────────┘
 ```
 
-Everything below the Git repo is a *consequence* of what's in Git. If the
+Everything below the Git repo is a *consequence* of what's in Git. If any
-box labelled "Talos Linux node" burned to the ground and you rebuilt it from
+one of the three node boxes burned to the ground and you replaced that
-scratch, replaying this repo against it would recreate everything except the
+machine, replaying this repo against it would recreate everything on it
-raw OS bootstrap (docs/02), the one-time Argo CD install (docs/03), and the
+except the raw OS bootstrap (docs/02) and the one-time Argo CD install
-handful of network-specific addresses you choose in docs/05 — all
+(docs/03) — both deliberately kept outside the loop, since something has
-deliberately kept outside the loop, since something has to start it and
+to start it. The handful of network-specific addresses you choose along
-nothing in Git can know your LAN's IP layout for you.
+the way (the VIP, the MetalLB pool) live in `docs/network.md`, not in Git
+history alone, since nothing in Git can know your LAN's IP layout for you.
 
 ## The reconciliation loop
 

docs/01-prerequisites.md

@@ -5,31 +5,62 @@ PC, or an amd64 VM). If your hardware is arm64 (Raspberry Pi, Apple Silicon
 VM), the tools below all publish arm64 builds too; swap the architecture in
 whichever install command you use and the rest of this guide is unchanged.
 
-## 1. A target machine
+## 1. Three target machines
 
-You need one machine to become your cluster node — a spare physical PC, a
+This repo builds a **three-node cluster**, with every node acting as both
-mini PC, or a VM (VirtualBox, UTM, Proxmox, or any hypervisor that can boot
+control plane (the brain: API server, scheduler, the cluster's own
-from an ISO). Minimum realistic spec: 2 CPU cores, 4GB RAM, 20GB disk.
+database) and worker (runs your actual apps) at once. Three, not one,
+because it's the smallest number where losing any single machine doesn't
+take the whole cluster down — see
+[`docs/00-what-is-all-this.md`](00-what-is-all-this.md) for why three
+specifically, not two or four.
 
-It needs to be reachable over the network from your everyday machine (the
+You need three machines — spare physical PCs, mini PCs, or VMs (VirtualBox,
-one you'll run `talosctl` and `kubectl` from), and it needs a stable IP
+UTM, Proxmox, or any hypervisor that can boot from an ISO). Minimum
-address — see the note about DHCP reservations in
+realistic spec per machine: 2 CPU cores, 4GB RAM, plus **two disks each**:
-[`talos/README.md`](../talos/README.md).
+one for Talos itself (20GB+ is plenty), and a second, separate disk of
+**at least 900GB** that will be handed entirely to Longhorn
+(see [`docs/06-longhorn.md`](06-longhorn.md)) for storage. If your spare
+hardware doesn't have a second disk that large, that's a real constraint —
+ask Claude about adjusting the size threshold in
+[`talos/patch-longhorn.yaml`](../talos/patch-longhorn.yaml) to fit what you
+actually have.
 
-You do **not** need to install an OS on it yet. That's what the next doc
+Each machine needs to be reachable over the network from your everyday
-covers — Talos gets installed by booting a special ISO and sending it
+machine (the one you'll run `talosctl` and `kubectl` from), and each needs
-configuration over the network, not through a normal installer.
+its own stable IP address — see the note about DHCP reservations in
-
+[`talos/README.md`](../talos/README.md). You'll also need one *additional*
-Download the Talos amd64 ISO from the
+unused address for the cluster's shared VIP (Virtual IP) — more on that in
-[Talos releases page](https://github.com/siderolabs/talos/releases) (look
-for `metal-amd64.iso` under the latest release) and boot your target machine
-from it. Leave it sitting at the boot screen — you'll configure it in
 [`docs/02-talos-bootstrap.md`](02-talos-bootstrap.md).
 
-> **Ask Claude:** if you're not sure whether to use a VM or spare physical
+You do **not** need to install an OS on any of them yet. That's what the
+next doc covers — Talos gets installed by booting a special ISO and
+sending it configuration over the network, not through a normal installer.
+
+### Building your Talos image
+
+Stock Talos doesn't include everything Longhorn needs — specifically, the
+`iscsi-tools` and `util-linux-tools` **system extensions** (optional
+add-ons baked into the boot image itself, since Talos has no package
+manager to install them onto a running node). You build a custom image,
+called a **schematic**, once, and use it for all three nodes:
+
+1. Go to [factory.talos.dev](https://factory.talos.dev).
+2. Under system extensions, select `siderolabs/iscsi-tools` and
+   `siderolabs/util-linux-tools`.
+3. Pick the current stable Talos version and `metal-amd64` as the platform.
+4. Download the resulting ISO (the site calls this a "schematic" —
+   your own custom Talos image, identified by a schematic ID).
+
+Boot **all three** machines from this same ISO, and leave each sitting at
+its boot screen — you'll configure them in
+[`docs/02-talos-bootstrap.md`](02-talos-bootstrap.md).
+
+> **Ask Claude:** if you're not sure whether to use VMs or spare physical
 > hardware, ask — the tradeoffs (snapshots and easy resets vs. more
-> realistic networking) are worth five minutes of discussion before you
+> realistic networking, and whether three VMs on one physical host really
-> commit.
+> gives you the resilience three nodes are supposed to buy you) are worth
+> five minutes of discussion before you commit.
 
 ## 2. talosctl
 
@@ -101,8 +132,12 @@ that references a repo URL means your fork's URL.
 
 ## Checklist
 
-- [ ] Target machine chosen, network-reachable, IP address noted
+- [ ] Three target machines chosen, each network-reachable, each with a
-- [ ] Talos ISO downloaded, machine booted from it and idle at the boot screen
+      stable IP address noted, each with a second disk ≥900GB for Longhorn
+- [ ] One additional unused IP address set aside for the cluster's VIP
+- [ ] Custom Talos schematic built at factory.talos.dev with `iscsi-tools`
+      and `util-linux-tools`, ISO downloaded
+- [ ] All three machines booted from that ISO, idle at the boot screen
 - [ ] `talosctl version --client` works
 - [ ] `kubectl version --client` works
 - [ ] `argocd version --client` works

docs/02-talos-bootstrap.md

@@ -1,7 +1,8 @@
 # 02 — Bootstrapping Talos
 
-At the end of this doc, you'll have a single-node Kubernetes cluster,
+At the end of this doc, you'll have a three-node Kubernetes cluster,
-running nothing but Kubernetes itself. No Argo CD yet — that's next.
+running nothing but Kubernetes itself (plus the storage prep Longhorn
+needs later). No Argo CD yet — that's next.
 
 ## Why Talos, specifically
 
@@ -17,44 +18,92 @@ same declarative way Kubernetes applies your app manifests. Less surface
 area, less to secure, less to keep in sync by hand — the same philosophy as
 the rest of this stack, one layer lower.
 
-## Single node, on purpose
+## Three nodes, each both control plane and worker
 
-A "real" Kubernetes cluster usually has multiple nodes — some running the
+A "real" Kubernetes cluster has multiple nodes — some running the
-**control plane** (the brain: API server, scheduler, the cluster's
+**control plane** (the brain: API server, scheduler, the cluster's own
-database) and others as **workers** (running your actual apps), so that
+database, called etcd) and others as **workers** (running your actual
-losing one machine doesn't take the whole cluster down. This repo starts
+apps) — so that losing one machine doesn't take the whole cluster down.
-with exactly one node acting as both, because that's the cheapest way to
+This repo starts with three nodes, each acting as *both* control plane and
-learn the concepts that matter for GitOps without needing multiple
+worker, which gets you real resilience — survive losing any one machine —
-machines. [`docs/08-growing-up.md`](08-growing-up.md) covers adding more
+without needing more than three.
-nodes once you're comfortable — the config here is deliberately compatible
+
-with that path, not a dead end.
+Why three specifically, and not two: etcd (the control plane's database)
+uses **quorum** — a majority vote among its members — to agree on cluster
+state, and a majority only means something with an odd number of voters.
+With one control-plane node, losing it loses the cluster outright. Two
+doesn't help: losing either one leaves one survivor, which is not a
+majority of two. Three is the smallest number where losing any *one*
+still leaves two — a majority — able to keep agreeing on state. This is
+also why [`docs/09-growing-up.md`](09-growing-up.md) — once you're past
+this doc — specifically warns against going to four control-plane nodes
+later: four doesn't improve on three's fault tolerance, it just makes
+quorum math worse.
+
+## The VIP: one address for a cluster that moves
+
+With three control-plane nodes, "where's the API server?" no longer has
+one fixed answer — it's whichever nodes happen to be up right now. Talos's
+**VIP** (Virtual IP) feature solves this: the three nodes continuously
+elect one of themselves to answer on one shared IP address you choose, and
+re-elect within seconds if that node goes down. Everything that talks to
+this cluster — `talosctl`, `kubectl`, the Application manifests in this
+repo — points at the VIP, never at one node's own address. See the
+`vip` block in
+[`talos/patch-controlplane.yaml`](../talos/patch-controlplane.yaml).
+
+> **Ask Claude:** "help me pick the VIP address for my cluster" before you
+> generate any config — same kind of decision as the MetalLB pool in
+> [`docs/05-metallb.md`](05-metallb.md): a real, unused address on your
+> LAN, outside your DHCP range, that Claude can't guess on its own.
+
+## Preparing storage for Longhorn
+
+Each node also gets a second disk reserved for
+[Longhorn](06-longhorn.md), this repo's pre-installed storage layer —
+handled by [`talos/patch-longhorn.yaml`](../talos/patch-longhorn.yaml),
+applied at the same time as the control-plane patch below. Read that
+file's comments; the short version is that Talos finds, formats, and
+mounts a disk for Longhorn on its own, using rules you set (not the system
+disk, at least 900GB), so there's no manual partitioning step here.
 
 ## Steps
 
 Full exact commands live in [`talos/README.md`](../talos/README.md) — read
 that alongside this doc; this section explains what's happening at each
-stage, that one has copy-paste blocks.
+stage, that one has copy-paste blocks. You'll repeat most of these steps
+**three times**, once per node — `talosctl README.md` calls out exactly
+which commands are "once" vs. "once per node."
 
 1. **Generate machine configuration.** `talosctl gen config` produces
-   Talos's config files for your cluster, including
+   Talos's config files for your cluster, with both
-   [`talos/patch-controlplane.yaml`](../talos/patch-controlplane.yaml) —
+   [`talos/patch-controlplane.yaml`](../talos/patch-controlplane.yaml)
-   read that file, it's short and explains itself — which lets your one
+   (scheduling + the VIP) and
-   node run workloads as well as the control plane.
+   [`talos/patch-longhorn.yaml`](../talos/patch-longhorn.yaml) (storage
+   prep) baked in — read both files, they explain themselves. This
+   produces one `controlplane.yaml`, applied identically to all three
+   nodes; there's no separate worker config in this setup, since every
+   node is both.
 
-2. **Apply the config to your node.** The node, freshly booted from the
+2. **Apply the config to each node.** Each node, freshly booted from your
-   Talos ISO, is sitting in "maintenance mode" with no configuration.
+   custom Talos ISO, is sitting in "maintenance mode" with no
-   `talosctl apply-config` sends it the config generated in step 1. The
+   configuration. `talosctl apply-config` sends it the config from step 1.
-   node reboots, installs Talos to its disk, and comes up running for real.
+   Run this once per node, pointed at that node's own address. Each node
+   reboots, installs Talos to its system disk, formats and mounts its
+   second disk for Longhorn, and comes up running for real.
 
 3. **Bootstrap etcd.** `talosctl bootstrap` initializes the cluster's data
-   store. This is a one-time, one-node-ever operation — even in a
+   store. This is a one-time, one-node-ever operation, run against exactly
-   multi-node cluster you only ever bootstrap once.
+   one of your three nodes — even though all three will end up running
+   etcd, only one of them needs telling to *start* the cluster; the other
+   two join the existing one once it exists.
 
 4. **Retrieve your kubeconfig.** `talosctl kubeconfig` downloads the
    credentials you'll use to talk to Kubernetes with `kubectl` — the
-   Kubernetes equivalent of the `talosconfig` you got in step 1.
+   Kubernetes equivalent of the `talosconfig` you got in step 1. Point it
+   at the VIP, not at any one node.
 
-5. **Verify.** `kubectl get nodes` should show one node, `Ready`.
+5. **Verify.** `kubectl get nodes` should show all three nodes, `Ready`.
 
 > **Ask Claude:** "what's the difference between a kubeconfig and a
 > talosconfig, and why are they separate files?" is a good one to ask once
@@ -62,11 +111,14 @@ stage, that one has copy-paste blocks.
 
 ## Checklist
 
-- [ ] `_out/controlplane.yaml`, `worker.yaml`, and `talosconfig` generated
+- [ ] VIP address chosen, with Claude's help, for your LAN
-- [ ] Config applied to the node, node has rebooted and installed Talos
+- [ ] `_out/controlplane.yaml` and `talosconfig` generated, with both
-- [ ] `talosctl bootstrap` run exactly once
+      patches applied
-- [ ] kubeconfig retrieved
+- [ ] Config applied to **all three** nodes; each has rebooted, installed
-- [ ] `kubectl get nodes` shows one `Ready` node
+      Talos, and mounted its Longhorn disk
+- [ ] `talosctl bootstrap` run exactly once, against exactly one node
+- [ ] kubeconfig retrieved, pointed at the VIP
+- [ ] `kubectl get nodes` shows all three nodes `Ready`
 
-Once your node is `Ready`, continue to
+Once all three nodes are `Ready`, continue to
 [`docs/03-installing-argocd.md`](03-installing-argocd.md).

docs/03-installing-argocd.md

@@ -46,7 +46,7 @@ each stage does:
 > **Ask Claude:** "why is a port-forward good enough for now, and what
 > would I need instead to reach this Argo CD from somewhere other than my
 > own machine?" — a preview of the Ingress topic in
-> [`docs/08-growing-up.md`](08-growing-up.md).
+> [`docs/09-growing-up.md`](09-growing-up.md).
 
 ## Checklist
 

docs/04-app-of-apps.md

@@ -22,30 +22,33 @@ new app.
 argocd/app-of-apps/app-of-apps.yaml   (you apply this ONE file, by hand, once)
         │
         │  watches apps/, non-recursively
-        ├──────────────┬───────────────┐
+        ├───────────────┬────────────────┬───────────────┐
-        ▼               ▼               ▼
+        ▼               ▼                ▼               ▼
-   apps/demo.yaml   apps/metallb.yaml   (more, later — just add a file)
+   apps/demo.yaml  apps/metallb.yaml apps/longhorn.yaml  (more, later)
-        │               │
+        │               │                │
-        ▼               ▼
+        ▼               ▼                ▼
-   apps/demo/*.yaml  apps/metallb/*    (the actual manifests / Helm chart)
+   apps/demo/*.yaml  apps/metallb/*   apps/longhorn/*    (manifests / Helm charts)
 ```
 
-Three layers, each watching the next. You only ever touch the bottom two —
+Four layers wide, each watching the next. You only ever touch the bottom
-the app-of-apps is set-and-forget. Notice `apps/metallb.yaml` is already
+two — the app-of-apps is set-and-forget. Notice `apps/metallb.yaml` and
-here — see [`docs/05-metallb.md`](05-metallb.md) right after this one.
+`apps/longhorn.yaml` are already here — see
+[`docs/05-metallb.md`](05-metallb.md) and
+[`docs/06-longhorn.md`](06-longhorn.md) right after this one.
 
 ## Why `apps/` isn't recursive
 
 Look at [`argocd/app-of-apps/app-of-apps.yaml`](../argocd/app-of-apps/app-of-apps.yaml)
 — specifically `directory.recurse: false`. The `apps/` folder contains both
-Application manifests (`apps/demo.yaml`, `apps/metallb.yaml`) *and*
+Application manifests (`apps/demo.yaml`, `apps/metallb.yaml`,
-subfolders holding the real workload manifests those Applications point to
+`apps/longhorn.yaml`) *and* subfolders holding the real workload manifests
-(`apps/demo/*.yaml`, `apps/metallb/*`). Non-recursive means the
+those Applications point to (`apps/demo/*.yaml`, `apps/metallb/*`,
-app-of-apps only ever looks at files directly inside `apps/`, never inside
+`apps/longhorn/*`). Non-recursive means the app-of-apps only ever looks at
-its subfolders — so it only ever sees Application definitions, and leaves
+files directly inside `apps/`, never inside its subfolders — so it only
-the actual workload manifests to the Application that owns them. Without
+ever sees Application definitions, and leaves the actual workload
-this, the app-of-apps would try to apply raw Deployments and Helm chart
+manifests to the Application that owns them. Without this, the
-files itself, duplicating and conflicting with what those Applications are
+app-of-apps would try to apply raw Deployments and Helm chart files
+itself, duplicating and conflicting with what those Applications are
 already doing.
 
 ## Every Application needs its own `repoURL`
@@ -59,19 +62,19 @@ that has it, not just the one you apply by hand:
 grep -rl YOUR_GITHUB_USERNAME .
 ```
 
-You should see three files: `argocd/app-of-apps/app-of-apps.yaml`,
+You should see four files: `argocd/app-of-apps/app-of-apps.yaml`,
-`apps/demo.yaml`, and `apps/metallb.yaml`. Edit `spec.source.repoURL` in
+`apps/demo.yaml`, `apps/metallb.yaml`, and `apps/longhorn.yaml`. Edit
-each to point at your fork.
+`spec.source.repoURL` in each to point at your fork.
 
 ## Steps
 
-1. Fix `repoURL` in all three files (above).
+1. Fix `repoURL` in all four files (above).
 2. Commit and push — this part matters more than it looks: the
-   app-of-apps reads `apps/demo.yaml` and `apps/metallb.yaml` *from Git*,
+   app-of-apps reads `apps/demo.yaml`, `apps/metallb.yaml`, and
-   not from your local edits, so they have to be pushed before you apply
+   `apps/longhorn.yaml` *from Git*, not from your local edits, so they
-   anything.
+   have to be pushed before you apply anything.
    ```sh
-   git add argocd/app-of-apps/app-of-apps.yaml apps/demo.yaml apps/metallb.yaml
+   git add argocd/app-of-apps/app-of-apps.yaml apps/demo.yaml apps/metallb.yaml apps/longhorn.yaml
    git commit -m "Point Applications at my fork"
    git push
    ```
@@ -79,13 +82,15 @@ each to point at your fork.
    ```sh
    kubectl apply -f argocd/app-of-apps/app-of-apps.yaml
    ```
-4. Watch it sync, then watch `demo` and `metallb` appear underneath it:
+4. Watch it sync, then watch `demo`, `metallb`, and `longhorn` appear
+   underneath it:
    ```sh
    argocd app list
    ```
-   You should see `app-of-apps`, `demo`, and `metallb`, all `Synced` and
+   You should see `app-of-apps`, `demo`, `metallb`, and `longhorn`, all
-   `Healthy` within a minute or two (MetalLB's own install takes a little
+   `Synced` and `Healthy` within a few minutes (MetalLB and Longhorn's own
-   longer than the demo app).
+   installs take longer than the demo app — Longhorn especially, since it
+   has to find and prepare a disk on every node).
 5. Confirm the demo app is actually running:
    ```sh
    kubectl -n demo get pods,svc
@@ -95,23 +100,23 @@ each to point at your fork.
    hands those out real addresses on bare metal until MetalLB is
    configured, which is exactly the next doc.
 
-If `app-of-apps`, `demo`, and `metallb` all show `Synced`/`Healthy`, the
+If `app-of-apps`, `demo`, `metallb`, and `longhorn` all show
-loop — Git → Argo CD → cluster — is already proven end to end; you just
+`Synced`/`Healthy`, the loop — Git → Argo CD → cluster — is already proven
-can't see the demo app in a browser yet.
+end to end; you just can't see the demo app in a browser yet.
 
 > **Ask Claude:** "walk me through exactly what happened, in order, from the
 > moment I ran `kubectl apply -f argocd/app-of-apps/app-of-apps.yaml` to
-> these three Applications existing." Making sure you can narrate that
+> these four Applications existing." Making sure you can narrate that
 > chain yourself is the real checkpoint for this doc — everything from
 > here on is a variation on it.
 
 ## Checklist
 
-- [ ] `repoURL` fixed in all three files listed above
+- [ ] `repoURL` fixed in all four files listed above
 - [ ] Fix committed and pushed
 - [ ] `app-of-apps` applied and shows `Synced`/`Healthy`
-- [ ] `demo` and `metallb` Applications appeared automatically, both
+- [ ] `demo`, `metallb`, and `longhorn` Applications appeared
-      `Synced`/`Healthy`
+      automatically, all `Synced`/`Healthy`
 - [ ] `kubectl -n demo get pods,svc` shows the demo app's Pod running
       (Service `EXTERNAL-IP` still `<pending>` — expected for now)
 

docs/05-metallb.md

@@ -34,7 +34,7 @@ Helm chart that does two things at once (see
 
 - Pulls in the official upstream MetalLB chart as a dependency — this is
   also the real, working example of a Helm-based app mentioned in
-  [`docs/07-adding-new-apps.md`](07-adding-new-apps.md).
+  [`docs/08-adding-new-apps.md`](08-adding-new-apps.md).
 - Carries two templates of its own
   ([`apps/metallb/templates/`](../apps/metallb/templates)): an
   `IPAddressPool` and an `L2Advertisement` — MetalLB's own custom
@@ -152,4 +152,4 @@ the only place this address gets recorded at all.
 - [ ] `kubectl -n demo get svc demo` shows a real `EXTERNAL-IP`, page loads in a browser
 - [ ] `argocd-server` patched to `LoadBalancer`, reachable at its own address without port-forwarding
 
-Continue to [`docs/06-your-first-change.md`](06-your-first-change.md).
+Continue to [`docs/06-longhorn.md`](06-longhorn.md).

docs/06-longhorn.md

@@ -0,0 +1,100 @@
+# 06 — Longhorn: real, replicated storage
+
+The app-of-apps you applied in [`docs/04-app-of-apps.md`](04-app-of-apps.md)
+already brought up `longhorn` alongside `demo` and `metallb`. This doc
+explains what it does and how to actually use it — there's no address to
+pick this time, unlike MetalLB; the interesting decisions already happened
+back in [`docs/02-talos-bootstrap.md`](02-talos-bootstrap.md).
+
+## Why this is needed at all
+
+A Kubernetes Pod's own filesystem disappears the moment that Pod does —
+fine for a stateless web server, useless for anything that needs to keep
+data around (a database, uploaded files, anything you'd be upset to lose).
+Kubernetes' answer is a **PersistentVolumeClaim (PVC)**: an app asks for
+some amount of storage, and something has to actually provide it. On a
+cloud provider, that something is the cloud's own disk service. On bare
+metal — your three Talos nodes — nothing provides that on its own, the
+same gap MetalLB fills for IP addresses rather than storage.
+
+**Longhorn** is that provider: it turns a plain disk on each node into one
+pool of storage, and — because your cluster has three nodes — it
+replicates each volume across more than one of them. Lose a node, and the
+data backing any volume that had a replica elsewhere is still there. This
+is also why Longhorn is genuinely a **three-node** story: with one node,
+"replicated storage" has nothing to replicate *to*.
+
+## What's already running
+
+**`apps/longhorn.yaml`** → one Application, pointing at `apps/longhorn`, a
+Helm chart that pulls in the official upstream Longhorn chart (see
+[`apps/longhorn/Chart.yaml`](../apps/longhorn/Chart.yaml)) — the same
+umbrella-chart pattern as MetalLB, just without any extra templates of its
+own this time.
+
+The interesting configuration already happened at the Talos layer, back in
+[`talos/patch-longhorn.yaml`](../talos/patch-longhorn.yaml): each node
+found, formatted, and mounted its own ≥900GB disk at
+`/var/mnt/longhorn`. [`apps/longhorn/values.yaml`](../apps/longhorn/values.yaml)
+points Longhorn's `defaultDataPath` at that exact path — this is the thing
+that makes Longhorn use *that* disk instead of carving space out of each
+node's root filesystem (small, and on Talos, not meant for this kind of
+data anyway).
+
+## Verify it's actually working
+
+```sh
+kubectl -n longhorn-system get pods
+```
+
+Expect a `longhorn-manager` and `longhorn-csi-plugin` Pod on each of your
+three nodes, plus one `instance-manager` per node once a volume exists,
+all `Running`.
+
+```sh
+kubectl get storageclass
+```
+
+Longhorn's chart creates a default `StorageClass` called `longhorn`
+automatically — this is what makes it the thing Kubernetes reaches for
+whenever an app asks for storage without naming one specifically.
+
+## Try it: give the demo app real storage
+
+The demo app's HTML currently lives in a ConfigMap (see
+[`docs/07-your-first-change.md`](07-your-first-change.md)) — fine for a
+few lines of text, but not how a real app would store real data. As a
+hands-on check that Longhorn actually works, ask Claude to help you:
+
+1. Add a small `PersistentVolumeClaim` to `apps/demo/`, requesting a
+   couple of gigabytes with no `storageClassName` set (so it uses
+   Longhorn's default).
+2. Mount it into the demo Deployment somewhere harmless, like
+   `/usr/share/nginx/html/data`.
+3. Push it, let Argo CD sync, and confirm:
+   ```sh
+   kubectl -n demo get pvc
+   ```
+   should show your claim `Bound`, and
+   ```sh
+   kubectl -n longhorn-system get volumes
+   ```
+   should show a matching volume, `healthy`, with more than one replica.
+
+This isn't part of the checklist below — treat it as an optional exercise
+once the pre-installed pieces are confirmed working, not a requirement to
+move on.
+
+> **Ask Claude:** "what would happen to that PVC's data if I lost one of
+> my three nodes right now?" — worth being able to answer before you trust
+> this with anything real.
+
+## Checklist
+
+- [ ] `kubectl -n longhorn-system get pods` shows manager/CSI Pods
+      `Running` on all three nodes
+- [ ] `kubectl get storageclass` shows a default `longhorn` StorageClass
+- [ ] (Optional) Gave the demo app a real PVC and confirmed it's `Bound`
+      with a healthy, replicated Longhorn volume behind it
+
+Continue to [`docs/07-your-first-change.md`](07-your-first-change.md).

docs/07-your-first-change.md

@@ -1,4 +1,4 @@
-# 06 — Your first change
+# 07 — Your first change
 
 Everything so far proved Argo CD can deploy what's in Git. This doc proves
 the part that actually matters day to day: changing what's in Git and
@@ -90,5 +90,5 @@ for the loop that just ran.
 - [ ] Saw the new text in the browser
 - [ ] (Optional) Watched self-healing revert a hand-made change
 
-Continue to [`docs/07-adding-new-apps.md`](07-adding-new-apps.md) to add
+Continue to [`docs/08-adding-new-apps.md`](08-adding-new-apps.md) to add
 something of your own.

docs/08-adding-new-apps.md

@@ -1,4 +1,4 @@
-# 07 — Adding a new app
+# 08 — Adding a new app
 
 This is the payoff for the app-of-apps structure: adding something new is
 "drop a file here," not a special procedure.
@@ -32,8 +32,13 @@ To add a new app called, say, `whoami`:
    `whoami.yaml`, creates the `whoami` Application, which in turn deploys
    everything in `apps/whoami/`.
 
-4. Confirm: `argocd app list` should show `whoami` alongside `demo` and
+4. Confirm: `argocd app list` should show `whoami` alongside `demo`,
-   `metallb`.
+   `metallb`, and `longhorn`.
+
+If your new app needs storage, request it with a plain
+`PersistentVolumeClaim` and no `storageClassName` — Longhorn's chart set
+itself as the cluster's default StorageClass, so it's used automatically.
+See [`docs/06-longhorn.md`](06-longhorn.md).
 
 ## Manifest-based vs. Helm-based apps
 
@@ -83,5 +88,5 @@ address shows up in it.
 - [ ] `argocd app list` shows the new app, `Synced`/`Healthy`
 - [ ] If it has a `LoadBalancer` Service, its address is recorded in `docs/network.md`
 
-Continue to [`docs/08-growing-up.md`](08-growing-up.md) for what to add
+Continue to [`docs/09-growing-up.md`](09-growing-up.md) for what to add
 next as this cluster grows beyond a learning setup.

docs/09-growing-up.md

@@ -1,4 +1,4 @@
-# 08 — Growing up: what to add next
+# 09 — Growing up: what to add next
 
 Everything up to here is deliberately minimal — enough to understand the
 whole loop, nothing more. This doc is a map of the natural next steps, each
@@ -7,26 +7,43 @@ for you; each is its own project once you're ready.
 
 ## More nodes
 
-Right now, one machine is both control plane and worker
+Your three nodes already give you real resilience — losing any one still
-(`allowSchedulingOnControlPlanes: true` in
+leaves a majority of control-plane voters (see
-[`talos/patch-controlplane.yaml`](../talos/patch-controlplane.yaml)). That's
+[`docs/02-talos-bootstrap.md`](02-talos-bootstrap.md#three-nodes-each-both-control-plane-and-worker))
-a single point of failure: if that machine dies, the cluster is gone.
+and, once Longhorn has real volumes, a replica of your data likely
+survives too. Growing further is about *capacity*, not fault tolerance you
+don't already have — so beyond this point, add plain **worker** nodes, not
+more control-plane ones:
 
-To grow beyond it:
+1. Boot a new machine from the same custom Talos ISO as the first three
-
+   (see [`docs/01-prerequisites.md`](01-prerequisites.md)).
-1. Boot a second machine from the Talos ISO, same as the first.
+2. Apply the **worker** config instead of the control-plane one — you
-2. Generate a **worker** config instead of a control-plane one — you
    already have `_out/worker.yaml` from
-   [`docs/02-talos-bootstrap.md`](02-talos-bootstrap.md), unused until now.
+   [`docs/02-talos-bootstrap.md`](02-talos-bootstrap.md), unused until now:
-3. `talosctl apply-config --insecure --nodes <NEW_NODE_IP> --file _out/worker.yaml`
+   ```sh
-4. `kubectl get nodes` should show two nodes shortly after.
+   talosctl apply-config --insecure --nodes <NEW_NODE_IP> --file _out/worker.yaml
+   ```
+3. `kubectl get nodes` should show a fourth node shortly after, without
+   the `control-plane` role.
 
-For real resilience against a control-plane node failing (not just adding
+### Why not a fourth control-plane node
-worker capacity), you'd want an odd number of control-plane nodes — 3 is
+
-the normal minimum — because etcd (the cluster's data store) uses majority
+It's tempting to reach for a fourth control-plane node when you're growing
-voting to agree on state, and majority voting needs an odd number to avoid
+a cluster you already made resilient with three — but it actually makes
-ties. That's a bigger step than this doc covers in detail; ask Claude when
+things worse, not better. etcd (the control-plane database) needs a
-you're actually ready to take it.
+**majority** of its members to agree before it can do anything, and
+majority only means something clean with an odd number: with 3 members, 1
+can fail and 2 (a majority) still agree — you can lose one. With 4, you
+still need 3 to agree, so you can *still* only lose one — you paid for an
+extra machine and gained nothing. Worse, 4 is *more* likely to have
+exactly a 2-2 split during a network hiccup, which stalls the cluster
+until it's resolved. If you ever do want more control-plane fault
+tolerance than three nodes gives you, the next meaningful number is 5 —
+not 4.
+
+> **Ask Claude:** "walk me through the quorum math for 3 vs. 4 vs. 5
+> control-plane nodes" if you want to see this worked out concretely
+> rather than taking it on faith.
 
 ## Ingress
 
@@ -40,7 +57,7 @@ so big, and there's no way to route by hostname (`app1.example.com` vs
 
 An **Ingress controller** is itself just another app you'd deploy the same
 way as anything else in this repo (see
-[`docs/07-adding-new-apps.md`](07-adding-new-apps.md)) — commonly as a
+[`docs/08-adding-new-apps.md`](08-adding-new-apps.md)) — commonly as a
 Helm-based app, since most ingress controllers are distributed that way,
 same as MetalLB. It typically sits behind one single MetalLB
 `LoadBalancer` Service (one IP, total, for every app), and you add an

docs/checklist.md

@@ -1,25 +1,27 @@
 # Guided bootstrap checklist
 
 A single-page, ordered checklist covering the entire path from bare
-machine to a working GitOps loop. It's meant to be worked through **with
+machines to a working GitOps loop. It's meant to be worked through **with
 Claude**, one item at a time — ask Claude to walk you through this list,
 and it will run each step with you, wait for your output, confirm it
 worked, and only then move to the next one. See `CLAUDE.md` at the repo
 root for how Claude is primed to do this.
 
 Each item names the doc it's drawn from, in case you want the full
-explanation before or after running it. Replace `<NODE_IP>`,
+explanation before or after running it. Replace `<NODE1_IP>`/`<NODE2_IP>`/`<NODE3_IP>`,
-`YOUR_GITHUB_USERNAME`, and the MetalLB IP addresses with your real values
+`<VIP_IP>`, `YOUR_GITHUB_USERNAME`, and the MetalLB IP addresses with your
-throughout — Claude will remind you if you paste a command still
+real values throughout — Claude will remind you if you paste a command
-containing a placeholder, and will ask you for the IP addresses directly
+still containing a placeholder, and will ask you for all of these directly
-in Phase 5 rather than guessing them.
+in Phases 2 and 5 rather than guessing them.
 
 ## Phase 1 — Prerequisites ([`01`](01-prerequisites.md))
 
-- [ ] Target machine chosen, reachable on the network, with a stable
+- [ ] Three target machines chosen, each network-reachable, each with a
-      (ideally DHCP-reserved) IP address
+      stable (ideally DHCP-reserved) IP address, each with a second disk
-- [ ] Talos amd64 ISO downloaded, machine booted from it, sitting at the
+      ≥900GB set aside for Longhorn
-      boot screen
+- [ ] Custom Talos schematic built at factory.talos.dev with `iscsi-tools`
+      and `util-linux-tools`, ISO downloaded
+- [ ] All three machines booted from that ISO, sitting at the boot screen
 - [ ] `talosctl version --client` runs successfully
 - [ ] `kubectl version --client` runs successfully
 - [ ] `argocd version --client` runs successfully
@@ -27,12 +29,16 @@ in Phase 5 rather than guessing them.
 
 ## Phase 2 — Talos bootstrap ([`02`](02-talos-bootstrap.md), [`talos/README.md`](../talos/README.md))
 
-- [ ] `talosctl gen config gitops-starter https://<NODE_IP>:6443 --output-dir _out --config-patch-control-plane @talos/patch-controlplane.yaml` run — `_out/controlplane.yaml`, `worker.yaml`, `talosconfig` exist
+- [ ] **Ask Claude** to help pick: your three nodes' IP addresses (or
+      confirm DHCP reservations) and a separate VIP address — all based on
+      your actual LAN/DHCP setup
+- [ ] `talosctl gen config gitops-starter https://<VIP_IP>:6443 --output-dir _out --config-patch-control-plane @talos/patch-controlplane.yaml --config-patch-control-plane @talos/patch-longhorn.yaml` run — `_out/controlplane.yaml`, `worker.yaml`, `talosconfig` exist
 - [ ] `TALOSCONFIG` exported (or `talosctl config endpoint`/`node` set) to `_out/talosconfig`
-- [ ] `talosctl apply-config --insecure --nodes <NODE_IP> --file _out/controlplane.yaml` run, node rebooted
+- [ ] For **each** of the three nodes: `talosctl get disks --insecure --nodes <NODE_IP>` confirms two disks, then `talosctl apply-config --insecure --nodes <NODE_IP> --file _out/controlplane.yaml` run, node rebooted
-- [ ] `talosctl bootstrap --nodes <NODE_IP>` run — **exactly once**
+- [ ] `talosctl bootstrap --nodes <NODE1_IP>` run — **exactly once**, against **exactly one** node
-- [ ] `talosctl kubeconfig --nodes <NODE_IP> .` run, `KUBECONFIG` pointed at the result
+- [ ] `talosctl kubeconfig --nodes <VIP_IP> .` run, `KUBECONFIG` pointed at the result
-- [ ] `kubectl get nodes` shows one node, `Ready`
+- [ ] `kubectl get nodes` shows **all three** nodes, `Ready`
+- [ ] `docs/network.md` updated with all three node addresses and the VIP
 
 ## Phase 3 — Argo CD install ([`03`](03-installing-argocd.md), [`argocd/README.md`](../argocd/README.md))
 
@@ -45,13 +51,15 @@ in Phase 5 rather than guessing them.
 
 ## Phase 4 — Hand the cluster to Git ([`04`](04-app-of-apps.md))
 
-- [ ] `spec.source.repoURL` fixed in all three placeholder files: run
+- [ ] `spec.source.repoURL` fixed in all four placeholder files: run
       `grep -rl YOUR_GITHUB_USERNAME .` to find them
-      (`argocd/app-of-apps/app-of-apps.yaml`, `apps/demo.yaml`, `apps/metallb.yaml`)
+      (`argocd/app-of-apps/app-of-apps.yaml`, `apps/demo.yaml`,
+      `apps/metallb.yaml`, `apps/longhorn.yaml`)
 - [ ] Edit committed and pushed to your fork's `main` branch
 - [ ] `kubectl apply -f argocd/app-of-apps/app-of-apps.yaml`
 - [ ] `argocd app list` shows `app-of-apps`, `Synced`/`Healthy`
-- [ ] `argocd app list` shows `demo` and `metallb`, appeared automatically, both `Synced`/`Healthy`
+- [ ] `argocd app list` shows `demo`, `metallb`, and `longhorn`, appeared
+      automatically, all `Synced`/`Healthy`
 - [ ] `kubectl -n demo get pods,svc` shows the demo app's Pod running
       (Service `EXTERNAL-IP` still `<pending>` — expected for now)
 
@@ -68,7 +76,15 @@ in Phase 5 rather than guessing them.
 - [ ] `argocd-server` Service patched to `LoadBalancer` with your chosen Argo CD address (`kubectl -n argocd patch svc argocd-server ...`)
 - [ ] Argo CD UI reachable at `https://<that-address>` with no port-forward running
 
-## Phase 6 — Prove the loop ([`06`](06-your-first-change.md))
+## Phase 6 — Longhorn: real storage ([`06`](06-longhorn.md))
+
+- [ ] `kubectl -n longhorn-system get pods` shows manager/CSI Pods
+      `Running` on all three nodes
+- [ ] `kubectl get storageclass` shows a default `longhorn` StorageClass
+- [ ] (Optional) Gave the demo app a real PVC and confirmed it's `Bound`
+      with a healthy, replicated Longhorn volume behind it
+
+## Phase 7 — Prove the loop ([`07`](07-your-first-change.md))
 
 - [ ] Edited the `<h1>` text in `apps/demo/configmap.yaml`
 - [ ] Committed and pushed
@@ -85,6 +101,6 @@ push" and the page updating. If you can't yet, that's the sign to ask
 Claude for a walkthrough of whichever phase felt shaky before calling this
 finished.
 
-From here: [`07-adding-new-apps.md`](07-adding-new-apps.md) to add
+From here: [`08-adding-new-apps.md`](08-adding-new-apps.md) to add
-something of your own, or [`08-growing-up.md`](08-growing-up.md) for the
+something of your own, or [`09-growing-up.md`](09-growing-up.md) for the
 bigger next steps (more nodes, ingress, secrets management).

docs/glossary.md

@@ -29,7 +29,7 @@ steps — installing Argo CD, applying the app-of-apps — that get the
 GitOps loop started before Git takes over.
 
 **cluster** — A group of one or more machines (nodes) working together as
-one Kubernetes system. This repo's cluster starts as a single node.
+one Kubernetes system. This repo's cluster starts as three nodes.
 
 **CNI (Container Network Interface)** — The plugin responsible for giving
 Pods network connectivity and IP addresses. Talos ships Flannel, a CNI, by
@@ -42,9 +42,9 @@ demo app's HTML page.
 
 **control plane** — The set of components that run the cluster itself
 (API server, scheduler, etcd) as opposed to your applications. Normally
-kept on separate machines from your workloads; this repo's single-node
+kept on separate machines from your workloads; this repo's three nodes
-setup deliberately blurs that line for simplicity — see
+deliberately blur that line, each acting as control plane and worker at
-[`talos/patch-controlplane.yaml`](../talos/patch-controlplane.yaml).
+once — see [`talos/patch-controlplane.yaml`](../talos/patch-controlplane.yaml).
 
 **CRD (Custom Resource Definition)** — A way to teach Kubernetes about a
 new *kind* of object beyond its built-ins (Pod, Service, etc.). Argo CD's
@@ -74,13 +74,19 @@ the real system to match it. The core idea this whole repo demonstrates.
 **Ingress** — A Kubernetes object that routes external HTTP(S) traffic to
 services inside the cluster by hostname/path, via an Ingress controller.
 Not used in this repo's v1 — see
-[`docs/08-growing-up.md`](08-growing-up.md).
+[`docs/09-growing-up.md`](09-growing-up.md).
 
 **IPAddressPool** — A MetalLB custom resource (see CRD) declaring which IP
 addresses it's allowed to hand out to `LoadBalancer` Services. Configured
 in [`apps/metallb/templates/ipaddresspool.yaml`](../apps/metallb/templates/ipaddresspool.yaml)
 — see [`docs/05-metallb.md`](05-metallb.md).
 
+**iSCSI** — A protocol for accessing block storage over a network as if it
+were a local disk. Longhorn's underlying engine uses it, which is why
+nodes need the `iscsi-tools` Talos system extension (see below) baked
+into their image before Longhorn will work — see
+[`docs/01-prerequisites.md`](01-prerequisites.md).
+
 **kubeconfig** — A file holding the credentials and connection info
 `kubectl` (and other Kubernetes tools) use to talk to a specific cluster.
 Retrieved with `talosctl kubeconfig` in this setup.
@@ -96,6 +102,12 @@ cloud provider this provisions an actual cloud load balancer; on bare
 metal (this repo) MetalLB fills that role instead — see
 [`docs/05-metallb.md`](05-metallb.md).
 
+**Longhorn** — Software installed in this repo (`apps/longhorn.yaml`) that
+turns a plain disk on each node into one pool of storage, replicated
+across nodes, so apps can request real persistent storage on bare metal —
+a job a cloud provider's disk service would normally do. See
+[`docs/06-longhorn.md`](06-longhorn.md).
+
 **machine config** — Talos's equivalent of a Kubernetes manifest, but for
 the operating system itself: a declarative file describing how a node
 should be configured, generated with `talosctl gen config` and sent to the
@@ -116,19 +128,37 @@ independently.
 **node** — A single machine (physical or virtual) that's part of a
 Kubernetes cluster.
 
+**PersistentVolumeClaim (PVC)** — How an app asks Kubernetes for storage:
+"I need this much space," without caring which disk or which node
+provides it. Longhorn is what actually fulfills that request in this
+repo — see [`docs/06-longhorn.md`](06-longhorn.md).
+
 **Pod** — The smallest deployable unit in Kubernetes: one or more
 containers that run together, sharing network and storage. You'll rarely
 create Pods directly — a Deployment creates and manages them for you.
 
+**quorum** — The minimum number of etcd members (a majority) that have to
+agree before the cluster's control plane can do anything. This is why
+control-plane node counts matter, and why 3 is a meaningfully different
+number than 2 or 4 — see
+[`docs/02-talos-bootstrap.md`](02-talos-bootstrap.md#three-nodes-each-both-control-plane-and-worker)
+and [`docs/09-growing-up.md`](09-growing-up.md#why-not-a-fourth-control-plane-node).
+
 **reconciliation** — The repeating process of comparing desired state
 (Git, for Argo CD) against actual state (the live cluster) and correcting
 any difference. See the diagram in
 [`docs/00-what-is-all-this.md`](00-what-is-all-this.md#the-reconciliation-loop).
 
+**schematic** — A custom Talos boot image, built at
+[factory.talos.dev](https://factory.talos.dev) by selecting which system
+extensions (see below) it should include. This repo's nodes need one with
+`iscsi-tools` and `util-linux-tools` for Longhorn to work — see
+[`docs/01-prerequisites.md`](01-prerequisites.md).
+
 **Secret** — A Kubernetes object for holding sensitive data. Its values are
 only base64-*encoded*, not encrypted — never commit a raw Secret manifest
 containing real credentials to Git. See
-[`docs/08-growing-up.md`](08-growing-up.md) for how to do this safely.
+[`docs/09-growing-up.md`](09-growing-up.md) for how to do this safely.
 
 **selfHeal** — An Argo CD sync policy setting: if the live cluster drifts
 from what's in Git (e.g. someone runs `kubectl edit`), Argo CD reverts it
@@ -141,6 +171,12 @@ uses a `LoadBalancer` Service for the demo app, backed by MetalLB — see
 [`apps/demo/service.yaml`](../apps/demo/service.yaml) and
 [`docs/05-metallb.md`](05-metallb.md).
 
+**StorageClass** — A Kubernetes object describing a "kind" of storage
+that's available (who provides it, how). Longhorn's chart registers itself
+as the cluster's default StorageClass, so a plain
+`PersistentVolumeClaim` with no class named gets Longhorn storage
+automatically — see [`docs/06-longhorn.md`](06-longhorn.md).
+
 **sync** — Argo CD's term for actually applying a change (a reconciliation
 pass that found and fixed a difference). "Synced" means the cluster
 currently matches Git.
@@ -153,9 +189,27 @@ waits for a wave to be healthy before starting the next. Used in
 apply before the `IPAddressPool`/`L2Advertisement` that depend on them —
 see [`docs/05-metallb.md`](05-metallb.md).
 
-**Talos Linux** — The operating system this repo's node(s) run: built only
+**system extension** — An optional add-on baked into a Talos boot image at
+build time (see "schematic," above), since Talos has no package manager to
+install one onto an already-running node. This repo's nodes need
+`iscsi-tools` and `util-linux-tools` for Longhorn.
+
+**Talos Linux** — The operating system this repo's nodes run: built only
 to run Kubernetes, configured entirely declaratively over the network, with
 no SSH or shell. See [`docs/02-talos-bootstrap.md`](02-talos-bootstrap.md).
 
 **talosconfig** — The credentials file `talosctl` uses to talk to a Talos
 node's API — the Talos-level equivalent of a kubeconfig.
+
+**UserVolumeConfig** — A standalone Talos configuration resource (not part
+of the normal `machine:`/`cluster:` structure) that tells Talos to find,
+format, and mount a disk on its own, based on rules you give it (which
+disk to pick, minimum size). Used in
+[`talos/patch-longhorn.yaml`](../talos/patch-longhorn.yaml) to prepare
+each node's storage for Longhorn.
+
+**VIP (Virtual IP)** — One IP address that Talos keeps pointed at whichever
+of several control-plane nodes is currently healthy, re-electing within
+seconds if that node goes down. Everything that talks to this cluster
+should use the VIP, not any one node's own address — see
+[`docs/02-talos-bootstrap.md`](02-talos-bootstrap.md#the-vip-one-address-for-a-cluster-that-moves).

docs/network.md

@@ -13,6 +13,21 @@ assigned or changed, so you (or Claude) never have to reverse-engineer
 > instruction that tells Claude to do this on its own when it makes such a
 > change itself.
 
+## The three nodes and the VIP
+
+None of these are from the MetalLB pool below — they're the machines
+themselves and the shared address Talos elects between them. Set in
+[`talos/README.md`](../talos/README.md); see
+[`docs/02-talos-bootstrap.md`](02-talos-bootstrap.md) for what the VIP is
+and why it's separate from any one node's address.
+
+| Address | What | Port | URL |
+|---|---|---|---|
+| `EDIT ME` | Node 1 | 6443 | `https://EDIT ME:6443` |
+| `EDIT ME` | Node 2 | 6443 | `https://EDIT ME:6443` |
+| `EDIT ME` | Node 3 | 6443 | `https://EDIT ME:6443` |
+| `EDIT ME` | Cluster VIP (use this one, not a node's own address) | 6443 | `https://EDIT ME:6443` |
+
 ## MetalLB pool
 
 | | |
@@ -20,16 +35,15 @@ assigned or changed, so you (or Claude) never have to reverse-engineer
 | Range | `EDIT ME — e.g. 192.168.1.240-192.168.1.250` |
 | Configured in | [`apps/metallb/templates/ipaddresspool.yaml`](../apps/metallb/templates/ipaddresspool.yaml) |
 
-## Allocated addresses
+## Allocated addresses (from the MetalLB pool)
 
 | Address | What | Namespace | Port(s) | URL | Source |
 |---|---|---|---|---|---|
-| `<NODE_IP>` | Talos node — Kubernetes API server | — | 6443 | `https://<NODE_IP>:6443` | Not from the MetalLB pool — the node's own address, set in [`talos/README.md`](../talos/README.md) |
 | `EDIT ME` | Argo CD UI/API (`argocd-server`) | `argocd` | 443 | `https://EDIT ME` | Patched to `LoadBalancer` by hand — see [`docs/05-metallb.md`](05-metallb.md) |
 | `EDIT ME` | Demo app | `demo` | 80 | `http://EDIT ME` | [`apps/demo/service.yaml`](../apps/demo/service.yaml) |
 
 Add a row here every time you add a new app with a `LoadBalancer` Service
-(see [`docs/07-adding-new-apps.md`](07-adding-new-apps.md)). Leave the URL
+(see [`docs/08-adding-new-apps.md`](08-adding-new-apps.md)). Leave the URL
 column's port number off only when it's the protocol default (80 for
 `http://`, 443 for `https://`) — include it otherwise.
 

talos/README.md

@@ -5,64 +5,81 @@ This is the detailed, copy-pasteable companion to
 first for the explanations — this one is here so you have every exact
 command in one place once you understand what they do.
 
-Throughout, replace `<NODE_IP>` with your machine's real IP address. If it's
+Throughout, replace `<NODE1_IP>`, `<NODE2_IP>`, `<NODE3_IP>` with your three
-a VM, give it a DHCP reservation (a fixed IP tied to its MAC address) in your
+machines' real IP addresses, and `<VIP_IP>` with the shared address you
-router or hypervisor before you start, so the address never changes under
+picked (with Claude's help) for the cluster's VIP — see
-the cluster. Talos bakes the IP into certificates during setup, and changing
+[`docs/02-talos-bootstrap.md`](../docs/02-talos-bootstrap.md#the-vip-one-address-for-a-cluster-that-moves).
-it later means starting over.
+Give each machine a DHCP reservation (a fixed IP tied to its MAC address)
+in your router or hypervisor before you start, so none of these addresses
+change under the cluster. Talos bakes them into certificates during setup,
+and changing one later means starting over for that node.
+
+Steps 1–2 below run **once, total**. Steps 3 runs **once per node** (three
+times). Steps 4–6 run **once, total**, against whichever node you like
+(or, once it exists, the VIP).
 
 All commands assume you're running them from the root of this repo.
 
 ## 1. Generate machine configuration
 
 ```sh
-talosctl gen config gitops-starter https://<NODE_IP>:6443 \
+talosctl gen config gitops-starter https://<VIP_IP>:6443 \
   --output-dir _out \
-  --config-patch-control-plane @talos/patch-controlplane.yaml
+  --config-patch-control-plane @talos/patch-controlplane.yaml \
+  --config-patch-control-plane @talos/patch-longhorn.yaml
 ```
 
+Note the cluster endpoint here is the **VIP**, not any one node's address
+— every node needs to agree in advance on where the cluster "lives," and
+that address has to be the one that keeps working no matter which node is
+currently answering.
+
 This writes three files into `_out/` (already covered by this repo's
 `.gitignore` — they contain private keys and must never be committed):
 
-- `controlplane.yaml` — the full machine configuration for a control-plane
+- `controlplane.yaml` — the machine configuration for a control-plane
-  node, with our single-node patch baked in.
+  node, with both patches baked in. Applied identically to all three
-- `worker.yaml` — a worker machine configuration. Unused for now (single
+  nodes — there's no separate worker config in this setup, since every
-  node), kept for when you follow docs/08-growing-up.md.
+  node is both.
+- `worker.yaml` — a plain worker config, generated automatically but
+  unused here since every node in this repo's setup is both control plane
+  and worker. Kept around for [`docs/09-growing-up.md`](../docs/09-growing-up.md),
+  where a fourth-and-beyond node uses exactly this file.
 - `talosconfig` — *your* client credentials for talking to the Talos API on
   this cluster (equivalent in spirit to a kubeconfig, but for Talos itself
   rather than Kubernetes).
 
 `gitops-starter` is the cluster name — an arbitrary label baked into the
 generated certificates and config, used to tell clusters apart if you ever
-manage more than one. `https://<NODE_IP>:6443` is the cluster's Kubernetes
+manage more than one. Port 6443 is the standard Kubernetes API server port.
-API endpoint: port 6443 is the standard Kubernetes API server port, and this
-same address gets written into the certificates and your future kubeconfig.
 
 ## 2. Point talosctl at this cluster
 
 ```sh
 export TALOSCONFIG=_out/talosconfig
-talosctl config endpoint <NODE_IP>
+talosctl config endpoint <NODE1_IP> <NODE2_IP> <NODE3_IP>
-talosctl config node <NODE_IP>
 ```
 
-`endpoint` is which machine talosctl *talks to*. `node` is which machine a
+`endpoint` is which machine(s) talosctl is willing to *talk to* — giving
-command *applies to*. On a single-node cluster they're the same address, but
+it all three means it can still reach the cluster if any one of them is
-Talos always keeps the two concepts separate, because on a multi-node
+down. You'll pass `--nodes` per-command below for which machine a given
-cluster you often want to ask one node about another.
+command actually *applies to* — Talos always keeps those two concepts
+separate.
 
-## 3. Apply the configuration
+## 3. Apply the configuration — once per node
 
-Boot the target machine from the Talos ISO (see
+Boot each machine from your custom Talos ISO (see
-[docs/01-prerequisites.md](../docs/01-prerequisites.md) for where to get it).
+[docs/01-prerequisites.md](../docs/01-prerequisites.md)). Each boots into
-It boots into "maintenance mode" — alive enough to accept a configuration,
+"maintenance mode" — alive enough to accept a configuration, with nothing
-with nothing installed yet. Confirm it's reachable, then apply:
+installed yet. For **each** node in turn:
 
 ```sh
 talosctl get disks --insecure --nodes <NODE_IP>
 ```
 
-Note the disk name you want Talos installed to (e.g. `/dev/sda`). Then:
+Confirm you see two disks: the one Talos will install itself to, and the
+≥900GB one `talos/patch-longhorn.yaml`'s `UserVolumeConfig` will claim for
+Longhorn. Then:
 
 ```sh
 talosctl apply-config --insecure \
@@ -73,32 +90,37 @@ talosctl apply-config --insecure \
 `--insecure` is required for this one command only: the node has no
 certificates yet, so talosctl can't authenticate to it normally. This is
 the one moment during setup where you're trusting the network — do this on
-a network you control. The node will reboot, install Talos to disk, and
+a network you control. The node will reboot, install Talos to its system
-come back up running the configuration you gave it — now *with*
+disk, format and mount its second disk for Longhorn, and come back up
-certificates, so every command after this uses the secure path.
+running the configuration you gave it — now *with* certificates, so every
+command after this uses the secure path.
 
-## 4. Bootstrap etcd
+Repeat this for `<NODE1_IP>`, `<NODE2_IP>`, and `<NODE3_IP>` before moving on.
 
-Talos is installed and Kubernetes components are starting, but the cluster
+## 4. Bootstrap etcd — once, total
-has no data store yet. This one-time command initializes it:
+
+Talos is installed on all three nodes and Kubernetes components are
+starting, but the cluster has no data store yet. This one-time command
+initializes it:
 
 ```sh
-talosctl bootstrap --nodes <NODE_IP>
+talosctl bootstrap --nodes <NODE1_IP>
 ```
 
-Run this exactly once per cluster, against exactly one control-plane node,
+Run this **exactly once**, against **exactly one** of your three nodes —
-even if you later add more. Running it again on an existing cluster can
+even though all three will end up running etcd as peers, only one needs
-corrupt it.
+telling to *start* the cluster; the other two discover and join the
+existing one. Running it again against an existing cluster can corrupt it.
 
 ## 5. Get your kubeconfig
 
 ```sh
-talosctl kubeconfig --nodes <NODE_IP> .
+talosctl kubeconfig --nodes <VIP_IP> .
 ```
 
 Downloads a kubeconfig into the current directory (also gitignored —
-it grants full cluster access). Merge it with `~/.kube/config` or point
+it grants full cluster access), pointed at the VIP rather than any one
-`KUBECONFIG` at it directly:
+node. Merge it with `~/.kube/config` or point `KUBECONFIG` at it directly:
 
 ```sh
 export KUBECONFIG=$PWD/kubeconfig
@@ -110,14 +132,16 @@ export KUBECONFIG=$PWD/kubeconfig
 kubectl get nodes
 ```
 
-Expect to see one node, `Ready`, within a couple of minutes (it takes a
+Expect to see **three** nodes, `Ready`, within a couple of minutes (it
-moment for the default CNI — Talos ships Flannel out of the box — to come
+takes a moment for the default CNI — Talos ships Flannel out of the box —
-up and clear the node's networking checks).
+to come up and clear each node's networking checks).
 
 ```
 NAME              STATUS   ROLES           AGE   VERSION
 talos-...         Ready    control-plane   90s   v1.3x.x
+talos-...         Ready    control-plane   85s   v1.3x.x
+talos-...         Ready    control-plane   80s   v1.3x.x
 ```
 
-If it's stuck on `NotReady`, give it another minute before troubleshooting —
+If any node is stuck on `NotReady`, give it another minute before
-first boot does a lot of one-time work.
+troubleshooting — first boot does a lot of one-time work.

talos/patch-controlplane.yaml

@@ -1,15 +1,43 @@
 # A patch applied on top of the controlplane.yaml that `talosctl gen config`
 # produces (see talos/README.md and docs/02-talos-bootstrap.md for the full
-# command). Talos's generated defaults assume a multi-node cluster where
+# command). Applied identically to all three control-plane nodes.
-# control-plane nodes only run the Kubernetes control plane itself (API
-# server, scheduler, etcd, etc.) and never your actual workloads. For a
-# single-node learning cluster, that would mean nothing can ever be
-# scheduled — there's no other node to run it on.
 #
-# This one field undoes that restriction, letting the single node be both
+# --- allowSchedulingOnControlPlanes ---
-# the control plane AND a worker.
+# Talos's generated defaults assume control-plane nodes only run the
+# Kubernetes control plane itself (API server, scheduler, etcd, etc.) and
+# never your actual workloads — normal practice, so a workload crashing
+# never has a chance to take the control plane down with it. This repo's
+# three nodes are deliberately both control plane AND worker at once (no
+# separate worker nodes), to get real multi-node resilience without needing
+# more than three machines. This one field undoes the default restriction.
 #
 # >> Ask Claude: "why does Kubernetes normally keep workloads off control
-#    plane nodes?" if you want the reasoning before you turn it off.
+#    plane nodes, and what am I giving up by turning that off here?" if you
+#    want the reasoning before you turn it off.
+#
+# --- vip ---
+# With three control-plane nodes, "the cluster's API server" isn't one
+# fixed machine anymore — it's whichever of the three happen to be up.
+# Talos's VIP (Virtual IP) feature solves this: the three nodes
+# continuously elect one of themselves to answer on this one shared
+# address, and re-elect within seconds if that node goes down. Point
+# everything that talks to the cluster (talosctl, kubectl, this repo's
+# Application manifests) at the VIP, never at one node's own address —
+# that's what makes losing any single node a non-event instead of an
+# outage.
+#
+# Replace both the interface name and the IP below with your real values
+# before generating config — see docs/02-talos-bootstrap.md.
+#
+# >> Ask Claude to help you pick the VIP address — like the MetalLB pool in
+#    docs/05-metallb.md, it has to be a real, unused address on your LAN,
+#    outside your DHCP range.
 cluster:
   allowSchedulingOnControlPlanes: true
+machine:
+  network:
+    interfaces:
+      - interface: eth0 # EDIT ME — match your node's real NIC name (`talosctl get links --insecure --nodes <NODE_IP>` from maintenance mode will show it)
+        dhcp: true
+        vip:
+          ip: 192.168.1.239 # EDIT ME — replace with your chosen VIP address

talos/patch-longhorn.yaml

@@ -0,0 +1,60 @@
+# Prepares each node for Longhorn (apps/longhorn.yaml) — the storage layer
+# this repo ships pre-installed. Applied identically to all three
+# control-plane nodes, alongside patch-controlplane.yaml.
+#
+# This file has TWO documents (separated by `---`), because it's making two
+# different kinds of change: a patch to the normal machine config, and a
+# standalone Talos resource that doesn't live inside `machine:`/`cluster:`
+# at all.
+#
+# --- Document 1: mount the volume into the kubelet's view ---
+# Talos runs the kubelet in its own isolated mount namespace. Without this,
+# a volume mounted at /var/mnt/longhorn on the node itself would be
+# invisible to any Pod using a hostPath-style mount there — which is
+# exactly how Longhorn's own manager and engine Pods access node storage.
+# This bind-mounts that same path into the kubelet's namespace so Pods can
+# actually see it. `rshared` matters specifically: it lets mounts Longhorn
+# creates later *underneath* this path (one per volume) propagate back out
+# to the host, instead of being invisible outside the container that made
+# them.
+#
+# --- Document 2: UserVolumeConfig ---
+# A Talos UserVolumeConfig tells Talos itself to find, format, and mount a
+# disk — no manual partitioning on your part. `diskSelector.match:
+# '!system_disk'` means "any disk that isn't the one Talos is installed
+# on" (never touches your OS disk), and `minSize: 900GiB` means "only
+# consider disks at least that large" — so if a node has more than one
+# extra disk, Talos won't accidentally pick a small one. Adjust `minSize`
+# if your nodes' data disks are smaller than 900GiB; just make sure it's
+# still large enough to exclude anything that *isn't* meant for Longhorn.
+# `grow: false` keeps the volume at its initial size even if the
+# underlying disk is later resized — deliberate here, so growing storage
+# is always a conscious decision (see docs/09-growing-up.md).
+#
+# The `name: longhorn` below is what determines the mount path Talos uses
+# — user volumes are always mounted at /var/mnt/<name>, which is why
+# Document 1 above and apps/longhorn/values.yaml both hardcode
+# /var/mnt/longhorn. Renaming this would mean updating both.
+#
+# >> Ask Claude: "how do I find out which of my disks Talos will actually
+#    pick for this?" before you boot for real — worth confirming on your
+#    specific hardware rather than assuming.
+machine:
+  kubelet:
+    extraMounts:
+      - destination: /var/mnt/longhorn
+        type: bind
+        source: /var/mnt/longhorn
+        options:
+          - bind
+          - rshared
+          - rw
+---
+apiVersion: v1alpha1
+kind: UserVolumeConfig
+name: longhorn
+provisioning:
+  diskSelector:
+    match: "!system_disk"
+  minSize: 900GiB
+  grow: false