From 39e04e43d06efcfa2e81ff9b08b75d0d1d94ac39 Mon Sep 17 00:00:00 2001
From: ahmza <hamza@ahmza.com>
Date: Thu, 3 Sep 2026 03:28:13 +0100
Subject: [PATCH] Update Longhorn docs for two-worker topology

---
 docs/06-longhorn.md | 64 +++++++++++++++++++++++++++++++--------------
 1 file changed, 44 insertions(+), 20 deletions(-)

diff --git a/docs/06-longhorn.md b/docs/06-longhorn.md
index 104fc83..64de5c8 100644
--- a/docs/06-longhorn.md
+++ b/docs/06-longhorn.md
@@ -18,11 +18,21 @@ metal — your three Talos nodes — nothing provides that on its own, the
 same gap MetalLB fills for IP addresses rather than storage.
 
 **Longhorn** is that provider: it turns a plain disk on each node into one
-pool of storage, and — because your cluster has three nodes — it
-replicates each volume across more than one of them. Lose a node, and the
-data backing any volume that had a replica elsewhere is still there. This
-is also why Longhorn is genuinely a **three-node** story: with one node,
-"replicated storage" has nothing to replicate *to*.
+pool of storage, and replicates each volume across more than one node. Lose
+a node, and the data backing any volume that had a replica elsewhere is
+still there. This is why Longhorn needs more than one storage-providing
+node at all: with only one, "replicated storage" has nothing to replicate
+*to*.
+
+This cluster deliberately keeps storage off the three Raspberry Pi control
+planes — they have no data disk and are not meant to run application
+workloads (see [`docs/02-talos-bootstrap.md`](02-talos-bootstrap.md)).
+Longhorn instead runs only on the two dedicated Proxmox workers, each with
+its own local NVMe-backed disk on a separate physical host (Gimli and
+Titus). `defaultReplicaCount: 2` in
+[`apps/longhorn/values.yaml`](../apps/longhorn/values.yaml) matches that:
+with exactly two storage nodes, two replicas is the most redundancy
+possible — losing either worker still leaves one complete copy.
 
 ## What's already running
 
@@ -33,23 +43,35 @@ umbrella-chart pattern as MetalLB, just without any extra templates of its
 own this time.
 
 The interesting configuration already happened at the Talos layer, back in
-[`talos/patch-longhorn.yaml`](../talos/patch-longhorn.yaml): each node
-found, formatted, and mounted its own ≥900GB disk at
-`/var/mnt/longhorn`. [`apps/longhorn/values.yaml`](../apps/longhorn/values.yaml)
-points Longhorn's `defaultDataPath` at that exact path — this is the thing
-that makes Longhorn use *that* disk instead of carving space out of each
-node's root filesystem (small, and on Talos, not meant for this kind of
-data anyway).
+[`talos/patch-longhorn.yaml`](../talos/patch-longhorn.yaml): each worker
+found, formatted, and mounted its own ≥300GiB disk at `/var/mnt/longhorn`.
+[`apps/longhorn/values.yaml`](../apps/longhorn/values.yaml) points Longhorn's
+`defaultDataPath` at that exact path — this is the thing that makes Longhorn
+use *that* disk instead of carving space out of each node's root filesystem
+(small, and on Talos, not meant for this kind of data anyway).
+[`talos/patch-worker.yaml`](../talos/patch-worker.yaml) also labels each
+worker `node.longhorn.io/create-default-disk=true`, which is what actually
+limits default-disk creation to these two nodes instead of every node in
+the cluster — without it, Longhorn would try (and fail) to create a
+default disk on the Raspberry Pi control planes too.
 
 ## Verify it's actually working
 
 ```sh
-kubectl -n longhorn-system get pods
+kubectl -n longhorn-system get pods -o wide
 ```
 
-Expect a `longhorn-manager` and `longhorn-csi-plugin` Pod on each of your
-three nodes, plus one `instance-manager` per node once a volume exists,
-all `Running`.
+Expect a `longhorn-manager` and `longhorn-csi-plugin` Pod on each of the
+two workers (not the control planes), plus one `instance-manager` per
+worker once a volume exists, all `Running`. Confirm Longhorn agrees which
+nodes are eligible storage nodes:
+
+```sh
+kubectl -n longhorn-system get nodes.longhorn.io -o wide
+```
+
+Only the two workers should show `READY=true`, `ALLOWSCHEDULING=true`,
+`SCHEDULABLE=true`.
 
 ```sh
 kubectl get storageclass
@@ -86,13 +108,15 @@ once the pre-installed pieces are confirmed working, not a requirement to
 move on.
 
 > **Ask Claude:** "what would happen to that PVC's data if I lost one of
-> my three nodes right now?" — worth being able to answer before you trust
-> this with anything real.
+> my two storage workers right now?" — worth being able to answer before you
+> trust this with anything real.
 
 ## Checklist
 
-- [ ] `kubectl -n longhorn-system get pods` shows manager/CSI Pods
-      `Running` on all three nodes
+- [ ] `kubectl -n longhorn-system get pods -o wide` shows manager/CSI Pods
+      `Running` on both workers, and none on the control planes
+- [ ] `kubectl -n longhorn-system get nodes.longhorn.io` shows exactly the
+      two workers as schedulable storage nodes
 - [ ] `kubectl get storageclass` shows a default `longhorn` StorageClass
 - [ ] (Optional) Gave the demo app a real PVC and confirmed it's `Bound`
       with a healthy, replicated Longhorn volume behind it