Move to a three-node cluster with a Talos VIP; add Longhorn
Every node is now both control plane and worker, giving real tolerance for losing any single machine (etcd quorum needs 3, not 1). A Talos VIP gives the cluster one stable address regardless of which node answers. Also pre-installs Longhorn for replicated storage, backed by a Talos UserVolume on each node's second disk (>=900GB, non-system) rather than the default path — mirroring the pattern from the homescale repo. Docs renumbered to make room for 06-longhorn.md; checklist, glossary, network map, and CLAUDE.md's guided-bootstrap instructions updated to match, including asking the user for node IPs, VIP, and disk info before generating any Talos config. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Sonnet 5
parent
92c3ad3191
commit
110bd97cca
@@ -0,0 +1,100 @@
|
||||
# 06 — Longhorn: real, replicated storage
|
||||
|
||||
The app-of-apps you applied in [`docs/04-app-of-apps.md`](04-app-of-apps.md)
|
||||
already brought up `longhorn` alongside `demo` and `metallb`. This doc
|
||||
explains what it does and how to actually use it — there's no address to
|
||||
pick this time, unlike MetalLB; the interesting decisions already happened
|
||||
back in [`docs/02-talos-bootstrap.md`](02-talos-bootstrap.md).
|
||||
|
||||
## Why this is needed at all
|
||||
|
||||
A Kubernetes Pod's own filesystem disappears the moment that Pod does —
|
||||
fine for a stateless web server, useless for anything that needs to keep
|
||||
data around (a database, uploaded files, anything you'd be upset to lose).
|
||||
Kubernetes' answer is a **PersistentVolumeClaim (PVC)**: an app asks for
|
||||
some amount of storage, and something has to actually provide it. On a
|
||||
cloud provider, that something is the cloud's own disk service. On bare
|
||||
metal — your three Talos nodes — nothing provides that on its own, the
|
||||
same gap MetalLB fills for IP addresses rather than storage.
|
||||
|
||||
**Longhorn** is that provider: it turns a plain disk on each node into one
|
||||
pool of storage, and — because your cluster has three nodes — it
|
||||
replicates each volume across more than one of them. Lose a node, and the
|
||||
data backing any volume that had a replica elsewhere is still there. This
|
||||
is also why Longhorn is genuinely a **three-node** story: with one node,
|
||||
"replicated storage" has nothing to replicate *to*.
|
||||
|
||||
## What's already running
|
||||
|
||||
**`apps/longhorn.yaml`** → one Application, pointing at `apps/longhorn`, a
|
||||
Helm chart that pulls in the official upstream Longhorn chart (see
|
||||
[`apps/longhorn/Chart.yaml`](../apps/longhorn/Chart.yaml)) — the same
|
||||
umbrella-chart pattern as MetalLB, just without any extra templates of its
|
||||
own this time.
|
||||
|
||||
The interesting configuration already happened at the Talos layer, back in
|
||||
[`talos/patch-longhorn.yaml`](../talos/patch-longhorn.yaml): each node
|
||||
found, formatted, and mounted its own ≥900GB disk at
|
||||
`/var/mnt/longhorn`. [`apps/longhorn/values.yaml`](../apps/longhorn/values.yaml)
|
||||
points Longhorn's `defaultDataPath` at that exact path — this is the thing
|
||||
that makes Longhorn use *that* disk instead of carving space out of each
|
||||
node's root filesystem (small, and on Talos, not meant for this kind of
|
||||
data anyway).
|
||||
|
||||
## Verify it's actually working
|
||||
|
||||
```sh
|
||||
kubectl -n longhorn-system get pods
|
||||
```
|
||||
|
||||
Expect a `longhorn-manager` and `longhorn-csi-plugin` Pod on each of your
|
||||
three nodes, plus one `instance-manager` per node once a volume exists,
|
||||
all `Running`.
|
||||
|
||||
```sh
|
||||
kubectl get storageclass
|
||||
```
|
||||
|
||||
Longhorn's chart creates a default `StorageClass` called `longhorn`
|
||||
automatically — this is what makes it the thing Kubernetes reaches for
|
||||
whenever an app asks for storage without naming one specifically.
|
||||
|
||||
## Try it: give the demo app real storage
|
||||
|
||||
The demo app's HTML currently lives in a ConfigMap (see
|
||||
[`docs/07-your-first-change.md`](07-your-first-change.md)) — fine for a
|
||||
few lines of text, but not how a real app would store real data. As a
|
||||
hands-on check that Longhorn actually works, ask Claude to help you:
|
||||
|
||||
1. Add a small `PersistentVolumeClaim` to `apps/demo/`, requesting a
|
||||
couple of gigabytes with no `storageClassName` set (so it uses
|
||||
Longhorn's default).
|
||||
2. Mount it into the demo Deployment somewhere harmless, like
|
||||
`/usr/share/nginx/html/data`.
|
||||
3. Push it, let Argo CD sync, and confirm:
|
||||
```sh
|
||||
kubectl -n demo get pvc
|
||||
```
|
||||
should show your claim `Bound`, and
|
||||
```sh
|
||||
kubectl -n longhorn-system get volumes
|
||||
```
|
||||
should show a matching volume, `healthy`, with more than one replica.
|
||||
|
||||
This isn't part of the checklist below — treat it as an optional exercise
|
||||
once the pre-installed pieces are confirmed working, not a requirement to
|
||||
move on.
|
||||
|
||||
> **Ask Claude:** "what would happen to that PVC's data if I lost one of
|
||||
> my three nodes right now?" — worth being able to answer before you trust
|
||||
> this with anything real.
|
||||
|
||||
## Checklist
|
||||
|
||||
- [ ] `kubectl -n longhorn-system get pods` shows manager/CSI Pods
|
||||
`Running` on all three nodes
|
||||
- [ ] `kubectl get storageclass` shows a default `longhorn` StorageClass
|
||||
- [ ] (Optional) Gave the demo app a real PVC and confirmed it's `Bound`
|
||||
with a healthy, replicated Longhorn volume behind it
|
||||
|
||||
Continue to [`docs/07-your-first-change.md`](07-your-first-change.md).
|
||||
Reference in New Issue
Block a user