Move to a three-node cluster with a Talos VIP; add Longhorn
Every node is now both control plane and worker, giving real tolerance for losing any single machine (etcd quorum needs 3, not 1). A Talos VIP gives the cluster one stable address regardless of which node answers. Also pre-installs Longhorn for replicated storage, backed by a Talos UserVolume on each node's second disk (>=900GB, non-system) rather than the default path — mirroring the pattern from the homescale repo. Docs renumbered to make room for 06-longhorn.md; checklist, glossary, network map, and CLAUDE.md's guided-bootstrap instructions updated to match, including asking the user for node IPs, VIP, and disk info before generating any Talos config. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Maxwell Morris
co-authored by
Claude Sonnet 5
Claude Sonnet 5
parent
92c3ad3191
commit
110bd97cca
+46
-33
@@ -19,17 +19,20 @@ the real cluster match it. You stop making changes by hand. You edit a file,
|
||||
push it, and a robot applies it for you — the same way every time, with a
|
||||
record of every change in `git log`.
|
||||
|
||||
## The five pieces, in plain terms
|
||||
## The six pieces, in plain terms
|
||||
|
||||
**Kubernetes** is software that runs on one or more machines and manages
|
||||
containers for you: starts them, restarts them if they crash, and moves
|
||||
them to a different machine if one dies (once you have more than one — see
|
||||
[`docs/08-growing-up.md`](08-growing-up.md)). You describe the end state you
|
||||
want ("run one copy of this"); Kubernetes' job is to make that true and keep
|
||||
it true. This is what people mean by **declarative**: you state the
|
||||
destination, not the steps to get there.
|
||||
them to a different machine if one dies. You describe the end state you
|
||||
want ("run one copy of this"); Kubernetes' job is to make that true and
|
||||
keep it true. This is what people mean by **declarative**: you state the
|
||||
destination, not the steps to get there. This repo runs three machines
|
||||
from the start, specifically so that "moves them to a different machine
|
||||
if one dies" is something you actually get, not just a fact about
|
||||
Kubernetes in general — see
|
||||
[`docs/02-talos-bootstrap.md`](02-talos-bootstrap.md) for why three.
|
||||
|
||||
**Talos Linux** is the operating system your machine runs. It's built for
|
||||
**Talos Linux** is the operating system each machine runs. It's built for
|
||||
exactly one purpose — running Kubernetes — and nothing else. There's no SSH,
|
||||
no shell, no package manager to fiddle with by hand. You configure it by
|
||||
sending it a config file over the network (a "machine config"), the same
|
||||
@@ -47,15 +50,23 @@ manages. Rather than one Application resource pointing at everything, one
|
||||
Application — the app-of-apps — watches a folder (`apps/`) full of *other*
|
||||
Application definitions, each pointing at its own small piece. Add a new
|
||||
app by adding a new file — see
|
||||
[`docs/07-adding-new-apps.md`](07-adding-new-apps.md).
|
||||
[`docs/08-adding-new-apps.md`](08-adding-new-apps.md).
|
||||
|
||||
**MetalLB** solves a specific problem: on a cloud provider, asking for a
|
||||
public IP address for a service is one API call away; on your own bare
|
||||
metal machine, nothing hands those out automatically. MetalLB runs inside
|
||||
metal machines, nothing hands those out automatically. MetalLB runs inside
|
||||
the cluster and fills that gap, so apps here can get a real, dedicated IP
|
||||
address on your network instead of an awkward fixed port. It's pre-installed
|
||||
in this repo, not a "next step" — see [`docs/05-metallb.md`](05-metallb.md).
|
||||
|
||||
**Longhorn** solves the equivalent problem for storage: a cloud provider
|
||||
gives you durable, replicated disks on demand; bare metal just gives you
|
||||
whatever's physically bolted into each machine. Longhorn turns a plain
|
||||
disk on each of your three nodes into one pool of replicated storage any
|
||||
app can request a slice of — and because it's replicated *across* nodes,
|
||||
losing one machine doesn't lose the data either. Also pre-installed, not a
|
||||
"next step" — see [`docs/06-longhorn.md`](06-longhorn.md).
|
||||
|
||||
## Architecture — the shape of what you'll build
|
||||
|
||||
```
|
||||
@@ -64,9 +75,11 @@ in this repo, not a "next step" — see [`docs/05-metallb.md`](05-metallb.md).
|
||||
│ repo, on GitHub) │
|
||||
│ │
|
||||
│ apps/demo.yaml │
|
||||
│ apps/demo/*.yaml │──┐
|
||||
│ apps/metallb.yaml │ │ watched by
|
||||
│ apps/metallb/* │ │
|
||||
│ apps/demo/*.yaml │
|
||||
│ apps/metallb.yaml │──┐
|
||||
│ apps/metallb/* │ │ watched by
|
||||
│ apps/longhorn.yaml │ │
|
||||
│ apps/longhorn/* │ │
|
||||
└─────────────────────────┘ │
|
||||
▼
|
||||
┌───────────────────┐
|
||||
@@ -76,29 +89,29 @@ in this repo, not a "next step" — see [`docs/05-metallb.md`](05-metallb.md).
|
||||
└─────────┬──────────┘
|
||||
│ applies manifests
|
||||
▼
|
||||
┌─────────────────────────────┐
|
||||
│ Talos Linux node │
|
||||
│ (control plane + worker, │
|
||||
│ single node) │
|
||||
│ │
|
||||
│ ┌────────────┐ ┌──────────┐ │
|
||||
│ │demo │ metallb- │ │
|
||||
│ │namespace: │ system: │ │
|
||||
│ │nginx Pod ⇄ │ hands out │ │
|
||||
│ │Service │ a real IP │ │
|
||||
│ │(LoadBalancer)◄─address to │ │
|
||||
│ │ │ that Service│ │
|
||||
│ └────────────┘ └──────────┘ │
|
||||
└─────────────────────────────┘
|
||||
┌───────────────────────────────────────────────┐
|
||||
│ Three Talos Linux nodes (VIP: <VIP_IP>) │
|
||||
│ each: control plane + worker + etcd peer │
|
||||
│ │
|
||||
│ demo namespace: metallb-system: │
|
||||
│ nginx Pod ⇄ Service hands out a real IP │
|
||||
│ (LoadBalancer) ◄───────address to that Service │
|
||||
│ │
|
||||
│ longhorn-system: each node's 2nd disk │
|
||||
│ replicates data (≥900GB, UserVolume) │
|
||||
│ across all 3 nodes ◄──mounted at │
|
||||
│ /var/mnt/longhorn │
|
||||
└───────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
Everything below the Git repo is a *consequence* of what's in Git. If the
|
||||
box labelled "Talos Linux node" burned to the ground and you rebuilt it from
|
||||
scratch, replaying this repo against it would recreate everything except the
|
||||
raw OS bootstrap (docs/02), the one-time Argo CD install (docs/03), and the
|
||||
handful of network-specific addresses you choose in docs/05 — all
|
||||
deliberately kept outside the loop, since something has to start it and
|
||||
nothing in Git can know your LAN's IP layout for you.
|
||||
Everything below the Git repo is a *consequence* of what's in Git. If any
|
||||
one of the three node boxes burned to the ground and you replaced that
|
||||
machine, replaying this repo against it would recreate everything on it
|
||||
except the raw OS bootstrap (docs/02) and the one-time Argo CD install
|
||||
(docs/03) — both deliberately kept outside the loop, since something has
|
||||
to start it. The handful of network-specific addresses you choose along
|
||||
the way (the VIP, the MetalLB pool) live in `docs/network.md`, not in Git
|
||||
history alone, since nothing in Git can know your LAN's IP layout for you.
|
||||
|
||||
## The reconciliation loop
|
||||
|
||||
|
||||
Reference in New Issue
Block a user