Move to a three-node cluster with a Talos VIP; add Longhorn
Every node is now both control plane and worker, giving real tolerance for losing any single machine (etcd quorum needs 3, not 1). A Talos VIP gives the cluster one stable address regardless of which node answers. Also pre-installs Longhorn for replicated storage, backed by a Talos UserVolume on each node's second disk (>=900GB, non-system) rather than the default path — mirroring the pattern from the homescale repo. Docs renumbered to make room for 06-longhorn.md; checklist, glossary, network map, and CLAUDE.md's guided-bootstrap instructions updated to match, including asking the user for node IPs, VIP, and disk info before generating any Talos config. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Sonnet 5
parent
92c3ad3191
commit
110bd97cca
+83
-31
@@ -1,7 +1,8 @@
|
||||
# 02 — Bootstrapping Talos
|
||||
|
||||
At the end of this doc, you'll have a single-node Kubernetes cluster,
|
||||
running nothing but Kubernetes itself. No Argo CD yet — that's next.
|
||||
At the end of this doc, you'll have a three-node Kubernetes cluster,
|
||||
running nothing but Kubernetes itself (plus the storage prep Longhorn
|
||||
needs later). No Argo CD yet — that's next.
|
||||
|
||||
## Why Talos, specifically
|
||||
|
||||
@@ -17,44 +18,92 @@ same declarative way Kubernetes applies your app manifests. Less surface
|
||||
area, less to secure, less to keep in sync by hand — the same philosophy as
|
||||
the rest of this stack, one layer lower.
|
||||
|
||||
## Single node, on purpose
|
||||
## Three nodes, each both control plane and worker
|
||||
|
||||
A "real" Kubernetes cluster usually has multiple nodes — some running the
|
||||
**control plane** (the brain: API server, scheduler, the cluster's
|
||||
database) and others as **workers** (running your actual apps), so that
|
||||
losing one machine doesn't take the whole cluster down. This repo starts
|
||||
with exactly one node acting as both, because that's the cheapest way to
|
||||
learn the concepts that matter for GitOps without needing multiple
|
||||
machines. [`docs/08-growing-up.md`](08-growing-up.md) covers adding more
|
||||
nodes once you're comfortable — the config here is deliberately compatible
|
||||
with that path, not a dead end.
|
||||
A "real" Kubernetes cluster has multiple nodes — some running the
|
||||
**control plane** (the brain: API server, scheduler, the cluster's own
|
||||
database, called etcd) and others as **workers** (running your actual
|
||||
apps) — so that losing one machine doesn't take the whole cluster down.
|
||||
This repo starts with three nodes, each acting as *both* control plane and
|
||||
worker, which gets you real resilience — survive losing any one machine —
|
||||
without needing more than three.
|
||||
|
||||
Why three specifically, and not two: etcd (the control plane's database)
|
||||
uses **quorum** — a majority vote among its members — to agree on cluster
|
||||
state, and a majority only means something with an odd number of voters.
|
||||
With one control-plane node, losing it loses the cluster outright. Two
|
||||
doesn't help: losing either one leaves one survivor, which is not a
|
||||
majority of two. Three is the smallest number where losing any *one*
|
||||
still leaves two — a majority — able to keep agreeing on state. This is
|
||||
also why [`docs/09-growing-up.md`](09-growing-up.md) — once you're past
|
||||
this doc — specifically warns against going to four control-plane nodes
|
||||
later: four doesn't improve on three's fault tolerance, it just makes
|
||||
quorum math worse.
|
||||
|
||||
## The VIP: one address for a cluster that moves
|
||||
|
||||
With three control-plane nodes, "where's the API server?" no longer has
|
||||
one fixed answer — it's whichever nodes happen to be up right now. Talos's
|
||||
**VIP** (Virtual IP) feature solves this: the three nodes continuously
|
||||
elect one of themselves to answer on one shared IP address you choose, and
|
||||
re-elect within seconds if that node goes down. Everything that talks to
|
||||
this cluster — `talosctl`, `kubectl`, the Application manifests in this
|
||||
repo — points at the VIP, never at one node's own address. See the
|
||||
`vip` block in
|
||||
[`talos/patch-controlplane.yaml`](../talos/patch-controlplane.yaml).
|
||||
|
||||
> **Ask Claude:** "help me pick the VIP address for my cluster" before you
|
||||
> generate any config — same kind of decision as the MetalLB pool in
|
||||
> [`docs/05-metallb.md`](05-metallb.md): a real, unused address on your
|
||||
> LAN, outside your DHCP range, that Claude can't guess on its own.
|
||||
|
||||
## Preparing storage for Longhorn
|
||||
|
||||
Each node also gets a second disk reserved for
|
||||
[Longhorn](06-longhorn.md), this repo's pre-installed storage layer —
|
||||
handled by [`talos/patch-longhorn.yaml`](../talos/patch-longhorn.yaml),
|
||||
applied at the same time as the control-plane patch below. Read that
|
||||
file's comments; the short version is that Talos finds, formats, and
|
||||
mounts a disk for Longhorn on its own, using rules you set (not the system
|
||||
disk, at least 900GB), so there's no manual partitioning step here.
|
||||
|
||||
## Steps
|
||||
|
||||
Full exact commands live in [`talos/README.md`](../talos/README.md) — read
|
||||
that alongside this doc; this section explains what's happening at each
|
||||
stage, that one has copy-paste blocks.
|
||||
stage, that one has copy-paste blocks. You'll repeat most of these steps
|
||||
**three times**, once per node — `talosctl README.md` calls out exactly
|
||||
which commands are "once" vs. "once per node."
|
||||
|
||||
1. **Generate machine configuration.** `talosctl gen config` produces
|
||||
Talos's config files for your cluster, including
|
||||
[`talos/patch-controlplane.yaml`](../talos/patch-controlplane.yaml) —
|
||||
read that file, it's short and explains itself — which lets your one
|
||||
node run workloads as well as the control plane.
|
||||
Talos's config files for your cluster, with both
|
||||
[`talos/patch-controlplane.yaml`](../talos/patch-controlplane.yaml)
|
||||
(scheduling + the VIP) and
|
||||
[`talos/patch-longhorn.yaml`](../talos/patch-longhorn.yaml) (storage
|
||||
prep) baked in — read both files, they explain themselves. This
|
||||
produces one `controlplane.yaml`, applied identically to all three
|
||||
nodes; there's no separate worker config in this setup, since every
|
||||
node is both.
|
||||
|
||||
2. **Apply the config to your node.** The node, freshly booted from the
|
||||
Talos ISO, is sitting in "maintenance mode" with no configuration.
|
||||
`talosctl apply-config` sends it the config generated in step 1. The
|
||||
node reboots, installs Talos to its disk, and comes up running for real.
|
||||
2. **Apply the config to each node.** Each node, freshly booted from your
|
||||
custom Talos ISO, is sitting in "maintenance mode" with no
|
||||
configuration. `talosctl apply-config` sends it the config from step 1.
|
||||
Run this once per node, pointed at that node's own address. Each node
|
||||
reboots, installs Talos to its system disk, formats and mounts its
|
||||
second disk for Longhorn, and comes up running for real.
|
||||
|
||||
3. **Bootstrap etcd.** `talosctl bootstrap` initializes the cluster's data
|
||||
store. This is a one-time, one-node-ever operation — even in a
|
||||
multi-node cluster you only ever bootstrap once.
|
||||
store. This is a one-time, one-node-ever operation, run against exactly
|
||||
one of your three nodes — even though all three will end up running
|
||||
etcd, only one of them needs telling to *start* the cluster; the other
|
||||
two join the existing one once it exists.
|
||||
|
||||
4. **Retrieve your kubeconfig.** `talosctl kubeconfig` downloads the
|
||||
credentials you'll use to talk to Kubernetes with `kubectl` — the
|
||||
Kubernetes equivalent of the `talosconfig` you got in step 1.
|
||||
Kubernetes equivalent of the `talosconfig` you got in step 1. Point it
|
||||
at the VIP, not at any one node.
|
||||
|
||||
5. **Verify.** `kubectl get nodes` should show one node, `Ready`.
|
||||
5. **Verify.** `kubectl get nodes` should show all three nodes, `Ready`.
|
||||
|
||||
> **Ask Claude:** "what's the difference between a kubeconfig and a
|
||||
> talosconfig, and why are they separate files?" is a good one to ask once
|
||||
@@ -62,11 +111,14 @@ stage, that one has copy-paste blocks.
|
||||
|
||||
## Checklist
|
||||
|
||||
- [ ] `_out/controlplane.yaml`, `worker.yaml`, and `talosconfig` generated
|
||||
- [ ] Config applied to the node, node has rebooted and installed Talos
|
||||
- [ ] `talosctl bootstrap` run exactly once
|
||||
- [ ] kubeconfig retrieved
|
||||
- [ ] `kubectl get nodes` shows one `Ready` node
|
||||
- [ ] VIP address chosen, with Claude's help, for your LAN
|
||||
- [ ] `_out/controlplane.yaml` and `talosconfig` generated, with both
|
||||
patches applied
|
||||
- [ ] Config applied to **all three** nodes; each has rebooted, installed
|
||||
Talos, and mounted its Longhorn disk
|
||||
- [ ] `talosctl bootstrap` run exactly once, against exactly one node
|
||||
- [ ] kubeconfig retrieved, pointed at the VIP
|
||||
- [ ] `kubectl get nodes` shows all three nodes `Ready`
|
||||
|
||||
Once your node is `Ready`, continue to
|
||||
Once all three nodes are `Ready`, continue to
|
||||
[`docs/03-installing-argocd.md`](03-installing-argocd.md).
|
||||
|
||||
Reference in New Issue
Block a user