vps-k8s/docs/00-what-is-all-this.md

00 — What is all this?

You're comfortable with servers, SSH, config files, and the command line. You've probably never touched Kubernetes. This doc gives you the mental model before you type a single command — everything after this builds on it.

The problem this stack solves

Say you run a few services on a server. The traditional way: SSH in, install things, edit config files by hand, restart services, and remember what you did (or don't, and regret it later). It works, but nothing records why the server looks the way it does — the truth lives only on that one machine, in your head, and in shell history if you're lucky.

This stack solves that by making a Git repository the single source of truth for "what should be running," and using software to continuously make the real cluster match it. You stop making changes by hand. You edit a file, push it, and a robot applies it for you — the same way every time, with a record of every change in git log.

The six pieces, in plain terms

Kubernetes is software that runs on one or more machines and manages containers for you: starts them, restarts them if they crash, and moves them to a different machine if one dies. You describe the end state you want ("run one copy of this"); Kubernetes' job is to make that true and keep it true. This is what people mean by declarative: you state the destination, not the steps to get there. This repo runs three machines from the start, specifically so that "moves them to a different machine if one dies" is something you actually get, not just a fact about Kubernetes in general — see docs/02-talos-bootstrap.md for why three.

Talos Linux is the operating system each machine runs. It's built for exactly one purpose — running Kubernetes — and nothing else. There's no SSH, no shell, no package manager to fiddle with by hand. You configure it by sending it a config file over the network (a "machine config"), the same declarative idea as Kubernetes itself, just one layer lower. Less to install, less to secure, less that can drift out of a known state.

Argo CD is the piece that watches a Git repository — this one — and continuously makes the cluster match whatever it finds there. This is called GitOps: Git isn't just where your config lives, it's where change happens. You don't run kubectl apply for day-to-day changes; you edit a file, commit, push, and Argo CD does the applying.

App of apps is the pattern this repo uses to organize what Argo CD manages. Rather than one Application resource pointing at everything, one Application — the app-of-apps — watches a folder (apps/) full of other Application definitions, each pointing at its own small piece. Add a new app by adding a new file — see docs/08-adding-new-apps.md.

MetalLB solves a specific problem: on a cloud provider, asking for a public IP address for a service is one API call away; on your own bare metal machines, nothing hands those out automatically. MetalLB runs inside the cluster and fills that gap, so apps here can get a real, dedicated IP address on your network instead of an awkward fixed port. It's pre-installed in this repo, not a "next step" — see docs/05-metallb.md.

Longhorn solves the equivalent problem for storage: a cloud provider gives you durable, replicated disks on demand; bare metal just gives you whatever's physically bolted into each machine. Longhorn turns a plain disk on each of your three nodes into one pool of replicated storage any app can request a slice of — and because it's replicated across nodes, losing one machine doesn't lose the data either. Also pre-installed, not a "next step" — see docs/06-longhorn.md.

Architecture — the shape of what you'll build

┌─────────────────────────┐
│   Git repository (this  │
│   repo, on GitHub)      │
│                         │
│   apps/demo.yaml        │
│   apps/demo/*.yaml      │
│   apps/metallb.yaml     │──┐
│   apps/metallb/*        │  │  watched by
│   apps/longhorn.yaml    │  │
│   apps/longhorn/*       │  │
└─────────────────────────┘  │
                              ▼
                    ┌───────────────────┐
                    │      Argo CD       │  runs inside the cluster
                    │  (installed once,  │
                    │   imperatively)    │
                    └─────────┬──────────┘
                              │ applies manifests
                              ▼
        ┌───────────────────────────────────────────────┐
        │      Three Talos Linux nodes (VIP: <VIP_IP>)    │
        │      each: control plane + worker + etcd peer   │
        │                                                 │
        │  demo namespace:        metallb-system:         │
        │  nginx Pod ⇄ Service    hands out a real IP      │
        │  (LoadBalancer) ◄───────address to that Service  │
        │                                                 │
        │  longhorn-system:       each node's 2nd disk    │
        │  replicates data        (≥900GB, UserVolume)    │
        │  across all 3 nodes  ◄──mounted at              │
        │                         /var/mnt/longhorn        │
        └───────────────────────────────────────────────┘

Everything below the Git repo is a consequence of what's in Git. If any one of the three node boxes burned to the ground and you replaced that machine, replaying this repo against it would recreate everything on it except the raw OS bootstrap (docs/02) and the one-time Argo CD install (docs/03) — both deliberately kept outside the loop, since something has to start it. The handful of network-specific addresses you choose along the way (the VIP, the MetalLB pool) live in docs/network.md, not in Git history alone, since nothing in Git can know your LAN's IP layout for you.

The reconciliation loop

This is the mechanism that makes GitOps actually work, not just a nice diagram. Argo CD repeats this forever, by default every few minutes (and instantly, if you ask it to):

   ┌────────────────────────────────────────────────┐
   │                                                  │
   │   1. Read desired state from Git                │
   │              │                                   │
   │              ▼                                   │
   │   2. Read actual state from the cluster          │
   │              │                                   │
   │              ▼                                   │
   │   3. Diff them                                   │
   │              │                                   │
   │         ┌────┴────┐                              │
   │         │          │                              │
   │    no diff    diff found                          │
   │         │          │                              │
   │         │          ▼                              │
   │         │   4. Apply the change so the             │
   │         │      cluster matches Git                │
   │         │          │                              │
   │         └────┬─────┘                              │
   │              ▼                                   │
   │        wait, then repeat                          │
   │                                                  │
   └────────────────────────────────────────────────┘

"Reconciliation" just means step 3 and 4: noticing a difference between what should exist and what does, and fixing it. This loop is also what makes Argo CD self-healing — if someone changes the cluster by hand (kubectl edit during an emergency, say), the very next loop notices the drift from Git and reverts it. Git wins, always. That's the whole point.

Ask Claude: before moving on, try asking "why does Git winning matter — what goes wrong without it?" It's worth being able to answer in your own words before you start building.

What's next

Continue to docs/01-prerequisites.md, or jump straight to docs/checklist.md if you'd rather work through a condensed step-by-step list with Claude alongside you.