k8s-starter-pack/docs/04-app-of-apps.md

# 04 — The app-of-apps pattern

This is the last manual step. After this, Git drives everything.

## The idea

An Argo CD `Application` is a resource that says "watch this path in this
Git repo, and keep the cluster matching what's there." The simplest
possible setup is one Application per app you want to run — but then every
new app means one more manual `kubectl apply`, which quietly puts you back
to making changes by hand.

**App-of-apps** fixes this by making Applications themselves something Argo
CD manages. One Application — the app-of-apps — watches a folder
([`apps/`](../apps)) full of *other* Application manifests. Add a new app by
adding a new file to that folder and pushing it — the app-of-apps notices
the new file and creates the Application it describes, which in turn
deploys the app. No `kubectl apply` required past this point, ever, for a
new app.

```
argocd/app-of-apps/app-of-apps.yaml   (you apply this ONE file, by hand, once)
        │
        │  watches apps/, non-recursively
        ├───────────────┬────────────────┬───────────────┐
        ▼               ▼                ▼               ▼
   apps/demo.yaml  apps/metallb.yaml apps/longhorn.yaml  (more, later)
        │               │                │
        ▼               ▼                ▼
   apps/demo/*.yaml  apps/metallb/*   apps/longhorn/*    (manifests / Helm charts)
```

Four layers wide, each watching the next. You only ever touch the bottom
two — the app-of-apps is set-and-forget. Notice `apps/metallb.yaml` and
`apps/longhorn.yaml` are already here — see
[`docs/05-metallb.md`](05-metallb.md) and
[`docs/06-longhorn.md`](06-longhorn.md) right after this one.

## Why `apps/` isn't recursive

Look at [`argocd/app-of-apps/app-of-apps.yaml`](../argocd/app-of-apps/app-of-apps.yaml)
— specifically `directory.recurse: false`. The `apps/` folder contains both
Application manifests (`apps/demo.yaml`, `apps/metallb.yaml`,
`apps/longhorn.yaml`) *and* subfolders holding the real workload manifests
those Applications point to (`apps/demo/*.yaml`, `apps/metallb/*`,
`apps/longhorn/*`). Non-recursive means the app-of-apps only ever looks at
files directly inside `apps/`, never inside its subfolders — so it only
ever sees Application definitions, and leaves the actual workload
manifests to the Application that owns them. Without this, the
app-of-apps would try to apply raw Deployments and Helm chart files
itself, duplicating and conflicting with what those Applications are
already doing.

## Every Application needs its own `repoURL`

Applications don't inherit `repoURL` from whatever applied them — each one
declares its own source repo independently. All four initial Applications
therefore point explicitly at this cluster's public Gitea repository:
`https://tea.ahmza.com/ahmza/k8s-starter-pack.git`. Anonymous read access is
required because Argo CD clones it from inside the cluster without a stored
Git credential.

Verify every source before applying the app-of-apps:

```sh
grep -R "repoURL:" argocd/app-of-apps apps/*.yaml
```

The command should show the same Gitea URL in
`argocd/app-of-apps/app-of-apps.yaml`, `apps/demo.yaml`, `apps/metallb.yaml`,
and `apps/longhorn.yaml`.

## Steps

1. Verify `repoURL` in all four files (above) and confirm anonymous access
   with `git ls-remote https://tea.ahmza.com/ahmza/k8s-starter-pack.git HEAD`.
2. Commit and push any pending configuration first. This matters more than it
   looks: the app-of-apps reads its children *from Git*, not from your local
   checkout, so local-only edits cannot affect the cluster.
3. Apply the app-of-apps — the one manifest you apply by hand:
   ```sh
   kubectl apply -f argocd/app-of-apps/app-of-apps.yaml
   ```
4. Watch it sync, then watch `demo`, `metallb`, and `longhorn` appear
   underneath it:
   ```sh
   argocd app list
   ```
   You should see `app-of-apps`, `demo`, `metallb`, and `longhorn`, all
   `Synced` and `Healthy` within a few minutes (MetalLB and Longhorn's own
   installs take longer than the demo app — Longhorn especially, since it
   has to find and prepare a disk on every node).
5. Confirm the demo app is actually running:
   ```sh
   kubectl -n demo get pods,svc
   ```
   You'll notice the Service's `EXTERNAL-IP` column says `<pending>` —
   that's expected. The demo app is a `LoadBalancer` Service, and nothing
   hands those out real addresses on bare metal until MetalLB is
   configured, which is exactly the next doc.

If `app-of-apps`, `demo`, `metallb`, and `longhorn` all show
`Synced`/`Healthy`, the loop — Git → Argo CD → cluster — is already proven
end to end; you just can't see the demo app in a browser yet.

> **Ask Claude:** "walk me through exactly what happened, in order, from the
> moment I ran `kubectl apply -f argocd/app-of-apps/app-of-apps.yaml` to
> these four Applications existing." Making sure you can narrate that
> chain yourself is the real checkpoint for this doc — everything from
> here on is a variation on it.

## Checklist

- [ ] The public Gitea `repoURL` is present in all four files listed above
- [ ] Anonymous `git ls-remote` succeeds and all pending changes are pushed
- [ ] `app-of-apps` applied and shows `Synced`/`Healthy`
- [ ] `demo`, `metallb`, and `longhorn` Applications appeared
      automatically, all `Synced`/`Healthy`
- [ ] `kubectl -n demo get pods,svc` shows the demo app's Pod running
      (Service `EXTERNAL-IP` still `<pending>` — expected for now)

Continue to [`docs/05-metallb.md`](05-metallb.md) to give the demo app —
and Argo CD's own UI — a real address.