k8s-starter-pack/docs/10-secrets.md

# 10 — Secrets: SOPS + age

This repo is public. A Kubernetes `Secret` only **base64-encodes** its data
— that's encoding, not encryption, trivially reversible by anyone who can
read the YAML. Never commit one directly. This doc sets up the tool this
repo actually uses instead: **SOPS** (Secrets OPerationS) encrypting with
**age**, decrypted inside the cluster by a small operator so committed files
stay encrypted end to end, right up until the moment they become a real
Kubernetes `Secret`.

## The pieces

- **age** — a modern, simple encryption tool. A keypair: a **public key**
  (safe to share, can only encrypt) and a **private key** (must never leave
  your machine and the cluster, can decrypt).
- **sops** — a CLI that encrypts specific fields inside a YAML/JSON file
  using a recipient (here, your age public key), leaving the rest of the
  file's structure readable so diffs in Git still make sense.
- **[sops-secrets-operator](https://github.com/isindir/sops-secrets-operator)**
  — a small controller running inside the cluster, installed exactly like
  any other app in this repo (see
  [`apps/sops-secrets-operator/`](../apps/sops-secrets-operator)). It
  watches for a custom resource called `SopsSecret` — an ordinary,
  SOPS-encrypted YAML file, safe to commit — and decrypts it into a real
  `Secret` automatically.

## The one manual step: getting the private key into the cluster

Every other piece of this repo reaches the cluster through Git and Argo
CD. The age **private** key is the one deliberate exception — same idea as
Argo CD's own install in [`docs/03-installing-argocd.md`](03-installing-argocd.md):
something has to exist outside the loop to start it, and a decryption key
is exactly the kind of thing that must never be *in* Git to begin with.

1. Generate the keypair once, on your own machine:
   ```sh
   mkdir -p ~/.config/sops/age
   age-keygen -o ~/.config/sops/age/keys.txt
   ```
   This prints a `Public key: age1...` line — copy it into
   [`.sops.yaml`](../.sops.yaml) at the repo root (already done for this
   cluster). The public key is safe to commit; the file it's printed from
   is not.

2. Put the private key into the cluster, by hand, once:
   ```sh
   kubectl create namespace sops
   kubectl create secret generic sops-age-key-file \
     --namespace sops \
     --from-file=key=$HOME/.config/sops/age/keys.txt
   ```
   [`apps/sops-secrets-operator/values.yaml`](../apps/sops-secrets-operator/values.yaml)
   mounts this exact Secret into the operator so it can decrypt — the
   operator's own Application manifest never contains the key itself, only
   a reference to a Secret you created directly.

3. Back up `~/.config/sops/age/keys.txt` somewhere safe and private (a
   password manager, not this repo). If you lose both this file and the
   in-cluster Secret, every `SopsSecret` ever committed becomes
   permanently unreadable — there is no recovery path.

## Creating a new secret

Say you need to store a Cloudflare API token for `cert-manager`
(see [`docs/12-cert-manager.md`](12-cert-manager.md)). Write the *plaintext*
version first, as a `SopsSecret` — this is the custom resource the operator
watches for, not a plain `Secret`:

```yaml
# apps/cert-manager/cloudflare-api-token.sops.yaml
apiVersion: isindir.github.com/v1alpha3
kind: SopsSecret
metadata:
  name: cloudflare-api-token
  namespace: cert-manager
spec:
  secretTemplates:
    - name: cloudflare-api-token
      stringData:
        api-token: PASTE_THE_REAL_TOKEN_HERE
```

Then encrypt it **in place** — this is the step that actually protects it,
and it must happen before this file is ever `git add`ed:

```sh
sops --encrypt --in-place apps/cert-manager/cloudflare-api-token.sops.yaml
```

`sops` reads `.sops.yaml`, sees the filename ends in `.sops.yaml`, and —
because of that file's `encrypted_regex: ^(data|stringData)$` — encrypts
only the values under `stringData`, for your age public key. `apiVersion`,
`kind`, and everything under `metadata` (name, namespace) stay in plain
text on purpose: Kubernetes object names have to be plain DNS-safe
strings, and encrypting them would produce a `SopsSecret` that fails to
apply at all. Open the file afterward — `stringData`'s values are now
unintelligible ciphertext with `sops` metadata appended; everything else is
still readable. That's what's safe to commit.

Once pushed and synced, the operator finds this `SopsSecret`, decrypts it
using the private key mounted from step 2, and creates an ordinary
`Secret` named `cloudflare-api-token` in the `cert-manager` namespace —
exactly as if you'd run `kubectl create secret` by hand, just reproducible
and driven from Git like everything else here.

## Editing an existing secret

Never hand-edit the ciphertext. Decrypt to your terminal, edit, and
re-encrypt:

```sh
sops apps/cert-manager/cloudflare-api-token.sops.yaml
```

Run with no flags, `sops` decrypts the file into a temporary copy, opens
your `$EDITOR`, and re-encrypts it back to disk on save — you never see or
create a separate plaintext file on your own disk.

## Checklist

- [ ] `age-keygen` run once; `~/.config/sops/age/keys.txt` backed up
      somewhere safe, outside this repo
- [ ] Public key present in [`.sops.yaml`](../.sops.yaml)
- [ ] `sops-age-key-file` Secret created by hand in the `sops` namespace
- [ ] `sops-secrets-operator` Application `Synced`/`Healthy`
- [ ] `argocd app list` shows `sops-secrets-operator` alongside the others

Continue to [`docs/11-ingress.md`](11-ingress.md), which comes first
(cert-manager depends on Ingress existing).