10 — Secrets: SOPS + age

This repo is public. A Kubernetes Secret only base64-encodes its data — that's encoding, not encryption, trivially reversible by anyone who can read the YAML. Never commit one directly. This doc sets up the tool this repo actually uses instead: SOPS (Secrets OPerationS) encrypting with age, decrypted inside the cluster by a small operator so committed files stay encrypted end to end, right up until the moment they become a real Kubernetes Secret.

The pieces

age — a modern, simple encryption tool. A keypair: a public key (safe to share, can only encrypt) and a private key (must never leave your machine and the cluster, can decrypt).
sops — a CLI that encrypts specific fields inside a YAML/JSON file using a recipient (here, your age public key), leaving the rest of the file's structure readable so diffs in Git still make sense.
sops-secrets-operator — a small controller running inside the cluster, installed exactly like any other app in this repo (see apps/sops-secrets-operator/). It watches for a custom resource called SopsSecret — an ordinary, SOPS-encrypted YAML file, safe to commit — and decrypts it into a real Secret automatically.

The one manual step: getting the private key into the cluster

Every other piece of this repo reaches the cluster through Git and Argo CD. The age private key is the one deliberate exception — same idea as Argo CD's own install in docs/03-installing-argocd.md: something has to exist outside the loop to start it, and a decryption key is exactly the kind of thing that must never be in Git to begin with.

Generate the keypair once, on your own machine:
```
mkdir -p ~/.config/sops/age
age-keygen -o ~/.config/sops/age/keys.txt
```
This prints a Public key: age1... line — copy it into .sops.yaml at the repo root (already done for this cluster). The public key is safe to commit; the file it's printed from is not.
Put the private key into the cluster, by hand, once:
```
kubectl create namespace sops
kubectl create secret generic sops-age-key-file \
  --namespace sops \
  --from-file=key=$HOME/.config/sops/age/keys.txt
```
apps/sops-secrets-operator/values.yaml mounts this exact Secret into the operator so it can decrypt — the operator's own Application manifest never contains the key itself, only a reference to a Secret you created directly.
Back up ~/.config/sops/age/keys.txt somewhere safe and private (a password manager, not this repo). If you lose both this file and the in-cluster Secret, every SopsSecret ever committed becomes permanently unreadable — there is no recovery path.

Creating a new secret

Say you need to store a Cloudflare API token for cert-manager (see docs/12-cert-manager.md). Write the plaintext version first, as a SopsSecret — this is the custom resource the operator watches for, not a plain Secret:

# apps/cert-manager/cloudflare-api-token.sops.yaml
apiVersion: isindir.github.com/v1alpha3
kind: SopsSecret
metadata:
  name: cloudflare-api-token
  namespace: cert-manager
spec:
  secretTemplates:
    - name: cloudflare-api-token
      stringData:
        api-token: PASTE_THE_REAL_TOKEN_HERE

Then encrypt it in place — this is the step that actually protects it, and it must happen before this file is ever git added:

sops --encrypt --in-place apps/cert-manager/cloudflare-api-token.sops.yaml

sops reads .sops.yaml, sees the filename ends in .sops.yaml, and — because of that file's encrypted_regex: ^(data|stringData)$ — encrypts only the values under stringData, for your age public key. apiVersion, kind, and everything under metadata (name, namespace) stay in plain text on purpose: Kubernetes object names have to be plain DNS-safe strings, and encrypting them would produce a SopsSecret that fails to apply at all. Open the file afterward — stringData's values are now unintelligible ciphertext with sops metadata appended; everything else is still readable. That's what's safe to commit.

Once pushed and synced, the operator finds this SopsSecret, decrypts it using the private key mounted from step 2, and creates an ordinary Secret named cloudflare-api-token in the cert-manager namespace — exactly as if you'd run kubectl create secret by hand, just reproducible and driven from Git like everything else here.

Editing an existing secret

Never hand-edit the ciphertext. Decrypt to your terminal, edit, and re-encrypt:

sops apps/cert-manager/cloudflare-api-token.sops.yaml

Run with no flags, sops decrypts the file into a temporary copy, opens your $EDITOR, and re-encrypts it back to disk on save — you never see or create a separate plaintext file on your own disk.

Checklist

age-keygen run once; ~/.config/sops/age/keys.txt backed up somewhere safe, outside this repo
Public key present in .sops.yaml
sops-age-key-file Secret created by hand in the sops namespace
sops-secrets-operator Application Synced/Healthy
argocd app list shows sops-secrets-operator alongside the others

Continue to docs/11-ingress.md, which comes first (cert-manager depends on Ingress existing).

5.5 KiB Raw Blame History