Add SOPS+age secrets pipeline

2026-09-03 04:24:38 +01:00
parent 41dfd195a1
commit 0a733a7d76
5 changed files with 207 additions and 0 deletions
@@ -0,0 +1,13 @@
+# Tells the `sops` CLI which recipient to encrypt for, and which files that
+# applies to. Files matching path_regex below get encrypted for this age
+# public key when you run `sops -e`. Only the public key lives here — it can
+# encrypt, but cannot decrypt anything. The matching private key lives only
+# on your machine (~/.config/sops/age/keys.txt) and inside the cluster (a
+# Secret created by hand, never committed) — see docs/10-secrets.md.
+#
+# Convention: any file ending in .sops.yaml is a secret meant to be committed
+# encrypted. Plain .yaml files are never treated as secrets by this rule, so
+# an accidental `sops -e` on the wrong file won't silently do nothing useful.
+creation_rules:
+  - path_regex: .*\.sops\.yaml$
+    age: age1udejf5qa66rd6yvj5r8dz0fjnqfh0quf4t339c3cpqr26mkx2ydq3zel07
@@ -0,0 +1,34 @@
+# sops-secrets-operator, installed like MetalLB and Longhorn: an Application
+# pointing at apps/sops-secrets-operator, a Helm chart that pulls in the
+# upstream operator chart. Once this syncs, any SopsSecret custom resource
+# committed anywhere in this repo gets decrypted into a real Kubernetes
+# Secret automatically. See docs/10-secrets.md for the full explanation,
+# including the one manual step this needs before it can decrypt anything.
+apiVersion: argoproj.io/v1alpha1
+kind: Application
+metadata:
+  name: sops-secrets-operator
+  namespace: argocd
+  # Safe to cascade: this operator holds no persistent data of its own.
+  # Deleting it stops decrypting SopsSecrets; the encrypted source files and
+  # the age key stay intact.
+  finalizers:
+    - resources-finalizer.argocd.argoproj.io
+spec:
+  project: default
+
+  source:
+    repoURL: https://tea.ahmza.com/ahmza/k8s-starter-pack.git
+    targetRevision: main
+    path: apps/sops-secrets-operator
+
+  destination:
+    server: https://kubernetes.default.svc
+    namespace: sops
+
+  syncPolicy:
+    automated:
+      prune: true
+      selfHeal: true
+    syncOptions:
+      - CreateNamespace=true
@@ -0,0 +1,20 @@
+# An "umbrella" chart, same pattern as apps/metallb/ and apps/longhorn/:
+# declares a dependency on the upstream sops-secrets-operator chart. Argo CD
+# detects this as a Helm source because a Chart.yaml exists at this path.
+#
+# This operator watches for SopsSecret custom resources (ordinary YAML files,
+# encrypted with `sops`, safe to commit to this public repo) and decrypts
+# them into real Kubernetes Secrets inside the cluster. See docs/10-secrets.md
+# for the full explanation and the one manual bootstrap step this needs.
+apiVersion: v2
+name: sops-secrets-operator-app
+description: Installs sops-secrets-operator, which turns committed SOPS-encrypted SopsSecret resources into real Kubernetes Secrets
+version: 0.1.0
+dependencies:
+  - name: sops-secrets-operator
+    # A version range rather than an exact pin, so this doesn't silently go
+    # stale. Check https://artifacthub.io/packages/helm/sops-secrets-operator/sops-secrets-operator
+    # for the current release and pin an exact version for a fully
+    # reproducible install.
+    version: ">=0.28.0"
+    repository: https://isindir.github.io/sops-secrets-operator/
@@ -0,0 +1,15 @@
+# Values for the upstream sops-secrets-operator chart, nested under
+# `sops-secrets-operator:` (the dependency's name in Chart.yaml).
+#
+# The operator needs the age *private* key to decrypt SopsSecret resources.
+# That key must never be committed — it's mounted from a Secret you create
+# by hand, once, directly on the cluster (see docs/10-secrets.md), the same
+# kind of one-time manual bootstrap step as Argo CD's own install.
+sops-secrets-operator:
+  secretsAsFiles:
+    - name: sops-age-key-file
+      mountPath: /etc/sops-age-key-file
+      secretName: sops-age-key-file
+  extraEnv:
+    - name: SOPS_AGE_KEY_FILE
+      value: /etc/sops-age-key-file/key
@@ -0,0 +1,125 @@
+# 10 — Secrets: SOPS + age
+
+This repo is public. A Kubernetes `Secret` only **base64-encodes** its data
+— that's encoding, not encryption, trivially reversible by anyone who can
+read the YAML. Never commit one directly. This doc sets up the tool this
+repo actually uses instead: **SOPS** (Secrets OPerationS) encrypting with
+**age**, decrypted inside the cluster by a small operator so committed files
+stay encrypted end to end, right up until the moment they become a real
+Kubernetes `Secret`.
+
+## The pieces
+
+- **age** — a modern, simple encryption tool. A keypair: a **public key**
+  (safe to share, can only encrypt) and a **private key** (must never leave
+  your machine and the cluster, can decrypt).
+- **sops** — a CLI that encrypts specific fields inside a YAML/JSON file
+  using a recipient (here, your age public key), leaving the rest of the
+  file's structure readable so diffs in Git still make sense.
+- **[sops-secrets-operator](https://github.com/isindir/sops-secrets-operator)**
+  — a small controller running inside the cluster, installed exactly like
+  any other app in this repo (see
+  [`apps/sops-secrets-operator/`](../apps/sops-secrets-operator)). It
+  watches for a custom resource called `SopsSecret` — an ordinary,
+  SOPS-encrypted YAML file, safe to commit — and decrypts it into a real
+  `Secret` automatically.
+
+## The one manual step: getting the private key into the cluster
+
+Every other piece of this repo reaches the cluster through Git and Argo
+CD. The age **private** key is the one deliberate exception — same idea as
+Argo CD's own install in [`docs/03-installing-argocd.md`](03-installing-argocd.md):
+something has to exist outside the loop to start it, and a decryption key
+is exactly the kind of thing that must never be *in* Git to begin with.
+
+1. Generate the keypair once, on your own machine:
+   ```sh
+   mkdir -p ~/.config/sops/age
+   age-keygen -o ~/.config/sops/age/keys.txt
+   ```
+   This prints a `Public key: age1...` line — copy it into
+   [`.sops.yaml`](../.sops.yaml) at the repo root (already done for this
+   cluster). The public key is safe to commit; the file it's printed from
+   is not.
+
+2. Put the private key into the cluster, by hand, once:
+   ```sh
+   kubectl create namespace sops
+   kubectl create secret generic sops-age-key-file \
+     --namespace sops \
+     --from-file=key=$HOME/.config/sops/age/keys.txt
+   ```
+   [`apps/sops-secrets-operator/values.yaml`](../apps/sops-secrets-operator/values.yaml)
+   mounts this exact Secret into the operator so it can decrypt — the
+   operator's own Application manifest never contains the key itself, only
+   a reference to a Secret you created directly.
+
+3. Back up `~/.config/sops/age/keys.txt` somewhere safe and private (a
+   password manager, not this repo). If you lose both this file and the
+   in-cluster Secret, every `SopsSecret` ever committed becomes
+   permanently unreadable — there is no recovery path.
+
+## Creating a new secret
+
+Say you need to store a Cloudflare API token for `cert-manager`
+(see [`docs/12-cert-manager.md`](12-cert-manager.md)). Write the *plaintext*
+version first, as a `SopsSecret` — this is the custom resource the operator
+watches for, not a plain `Secret`:
+
+```yaml
+# apps/cert-manager/cloudflare-api-token.sops.yaml
+apiVersion: isindir.github.com/v1alpha3
+kind: SopsSecret
+metadata:
+  name: cloudflare-api-token
+  namespace: cert-manager
+spec:
+  secretTemplates:
+    - name: cloudflare-api-token
+      stringData:
+        api-token: PASTE_THE_REAL_TOKEN_HERE
+```
+
+Then encrypt it **in place** — this is the step that actually protects it,
+and it must happen before this file is ever `git add`ed:
+
+```sh
+sops --encrypt --in-place apps/cert-manager/cloudflare-api-token.sops.yaml
+```
+
+`sops` reads `.sops.yaml`, sees the filename ends in `.sops.yaml`, and
+encrypts every value under `stringData` for your age public key. Open the
+file afterward — the keys (`api-token`) stay readable, the values are now
+unintelligible ciphertext with `sops` metadata appended. That's what's safe
+to commit.
+
+Once pushed and synced, the operator finds this `SopsSecret`, decrypts it
+using the private key mounted from step 2, and creates an ordinary
+`Secret` named `cloudflare-api-token` in the `cert-manager` namespace —
+exactly as if you'd run `kubectl create secret` by hand, just reproducible
+and driven from Git like everything else here.
+
+## Editing an existing secret
+
+Never hand-edit the ciphertext. Decrypt to your terminal, edit, and
+re-encrypt:
+
+```sh
+sops apps/cert-manager/cloudflare-api-token.sops.yaml
+```
+
+Run with no flags, `sops` decrypts the file into a temporary copy, opens
+your `$EDITOR`, and re-encrypts it back to disk on save — you never see or
+create a separate plaintext file on your own disk.
+
+## Checklist
+
+- [ ] `age-keygen` run once; `~/.config/sops/age/keys.txt` backed up
+      somewhere safe, outside this repo
+- [ ] Public key present in [`.sops.yaml`](../.sops.yaml)
+- [ ] `sops-age-key-file` Secret created by hand in the `sops` namespace
+- [ ] `sops-secrets-operator` Application `Synced`/`Healthy`
+- [ ] `argocd app list` shows `sops-secrets-operator` alongside the others
+
+Continue to [`docs/11-ingress.md`](11-ingress.md), which comes first
+(cert-manager depends on Ingress existing).