ahmza.com/k8s-starter-pack
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

add NetBird connectivity to Talos nodes

Browse Source
This commit is contained in:
ahmza
2026-10-16 00:51:47 +00:00
parent fc195c5379
commit 6f43b5e343
5 changed files with 95 additions and 50 deletions
Download Patch File Download Diff File Expand all files Collapse all files
talos/README.md
+49 -42
View File
@@ -5,18 +5,14 @@ This is the detailed, copy-pasteable companion to
first for the explanations — this one is here so you have every exact
command in one place once you understand what they do.
Throughout, replace `<NODE1_IP>`, `<NODE2_IP>`, `<NODE3_IP>` with your three
machines' real IP addresses, and `<VIP_IP>` with the shared address you
picked (with Claude's help) for the cluster's VIP — see
[`docs/02-talos-bootstrap.md`](../docs/02-talos-bootstrap.md#the-vip-one-address-for-a-cluster-that-moves).
Give each machine a DHCP reservation (a fixed IP tied to its MAC address)
in your router or hypervisor before you start, so none of these addresses
change under the cluster. Talos bakes them into certificates during setup,
and changing one later means starting over for that node.
Throughout, use the three control-plane and two worker addresses recorded in
[`docs/network.md`](../docs/network.md), plus the cluster's shared VIP. Give
each machine a DHCP reservation (a fixed IP tied to its MAC address) before
installation so its address remains stable.
Steps 1–2 below run **once, total**. Steps 3 runs **once per node** (three
times). Steps 4–6 run **once, total**, against whichever node you like
(or, once it exists, the VIP).
Steps 1–2 below run **once, total**. Step 3 runs **once per node** (five
times, using the matching role configuration). Steps 4–6 run **once, total**;
Talos API commands target a control-plane node, while Kubernetes uses the VIP.
All commands assume you're running them from the root of this repo.
@@ -26,7 +22,10 @@ All commands assume you're running them from the root of this repo.
talosctl gen config gitops-starter https://<VIP_IP>:6443 \
--output-dir _out \
--config-patch-control-plane @talos/patch-controlplane.yaml \
--config-patch-control-plane @talos/patch-longhorn.yaml
--config-patch-control-plane @talos/patch-netbird.yaml \
--config-patch-worker @talos/patch-worker.yaml \
--config-patch-worker @talos/patch-longhorn.yaml \
--config-patch-worker @talos/patch-netbird.yaml
```
Note the cluster endpoint here is the **VIP**, not any one node's address
@@ -38,13 +37,12 @@ This writes three files into `_out/` (already covered by this repo's
`.gitignore` — they contain private keys and must never be committed):
- `controlplane.yaml` — the machine configuration for a control-plane
node, with both patches baked in. Applied identically to all three
nodes — there's no separate worker config in this setup, since every
node is both.
- `worker.yaml` — a plain worker config, generated automatically but
unused here since every node in this repo's setup is both control plane
and worker. Kept around for [`docs/09-growing-up.md`](../docs/09-growing-up.md),
where a fourth-and-beyond node uses exactly this file.
node, with the Raspberry Pi/VIP and non-secret NetBird patches baked in.
It is applied identically to all three control-plane nodes.
- `worker.yaml` — the machine configuration for the two Proxmox workers,
with their role, Longhorn data-disk, and non-secret NetBird patches baked
in. The NetBird setup key is deliberately absent from both generated files;
enrollment uses a short-lived key applied separately and revoked afterward.
- `talosconfig` — *your* client credentials for talking to the Talos API on
this cluster (equivalent in spirit to a kubeconfig, but for Talos itself
rather than Kubernetes).
@@ -68,34 +66,39 @@ separate.
## 3. Apply the configuration — once per node
Boot each machine from your custom Talos ISO (see
Boot each machine from its architecture-appropriate custom Talos image (see
[docs/01-prerequisites.md](../docs/01-prerequisites.md)). Each boots into
"maintenance mode" — alive enough to accept a configuration, with nothing
installed yet. For **each** node in turn:
"maintenance mode" — alive enough to accept configuration, with nothing
installed yet. Inspect its disks first:
```sh
talosctl get disks --insecure --nodes <NODE_IP>
```
Confirm you see two disks: the one Talos will install itself to, and the
≥900GB one `talos/patch-longhorn.yaml`'s `UserVolumeConfig` will claim for
Longhorn. Then:
For each Raspberry Pi control plane, confirm `/dev/mmcblk0` is the intended
system disk, then apply the control-plane configuration:
```sh
talosctl apply-config --insecure \
--nodes <NODE_IP> \
--nodes <CONTROL_PLANE_IP> \
--file _out/controlplane.yaml
```
`--insecure` is required for this one command only: the node has no
certificates yet, so talosctl can't authenticate to it normally. This is
the one moment during setup where you're trusting the network — do this on
a network you control. The node will reboot, install Talos to its system
disk, format and mount its second disk for Longhorn, and come back up
running the configuration you gave it — now *with* certificates, so every
command after this uses the secure path.
For each Proxmox worker, confirm `/dev/sda` is the Talos system disk and
`/dev/sdb` is the dedicated Longhorn disk selected by
`talos/patch-longhorn.yaml`, then apply the worker configuration:
Repeat this for `<NODE1_IP>`, `<NODE2_IP>`, and `<NODE3_IP>` before moving on.
```sh
talosctl apply-config --insecure \
--nodes <WORKER_IP> \
--file _out/worker.yaml
```
`--insecure` is required for this one command only: the node has no
certificates yet, so talosctl cannot authenticate to it normally. This is
the one moment during setup where you trust the local network. After applying
configuration, the node installs Talos and returns with authenticated API
access. Configure all three control planes and both workers before continuing.
## 4. Bootstrap etcd — once, total
@@ -115,12 +118,14 @@ existing one. Running it again against an existing cluster can corrupt it.
## 5. Get your kubeconfig
```sh
talosctl kubeconfig --nodes <VIP_IP> .
talosctl kubeconfig --nodes <CONTROL_PLANE_IP> .
```
Downloads a kubeconfig into the current directory (also gitignored —
it grants full cluster access), pointed at the VIP rather than any one
node. Merge it with `~/.kube/config` or point `KUBECONFIG` at it directly:
Downloads a kubeconfig through a control-plane node's Talos API into the
current directory (also gitignored — it grants full cluster access). The
resulting Kubernetes configuration points at the cluster VIP rather than one
specific control plane. Merge it with `~/.kube/config` or point `KUBECONFIG`
at it directly:
```sh
export KUBECONFIG=$PWD/kubeconfig
@@ -132,15 +137,17 @@ export KUBECONFIG=$PWD/kubeconfig
kubectl get nodes
```
Expect to see **three** nodes, `Ready`, within a couple of minutes (it
takes a moment for the default CNI — Talos ships Flannel out of the box —
to come up and clear each node's networking checks).
Expect to see **five** nodes, `Ready`, within a couple of minutes (it takes
a moment for the default CNI — Talos ships Flannel out of the box — to clear
each node's networking checks).
```
```text
NAME STATUS ROLES AGE VERSION
talos-... Ready control-plane 90s v1.3x.x
talos-... Ready control-plane 85s v1.3x.x
talos-... Ready control-plane 80s v1.3x.x
talos-... Ready worker 75s v1.3x.x
talos-... Ready worker 70s v1.3x.x
```
If any node is stuck on `NotReady`, give it another minute before
talos/patch-controlplane.yaml
+6 -4
View File
@@ -12,9 +12,10 @@
# --- install image ---
# All three control planes are Raspberry Pi 4 systems currently installed on
# their microSD (`mmcblk0`). The Image Factory schematic contains the
# `rpi_generic` ARM64 overlay needed to boot this hardware. Keep this custom
# installer reference during upgrades or the board-specific boot support may
# be lost.
# `rpi_generic` ARM64 overlay needed to boot this hardware and the NetBird
# system extension used for private overlay connectivity. Keep this custom
# installer reference during upgrades or the board-specific boot support and
# NetBird binary may be lost.
#
# --- vip ---
# With three control-plane nodes, "the cluster's API server" isn't one
@@ -38,7 +39,8 @@ cluster:
machine:
install:
disk: /dev/mmcblk0
image: factory.talos.dev/metal-installer/ee21ef4a5ef808a9b7484cc0dda0f25075021691c8c09a276591eedb638ea1f9:v1.13.8
image: "factory.talos.dev/metal-installer/\
d093bb68c464b87cc637765535dde5f68fbb040da44280617dd11a42d6c03988:v1.13.8"
network:
interfaces:
# Raspberry Pi 4's built-in Ethernet interface is named `end0` by
talos/patch-netbird.yaml
+20
View File
@@ -0,0 +1,20 @@
# Persistent, non-secret configuration for the official SideroLabs NetBird
# system extension. Apply this patch to every Talos node alongside its normal
# role-specific patches.
#
# Enrollment still requires NB_SETUP_KEY, but that short-lived credential must
# never be committed to this public repository. Supply it separately from a
# silent shell prompt during first enrollment, revoke it after all nodes join,
# then remove it from the live ExtensionServiceConfig.
apiVersion: v1alpha1
kind: ExtensionServiceConfig
name: netbird
environment:
# Both endpoints use this installation's combined self-hosted NetBird URL.
- NB_MANAGEMENT_URL=https://netbird.ahmza.com
- NB_ADMIN_URL=https://netbird.ahmza.com
# Talos mounts /etc read-only. Prevent NetBird from repeatedly trying to
# rewrite resolv.conf and the OpenSSH client configuration there. Overlay
# networking remains active; NetBird-managed DNS is deliberately disabled.
- NB_DISABLE_DNS=true
- NB_DISABLE_SSH_CONFIG=true
talos/patch-worker.yaml
+5 -4
View File
@@ -8,9 +8,9 @@
# claiming the wrong virtual disk.
#
# The custom Image Factory installer contains Longhorn's `iscsi-tools` and
# `util-linux-tools` extensions plus `qemu-guest-agent`. Keep this schematic
# reference during future upgrades; switching to the stock installer would
# silently remove those extensions.
# `util-linux-tools` extensions plus `qemu-guest-agent` and NetBird. Keep this
# schematic reference during future upgrades; switching to the stock installer
# would silently remove those extensions.
#
# Both VMs use `ens18`. Their DHCP-reserved MAC addresses keep worker01 at
# 10.10.10.203 and worker02 at 10.10.10.204 while DHCP supplies their gateway
@@ -26,7 +26,8 @@ machine:
node.longhorn.io/create-default-disk: "true"
install:
disk: /dev/sda
image: factory.talos.dev/metal-installer/88d1f7a5c4f1d3aba7df787c448c1d3d008ed29cfb34af53fa0df4336a56040b:v1.13.8
image: "factory.talos.dev/metal-installer/\
81fb1af7c03952593ba459c6c8b7fcc1268b6fe0cc80e6035ac629a741c183cc:v1.13.8"
network:
interfaces:
- interface: ens18