fix(ci): self-heal a stale cluster CA in the deploy job #9

Merged

hermes merged 4 commits from fix/dockerfile-python314 into master 2026-11-09 12:16:00 +10:00

Conversation 4 Commits 4 Files Changed 3

+161 -1

4 Commits

Author	SHA1	Message	Date
hermes	3e20b558a5	fix(ci): harden the CA refresh script and add shellcheck to CI ... Round-2 review findings, all low severity. - `curl` now has `--connect-timeout` and `--max-time`, so an unreachable API server fails the job instead of hanging the runner indefinitely. - A guard for a kubeconfig with no cluster entries, or an empty one, with a clear message rather than an opaque kubectl error. - The fingerprint mismatch message now prints both the expected and the actual value, which cuts the time to diagnose the next CA rotation. - `set -euo pipefail` was already there; the script now passes shellcheck clean, and a shellcheck step was added to the CI test job so the operational scripts get the same treatment as the Python code.	2026-11-09 12:15:50 +10:00
hermes	a4b3d40e2d	fix(ci): pin the cluster CA in the repo rather than in a secret ... The CA fingerprint is not a secret. It is a public value that anyone can read straight off the certificate, so pinning it in a Gitea secret was the wrong shape: - A change to it would be invisible - no commit, no review. - Nobody with secret-write access could silently repoint the CI runner at a different CA. It now lives in `scripts/cluster-ca.sha256`, so changing it is a reviewable commit. The script reads it from there, and still accepts an override as its first argument. This also removes the need for a `KUBE_SERVER_CA_FINGERPRINT` secret, which the bot token cannot create anyway - the Actions secrets API is owner-only. The behaviour is unchanged: the CA is fetched over an unverified connection but only USED if it matches the pinned fingerprint, so an intercepted CA is refused. Re-tested against the live API server - the matching fingerprint refreshes the CA and kubectl then works; a wrong one is refused and the stale CA is left in place.	2026-11-09 12:13:27 +10:00
hermes	ce6f5b7962	fix(ci): pin the cluster CA fingerprint instead of trusting a fetched CA ... The review was right on both counts, and both are fixed. 1. MITM. Fetching the CA over an unverified connection (`curl -sk`) and then trusting it is a privilege-escalation hole: the runner holds cluster-admin credentials, so an attacker able to intercept the fetch could hand us a CA of their choosing and then read everything sent over the connection. The CA is now PINNED. It is still fetched over an insecure connection, but it is only USED if its SHA-256 fingerprint matches `KUBE_SERVER_CA_FINGERPRINT`. An intercepted CA will not hash to the pinned value, so it is refused. If the secret is not set, the job fails rather than trusting an unverified CA. 2. The hardcoded cluster name. `kubectl config set-cluster default` creates a NEW cluster entry when the kubeconfig's entry is called something else, leaving the original (and its stale CA) in use. The name is now read from the kubeconfig: `{.clusters[0].name}`. The bash is also moved out of the YAML into `scripts/kube-ca-refresh.sh`, so it can be read, reviewed and tested on its own rather than as a wall of shell inside a workflow file. All three paths are tested against the live API server: - no fingerprint supplied -> refuses, exit 1 - wrong fingerprint -> refuses, exit 1 - correct fingerprint -> refreshes the CA, exit 0, kubectl then works	2026-11-09 12:11:56 +10:00
hermes	687339bb9e	fix(ci): self-heal a stale cluster CA in the deploy job ... The deploy job failed on its first run: error: error validating "kube/beer_namespace.yaml": error validating data: failed to download openapi: Get "https://192.168.178.160:6443/openapi/v2": tls: failed to verify certificate: x509: certificate signed by unknown authority This is the same class of problem as the buildx driver earlier: the certificate-authority-data embedded in the kubeconfig secret does not match the CA the API server is currently presenting. Rather than requiring the secret to be regenerated by hand every time the cluster's CA is rotated, the job now refreshes it: - The API server publishes its own CA at `/cacerts`. - If kubectl cannot verify the server, the job fetches that CA (over an insecure connection — verifying it is the problem being solved) and pins it explicitly with `kubectl config set-cluster`. - It then retries, and fails with a clear message if that still does not work. Verified by deliberately corrupting the CA in a working kubeconfig: the failure reproduces exactly, and the recovery path restores connectivity (`Server Version: v1.34.5+k3s1`). The secret is also switched from `KUBEC_CONFIG_BUILDX` to `KUBEC_CONFIG_BUILDX_NEW_2`, which is the one the working `wedding-photos` workflow uses.	2026-11-09 12:08:49 +10:00