fix(ci): pin the cluster CA fingerprint instead of trusting a fetched CA
Build and Push Image / Unit tests (push) Successful in 46s
Build and Push Image / Build and push image (push) Skipped
Build and Push Image / Deploy to the cluster (push) Skipped

The review was right on both counts, and both are fixed.

1. MITM. Fetching the CA over an unverified connection (`curl -sk`) and then
   trusting it is a privilege-escalation hole: the runner holds cluster-admin
   credentials, so an attacker able to intercept the fetch could hand us a CA
   of their choosing and then read everything sent over the connection.

   The CA is now PINNED. It is still fetched over an insecure connection, but
   it is only USED if its SHA-256 fingerprint matches
   `KUBE_SERVER_CA_FINGERPRINT`. An intercepted CA will not hash to the
   pinned value, so it is refused. If the secret is not set, the job fails
   rather than trusting an unverified CA.

2. The hardcoded cluster name. `kubectl config set-cluster default` creates a
   NEW cluster entry when the kubeconfig's entry is called something else,
   leaving the original (and its stale CA) in use. The name is now read from
   the kubeconfig: `{.clusters[0].name}`.

The bash is also moved out of the YAML into `scripts/kube-ca-refresh.sh`, so
it can be read, reviewed and tested on its own rather than as a wall of
shell inside a workflow file.

All three paths are tested against the live API server:

- no fingerprint supplied   -> refuses, exit 1
- wrong fingerprint        -> refuses, exit 1
- correct fingerprint      -> refreshes the CA, exit 0, kubectl then works
This commit is contained in:
2026-11-09 12:11:56 +10:00
parent 687339bb9e
commit ce6f5b7962
2 changed files with 97 additions and 44 deletions
+5 -44
View File
@@ -123,50 +123,11 @@ jobs:
- name: Verify the cluster is reachable - name: Verify the cluster is reachable
run: | run: |
# Fail fast, and with a useful message, if the kubeconfig cannot # The kubeconfig embeds the k3s server CA. If the cluster's CA has
# talk to the API server. # been rotated since the secret was created, kubectl cannot verify
# # the server. The script refreshes the CA, but only if it matches
# The certificate-authority-data inside the kubeconfig is the k3s # the fingerprint pinned in the secret - see the script for why.
# server CA. If the cluster's CA has been rotated since the secret bash scripts/kube-ca-refresh.sh "${{ secrets.KUBE_SERVER_CA_FINGERPRINT }}"
# was created, kubectl fails with:
# tls: failed to verify certificate: x509: certificate signed by
# unknown authority
# which is a confusing way to learn that a secret is stale.
if kubectl version --request-timeout=15s >/dev/null 2>&1; then
echo "cluster reachable"
exit 0
fi
echo "the kubeconfig could not verify the API server - refreshing the CA"
# The API server publishes its own CA at /cacerts. Fetch it over an
# insecure connection - we cannot verify it yet, that is the problem
# being solved - and pin it explicitly. This makes the deploy
# self-healing after a CA rotation, instead of needing the secret
# to be regenerated by hand.
SERVER=$(kubectl config view --raw -o jsonpath='{.clusters[0].cluster.server}')
echo "server: $SERVER"
curl -sk "$SERVER/cacerts" -o $HOME/.kube/ca.crt
if [ ! -s $HOME/.kube/ca.crt ]; then
echo "ERROR: could not fetch the cluster CA from $SERVER/cacerts"
exit 1
fi
kubectl config set-cluster default \
--embed-certs=true \
--certificate-authority=$HOME/.kube/ca.crt >/dev/null
rm -f $HOME/.kube/ca.crt
if kubectl version --request-timeout=15s >/dev/null 2>&1; then
echo "cluster reachable after refreshing the CA"
exit 0
fi
echo "ERROR: cannot reach the cluster API server even with a refreshed CA."
echo " The kubeconfig in the KUBEC_CONFIG_BUILDX_NEW_2 secret may be"
echo " stale or invalid, and needs regenerating."
exit 1
- name: Install kubectl - name: Install kubectl
run: | run: |
+92
View File
@@ -0,0 +1,92 @@
#!/usr/bin/env bash
#
# Make sure kubectl can talk to the cluster API server.
#
# The kubeconfig embeds the k3s server CA. If the cluster's CA has been
# rotated since the secret was created, kubectl fails with:
#
# tls: failed to verify certificate: x509: certificate signed by unknown authority
#
# which reads like a network problem rather than a stale secret. This script
# refreshes the CA in place.
#
# The refresh is PINNED: the server's CA is fetched over an unverified
# connection, but it is only used if its SHA-256 fingerprint matches the value
# in the KUBE_SERVER_CA_FINGERPRINT secret. An attacker able to intercept
# the fetch cannot use it - their CA will not hash to the pinned fingerprint.
# The runner has cluster-admin credentials, so trusting an unverified CA here
# would be a privilege-escalation hole.
#
# Usage:
# kube-ca-refresh.sh <expected-fingerprint>
#
# Exit codes:
# 0 the cluster is reachable
# 1 the cluster is not reachable and could not be fixed safely
set -euo pipefail
EXPECTED="${1:-}"
if kubectl version --request-timeout=15s >/dev/null 2>&1; then
echo "cluster reachable"
exit 0
fi
if [ -z "$EXPECTED" ]; then
echo "ERROR: the kubeconfig cannot verify the API server, and no expected CA" >&2
echo " fingerprint was supplied, so the CA cannot be refreshed safely." >&2
echo " Set KUBE_SERVER_CA_FINGERPRINT to the SHA-256 fingerprint of the" >&2
echo " cluster's CA:" >&2
echo " openssl x509 -in server-ca.crt -noout -fingerprint -sha256" >&2
exit 1
fi
SERVER=$(kubectl config view --raw -o jsonpath='{.clusters[0].cluster.server}')
# The cluster entry's NAME - patching the CA into a *new* entry named
# "default" would leave the original entry (and its stale CA) in use.
CLUSTER_NAME=$(kubectl config view --raw -o jsonpath='{.clusters[0].name}')
echo "server: $SERVER cluster entry: $CLUSTER_NAME"
CA_FILE=$(mktemp)
trap 'rm -f "$CA_FILE"' EXIT
curl -sk "$SERVER/cacerts" -o "$CA_FILE"
if [ ! -s "$CA_FILE" ]; then
echo "ERROR: could not fetch the cluster CA from $SERVER/cacerts" >&2
exit 1
fi
# Normalise both to lowercase hex without separators, so the comparison does
# not depend on the colon-separated format openssl prints.
normalise() {
tr -d ':' | tr 'A-Z' 'a-z'
}
ACTUAL=$(openssl x509 -in "$CA_FILE" -noout -fingerprint -sha256 | cut -d= -f2 | normalise)
WANTED=$(printf '%s' "$EXPECTED" | normalise)
echo "expected CA fingerprint: $WANTED"
echo "actual CA fingerprint: $ACTUAL"
if [ "$ACTUAL" != "$WANTED" ]; then
echo "ERROR: the cluster CA does not match the pinned fingerprint." >&2
echo " Refusing to use it. Either the CA has been rotated and the" >&2
echo " fingerprint secret needs updating, or the connection was" >&2
echo " intercepted." >&2
exit 1
fi
# The fingerprint matches the pinned value, so this is the CA we expect.
kubectl config set-cluster "$CLUSTER_NAME" \
--embed-certs=true \
--certificate-authority="$CA_FILE" >/dev/null
if kubectl version --request-timeout=15s >/dev/null 2>&1; then
echo "cluster reachable after refreshing the CA"
exit 0
fi
echo "ERROR: cannot reach the cluster API server even with a refreshed CA." >&2
echo " The kubeconfig secret may be stale or invalid." >&2
exit 1