ab3050194d9787e28e0a0a6bea4318c07366db1f
15
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 hermes
|
563e22fb00 |
fix(ci): use the house CI pattern instead of a bespoke one
I had built a parallel mechanism for this repo rather than copying the workflow that is already known to work on this cluster. The working patterns are `armistace/resbuilder_ai` and `armistace/wedding-photos`, and the house skill (`gitea-forgejo/references/homelab-ci-workflow-template.md`) documents them. This rewrites the workflow to match them. What changed, and why each piece was wrong: - ONE job, not three. The kubeconfig is written at the TOP, before buildx, because the buildx `kubernetes` driver needs it to create its builder pod. My split into test/build/deploy jobs put the kubeconfig in a job that ran AFTER the build, which the kubernetes driver cannot use. - `driver: kubernetes` with `namespace=gitea-runner`, not the `remote` driver. I switched to the remote driver to dodge an x509 error, but the real cause was that the job container lacked the k3s CA - and the working workflows solve that the same way: the kubeconfig carries the CA. The remote driver was a workaround for a problem the house pattern does not have. - The CA-refresh script and the pinned-fingerprint file are deleted. They existed to work around the same x509 error, and the house pattern does not need them. - The deploy step installs kubectl and applies `kube/*.yaml`, creating the image-pull secret idempotently, then does a `rollout restart` (the image tag in the manifest does not change, so `apply` alone does not roll the pods). - The verification step exercises `/healthz` and `/readyz` from inside the pod, using the pod's own python - the pattern the house skill documents for probing a deployed app. There is no `curl` in the slim image. The Deployment manifest also changes: - `runAsUser: 999` at the POD level, matching the `beer` user in the image. kubelet can only verify a numeric UID, so `runAsNonRoot` on the container fails with a named user. - `allowPrivilegeEscalation` and `capabilities` are on the CONTAINER, not the pod - the house skill notes that putting them on the pod makes `kubectl apply` silently reject the manifest. - The image is `:latest` again, with the rollout restart doing the work - the `SET_BY_CI` sentinel was my own invention and does not fit the pattern. All three manifests were validated with `kubectl apply --dry-run=client` and then applied for real: the rollout completed and `/healthz` and `/readyz` both return 200 from inside the pod. |
||
|
hermes
|
bf0f88dd70 |
fix: build the image on Python 3.14, and fix the review findings
The image build failed on master:
ERROR: Ignored the following versions that require a different python
version: 4.19.0 Requires-Python >=3.11
ERROR: Could not find a version that satisfies the requirement
pymongo==4.19.0
The base image was `python:3.10` but the pinned pymongo 4.19 needs Python
>= 3.11. The image is now `python:3.14-slim` — the current stable release,
so it is not carrying the CVEs fixed in 3.11, 3.12 and 3.13.
All the runtime dependencies ship a cp314 wheel (pymongo included), so no
compiler toolchain is needed. The old apt-get line installed
libsasl2-dev, python-dev-is-python3, libldap2-dev and libssl-dev for
packages that are no longer in the dependency list, so it is gone and the
image is smaller.
The build now asserts the interpreter version, so a base image change that
breaks the pinned requirements fails the build with a clear message rather
than a pip resolution error.
Review findings, all from the independent review of this PR
------------------------------------------------------------
- Mongo indexes: the run metadata is always looked up by `beer_run_id` and
the readings by `(beer_run_id, date)`. Both are now indexed.
- Content-Security-Policy: the app serves only its own vendored assets, so
a strict `default-src 'self'` policy is enforceable. Adding it required
removing the inline `style=` attributes from the templates (they became
`.mb-lg` / `.mt-lg` utility classes) and the one inline `<script> (the run
id now travels as a `data-run-id` attribute). A test asserts no template
has either.
- Job timeouts: the `test` and `build` jobs now have `timeout-minutes`, so
a wedged build stops occupying a runner.
- OG range: raised from 1300 to 1500 points, so a strong specialty brew
(a 1.150 wort is real) can be recorded.
Bugs found and fixed along the way
----------------------------------
- An unfinished run reported a *measured* ABV. `abv_percent()` now returns
None for a run that is still in the fermenter — its figure is a projection,
not a measurement. This was showing the two active runs in the "alcohol by
brew" chart as if they were finished.
- The `/api/runs` endpoints the charts fetch from did not exist, so every
chart was silently empty. They are back, with tests.
- The "brews over time" chart left out the months with no brews, which
spaced the busy months evenly and made the chart read as continuous
activity. The quiet months are now included with a zero.
- The run-detail chart was dropped when PR 1 was scoped down, leaving dead
code behind. It is restored.
Tests: 105 (up from 91).
|
||
|
hermes
|
3e20b558a5 |
fix(ci): harden the CA refresh script and add shellcheck to CI
Round-2 review findings, all low severity. - `curl` now has `--connect-timeout` and `--max-time`, so an unreachable API server fails the job instead of hanging the runner indefinitely. - A guard for a kubeconfig with no cluster entries, or an empty one, with a clear message rather than an opaque kubectl error. - The fingerprint mismatch message now prints both the expected and the actual value, which cuts the time to diagnose the next CA rotation. - `set -euo pipefail` was already there; the script now passes shellcheck clean, and a shellcheck step was added to the CI test job so the operational scripts get the same treatment as the Python code. |
||
|
hermes
|
a4b3d40e2d |
fix(ci): pin the cluster CA in the repo rather than in a secret
The CA fingerprint is not a secret. It is a public value that anyone can read straight off the certificate, so pinning it in a Gitea secret was the wrong shape: - A change to it would be invisible - no commit, no review. - Nobody with secret-write access could silently repoint the CI runner at a different CA. It now lives in `scripts/cluster-ca.sha256`, so changing it is a reviewable commit. The script reads it from there, and still accepts an override as its first argument. This also removes the need for a `KUBE_SERVER_CA_FINGERPRINT` secret, which the bot token cannot create anyway - the Actions secrets API is owner-only. The behaviour is unchanged: the CA is fetched over an unverified connection but only USED if it matches the pinned fingerprint, so an intercepted CA is refused. Re-tested against the live API server - the matching fingerprint refreshes the CA and kubectl then works; a wrong one is refused and the stale CA is left in place. |
||
|
hermes
|
ce6f5b7962 |
fix(ci): pin the cluster CA fingerprint instead of trusting a fetched CA
The review was right on both counts, and both are fixed.
1. MITM. Fetching the CA over an unverified connection (`curl -sk`) and then
trusting it is a privilege-escalation hole: the runner holds cluster-admin
credentials, so an attacker able to intercept the fetch could hand us a CA
of their choosing and then read everything sent over the connection.
The CA is now PINNED. It is still fetched over an insecure connection, but
it is only USED if its SHA-256 fingerprint matches
`KUBE_SERVER_CA_FINGERPRINT`. An intercepted CA will not hash to the
pinned value, so it is refused. If the secret is not set, the job fails
rather than trusting an unverified CA.
2. The hardcoded cluster name. `kubectl config set-cluster default` creates a
NEW cluster entry when the kubeconfig's entry is called something else,
leaving the original (and its stale CA) in use. The name is now read from
the kubeconfig: `{.clusters[0].name}`.
The bash is also moved out of the YAML into `scripts/kube-ca-refresh.sh`, so
it can be read, reviewed and tested on its own rather than as a wall of
shell inside a workflow file.
All three paths are tested against the live API server:
- no fingerprint supplied -> refuses, exit 1
- wrong fingerprint -> refuses, exit 1
- correct fingerprint -> refreshes the CA, exit 0, kubectl then works
|
||
|
hermes
|
687339bb9e |
fix(ci): self-heal a stale cluster CA in the deploy job
The deploy job failed on its first run:
error: error validating "kube/beer_namespace.yaml": error validating
data: failed to download openapi: Get "https://192.168.178.160:6443/openapi/v2":
tls: failed to verify certificate: x509: certificate signed by unknown authority
This is the same class of problem as the buildx driver earlier: the
certificate-authority-data embedded in the kubeconfig secret does not match
the CA the API server is currently presenting.
Rather than requiring the secret to be regenerated by hand every time the
cluster's CA is rotated, the job now refreshes it:
- The API server publishes its own CA at `/cacerts`.
- If kubectl cannot verify the server, the job fetches that CA (over an
insecure connection — verifying it is the problem being solved) and pins
it explicitly with `kubectl config set-cluster`.
- It then retries, and fails with a clear message if that still does not
work.
Verified by deliberately corrupting the CA in a working kubeconfig: the
failure reproduces exactly, and the recovery path restores connectivity
(`Server Version: v1.34.5+k3s1`).
The secret is also switched from `KUBEC_CONFIG_BUILDX` to
`KUBEC_CONFIG_BUILDX_NEW_2`, which is the one the working `wedding-photos`
workflow uses.
|
||
|
hermes
|
958ae9b365 |
fix(ci): keep the credentials out of the process list
Two small changes to the deploy job: - The registry password went through `-u armistace:$PASSWORD` on the curl command line, where it is visible in the process list for the duration of the request. It now goes through a curl config file on stdin instead. - The kubeconfig is written with mode 600 rather than the default, and is removed at the end of the job (with `if: always()`, so it is removed even if a later step fails). It is cluster-admin-equivalent, so it should not be readable by anything else on the runner. Verified: `curl --config -` against the registry returns the same digest as the `-u` form. |
||
|
hermes
|
55d5e84150 |
fix(ci): pin the image to the commit SHA instead of the mutable :latest tag
The review flagged `:latest` as a critical issue, and it is right: a mutable
tag makes `kubectl rollout undo` meaningless and lets different pods in the
same deployment run different code if they restart at different times.
- The build job now pushes the image with BOTH `:latest` and
`:${{ github.sha }}`.
- The deployment manifest carries a sentinel tag rather than `:latest`, and
the deploy job sets the real SHA with `kubectl set image`. That makes the
deploy a genuine manifest change, so the Deployment rolls because its pod
template changed — not because it was told to. `kubectl rollout undo` now
means something: it goes back to the previous SHA.
- The digest check now compares against the SHA tag rather than `:latest`,
and it checks EVERY running pod instead of `items[0]`. During a rolling
update `items[0]` may still be an old pod that has not been terminated,
which would compare against the wrong digest. The job also fails if the
deployment is not fully rolled out.
- The smoke test now checks the app through the Service as well as directly
in the pod. The pod check alone would pass even if the Service selector
or the port mapping were wrong.
- `readOnlyRootFilesystem: true` — the app writes no files, so the root
filesystem can be read-only. /tmp is still writable via the emptyDir.
- The kubectl install is skipped when the runner already has one.
|
||
|
hermes
|
b73df93f20 |
feat(ci): add the deploy job so the app deploys from the repo
The workflow built and pushed the image but never deployed it - the cluster objects had been created by hand. This adds a `deploy` job that applies the manifests in `kube/` and rolls the deployment. The deploy job is separate from the build job on purpose. The `KUBEC_CONFIG_BUILDX` secret is cluster-admin-equivalent, and a build step that never talks to the cluster should not be able to read it. Three things this job does that a naive `kubectl apply` does not: 1. `kubectl rollout restart`. The image tag is `:latest`, which is immutable in the manifest, so `apply` alone does not roll the pods. Without the restart the deployment reports success while the old pods keep serving the old image - a deploy that silently changes nothing. 2. Verifies the running digest against the registry's `:latest`. CI green and pods `Ready` do not prove the new code is live; the digest comparison does. 3. Smoke tests the running app - `/healthz` and `/readyz` - from inside the pod, so a deploy that comes up but cannot serve is caught in CI rather than by the next person to open the page. The Secrets (`beer-db`, `beer-app`) are hand-provisioned and are deliberately not in the apply set, so a deploy cannot clobber them. |
||
|
hermes
|
a48fb1c566 |
ci: drop the kubeconfig from the build job
The independent review flagged that the build job wrote the `KUBEC_CONFIG_BUILDX` secret to disk even though the build does not need cluster access at all. That secret is cluster-admin-equivalent: a build step that never talks to the cluster should not be able to read it, and a compromised dependency in the build would otherwise have a direct path to the cluster. The "Create Kubeconfig" step is removed from the build job. The deploy step, added in a later PR, gets its own job with the kubeconfig - so the privilege is only present where it is actually used. Also checked the buildkit daemon's resources in response to the review: it already has limits (2 CPU, 4Gi memory, 2Gi ephemeral storage) and requests (500m, 1Gi). The crash that motivated the amd64-only change was qemu exhausting memory during multi-arch emulation, not an unconstrained daemon, so no change is needed there. |
||
|
hermes
|
e9ce0dcf9b |
ci: use the remote buildx driver, and build amd64 only
The build job failed at "Set up Docker Buildx":
Error: ERROR: error for bootstrap "builder-...":
Get "https://192.168.178.160:6443/apis/apps/v1/namespaces/gitea-runner/deployments/builder-...":
tls: failed to verify certificate: x509: certificate signed by unknown authority
Cause
------
The `kubernetes` buildx driver talks to the cluster API to create its builder
pods. That needs the k3s CA certificate, which the job container does not
have - the `catthehacker/ubuntu:act-latest` image ships the public CA set
only. So the bootstrap could not verify the API server.
Fix
---
Switch to the `remote` driver, which talks straight to the buildkit daemon
already running in the cluster:
driver: remote
endpoint: tcp://buildkit.gitea-runner.svc:1234
No cluster API access is needed at all, so no kubeconfig is needed for the
build. This is the same pattern the wedding-photos workflow uses.
`endpoint` must be a top-level key, not a driver-opt: the action appends it
as the POSITIONAL argument to `buildx create --driver remote <endpoint>`, and
passing it in driver-opts yields "ERROR: no remote endpoint provided".
Also
----
The build was multi-arch (linux/amd64,linux/arm64), which runs qemu
emulation and crashes the buildkit pod on this NUC (exit 255,
SandboxChanged), after which the build hangs for many minutes. The cluster
is amd64, so the image is built for linux/amd64 only.
Verified: `docker buildx create --driver remote tcp://buildkit.gitea-runner.svc:1234`
succeeds from the runner's dind container and reports buildkit v0.29.0 with
linux/amd64 among its supported platforms.
|
||
|
hermes
|
ba65ec5a8f |
ci: drop actions/setup-python, which hangs on this runner
The setup-python action downloads a Python build from GitHub. This runner cannot reach GitHub reliably for that, and the test job hung on it with no output. The runner image already carries a usable python3, so the tests use it directly and install into a local venv. |
||
|
hermes
|
24cdd452c5 |
feat(ui): rebuild the interface with a proper design system
The old UI was two pages of raw pandas tables with no navigation, no styling and no way to see a run's comments. This replaces it with a small, testable application and a beer-themed design. What changed ------------ Restructure - The domain logic now lives in `brewing.py` as pure functions over plain data, with no Flask or Mongo dependency. The gravity maths, the ABV formula and the run-aggregation rules are unit-testable on their own. - `app_factory.py` replaces the module-level app, and the routes move into blueprints under `views/`. A request-scoped Mongo repository is created per request and closed at teardown. - `mongo/repository.py` replaces `build_db.py` / `query_db.py` / `get_conn.py`. Reading a document no longer requires a second query to fetch the fields. Interface - A dark, beer-themed design system: warm amber `#be7f39` on near-black, a condensed display face (Anton) for headings and Inter for body text. Both fonts and the chart library are vendored into `static/`, so the container needs no outbound network at runtime — the old templates pulled Bootstrap from a CDN. - A real navigation bar, a login page, a brew-process page with a proper form, and a run-detail page listing every reading with its comment. - The run summary tables now show the beer name and the latest comment, which the old pandas tables did not. - Native form controls (checkbox, number spinners, date icon, scrollbars) are restyled for the dark theme. Correctness - The alcohol calculation now uses the standard homebrew formula, `(OG - FG) x 131.25`. The old code used `(max - min) / 7.36 + 0.5`, which is not a recognised brewing formula and overstated every result. - Gravity points and decimal readings now convert correctly in both directions (`1039` <-> `1.039`). - A run's display name resolves to the most-used label across its readings, because `beer_type` is stored per-reading and drifts across a run. - Re-submitting a reading for the same run and day updates it in place rather than creating a duplicate. Security / production - The container ran the Werkzeug debugger with `FLASK_DEBUG=1`, exposing an interactive console on any unhandled error. It now runs waitress with debugging off. - `SECRET_KEY` comes from the environment. - The app secret is no longer a hard-coded string in source. Housekeeping - Removes the bokeh/duckdb/pandas dependency chain and the static file it generated at import time; removes the dead `add_user.py` and `test.py`. - Splits `requirements.txt` (runtime) from `requirements-dev.txt` (tests). - Adds a pytest suite (64 tests) and a `test` job to the CI workflow so the image is only built when the tests pass. - Adds a README describing the data model and a RUNBOOK for the cluster side. Data compatibility ------------------ The existing `beer_db.beer_data` documents are read as they are and are not migrated or rewritten. No collection is dropped, and no document is modified by this change other than through the explicit "record a reading" form, which keeps the original `(date, beer_run_id)` uniqueness rule. |
||
 armistace
|
592de4793b |
fix workflow
Build and Push Image / Build and push image (push) Failing after 1m16s
|
||
|
armistace
|
1470f0979a |
initial
Build and Push Image / Build and push image (push) Has been cancelled
|