beer_data/RUNBOOK.md

# RUNBOOK — Brew Log (beer_data)

Operational notes for the `beer` namespace deployment.

## What runs where

| | |
|---|---|
| Namespace | `beer` |
| Deployment | `beer-deployment` (3 replicas) |
| Image | `git.aridgwayweb.com/armistace/beer-data:latest` |
| Service | `beer-service`, NodePort **30015** → container port 5000 |
| Database | MongoDB at `192.168.178.229`, database `beer_db` |
| Secret | `beercred` (registry pull credentials) |

Access: `http://<any-cluster-node>:30015/`

The pods are pinned to `archlinux-k3s-3` (`.161`) via `nodeSelector`.

## Environment variables

| Variable | Value | Source |
|---|---|---|
| `MONGO_HOST` | `192.168.178.229` | Deployment manifest |
| `MONGO_USER` | `root` | Deployment manifest |
| `MONGO_PASS` | *(see below)* | Deployment manifest |
| `SECRET_KEY` | *(see below)* | Deployment manifest — Flask session signing key. **Required** |
| `FLASK_ENV` | `production` | Dockerfile |
| `FLASK_DEBUG` | `0` | Dockerfile |

`SECRET_KEY` and `MONGO_PASS` must be set in the live Deployment. They are
**not** committed to this repo — read them from the running cluster:

```bash
kubectl -n beer get deploy beer-deployment -o jsonpath='{.spec.template.spec.containers[0].env}'
```

> **The app refuses to start without a real `SECRET_KEY`.** There is no default
> and no placeholder: a missing or placeholder value raises
> `ImproperlyConfigured` and the container exits. A known session-signing key
> would let anyone forge a login cookie, so a misconfigured deployment must
> fail loudly rather than run insecurely.
>
> Generate one with:
> ```bash
> python -c 'import secrets; print(secrets.token_hex(32))'
> ```

### Moving the credentials into a Kubernetes Secret

The live Deployment currently carries `MONGO_PASS` as a plain environment
value. Moving it to a Secret is the better shape and is **not** done here
because it would mean replacing the live hand-provisioned values — a change
that needs the operator's sign-off. When it is done:

```yaml
env:
  - name: MONGO_PASS
    valueFrom:
      secretKeyRef:
        name: beer-db
        key: mongo_pass
  - name: SECRET_KEY
    valueFrom:
      secretKeyRef:
        name: beer-app
        key: secret_key
```

## Deploying

Pushing to `master` triggers `.gitea/workflows/build_push.yml`, which runs the
test suite and then builds and pushes the image. **The workflow does not
currently deploy to the cluster** — apply the manifests yourself:

```bash
kubectl -n beer rollout restart deployment/beer-deployment
kubectl -n beer rollout status deployment/beer-deployment --timeout=180s
```

### Verify the deploy actually took

CI green and pods `Ready` do **not** prove the new code is running. Check the
running digest against the registry:

```bash
# what the pod is running
kubectl -n beer get pod -l app=beer-data \
  -o jsonpath='{.items[0].status.containerStatuses[0].imageID}'

# what the registry has as :latest
curl -s -u "armistace:$REG_PASSWORD" -I \
  "https://git.aridgwayweb.com/v2/armistace/beer-data/manifests/latest" \
  -H "Accept: application/vnd.docker.distribution.manifest.v2+json" \
  | grep -i docker-content-digest
```

If they differ, the registry `:latest` is stale — re-run the build rather than
re-applying the manifest.

### Check the app is alive

```bash
kubectl -n beer get pods
kubectl -n beer logs deploy/beer-deployment --tail=50
curl -s http://<node>:30015/healthz     # -> {"status":"ok"}
curl -s http://<node>:30015/readyz      # -> {"status":"ready"}
```

`/healthz` is the **liveness** probe and never touches Mongo — it answers "is
the process up". A database outage must not make the kubelet restart a healthy
process.

`/readyz` is the **readiness** probe and pings Mongo, returning 503 when the
database is unreachable. Wire it to the Service so a pod with no database is
pulled out of the endpoints rather than serving errors:

```yaml
readinessProbe:
  httpGet:
    path: /readyz
    port: 5000
  initialDelaySeconds: 5
  periodSeconds: 10
livenessProbe:
  httpGet:
    path: /healthz
    port: 5000
  initialDelaySeconds: 10
  periodSeconds: 20
```

## Data safety

The `beer_db.beer_data` collection holds the actual brewing records. There is
no migration step and no schema version — documents are read as they are
written, and every field is treated as optional.

To take a backup before any risky change:

```bash
mongodump --uri="mongodb://root:$MONGO_PASS@192.168.178.229:27017/beer_db" \
  --out=/tmp/beer_db_backup_$(date +%Y%m%d)
```

## Common problems

### The login page loads but nothing else does

The Mongo connection failed. Check the app logs for an auth error and confirm
the credentials:

```bash
kubectl -n beer logs deploy/beer-deployment --tail=100 | grep -i mongo
```

### A run shows a negative alcohol figure

That was the old formula (`(max - min) / 7.36 + 0.5`) applied to a run whose
current gravity is above the assumed original gravity of 1012. Fixed in the
current version: the standard `(OG - FG) × 131.25` formula is used, and a run
with no known OG shows a dash instead of a figure.

### The pod will not start after a deploy

The most likely cause is a missing `SECRET_KEY` — the app exits rather than
run with a default. Check the logs:

```bash
kubectl -n beer logs deploy/beer-deployment --tail=20
# ImproperlyConfigured: SECRET_KEY is not set. ...
```

### The chart on the run page is blank

The chart library is served from `/static/js/chart.umd.js` — it is vendored in
the repo, not loaded from a CDN, so the container works with no outbound
network. Check the browser console for a 404 on that path.