hermes/atheos-blog-lxc
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: provision Alpine LXC 136 running Atheos, wired to Gitea

Browse Source 
Provisions a LAN-only browser code editor (Atheos) in an Alpine 3.23 LXC on the
scotty PVE node, with armistace/blog cloned and ready to edit. Built via the
AI-DLC workflow; the full process trail is in aidlc-docs/.

Deliverables
- scripts/provision.sh          rebuild the guest from scratch (re-runnable)
- scripts/update-atheos.sh      move Atheos to latest upstream and RE-PIN
- scripts/build-and-test.sh     captures evidence for every acceptance criterion
- files/atheos.conf             nginx vhost (socket path byte-matches the FPM pool)
- files/ssh-config              ~atheos/.ssh/config deploy-key alias
- RUNBOOK.md                    operational guide (START AT STEP 1)
- docs/security-notes.md        credential model + wider-web hardening TODOs
- docs/resiliency-notes.md      what a non-PoC deployment would need

Design decisions
- D1 no git identity is configured by the agent; the operator sets it. Git will
  refuse the first commit until then, which is deliberate (requirement FR-11).
- D2 Atheos pinned to commit f90cd18 (build 611) with a verified tarball sha256,
  plus update-atheos.sh as the deliberate escape hatch.
- D3 nginx :80, web root /var/www/atheos, php-fpm over a unix socket.
- D6 git auth is a per-repository DEPLOY KEY scoped to armistace/blog, not a
  personal access token (a PAT's write:repository scope reaches every repo the
  account can write).

Verified on the live guest (40 checks, 0 failures)
- AC-2  Atheos served over HTTP 200 by nginx+php-fpm; socket paths byte-match
- AC-3  20 posts + themes/cleanblog present; owned by the php-fpm user
- AC-4  all five writable paths writable as the service user
- AC-5  remote targets armistace/blog; deploy key authenticates over SSH
- AC-6  survival proven with a real container restart, marker byte-identical
- AC-8  no python/pelican/pip/make and no DB engine - scope boundary enforced
- FR-11 git identity is empty; deploy key mode 600 owned by the service user

Traps documented in the RUNBOOK (each cost real debugging time)
- The Gitea SSH user is `gitea`, NOT `git` (app.ini SSH_USER=gitea). With `git`,
  sshd answers "Invalid user git" and rejects before reading authorized_keys,
  which is indistinguishable from a bad key.
- The Gitea SSH endpoint is the LAN address (:22 is not forwarded publicly).
- `uptime` and `free` report the PVE HOST inside an LXC without lxcfs, so a
  container that just restarted can claim days of uptime.
This commit is contained in:
hermes
2026-09-29 15:50:34 +10:00
parent 27272e6f6f
commit a6fb61d0ea
26 changed files with 2420 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
RUNBOOK.md
+298
View File
@@ -0,0 +1,298 @@
# RUNBOOK — atheos-blog-lxc
Atheos browser editor for `armistace/blog`, running in an Alpine LXC on Proxmox.
| | |
|---|---|
| **Guest** | LXC **136** on node `scotty` (192.168.178.69) |
| **IP** | **http://192.168.178.57/** |
| **OS** | Alpine 3.23.2 · unprivileged · 2 vCPU · 1 GB RAM · 20 GB root · `onboot=1` |
| **App** | Atheos IDE, pinned to commit `f90cd1806dbe0e25673a77e27390e8b1a240c3f6` (build 611) |
| **Repo** | `git.aridgwayweb.com/hermes/atheos-blog-lxc` (this provisioning repo) |
| **Workspace** | `/var/www/atheos/workspace/blog` → clone of `armistace/blog` |
| **Access** | LAN only. No NPM proxy host, no DNS record, no port-forward. |
---
## ⚠️ STEP 1 — DO THIS FIRST: set your git identity
**The agent deliberately did not configure a git identity** (design decision D1, requirement FR-11).
Git will **refuse to commit** until you set one. This is your first action after SSH-ing in.
```sh
ssh root@192.168.178.57
# become the user that runs PHP and owns the workspace
su -s /bin/sh atheos
# set YOUR identity (these values are placeholders - use your real ones)
git config --global user.name "Andrew Ridgway"
git config --global user.email "ar17787@gmail.com"
# verify
git config --global --get user.name
git config --global --get user.email
```
> Why you and not the agent: you own the `armistace` identity. Configuring it under your name
> programmatically would impersonate you and blur commit attribution.
The same identity must be set for the deploy key's user, which is the **same account** (`atheos`),
so the single `su` above covers both the editor's commits and anything you do by hand.
---
## STEP 2 — install the deploy key (Phase 2 prerequisite)
The guest already has a per-repo deploy keypair. Only the **public** half is shared:
```sh
# print it again if you need it
cat /home/atheos/.ssh/id_ed25519_blog.pub
```
Add it in Gitea:
1. Open `armistace/blog` → **Settings** → **Deploy Keys** → **Add Deploy Key**
2. Title: `atheos-blog-lxc`
3. Paste the public key
4. ✅ **TICK "Allow write access"** — required, otherwise push is rejected
Public key (as generated 2026-09-29):
```
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINj03Tep9pULTvVDCC3q1ZtocKTZSmDhPsB61Zic1SPk atheos-blog-lxc deploy key (armistace/blog)
```
Fingerprint to confirm you pasted the right one:
```
SHA256:qHvsjjSX79aPZRoNTRI4j+HH3hArAuINIRu9ewxx7t8 (ED25519)
```
---
## STEP 3 — switch the remote to SSH (Phase 2)
Once the key is installed, tell the agent (or run it yourself):
```sh
su -s /bin/sh atheos -c \
'git -C /var/www/atheos/workspace/blog remote set-url origin gitea-blog:armistace/blog.git'
# verify the transport actually authenticates
su -s /bin/sh atheos -c 'ssh -T gitea-blog' # expect a Gitea greeting, not "Permission denied"
su -s /bin/sh atheos -c 'git -C /var/www/atheos/workspace/blog fetch --dry-run'
```
### Why the remote uses `gitea-blog` and not a hostname
`gitea-blog` is an SSH config alias defined in `/home/atheos/.ssh/config`:
```
Host gitea-blog
HostName 192.168.178.155 # LAN address
User gitea # <-- NOT `git`! see the trap below
IdentityFile ~/.ssh/id_ed25519_blog
IdentitiesOnly yes
Port 22
```
> ### ⚠️ TRAP: the SSH user is `gitea`, not `git`
> Every GitHub/Gitea tutorial uses `git@host`. **This instance does not.** Its `app.ini` sets
> `SSH_USER = gitea`.
>
> If you use `User git`, sshd replies **`Invalid user git`** and rejects the connection
> **before it ever reads `authorized_keys`** — so the error looks exactly like a rejected key
> and sends you hunting for a problem that isn't there. Confirmed in the sshd log on LXC 109:
> ```
> Invalid user git from 192.168.178.57 port 39780
> Connection closed by invalid user git 192.168.178.57 [preauth]
> ```
> Correct user → `Hi there! You've successfully authenticated with the deploy key named atheos-blog-lxc`.
Two reasons for the alias:
1. **`IdentitiesOnly yes`** means no other key is ever offered — a stray `git push` cannot
authenticate with a different (possibly broader) identity.
2. **It's the LAN address because that's the only path that works.** Verified 2026-09-29:
| Endpoint | Result |
|---|---|
| `git.aridgwayweb.com:22` | **unreachable** — the public path does not forward :22 (only :443 via NPM) |
| `192.168.178.155:22` | **OPEN** — Gitea's `app.ini` sets `SSH_DOMAIN = 192.168.178.155`, `SSH_PORT = 22`, `START_SSH_SERVER = false` |
So Gitea serves SSH from the LXC's own `sshd` and advertises the LAN address. Since this guest is
LAN-only, LAN transport is also the correct choice — no internet round-trip.
---
## STEP 4 — complete the Atheos installer
Open **http://192.168.178.57/** in a browser. The Atheos first-run installer appears and creates
the **admin user** — this is yours to own; the agent seeded no accounts.
Verify afterwards:
- you can open the **Workspace** and see a `blog` project
- opening `src/content/*.md` shows your posts
- saving persists (the files are at `/var/www/atheos/workspace/blog`)
### Atheos git settings
Open Atheos → (user menu) → **Settings** to confirm the git integration. The `codegit` component
handles commit/push. If you see *"detected dubious ownership"*, the `safe.directory` entries are
missing — restore them with:
```sh
su -s /bin/sh atheos -c 'git config --global --add safe.directory /var/www/atheos/workspace'
su -s /bin/sh atheos -c 'git config --global --add safe.directory /var/www/atheos/workspace/blog'
```
---
## Day-to-day operations
### Edit and commit
Do everything in the browser at `http://192.168.178.57/`. Commit via Atheos's git panel, or by hand:
```sh
ssh root@192.168.178.57
su -s /bin/sh atheos
cd /var/www/atheos/workspace/blog
git status
git add -A && git commit -m "post: <what changed>"
git push
```
### The build is NOT here
This box **only writes markdown**. Building/publishing the Pelican site happens in the existing
pipeline elsewhere. There is deliberately **no Python, no pelican, no `make`** on this guest.
### Services (OpenRC — not systemd)
```sh
rc-service nginx status
rc-service php-fpm83 status
rc-update show | grep -E 'nginx|php-fpm83' # both must be in 'default'
```
### Logs
```sh
tail -f /var/log/nginx/atheos.error.log # PHP fatals surface here
tail -f /var/log/nginx/atheos.access.log
```
### Reboot / recovery
```sh
pct reboot 136 # from scotty, or
pct stop 136 && pct start 136
```
Services are enabled at the OpenRC `default` runlevel, so they return automatically
(verified — see `aidlc-docs` acceptance criteria AC-6).
### Rebuild from scratch
The whole guest is reproducible:
```sh
# on scotty (192.168.178.69), as root
pct destroy 136
sh scripts/provision.sh # from this repo, on the node
```
Nothing unique lives on the guest: the blog's source of truth is Gitea. Only **uncommitted edits**
would be lost — commit often.
### Update Atheos (and re-pin)
Atheos has no releases, so this install is pinned to a commit. To move to the latest upstream commit,
install it, and re-pin:
```sh
sh scripts/update-atheos.sh # on the guest; preserves workspace/, backs up, verifies, re-pins
```
---
## Security model (summary — full detail in `docs/security-notes.md`)
- **Git auth = a per-repository deploy key**, scoped to `armistace/blog`, write-enabled.
Not a PAT (whose `write:repository` scope reaches **every** repo the account can write).
- Private key: `/home/atheos/.ssh/id_ed25519_blog`, mode `600`, owned by the php-fpm user.
- The web app and the key share one unprivileged user, so a compromised Atheos **can** write `blog`.
That is the accepted floor — one repo, and (with branch protection) one branch.
- **LAN-only.** If this is ever exposed: keep branch protection on `blog`'s `master`, terminate TLS
at NPM, and put authentication *in front* of Atheos. See `docs/security-notes.md` §2 (S-1…S-14).
### Container resource reporting (a real trap — cost debugging time)
`uptime` and `free` on Alpine are **busybox** applets. Busybox reads via the **`sysinfo()` syscall**,
which lxcfs **cannot intercept** — so inside this container they report the **PVE host**, not the
container. A container that restarted seconds ago will cheerfully claim *"up 2 days"*.
lxcfs **is** working correctly here — the virtualised files are right:
```
/proc/meminfo → MemTotal: 1048576 kB ← our 1024 MB limit
/proc/uptime → 105 s ← container
/proc/cpuinfo → 2 processors ← container
```
The container is correct; **the tool is wrong**. Fix = `apk add procps-ng`, whose `/usr/bin/free`
and `/usr/bin/uptime` read those virtualised `/proc` files:
| Tool | Reports | |
|---|---|---|
| `procps-ng` `free` | `1024 MB` | ✅ correct |
| busybox `free` | `15701 MB` | ❌ host |
| `procps-ng` `uptime` | `up 2 min` | ✅ correct |
| busybox `uptime` | `up 2 days` | ❌ host |
> **Note:** `apk add lxcfs` inside the guest is **not** a fix — lxcfs is a host-provided FUSE
> filesystem, not a guest package. `provision.sh` installs `procps-ng` for this reason.
> (`df` is unaffected: it stats the filesystem directly, so it was always accurate.)
---
## Reaching the guest when SSH from your laptop fails
The guest has **root SSH enabled** with your fleet keys installed:
- `ssh-rsa ... aridgway@strategenics-25`
- `ssh-ed25519 ... oct2025`
If those keys aren't what you're using, add/push a key via the PVE node (which always works):
```sh
ssh root@192.168.178.69 "pct exec 136 -- sh -c 'echo \"<your-pubkey>\" >> /root/.ssh/authorized_keys'"
```
---
## Troubleshooting
| Symptom | Cause | Fix |
|---|---|---|
| **502 Bad Gateway** | nginx `fastcgi_pass` and the FPM pool `listen` disagree | `grep -E '^listen' /etc/php83/php-fpm.d/www.conf` and compare with the `fastcgi_pass` line in `/etc/nginx/http.d/atheos.conf`. Both must read `/run/php-fpm83/php-fpm.sock`. Then restart both services. |
| **"detected dubious ownership"** | `safe.directory` missing or the repo is owned by a different user | Re-add the two `safe.directory` entries (STEP 4) |
| **git commit refuses: "please tell me who you are"** | identity not set — **expected on a fresh build** | STEP 1 |
| **git push: Permission denied (publickey)** | deploy key not installed, **or the SSH user is wrong**, or "Allow write access" not ticked | Check all three. Especially: `grep User /home/atheos/.ssh/config` must say **`gitea`**, not `git` (see STEP 3 trap). On the Gitea host, confirm with `journalctl -u ssh \| tail` — `Invalid user git` means the username is wrong, not the key. |
| **push times out** | remote still pointing at `git.aridgwayweb.com` (:22 not forwarded) | STEP 3 — use the `gitea-blog` alias |
| **Atheos shows a blank page / 500** | PHP fatal | `tail -50 /var/log/nginx/atheos.error.log` |
| **Atheos installer says a path isn't writable** | ownership broken | `chown -R atheos:atheos /var/www/atheos` |
| **`free` / `uptime` show the PVE host's memory or days of uptime** | on Alpine these are **busybox** applets using the `sysinfo()` **syscall**, which lxcfs cannot intercept | `apk add procps-ng` — its `/usr/bin/free` and `/usr/bin/uptime` read the lxcfs-virtualised `/proc` files. Already installed by `provision.sh`; this bites if you rebuild by hand. |
| **PHP session errors** | rootfs full | `df -h /` (the rootfs is 20 GB and the app is tiny, so this indicates something else) |
---
## Change log
| Date | Change |
|---|---|
| 2026-09-29 | Initial build: LXC 136 on scotty, Alpine 3.23.2, nginx + php83-fpm (socket), Atheos pinned at `f90cd18` (build 611), `armistace/blog` cloned over HTTPS, per-repo deploy key generated. LAN-only. |