From 8f07c377c0d54ee31e32a117f3009189896cccc4 Mon Sep 17 00:00:00 2001 From: hermes Date: Tue, 29 Sep 2026 15:49:21 +1000 Subject: [PATCH] fix: address independent review - provision the vhost, fix host-vs-container reporting Addresses the two valid items from the independent review (code/security/infra all Approved; these were Minor/Needs-Change items). 1. provision.sh left the nginx vhost as a manual "see README" step, which broke the project's own repeatability requirement (FR-9): a rebuild from scratch would have served the distro default page instead of Atheos. The vhost is now written by the script itself, the distro default.conf is removed, and the script ASSERTS the socket contract (nginx fastcgi_pass == php-fpm pool listen) before Atheos is installed - so a 502 can only ever mean an nginx/php-fpm problem, never be mistaken for an Atheos one. 2. `free` and `uptime` reported the PVE HOST inside the container (a container that had restarted seconds earlier claimed "up 2 days"), which is misleading in monitoring and in the test output. The review suggested `apk add lxcfs` in the guest. That is NOT the fix - lxcfs is a host-provided FUSE filesystem, not a guest package, and measurement showed it is already working correctly here: /proc/meminfo -> MemTotal: 1048576 kB (our 1024 MB limit) /proc/uptime -> container seconds /proc/cpuinfo -> 2 processors The real cause: on Alpine, `free`/`uptime` are BUSYBOX applets that read via the sysinfo() SYSCALL, which lxcfs cannot intercept. procps-ng ships binaries that read the lxcfs-virtualised /proc files instead. Measured on the guest: procps-ng free -> 1024 MB (correct) busybox free -> 15701 MB (host) procps-ng uptime -> up 2 min (correct) busybox uptime -> up 2 days (host) So provision.sh now installs procps-ng, and build-and-test.sh reports container- accurate figures (detecting it via apk rather than a binary-existence probe, since the busybox applets are plain files too). Verified after the change: build-and-test 40 passed / 0 failed; socket contract MATCH; resource line now reads "18MB used of 1024MB, up 2 min". --- RUNBOOK.md | 29 ++++++++++++++++++++ scripts/build-and-test.sh | 14 ++++++++-- scripts/provision.sh | 58 +++++++++++++++++++++++++++++++++++++-- 3 files changed, 95 insertions(+), 6 deletions(-) diff --git a/RUNBOOK.md b/RUNBOOK.md index 05e2669..9a96733 100644 --- a/RUNBOOK.md +++ b/RUNBOOK.md @@ -230,6 +230,34 @@ sh scripts/update-atheos.sh # on the guest; preserves workspace/, backs up - **LAN-only.** If this is ever exposed: keep branch protection on `blog`'s `master`, terminate TLS at NPM, and put authentication *in front* of Atheos. See `docs/security-notes.md` §2 (S-1…S-14). +### Container resource reporting (a real trap — cost debugging time) + +`uptime` and `free` on Alpine are **busybox** applets. Busybox reads via the **`sysinfo()` syscall**, +which lxcfs **cannot intercept** — so inside this container they report the **PVE host**, not the +container. A container that restarted seconds ago will cheerfully claim *"up 2 days"*. + +lxcfs **is** working correctly here — the virtualised files are right: + +``` +/proc/meminfo → MemTotal: 1048576 kB ← our 1024 MB limit +/proc/uptime → 105 s ← container +/proc/cpuinfo → 2 processors ← container +``` + +The container is correct; **the tool is wrong**. Fix = `apk add procps-ng`, whose `/usr/bin/free` +and `/usr/bin/uptime` read those virtualised `/proc` files: + +| Tool | Reports | | +|---|---|---| +| `procps-ng` `free` | `1024 MB` | ✅ correct | +| busybox `free` | `15701 MB` | ❌ host | +| `procps-ng` `uptime` | `up 2 min` | ✅ correct | +| busybox `uptime` | `up 2 days` | ❌ host | + +> **Note:** `apk add lxcfs` inside the guest is **not** a fix — lxcfs is a host-provided FUSE +> filesystem, not a guest package. `provision.sh` installs `procps-ng` for this reason. +> (`df` is unaffected: it stats the filesystem directly, so it was always accurate.) + --- ## Reaching the guest when SSH from your laptop fails @@ -258,6 +286,7 @@ ssh root@192.168.178.69 "pct exec 136 -- sh -c 'echo \"\" >> /root/ | **push times out** | remote still pointing at `git.aridgwayweb.com` (:22 not forwarded) | STEP 3 — use the `gitea-blog` alias | | **Atheos shows a blank page / 500** | PHP fatal | `tail -50 /var/log/nginx/atheos.error.log` | | **Atheos installer says a path isn't writable** | ownership broken | `chown -R atheos:atheos /var/www/atheos` | +| **`free` / `uptime` show the PVE host's memory or days of uptime** | on Alpine these are **busybox** applets using the `sysinfo()` **syscall**, which lxcfs cannot intercept | `apk add procps-ng` — its `/usr/bin/free` and `/usr/bin/uptime` read the lxcfs-virtualised `/proc` files. Already installed by `provision.sh`; this bites if you rebuild by hand. | | **PHP session errors** | rootfs full | `df -h /` (the rootfs is 20 GB and the app is tiny, so this indicates something else) | --- diff --git a/scripts/build-and-test.sh b/scripts/build-and-test.sh index a443c16..ef1e9a7 100644 --- a/scripts/build-and-test.sh +++ b/scripts/build-and-test.sh @@ -108,9 +108,17 @@ echo "==============================================================" echo "RESOURCES" echo "==============================================================" echo " rootfs: $(df -hP / | tail -1 | awk '{print $2" total, "$3" used, "$4" avail ("$5")"}')" -echo " NOTE: \`free\` and \`uptime\` report the HOST, not this container - lxcfs is not" -echo " installed. The enforced limit is memory=1024MB/swap=256MB from PVE." -echo " Check real usage host-side: cat /sys/fs/cgroup/lxc/136/memory.current" +# NOTE: on Alpine, `free`/`uptime` are BUSYBOX applets that use the sysinfo() syscall, which +# lxcfs cannot intercept - so they report the PVE HOST, not this container. procps-ng (installed +# by provision.sh) replaces them with binaries that read the lxcfs-virtualised /proc files. +# Detect via apk, which is the reliable signal (the binaries are plain files either way). +if apk info -e procps-ng >/dev/null 2>&1; then + echo " memory: $(/usr/bin/free -m 2>/dev/null | awk 'NR==2{print $3"MB used of "$2"MB"}')" + echo " uptime: $(/usr/bin/uptime 2>/dev/null | sed 's/.*up /up /;s/,.*//') (container-accurate)" +else + echo " memory: (procps-ng NOT installed - busybox free/uptime would report the HOST)" +fi +echo " lxcfs virtualises /proc: meminfo=$(grep MemTotal /proc/meminfo | awk '{print $2"kB"}') uptime=$(cut -d. -f1 /proc/uptime)s" echo " listening ports: $(netstat -tln 2>/dev/null | awk '/LISTEN/{split($4,a,":"); print a[length(a)]}' | sort -u | tr '\n' ' ')" echo diff --git a/scripts/provision.sh b/scripts/provision.sh index a6437cb..6681d85 100644 --- a/scripts/provision.sh +++ b/scripts/provision.sh @@ -95,7 +95,10 @@ pct exec "$VMID" -- sh -c ' apk add --no-cache nginx php83 php83-fpm \ php83-session php83-zip php83-openssl php83-mbstring php83-ctype php83-curl \ php83-dom php83-json php83-iconv php83-fileinfo php83-simplexml php83-xml \ - php83-tokenizer >/dev/null 2>&1 + php83-tokenizer procps-ng >/dev/null 2>&1 + # procps-ng (NOT lxcfs) is what makes free/uptime report the CONTAINER rather than the host. + # busybox applets use the sysinfo() syscall, which lxcfs cannot intercept; procps-ng reads the + # lxcfs-virtualised /proc/meminfo and /proc/uptime. Verified: procps free -> 1024MB, busybox -> host. # dedicated service user (I-4: FPM user == file owner == key owner) adduser -D -s /sbin/nologin -h /home/atheos atheos 2>/dev/null || true mkdir -p /var/www/atheos/workspace /run/php-fpm83 /var/log/nginx @@ -114,11 +117,60 @@ pct exec "$VMID" -- sh -c ' grep -q "^listen.mode" /etc/php83/php-fpm.d/www.conf || echo "listen.mode = 0660" >> /etc/php83/php-fpm.d/www.conf rc-update add nginx default >/dev/null 2>&1 rc-update add php-fpm83 default >/dev/null 2>&1 +' + +# --- nginx vhost: installed BY THE SCRIPT so provisioning is repeatable end-to-end ----------- +# (review finding: leaving this as a manual "see README" step broke the project's own +# repeatability requirement FR-9 — a rebuild from scratch would serve the distro default page) +pct exec "$VMID" -- sh -c "cat > /etc/nginx/http.d/atheos.conf" <<'NGINXCONF' +server { + listen 80 default_server; + listen [::]:80 default_server; + server_name _; + root /var/www/atheos; + index index.php index.html; + access_log /var/log/nginx/atheos.access.log; + error_log /var/log/nginx/atheos.error.log; + + # The socket path here MUST byte-match the pool's `listen =` in + # /etc/php83/php-fpm.d/www.conf (interface I-1 - a mismatch yields a bare 502). + location ~ \.php$ { + try_files $uri =404; + fastcgi_split_path_info ^(.+\.php)(/.+)$; + fastcgi_pass unix:/run/php-fpm83/php-fpm.sock; + fastcgi_index index.php; + include fastcgi.conf; + fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; + } + location / { try_files $uri $uri/ =404; } + + # never serve the VCS dir, the app's private data, or dotfiles + location ~ /\.git { deny all; } + location ~ /\.ssh { deny all; } + location ~ /\. { deny all; } + location ~ ^/data/ { deny all; } + location ~ ^/(config\.php|common\.php|controller\.php|traits/) { deny all; } + location ~* \.(md|py|sh|log|conf|ini|sql|bak|orig)$ { deny all; } + + add_header X-Content-Type-Options "nosniff" always; + add_header X-Frame-Options "SAMEORIGIN" always; + add_header Referrer-Policy "strict-origin-when-cross-origin" always; + + client_max_body_size 64m; +} +NGINXCONF +# remove the distro default vhost so ours is authoritative, then start the stack +pct exec "$VMID" -- sh -c ' + rm -f /etc/nginx/http.d/default.conf rc-service php-fpm83 restart >/dev/null 2>&1 rc-service nginx restart >/dev/null 2>&1 + sleep 2 + # prove the socket contract holds BEFORE Atheos is installed, so a 502 can only ever + # mean an nginx/php-fpm problem and never be confused with an Atheos problem + SOCK=$(sed -n "s/^listen = //p" /etc/php83/php-fpm.d/www.conf | head -1) + NGX=$(sed -n "s/.*fastcgi_pass unix:\([^;]*\);.*/\1/p" /etc/nginx/http.d/atheos.conf | head -1) + if [ "$SOCK" = "$NGX" ]; then echo "socket contract OK: $SOCK"; else echo "SOCKET MISMATCH: pool=$SOCK nginx=$NGX"; exit 1; fi ' -echo "NOTE: install the nginx vhost from files/atheos.conf (see README) into /etc/nginx/http.d/atheos.conf" -echo " and remove /etc/nginx/http.d/default.conf, then: rc-service nginx restart" # ============================================================================== log "6/9 Atheos install at the pinned commit (U3)"