The instruction asks for "ADVICE:", but the model writes "**ADVICE:**",
"## ADVICE:" or puts it in a code block. The extraction has to cope,
because leftover markdown in the advice is visible to the brewer.
The marker is now matched case-insensitively and with the decoration the
model puts around it, and the leftover formatting is stripped.
plain marker -> 'Warm it to 20C and wait.'
bold marker -> 'Warm it to 20C and wait.'
in a code block -> 'Warm it to 20C and wait.'
marker mid-text -> 'Warm it to 20C and wait.'
markdown heading -> 'Warm it to 20C and wait.'
lowercase marker -> 'Warm it to 20C and wait.'
no marker -> 'Warm it to 20C and wait.'
bold + code block -> 'Warm it to 20C and wait.'
182 tests (up from 181).
The stripper tried to work out where the model's working ends and the
advice begins by classifying prose - marker words, hedging words,
length. It failed in BOTH directions, and the review caught both:
- it emptied legitimate advice (a 1,093-character reply containing one
hedging word and no marker phrase was reduced to ""), so the brewer
saw nothing at all; and
- it let the model's working through in full (a 1,053-character reply
with no hedging word was returned verbatim), so the brewer was shown
the model thinking about their beer.
Neither a marker list nor a length threshold is reliable. Prose is not
a format.
So the model is asked for the format instead. Its reply must have this
shape:
ADVICE:
<the advice itself>
Everything before the marker is working, everything from it on is the
advice. The split is exact, with no judgement calls.
Verified against the live model, four different brews:
Test Pale Ale marker@pos=0 finish=stop advice_len=656
Nocturna Christmas marker@pos=0 finish=stop advice_len=668
Ginger beer marker@pos=0 finish=stop advice_len=621
Cider raspberry marker@pos=0 finish=stop advice_len=539
All four start clean, with no marker left in the advice.
A reply with no marker is used as it stands - the safe direction, where
the brewer sees a slightly rambling reply rather than no advice at all.
An empty advice is now recorded as a FAILURE rather than stored as a
successful generation. Storing an empty string with advice_ok=True
made the page render "Thinking..." for ever, because the template's
`{% if reading.advice %}` was false - so the brewer never learned that
it had failed.
181 tests (up from 184 - the eight heuristic-era stripper tests are
replaced by six that test the marker behaviour).
A length threshold is a blunt instrument: a legitimate advice reply can
run to a thousand characters, and a short working reply can be a few
hundred - so a threshold gets both wrong. A 995-character advice reply
with no marker word in it was being emptied.
The reliable signal is the SHAPE. The model's working is about
INTERPRETING the numbers - what the reading might mean, what could be
causing it, what it is unsure about. The advice is about what to DO.
So a reply is working when it is full of the model's hedging about the
data AND contains no instruction at all. Length is only a tie-breaker
for a reply that is both hedged and short.
995-char legitimate advice -> kept
long pure working -> dropped
short legitimate advice -> kept
long working + advice -> advice kept
Checked against the live model - a real reply comes back clean, with no
reasoning in it:
At two weeks in, dropping from 1.055 to 1.020 is a healthy 35-point
fall - about 64% attenuation - which is right in the normal range for
a pale ale, though you likely still have 5 to 8 points to go...
184 tests (up from 183).
Two findings from the fifth review.
The pre-token fallback re-opened the hole
-----------------------------------------
A reading that has never been edited has no token. The guard fell back
to the gravity check for those, and a comment-only correction leaves the
gravity unchanged - so a late thread's write matched and overwrote the
fresh advice, for exactly the 47 existing readings.
Reproduced live before the fix:
pre-token reading token: None -> stale thread carries ""
fresh write modified: 1
stale write modified: 1 <<< stale advice overwrote the fresh one
final advice: 'STALE ADVICE'
The guard now matches on the ABSENCE of a token, which still tells the
generations apart: an edit writes a token, so a stale thread's "" stops
matching the moment the reading is corrected.
fresh write modified: 1 (must be 1)
stale write modified: 0 (must be 0)
The stripper could empty real advice
------------------------------------
"Actually the drop is fine.\\n\\nWait a couple of days and check again."
is advice, and judging it paragraph by paragraph emptied it - so the
brewer saw nothing at all, with no failure note.
The test is no longer a marker list, which real advice often does not
contain. A reply is all-working when it is LONG and has no instruction
in it: the model's working runs to thousands of characters, while the
advice is two to five sentences.
short reply with no marker -> kept
long reply, no instruction -> dropped
long reply with advice -> advice kept
single-paragraph working -> dropped
Also: the log message named the gravity, but the guard is the token.
182 tests (up from 178).
Two findings from the fourth review.
The stripper was a no-op in the case it was written for
---------------------------------------------------
The "all working" branch returned the text UNCHANGED, which is
byte-identical to the old fallthrough. And in the case where a preamble
marker WAS present and no paragraph read like advice, it returned the
whole reply - working included - so the branch made the "store the
working instead of the advice" failure MORE likely, not less.
Measured on a realistic 5-paragraph reasoning reply: the old code stored
80 chars (one paragraph), the new code stored 612 chars (five
paragraphs) of the model's working.
It now DROPS the working, so the reply is empty and the brewer sees a
clear "could not be generated" rather than the model thinking about
their beer.
"keep" alone is not an advice marker
------------------------------------
"I need to keep in mind that the hydrometer was read at a different
temperature" is the model thinking, not advice. The bare words "keep",
"raise", "lower", "take the" were matching ordinary reasoning phrasing,
so the "all working" detection almost never fired on real reasoning
text. They are now matched as phrases - "you should keep", "raise it
to", "take another reading" - and "i need to keep in mind" is
explicitly working.
The token is the primary guard
------------------------------
The gravity can be unparseable for some readings, so `expected_sg` can
be None - and then the write was becoming unconditional, reintroducing
the stale-overwrite bug on those readings.
The editing token is the primary guard: it changes on EVERY edit, so it
catches a stale generation whatever else changed - the gravity, the
comment, or nothing. The gravity is now a secondary check, used only
when there is no token.
177 tests (up from 174).
The review found a race between the background advice thread and the
brewer correcting a reading.
`set_reading_advice` wrote unconditionally by id. The advice is
generated in the background, so if the brewer corrects a reading while
the advice for the OLD gravity is still being generated, the slow thread
lands afterwards and overwrites the fresh advice with advice about a
reading that no longer exists. The reading ends up with the final
gravity from the correction and the advice from before it.
Reproduced against the live database:
wrote advice for 1050
wrote advice for 1020
late stale write matched: 0
final sg: 1020 | final advice: 'ADVICE FOR THE CORRECTION'
The write is now conditional on the reading still holding the gravity
the advice was generated for, so a late thread matches nothing.
Also
----
`finish_reason` is now checked. A reply that stopped at the token
ceiling is incomplete - it may be advice that stops mid-sentence - and
it was being stored as if it were the whole answer. It is now marked
as cut off.
173 tests (up from 171).
The review found a way to store the model's WORKING instead of its
advice.
The advice is at the END of the reply, so the length cap was cutting
from the front - which took the advice off and left only the working.
That is worse than no advice at all, because it looks like advice: the
brew sees a full paragraph of the model thinking about their beer, and
nothing about what to do.
Reproduced:
input len: 2515 -> final len: 1199
advice present? False
TAIL: 'padding padding padding padding padding'
Two changes:
- The length cap keeps the END of the reply and cuts at a sentence
boundary near the end, so the advice survives.
- The stripper now also catches a reply that is entirely working - no
marker phrase, and no paragraph that reads like advice - instead of
only catching one that says "let me think".
input: 2515 -> stripped: 2515 -> final: 1199
advice present? True
A short reply is untouched.
171 tests (up from 169).
The model is now `mistral-large-4:cloud`, on the same Ollama instance.
It was probed before being made the default - `deepseek-v4-flash:cloud`,
`deepseek-v3.2:cloud` and `qwen3.5:cloud` are all retired and fail at
call time while still appearing in the model list.
What the probes showed
----------------------
`think: false` does NOT stop the reasoning. It only changes where the
working goes: with thinking off, the model writes it into the CONTENT
instead of a separate field. Either way the reasoning comes out of the
same token allowance as the answer, and at 500-3000 tokens the reply
came back EMPTY - which looks like a broken model rather than a budget
problem. The budget is now 8000, and the timeout 120s.
The reply is also stripped of a reasoning preamble. The advice is the
prose at the END; the working comes first, in several paragraphs. The
preamble is found by walking back from the end of the reply and keeping
the paragraphs that read like advice to the brewer, rather than like
the model thinking out loud.
Verified against the live model, on a stalled brew (OG 1.042, 1.022 after
nine days, "no change since day 7"):
That 1.022 reading after nine days at 19C - with no movement since
day seven - looks like a stalled fermentation; a 1.042 wort should
be well on its way to 1.010-1.012 by now, not sitting at only about
48% attenuation. Gently swirl the fermenter to rouse the yeast cake
and keep the temperature steady at 19-20C, and if the gravity hasn't
budged in another 48 hours, pitch a fresh, healthy yeast starter...
And on a CORRECTED older reading - the case the review found broken -
the advice is about the corrected reading, with the later one as context:
A brew-day reading of 1.038 against an OG of 1.042 is only four points
low, which is almost certainly just the normal difference between
your kettle gravity and the post-transfer fermenter reading...
169 tests (up from 167).
Two findings from the re-review of PR #14.
A race that recreated the leak
-------------------------------
`get_shared_repository()` did an unsynchronised check-then-set on a
module global. Eight concurrent advice threads on a cold process each
saw None and each built their own repository - so eight MongoClients and
twenty-five threads. The fix that was meant to remove the leak
reintroduced it, just bounded by a race instead of by the number of
readings.
Measured before: 8 repositories built, 25 threads.
Measured after: 1 repository built, 4 threads.
There is a lock, and the check is repeated inside it.
The prompt was about the wrong reading
--------------------------------------
The advice targeted the right reading, but the PROMPT said "give advice
for the most recent reading" - and when the brewer corrects an OLDER
reading, the most recent one in the history is a different reading. So
the model was asked about the wrong number while the advice was filed
against the right one.
The reading is now named in the instruction (with its date and gravity),
marked in the reading list as the one to advise on, and the system prompt
says to advise about the marked reading because it may not be the most
recent one.
A corrected reading is also flagged as such, so the model knows it may
differ from what it said before.
167 tests (up from 164).
Stores a model's advice for each recorded reading, in its own field, so the
brewer's own comment is never overwritten.
Fire and forget: the advice is generated in a background thread, so
recording a reading never blocks on the model. A failure is recorded as a
visible note saying the advice could not be generated, because a silent
failure is indistinguishable from "still thinking".
The model is `glm-5.3:cloud` on the Ollama instance at 192.168.178.45:11434.
Several models on that instance are retired and fail at call time while
still appearing in the model list, so the default is one that was probed.
The system prompt tells the model never to call a reading impossible: a
gravity below 1.000 is always a mis-entry, but the rest of the run may be a
genuine long fermentation and still worth understanding.
150 tests.
hermes