fix(advice): the pre-token guard, and a stripper that cannot empty real advice

Two findings from the fifth review.

The pre-token fallback re-opened the hole
-----------------------------------------
A reading that has never been edited has no token.  The guard fell back
to the gravity check for those, and a comment-only correction leaves the
gravity unchanged - so a late thread's write matched and overwrote the
fresh advice, for exactly the 47 existing readings.

Reproduced live before the fix:

    pre-token reading token: None  ->  stale thread carries ""
    fresh write modified: 1
    stale write modified: 1   <<< stale advice overwrote the fresh one
    final advice: 'STALE ADVICE'

The guard now matches on the ABSENCE of a token, which still tells the
generations apart: an edit writes a token, so a stale thread's "" stops
matching the moment the reading is corrected.

    fresh write modified: 1 (must be 1)
    stale write modified: 0   (must be 0)

The stripper could empty real advice
------------------------------------
"Actually the drop is fine.\\n\\nWait a couple of days and check again."
is advice, and judging it paragraph by paragraph emptied it - so the
brewer saw nothing at all, with no failure note.

The test is no longer a marker list, which real advice often does not
contain.  A reply is all-working when it is LONG and has no instruction
in it: the model's working runs to thousands of characters, while the
advice is two to five sentences.

    short reply with no marker  -> kept
    long reply, no instruction  -> dropped
    long reply with advice      -> advice kept
    single-paragraph working    -> dropped

Also: the log message named the gravity, but the guard is the token.

182 tests (up from 178).
This commit is contained in:
2026-11-09 19:08:28 +10:00
parent 7d9d718b41
commit d18d037bb9
4 changed files with 153 additions and 14 deletions
+52 -1
View File
@@ -276,7 +276,10 @@ def _strip_reasoning(text: str) -> str:
# be a no-op, and the brewer would be shown the model thinking about
# their beer instead of any advice at all.
paragraphs = [p.strip() for p in text.split("\n\n") if p.strip()]
if len(paragraphs) > 1 and not any(_looks_like_advice(p) for p in paragraphs):
if paragraphs and _is_all_working(text, paragraphs):
# It is all working. It is dropped, so the brewer sees a clear
# "could not be generated" rather than the model thinking about
# their beer.
return ""
for marker in _PREAMBLE_MARKERS:
@@ -291,6 +294,9 @@ def _strip_reasoning(text: str) -> str:
# running text stops looking like working.
paragraphs = [p.strip() for p in tail.split("\n\n") if p.strip()]
if len(paragraphs) < 2:
# A single paragraph: the whole tail is one block of working
# with the advice inside it, so it is kept whole rather than
# thrown away.
continue
# Walk back from the end while the paragraph still reads like the
@@ -342,6 +348,51 @@ _ADVICE_MARKERS = (
)
#: A reply longer than this is the model's working, not the advice.
#: The advice is meant to be two to five sentences.
_WORKING_LENGTH = 600
def _is_all_working(text: str, paragraphs: list[str]) -> bool:
"""True when the reply is the model's working rather than advice.
Two signals, because either alone is unreliable:
- **Length.** The working runs to thousands of characters; the
advice is two to five sentences.
- **Shape.** The working interprets the numbers ("perhaps the yeast
is stressed, it could be that..."); the advice tells the brewer
what to do.
A reply that is SHORT is kept even if it has no marker word - real
advice is often phrased without one ("Actually the drop is fine.
Wait a couple of days and check again.").
"""
if len(text) <= _WORKING_LENGTH:
# Too short to be the model's working. It is advice, whatever
# it is worded like.
return False
if _has_advice_marker(text):
# It contains an instruction, so part of it is advice.
return False
# Long, and nothing in it reads like an instruction.
return True
def _has_advice_marker(text: str) -> bool:
"""True when the text contains any advice marker at all.
Used on the WHOLE reply rather than paragraph by paragraph, because
advice can be phrased without any of the markers - "Actually the
drop is fine.\n\nWait a couple of days and check again." is advice,
and judging it paragraph by paragraph emptied it.
"""
lowered = text.lower()
return any(marker in lowered for marker in _ADVICE_MARKERS)
def _looks_like_advice(paragraph: str) -> bool:
"""True when a paragraph reads like advice to the brewer.