armistace/beer_data
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(advice): judge the working by its shape, not by its length

Browse Source 
A length threshold is a blunt instrument: a legitimate advice reply can
run to a thousand characters, and a short working reply can be a few
hundred - so a threshold gets both wrong.  A 995-character advice reply
with no marker word in it was being emptied.

The reliable signal is the SHAPE.  The model's working is about
INTERPRETING the numbers - what the reading might mean, what could be
causing it, what it is unsure about.  The advice is about what to DO.

So a reply is working when it is full of the model's hedging about the
data AND contains no instruction at all.  Length is only a tie-breaker
for a reply that is both hedged and short.

    995-char legitimate advice  -> kept
    long pure working           -> dropped
    short legitimate advice     -> kept
    long working + advice       -> advice kept

Checked against the live model - a real reply comes back clean, with no
reasoning in it:

    At two weeks in, dropping from 1.055 to 1.020 is a healthy 35-point
    fall - about 64% attenuation - which is right in the normal range for
    a pale ale, though you likely still have 5 to 8 points to go...

184 tests (up from 183).
This commit is contained in:
hermes
2026-11-09 19:27:08 +10:00
parent 4bedd23c24
commit cbc71716e8
2 changed files with 61 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/test_brew_advice.py
+29
View File
@@ -688,6 +688,35 @@ def test_a_short_reply_is_advice_even_without_a_marker_word():
assert brew_advice._strip_reasoning(reply) == reply
def test_a_long_legitimate_advice_reply_is_kept():
"""Length alone is a blunt instrument.
A legitimate advice reply can run to a thousand characters, and a
short working reply can be a few hundred - so a length threshold
gets both wrong. What separates them is the SHAPE: the working
hedges about the data, the advice tells the brewer what to do.
"""
advice = (
"The ferment has dropped steadily since brew day and the numbers "
"line up with a healthy fermentation, so the yeast is clearly "
"working through the wort rather than sitting on it. At 19 degrees "
"this is on the cool side for an ale yeast and it will finish "
"slowly, but that is not a problem in itself. Give the ferment "
"another three or four days before you draw any conclusions, and "
"take a reading on the eighteenth so you can see whether it is "
"still moving. Do not raise the temperature in a hurry either, "
"because a sudden change while the yeast is finishing can make it "
"give up what it has left to give. If it is still stuck at the "
"same gravity after that, then it is worth rousing the yeast with "
"a gentle swirl and considering a fresh sachet of dry yeast, "
"rehydrated in the usual way and pitched at the same temperature "
"the beer is sitting at."
)
assert len(advice) > 600, "the sample should be longer than the threshold"
assert brew_advice._strip_reasoning(advice) == advice
def test_a_long_reply_with_no_instruction_is_dropped():
"""Long, and nothing in it reads like an instruction - all working."""
working = ("Let me think about this.\n\n" +