A length threshold is a blunt instrument: a legitimate advice reply can
run to a thousand characters, and a short working reply can be a few
hundred - so a threshold gets both wrong. A 995-character advice reply
with no marker word in it was being emptied.
The reliable signal is the SHAPE. The model's working is about
INTERPRETING the numbers - what the reading might mean, what could be
causing it, what it is unsure about. The advice is about what to DO.
So a reply is working when it is full of the model's hedging about the
data AND contains no instruction at all. Length is only a tie-breaker
for a reply that is both hedged and short.
995-char legitimate advice -> kept
long pure working -> dropped
short legitimate advice -> kept
long working + advice -> advice kept
Checked against the live model - a real reply comes back clean, with no
reasoning in it:
At two weeks in, dropping from 1.055 to 1.020 is a healthy 35-point
fall - about 64% attenuation - which is right in the normal range for
a pale ale, though you likely still have 5 to 8 points to go...
184 tests (up from 183).
hermes
cbc71716e8