feat(advice): use mistral-large-4, and cope with models that show their working
The model is now `mistral-large-4:cloud`, on the same Ollama instance.
It was probed before being made the default - `deepseek-v4-flash:cloud`,
`deepseek-v3.2:cloud` and `qwen3.5:cloud` are all retired and fail at
call time while still appearing in the model list.
What the probes showed
----------------------
`think: false` does NOT stop the reasoning. It only changes where the
working goes: with thinking off, the model writes it into the CONTENT
instead of a separate field. Either way the reasoning comes out of the
same token allowance as the answer, and at 500-3000 tokens the reply
came back EMPTY - which looks like a broken model rather than a budget
problem. The budget is now 8000, and the timeout 120s.
The reply is also stripped of a reasoning preamble. The advice is the
prose at the END; the working comes first, in several paragraphs. The
preamble is found by walking back from the end of the reply and keeping
the paragraphs that read like advice to the brewer, rather than like
the model thinking out loud.
Verified against the live model, on a stalled brew (OG 1.042, 1.022 after
nine days, "no change since day 7"):
That 1.022 reading after nine days at 19C - with no movement since
day seven - looks like a stalled fermentation; a 1.042 wort should
be well on its way to 1.010-1.012 by now, not sitting at only about
48% attenuation. Gently swirl the fermenter to rouse the yeast cake
and keep the temperature steady at 19-20C, and if the gravity hasn't
budged in another 48 hours, pitch a fresh, healthy yeast starter...
And on a CORRECTED older reading - the case the review found broken -
the advice is about the corrected reading, with the later one as context:
A brew-day reading of 1.038 against an OG of 1.042 is only four points
low, which is almost certainly just the normal difference between
your kettle gravity and the post-transfer fermenter reading...
169 tests (up from 167).
This commit is contained in:
@@ -220,7 +220,42 @@ def test_the_token_budget_leaves_room_for_the_reasoning(monkeypatch):
|
||||
brew_advice.generate_advice({"canonical_name": "Ale"},
|
||||
{"date": "2026-01-01"}, [])
|
||||
|
||||
assert captured["body"]["max_tokens"] >= 2000
|
||||
# The reasoning comes out of the same budget as the answer, and
|
||||
# `think: false` does not stop it - the models just move the working
|
||||
# into the content. The budget has to cover both.
|
||||
assert captured["body"]["max_tokens"] >= 8000
|
||||
|
||||
|
||||
def test_a_reasoning_preamble_is_cut_off():
|
||||
"""Some models put their working in the content even with thinking off.
|
||||
|
||||
The advice is the prose at the end; the working comes first, in
|
||||
several paragraphs. The preamble is removed by finding the advice,
|
||||
not by taking the text after the first blank line.
|
||||
"""
|
||||
reply = (
|
||||
"Let me look at this. The brewer has a Christmas beer, OG 1.042, "
|
||||
"fermenting at 19C.\n\n"
|
||||
"So on brew day, the reading is 1.038, but the OG is stated as 1.042. "
|
||||
"That is a discrepancy of 4 points.\n\n"
|
||||
"Wait, the reading on brew day is 1.038 but the recorded OG is 1.042.\n\n"
|
||||
"On brew day a gravity of 1.038 against a recorded original of 1.042 "
|
||||
"is a small discrepancy but not a problem. Take the OG from a "
|
||||
"well-mixed sample, and start the fermentation at 18-20C."
|
||||
)
|
||||
|
||||
stripped = brew_advice._strip_reasoning(reply)
|
||||
|
||||
assert stripped.startswith("On brew day a gravity of 1.038")
|
||||
assert "Let me look at this" not in stripped
|
||||
assert "That is a discrepancy of 4 points" not in stripped
|
||||
|
||||
|
||||
def test_a_reply_that_starts_with_the_advice_is_untouched():
|
||||
"""The stripper must not eat a good reply."""
|
||||
reply = ("The gravity looks fine for day nine. Give it three more days "
|
||||
"before you do anything, and take another reading on Friday.")
|
||||
assert brew_advice._strip_reasoning(reply) == reply
|
||||
|
||||
|
||||
def test_the_advice_is_truncated():
|
||||
|
||||
Reference in New Issue
Block a user