armistace/beer_data
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(advice): the review findings - the ids, the client leak, the wrong reading

Browse Source 
The review of PR #14 found a critical defect and two real bugs.  All
verified against the live database, not just read.

CRITICAL - the advice would never have been stored
--------------------------------------------------
`_as_doc_id` accepted only an ObjectId or a string that parses as one.
But **all 47 live readings have string `_id`s** - 24 random characters
from the original app's importer, like `l78ZidBJ3QaYn98tvB88x4uM`.  The
helper turned every one of them into None, so the update matched nothing
and the advice was silently never stored in production.

The mongomock fixture generates ObjectIds, so the tests could not see it.
`_as_doc_id` now returns both forms and the query uses `$in` (a plain
list would be an exact ARRAY match in Mongo, which is not what is
wanted).  Verified against the live database: the advice now lands on a
real reading, with the brewer's comment untouched.

There are now tests for BOTH id shapes, with the string shape built
explicitly, because the fixture would not produce it.

The advice was written about the wrong reading
----------------------------------------------
`latest` was taken from the date-sorted tail of the run, so editing an
OLDER reading generated advice about the NEWEST one.  The advice is now
written against the reading that was actually saved.

A leaked MongoClient per reading
--------------------------------
The background thread built a `BeerRepository()` - and therefore a
`MongoClient` - for every advice write, and never closed it.  A
MongoClient spawns three monitor threads and pymongo does NOT reap them
on `close()`; they go only when the client is garbage collected.

Measured: three threads and a connection pool accumulated per reading
recorded.  There is now one shared repository for the life of the
process, which is what the request-scoped repository does too.  Measured
again after the fix: 4 threads total, not 16 over five writes.

Also
----
- `BeerRepository` gained `close()` and the context-manager protocol.
- The advice writer now logs instead of dying silently if the write
  fails.

164 tests (up from 160).
This commit is contained in:
hermes
2026-11-09 16:29:39 +10:00
parent d0aef7a787
commit 6dd0b7b046
3 changed files with 242 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/test_brew_advice.py
+148
View File
@@ -3,6 +3,7 @@
from __future__ import annotations
import json
from contextlib import contextmanager
from datetime import date
import pytest
@@ -248,6 +249,153 @@ def test_editing_a_reading_clears_its_stale_advice(repo):
assert after["sg"] == "1030"
def test_the_advice_is_stored_on_a_reading_with_a_string_id(repo):
"""The original app's readings have 24-character STRING ids.
This is the shape every existing reading in the live database has -
the ids were generated by the old app as random characters, not by
Mongo. A helper that only accepted an ObjectId would have matched
nothing, and the advice would silently never be stored.
The mongomock fixture generates ObjectIds, so this has to be
constructed explicitly or the bug is invisible in the tests.
"""
repo.readings.insert_one({
"_id": "l78ZidBJ3QaYn98tvB88x4uM",
"beer_run_id": "1",
"beer_type": "Ale",
"sg": "1042",
"date": "2026-01-01",
"final_reading": "False",
"comment": "First ever try",
})
modified = repo.set_reading_advice("l78ZidBJ3QaYn98tvB88x4uM",
"THE ADVICE", True)
assert modified == 1, "the advice was not stored on a string-id reading"
after = repo.readings.find_one({"_id": "l78ZidBJ3QaYn98tvB88x4uM"})
assert after["advice"] == "THE ADVICE"
assert after["comment"] == "First ever try"
def test_the_advice_is_stored_on_a_reading_with_an_object_id(repo):
"""The other shape: a reading written by Mongo itself."""
seed_readings(repo, [("1", "Ale", "1042", "2026-01-01", False, "a comment")])
doc = repo.readings.find_one({"beer_run_id": "1"})
modified = repo.set_reading_advice(str(doc["_id"]), "THE ADVICE", True)
assert modified == 1
after = repo.readings.find_one({"beer_run_id": "1"})
assert after["advice"] == "THE ADVICE"
def test_writing_the_advice_does_not_leak_a_client_per_reading():
"""A MongoClient spawns three monitor threads and pymongo does not
reap them on close() - they go only when the client is collected.
So building a client per advice write accumulated three threads and a
connection pool for every reading recorded. The advice writer shares
one client for the life of the process instead.
"""
import inspect
from views import brews as brews_module
# The writer must NOT build a repository of its own - it has to use
# the shared one. A MongoClient spawns three monitor threads and
# pymongo does not reap them on close(), so a client per advice write
# accumulated three threads and a connection pool for every reading
# recorded.
source = inspect.getsource(brews_module._start_advice)
assert "BeerRepository()" not in source, (
"the advice writer builds its own repository - that leaks a Mongo "
"client (three monitor threads) per reading"
)
assert "get_shared_repository" in source, (
"the advice writer must use the shared repository"
)
@contextmanager
def _app_with_repo(repo):
"""An application context whose repository is *repo*.
The view reads the repository from ``app.extensions``, so pointing
that at a fixture is enough to drive the view's code directly.
"""
from app_factory import create_app
app = create_app({
"TESTING": True,
"WTF_CSRF_ENABLED": False,
"SECRET_KEY": "test-secret",
"MONGO_CLIENT": repo.client,
})
# The advice must be generated even though this is a test app.
app.config["TESTING"] = False
original = app.extensions["beer_repo"]
app.extensions["beer_repo"] = lambda: repo
with app.app_context():
try:
yield app
finally:
app.extensions["beer_repo"] = original
def test_the_advice_is_about_the_reading_that_was_saved():
"""Editing an OLDER reading must not generate advice about the newest.
The advice is written against a specific reading, so the reading being
saved has to be the one the advice is about.
"""
import time
import mongomock
from mongo.repository import BeerRepository
from views import brews as brews_module
from views.brews import _start_advice
captured = {}
def fake_generate(run, reading, history, writer=None):
captured["reading"] = reading
captured["history"] = history
return True, "THE ADVICE"
original = brews_module.brew_advice.generate_advice_in_background
brews_module.brew_advice.generate_advice_in_background = fake_generate
try:
# Two readings; the OLDER one is the one being saved.
repo = BeerRepository(client=mongomock.MongoClient())
old_id = repo.upsert_reading(beer_run_id="1", beer_type="Ale", sg=1042,
reading_date="2026-01-01", final_reading=False,
comment="first")
repo.upsert_reading(beer_run_id="1", beer_type="Ale", sg=1030,
reading_date="2026-01-10", final_reading=False,
comment="second")
# The advice is only generated when it is enabled, and it is off
# in testing.
with _app_with_repo(repo):
_start_advice("1", old_id)
for _ in range(40):
time.sleep(0.05)
if captured:
break
finally:
brews_module.brew_advice.generate_advice_in_background = original
assert captured, "the advice was never generated"
assert captured["reading"]["_id"] == old_id
assert captured["reading"]["sg"] == 1042, (
"the advice was generated about the wrong reading"
)
def test_the_reading_id_cannot_be_used_to_inject_a_query(repo):
"""The reading id is coerced before it reaches Mongo.