Repository navigation
perf: freeze the heap AppHarness started until it stops - #7959
benedikt-bartscher wants to merge 3 commits into
Conversation
The harness compiles and runs the app in the test process, where everything the app keeps stays alive until stop(). Every full garbage collection walked all of it: for a generated app with 400 states and 400 pages, 915k objects and 350-430 ms per collection, stalling the backend thread's event loop past the dev-mode blocking detector's 0.2 s threshold. start() now collects and freezes the heap once the app runs, as production workers do, and stop() unfreezes it so the next app's collections can free this one.
[Medium impact] The PR appears safe to merge; no new actionable issues were found. Summary
Reviews (3) · Last reviewed commit: "Compare the freeze count with the one th..." · Reviewed by Greptile |
There was a problem hiding this comment.
All reported issues were addressed across 3 files
Reply with feedback, questions, or to request a fix.
View guided diff | Re-trigger cubic
Merging this PR will degrade performance by 4.06%3 warnings
Warning Please fix the performance issues or acknowledge them on CodSpeed. Performance Changes
Tip Investigate this regression by commenting Comparing Footnotes
|
A cleanup step that raised skipped the unfreeze at the end of stop(), so the stopped app stayed frozen until another harness stopped cleanly. Unfreezing first covers every step. Shutdown runs after the test body, so a full collection during it only delays teardown.
Python 3.12 starts with its immortal tuples frozen, so the count before start() is not 0 there.
Summary
AppHarness.start()now ends withgc.collect(); gc.freeze(), once the app is built and running.stop()callsgc.unfreeze(), so the next harness in the session can collect this one's objects.stop(). Until then every full collection walks all of it. The collection holds the GIL, so the backend thread's event loop stalls for the whole pass.event_loop_blocking_threshold, default 0.2 s) reports such a stall as a blocking event handler. That can fail tests that check the console output.Measurement
The test app was generated: 400 states, each with 10 fields and 10 computed vars, plus 400 pages. It was started through
AppHarness.start()on Python 3.14.8. The table counts tracked objects and times a fullgc.collect()(range of 3 runs) at two points: right before the freeze, and afterstart()returned.gc.collect()start()(frozen)A larger app reaches 5.2M tracked objects under the harness, and a full collection there takes 2.2–2.5 s.
Costs
start()runs one extra full collection.stop()unfreezes everything, including anything the test session froze itself, and on Python 3.12 the interpreter's own immortal tuples, which is harmless. A conftest that freezes its own heap has to freeze again after a harness stops.stop(). Reference counting still frees everything else.Test plan
test_app_harness_freezes_the_started_app_until_stoppedchecks three states. The freeze count does not change during the start steps (Python 3.12 starts with its immortal tuples frozen), the heap is frozen while the app runs, and nothing is frozen afterstop(). It fails without the change.uv run pytest tests/unitstest_background_taskandtest_client_storageruff check,ruff format --check,pyright🤖 Generated with Claude Code