Skip to content

perf: freeze the heap AppHarness started until it stops - #7959

Open
benedikt-bartscher wants to merge 3 commits into
reflex-dev:mainfrom
benedikt-bartscher:harness-freeze-heap
Open

benedikt-bartscher wants to merge 3 commits into
reflex-dev:mainfrom
benedikt-bartscher:harness-freeze-heap

Conversation

@benedikt-bartscher

@benedikt-bartscher benedikt-bartscher commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • AppHarness.start() now ends with gc.collect(); gc.freeze(), once the app is built and running. stop() calls gc.unfreeze(), so the next harness in the session can collect this one's objects.
  • The harness compiles and runs the app inside the test process, and the app stays alive until stop(). Until then every full collection walks all of it. The collection holds the GIL, so the backend thread's event loop stalls for the whole pass.
  • The harness runs the app in dev mode, where the blocking detector (event_loop_blocking_threshold, default 0.2 s) reports such a stall as a blocking event handler. That can fail tests that check the console output.
  • Production workers already freeze the built app (perf: preload the app and fork production granian workers #7079, perf: freeze the heap in production backend workers once the app is built #7945).

Measurement

The test app was generated: 400 states, each with 10 fields and 10 computed vars, plus 400 pages. It was started through AppHarness.start() on Python 3.14.8. The table counts tracked objects and times a full gc.collect() (range of 3 runs) at two points: right before the freeze, and after start() returned.

tracked objects full gc.collect()
started, unfrozen 914,616 350–430 ms
after start() (frozen) 1 < 0.1 ms

A larger app reaches 5.2M tracked objects under the harness, and a full collection there takes 2.2–2.5 s.

Costs

  • start() runs one extra full collection.
  • stop() unfreezes everything, including anything the test session froze itself, and on Python 3.12 the interpreter's own immortal tuples, which is harmless. A conftest that freezes its own heap has to freeze again after a harness stops.
  • A frozen object that becomes unreachable through a reference cycle is only collected after stop(). Reference counting still frees everything else.

Test plan

  • test_app_harness_freezes_the_started_app_until_stopped checks three states. The freeze count does not change during the start steps (Python 3.12 starts with its immortal tuples frozen), the heap is frozen while the app runs, and nothing is frozen after stop(). It fails without the change.
  • uv run pytest tests/units
  • Integration: test_background_task and test_client_storage
  • ruff check, ruff format --check, pyright
  • Manual: the table above.

🤖 Generated with Claude Code

View guided diff

The harness compiles and runs the app in the test process, where
everything the app keeps stays alive until stop(). Every full garbage
collection walked all of it: for a generated app with 400 states and
400 pages, 915k objects and 350-430 ms per collection, stalling the
backend thread's event loop past the dev-mode blocking detector's
0.2 s threshold.

start() now collects and freezes the heap once the app runs, as
production workers do, and stop() unfreezes it so the next app's
collections can free this one.
@benedikt-bartscher
benedikt-bartscher requested a review from a team as a code owner November 21, 2026 21:43
@greptile-apps

greptile-apps Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; no new actionable issues were found.

Summary

AppHarness.start() collects and freezes the heap after startup. stop() unfreezes it before cleanup.

  • AppHarness keeps its running app out of full garbage-collection walks.

Reviews (3) · Last reviewed commit: "Compare the freeze count with the one th..." · Reviewed by Greptile

Comment thread reflex/testing.py Resolved

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 3 files

Reply with feedback, questions, or to request a fix.

View guided diff | Re-trigger cubic

Comment thread reflex/testing.py Resolved
Comment thread reflex/testing.py Resolved
@codspeed

codspeed Bot commented Nov 21, 2026 •

Copy link
Copy Markdown

Merging this PR will degrade performance by 4.06%

3 warnings

⚠️ 11 benchmarks are dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

⚠️ 1 benchmark with a significant change ran on different hardware

Its base and head results were measured on machines with a different CPU, core count or memory, so the change might not come from the code. Run base and head on the same runner type.

Learn more about Macro Runners

See affected benchmarks

⚠️ 1 benchmark with a significant change ran in a different software environment

Its base and head results were measured with a different OS or toolchain, such as compiler, runtime or library versions, so the change might come from this difference rather than from the code. Open the benchmark to see what changed.

See affected benchmarks

❌ 1 regressed benchmark
✅ 145 untouched benchmarks
⏩ 8 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Benchmark BASE HEAD Efficiency
❌ test_computed_dependency_fanout[50] 71.8 ms 74.9 ms -4.06%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing benedikt-bartscher:harness-freeze-heap (393a0a6) with main (df7f157)

Open in CodSpeed

Footnotes

  1. 8 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

A cleanup step that raised skipped the unfreeze at the end of stop(), so
the stopped app stayed frozen until another harness stopped cleanly.
Unfreezing first covers every step. Shutdown runs after the test body,
so a full collection during it only delays teardown.
Python 3.12 starts with its immortal tuples frozen, so the count before
start() is not 0 there.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant