Skip to content

Keep states defined in test_state tests out of later tests - #7963

Open
benedikt-bartscher wants to merge 1 commit into
reflex-dev:mainfrom
benedikt-bartscher:isolate-state-test-registrations
Open

benedikt-bartscher wants to merge 1 commit into
reflex-dev:mainfrom
benedikt-bartscher:isolate-state-test-registrations

Conversation

@benedikt-bartscher

@benedikt-bartscher benedikt-bartscher commented Nov 22, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Some tests in test_state.py leave registrations behind that break later tests, depending on the order the tests run in. CI runs the unit tests in default order, where test_app.py comes before test_state.py, so it doesn't hit this. Running a selection with -k, --lf, or the files in another order does.

  • test_computed_var_mutability defines states with a deliberately invalid deps=["hi"] and never removes them. Every later App compile then fails dependency validation. pytest tests/units/test_state.py tests/units/test_app.py -p no:randomly fails 17 test_app tests on main.
  • The router-dependency tests and test_composite_var_dep_tracks_fields_in_every_state register their local states in the class-level dependency maps of State and other module-level states. They remove those entries at the end of the test body, so a test that fails leaves them behind, and the next test that dirties one of those vars looks up a substate that no longer exists. Under random order, test_state.py alone fails 7 of 8 seeds (--randomly-seed=1..8).
  • The two redis config-knob tests call reload_config() with a redis://localhost config, which stays on the root registration context. Later tests then create a redis state manager and try to connect.

forked_registration_context already gives a test its own copy of the registrations and the config. It now also snapshots the dependency tracking (_var_dependencies, _potentially_dirty_states, _always_dirty_substates) of the states registered before the fork, and restores it on teardown. That covers the entries a state defined in the fork adds to classes that outlive it. The tests above use the fixture, and the manual cleanup at the end of their bodies is gone, so teardown runs even when an assertion fails.

Test-only change, no news fragment.

Test plan

  • pytest tests/units/test_state.py tests/units/test_app.py -p no:randomly: 17 failed before, 670 passed after
  • pytest tests/units/test_state.py --randomly-seed=N for N = 1..10: 336 passed for every seed (before: failures on 7 of seeds 1..8)
  • pytest tests/units/test_state.py tests/units/test_app.py --randomly-seed=N for N = 1..4: 670 passed
  • uv run pytest tests/units: 16458 passed
  • ruff check, ruff format --check, pyright

🤖 Generated with Claude Code

View guided diff

Several tests in test_state.py define states that register themselves in
the class-level dependency maps of module-level states, and clean those
maps up at the end of the test body, so a failing test leaves its entries
behind. test_computed_var_mutability does not clean up at all and leaves
states with an invalid dependency registered, which fails every later
app compile in the session. The redis config tests leave a redis config
in the root registration context.

forked_registration_context now also restores the dependency tracking of
the states registered before the fork, and these tests use it instead of
cleaning up by hand.
@benedikt-bartscher
benedikt-bartscher requested a review from a team as a code owner November 22, 2026 08:51

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 2 files

View guided diff | Re-trigger cubic

@greptile-apps

greptile-apps Bot commented Nov 22, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Low impact] This test-only change appears safe to merge.

Summary

This test-only change keeps temporary states and Redis settings out of later tests.

  • Each state test restores shared registrations and keeps config changes local.

Reviews (1) · Last reviewed commit: "Keep states defined in test_state tests ..." · Reviewed by Greptile

@codspeed

codspeed Bot commented Nov 22, 2026

Copy link
Copy Markdown

Merging this PR will degrade performance by 6.06%

2 warnings

⚠️ 15 benchmarks are dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

⚠️ 4 benchmarks with a significant change ran in a different software environment

Their base and head results were measured with a different OS or toolchain, such as compiler, runtime or library versions, so the changes might come from this difference rather than from the code. Open an affected benchmark to see what changed.

See affected benchmarks

❌ 4 regressed benchmarks
✅ 142 untouched benchmarks
⏩ 8 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Benchmark BASE HEAD Efficiency
❌ test_route_matching[100] 9.4 ms 10.1 ms -6.94%
❌ test_route_matching[1000] 90 ms 96.8 ms -6.93%
❌ test_route_matching[10] 1.2 ms 1.3 ms -5.53%
❌ test_state_access[write_own] 122.2 ms 128.3 ms -4.82%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing benedikt-bartscher:isolate-state-test-registrations (b35274c) with main (df7f157)

Open in CodSpeed

Footnotes

  1. 8 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant