Skip to content

Windows: full-suite debt of 194 failing tests (measured 2026-10-10), fix order by root cause #692

Open

Description

@cdeust

Summary

Cortex runs about 9000 tests on Linux and only 20 hand-picked test files on Windows. A one-off measurement of the full suite on Windows found 194 failing tests (188 failed, 6 errors) against 0 on the Linux control and 3 on macOS. This issue tracks the Windows debt as a sequence of root causes; the verified product defects and the bash question have their own issues.

Evidence

Measured on 2026-10-10 at main 18bd25b, SQLite backend, Python 3.12, dependencies from uv.lock, the process-leak guard active, no -x. Two identical runs gave the identical set of failing ids.

Failure classes (mutually exclusive, they sum to 194):

  1. 50 tests: the empty_env fixture of tests_py/scripts/test_launcher_platform_coverage.py (line 92) returns Scripts/python without .exe. Once fixed, uv's cross-platform resolution runs on Windows for the first time and may expose more.
  2. 42 tests: bare bash resolves to the WSL launcher (tracked in its own issue).
  3. 16 tests: default text encoding is cp1252; 4 of them are a product defect in auto_recall (tracked in its own issue), 12 are tests that call read_text() without an encoding or compare mojibake.
  4. 13 tests: benchmarks/lib/machine_load_snapshot.py:54 calls the POSIX-only os.getloadavg.
  5. 21 tests assume a POSIX path or branch: /a.py is not absolute on Windows (test_preemptive_context 9), os.getppid patched while _window_claude_pid takes the Windows branch (session_registry.py:210-212, test_session_registry 5; PR fix(session-registry): resolve the window's claude.exe on Windows and probe liveness without os.kill (#665) #678 made its own tests platform aware but not this file), ~/state under USERPROFILE, is_supported() false so dispatch takes the spool branch (test_capture_dispatch), a path regex over tmp_path (test_validate_memory 4).
  6. 4 tests use a POSIX primitive without a platform guard (os.O_NOFOLLOW, multiprocessing.get_context("fork"), select.select on a pipe).
  7. 2 tests: tests_py/_hermetic_hook_env.py:91 sets HOME only, and Path.home() reads USERPROFILE on Windows.
  8. 23 tests, suspected CRLF translation of fixtures written with write_text (test_migrate_decision_blocks 9, test_wiki_rebucket_file_docs 9, test_wiki_bulk_migrate 5); inferred from the code, not run.
  9. 23 tests not isolated (first error lines recorded): installer rerun fixtures 6, file priming aliases 6 (heat 0.5 against 0.6: a symlinked project alias is scoped differently, possibly a product defect in the ADR-1086 scoping), mcp client spawn 2, a mocked Path.resolve 2, a TTL on a coarse Windows clock 1, test_capture_drain 1 (tracked in its own issue), and 5 singles.

Expected

Fix the classes in this order, one pull request each, re-measuring on Windows after each one with the scratch workflow used for the measurement: 1 (50 tests), then 3 (the product defect first), 2, 4, 8, then 5, 6 and 7 together, then class 9 one by one. After the first four steps 73 ids remain, after 5 to 7 about 20. The required Windows job then grows one directory at a time (not straight to the full suite), and the per-system skip counts (Windows 78, macOS 6 more than the control) are pinned in CI so that a new platform skip is a visible decision.

Activity

  1. cdeust commented on Nov 4, 2026

    @cdeust
    OwnerAuthor

    Tranche classe 4 proposée dans #861 (commit d0f0ab9) : os.getloadavg est Unix-only, l'AttributeError sous Windows échappait au except OSError de benchmarks/lib/machine_load_snapshot.py. Le test qui échouait avant (os sans getloadavg) passe maintenant. Sous Linux, tests_py/benchmarks donne 386 passés et 12 ignorés. Rien n'a tourné sous Windows.

    Tranches restantes, dans l'ordre de l'issue : classe 3 (encodage cp1252, 12 tests ; le défaut produit auto_recall a sa propre issue), classe 8 (CRLF, 23 tests), classes 5 à 7 ensemble, puis la classe 9 une par une. Les classes 1 et 2 sont déjà fusionnées : #824 pour la classe 1, #738 pour la classe 2.


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    WindowsbugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions