Repository navigation
PYTHON-6188 Fix fuzz Regex divergence and replay the seed corpus in PR CI - #3148
Conversation
There was a problem hiding this comment.
🟢 Approval recommended
The reviewed changes address the reported regex divergence and ensure full corpus coverage in PR fuzz regression runs.
No open issues 🎉
What changed in this PR
This pull request fixes false cross-implementation divergences for BSON Regex values and expands PR fuzz regression runs to replay the full generated seed corpus deterministically.
Changes:
- Canonicalize
Regexpatterns and flags for consistent comparisons. - Generate and expose the seed corpus during regression runs.
- Replay committed reproducers and all corpus seeds, failing fast for missing or empty inputs.
- Document the expanded PR fuzz behavior.
| File | Description |
|---|---|
fuzz/README.md |
Documents deterministic full seed-corpus replay during PR regression tasks. |
fuzz/fuzz_shared.py |
Normalizes equivalent C-extension and pure-Python Regex values consistently. |
.evergreen/scripts/setup_tests.py |
Builds and exposes the seed corpus for all fuzz tasks, including regression runs. |
.evergreen/scripts/run_tests.py |
Validates and replays regression inputs plus the full seed corpus for each fuzz target. |
Review effort: Lite
NoahStapp
left a comment
There was a problem hiding this comment.
Some comment and documentation suggestions for clarity, otherwise looks good!
| f"No fuzz seed corpus found in {corpus_dir}; was setup_tests.py run?" | ||
| ) | ||
| inputs.extend(corpus_files) | ||
| # Each input runs exactly once without mutating; the -atheris_runs |
There was a problem hiding this comment.
What does it mean for an input to "mutate" in this context?
There was a problem hiding this comment.
As in static, without the random fuzzing (mutating).
There was a problem hiding this comment.
Could you rephrase to "random fuzzing" instead?
There was a problem hiding this comment.
Yep, it wouldn't let me do it in the UI, in progress
Co-authored-by: Noah Stapp <noah@noahstapp.com>
Co-authored-by: Noah Stapp <noah@noahstapp.com>
Co-authored-by: Noah Stapp <noah@noahstapp.com>
Co-authored-by: Noah Stapp <noah@noahstapp.com>
Co-authored-by: Noah Stapp <noah@noahstapp.com>
PYTHON-6188
Changes in this PR
The nightly fuzz run fails because a document containing a
Regexcan compare unequal between the C extension and pure Python even though both decoded it correctly. Corpus fuzzing also ran only on the nightly, so a change that broke a seed would not fail PR CI.Regexvalues compare equal, and commit the failing document as a regression reproducer.fuzz-regressiontask, not just the committed reproducers; fail fast when the corpus is missing or empty.Test Plan
Checklist
Checklist for Author
Checklist for Reviewer