Skip to content

PYTHON-6188 Fix fuzz Regex divergence and replay the seed corpus in PR CI - #3148

Merged
blink1073 merged 7 commits into
mongodb:mainfrom
blink1073:PYTHON-6188
Oct 27, 2026
Merged

blink1073 merged 7 commits into
mongodb:mainfrom
blink1073:PYTHON-6188

Conversation

@blink1073

Copy link
Copy Markdown
Member

PYTHON-6188

Changes in this PR

The nightly fuzz run fails because a document containing a Regex can compare unequal between the C extension and pure Python even though both decoded it correctly. Corpus fuzzing also ran only on the nightly, so a change that broke a seed would not fail PR CI.

  • Fix the comparison so equivalent Regex values compare equal, and commit the failing document as a regression reproducer.
  • Replay the full seed corpus in the PR fuzz-regression task, not just the committed reproducers; fail fast when the corpus is missing or empty.

Test Plan

  • The committed reproducer fails before the fix and passes after.
  • After the fix, all three fuzz targets replay the full 485-seed corpus with zero divergences; the corpus previously stopped the nightly at seed 38.

Checklist

Checklist for Author

  • Did you update the changelog (if necessary)? (not needed: the change is dev-only fuzz and CI tooling)
  • Is there test coverage?
  • Is any followup work tracked in a JIRA ticket? (none)

Checklist for Reviewer

  • Does the title of the PR reference a JIRA Ticket?
  • Do you fully understand the implementation? (Would you be comfortable explaining how this code works to someone else?)
  • Is all relevant documentation (README or docstring) updated?

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The reviewed changes address the reported regex divergence and ensure full corpus coverage in PR fuzz regression runs.

No open issues 🎉

What changed in this PR

This pull request fixes false cross-implementation divergences for BSON Regex values and expands PR fuzz regression runs to replay the full generated seed corpus deterministically.

Changes:

  • Canonicalize Regex patterns and flags for consistent comparisons.
  • Generate and expose the seed corpus during regression runs.
  • Replay committed reproducers and all corpus seeds, failing fast for missing or empty inputs.
  • Document the expanded PR fuzz behavior.
File Description
fuzz/​README.md Documents deterministic full seed-corpus replay during PR regression tasks.
fuzz/​fuzz_shared.py Normalizes equivalent C-extension and pure-Python Regex values consistently.
.evergreen/​scripts/​setup_tests.py Builds and exposes the seed corpus for all fuzz tasks, including regression runs.
.evergreen/​scripts/​run_tests.py Validates and replays regression inputs plus the full seed corpus for each fuzz target.

Review effort: Lite

@blink1073
blink1073 marked this pull request as ready for review October 27, 2026 00:28
@blink1073
blink1073 requested a review from a team as a code owner October 27, 2026 00:28
@blink1073
blink1073 requested a review from NoahStapp October 27, 2026 00:28

@NoahStapp NoahStapp left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comment and documentation suggestions for clarity, otherwise looks good!

Comment thread .evergreen/scripts/run_tests.py Outdated
Comment thread .evergreen/scripts/run_tests.py Outdated
Comment thread .evergreen/scripts/run_tests.py Outdated
f"No fuzz seed corpus found in {corpus_dir}; was setup_tests.py run?"
)
inputs.extend(corpus_files)
# Each input runs exactly once without mutating; the -atheris_runs

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does it mean for an input to "mutate" in this context?

@blink1073 blink1073 Oct 27, 2026 •

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As in static, without the random fuzzing (mutating).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you rephrase to "random fuzzing" instead?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, it wouldn't let me do it in the UI, in progress

Comment thread .evergreen/scripts/setup_tests.py Outdated
Comment thread fuzz/README.md Outdated
Comment thread fuzz/README.md Outdated
blink1073 and others added 6 commits October 27, 2026 08:32
Co-authored-by: Noah Stapp <noah@noahstapp.com>
Co-authored-by: Noah Stapp <noah@noahstapp.com>
Co-authored-by: Noah Stapp <noah@noahstapp.com>
Co-authored-by: Noah Stapp <noah@noahstapp.com>
Co-authored-by: Noah Stapp <noah@noahstapp.com>
@blink1073
blink1073 requested a review from NoahStapp October 27, 2026 13:43
@blink1073
blink1073 merged commit 4c35fb8 into mongodb:main Oct 27, 2026
80 checks passed
@blink1073
blink1073 deleted the PYTHON-6188 branch October 27, 2026 15:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants