Repository navigation
DOC: Make framework.md concise, prescriptive contributor guidance - #4175
Open
Richard Lundeen (richlundeen) wants to merge 12 commits into
Open
Richard Lundeen (richlundeen) wants to merge 12 commits into
Richard Lundeen (richlundeen) wants to merge 12 commits into
Conversation
Contributors and coding agents use framework.md to decide where code belongs. Rewrite it around principles and core responsibilities instead of feature-level detail. - Add principles for code, docs, and tests, including failing loudly rather than running partially - Replace the two architecture diagrams with one that shows surfaces, run inputs, technique configuration, and the shared library - Keep the backend and frontend thin: new behavior and types belong in the library and pyrit.models - Update the framework plans to match the current code - Link short docs and unit test rules back to the principles Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
No active review issues were identified in the architecture and contributor-guidance documentation changes.
No open issues 🎉
What changed in this PR
Recasts PyRIT’s architecture documentation as concise contributor guidance for deciding where code belongs, with explicit component responsibilities and thin application surfaces.
Changes:
- Adds shared principles for code, documentation, and tests, and updates component responsibilities and planned refactoring.
- Consolidates the architecture diagrams into one showing surfaces, run inputs, technique configuration, and shared services.
- Aligns the notebook, unit-test, and style guides with those principles.
| File | Description |
|---|---|
doc/contributing/7_notebooks.md |
Adds concise documentation-writing rules linked to the shared principles. |
doc/contributing/5_unit_tests.md |
Clarifies deterministic, isolated, behavior-focused testing guidance. |
doc/contributing/3_style_guide.md |
Prefers self-explanatory code and comments that explain why. |
doc/code/framework.md |
Defines contributor principles, component ownership, and architectural plans. |
doc/assets/pyrit-components.svg |
Illustrates application surfaces, configuration boundaries, and shared services. |
Review effort: Balanced
Mark the page as a north star and limit updates to agreed principle or responsibility changes and completed plans, so feature work stays in the component guides. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…runner Scenario subclasses often rebuild run logic that the Scenario base class already owns. State that a scenario declares its techniques, datasets, and scorer, implements _build_atomic_attacks_async, and leaves the rest of the run to the base class. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Richard Lundeen (richlundeen)
marked this pull request as ready for review
November 10, 2026 22:02
State the security model: operators are trusted with test data, so don't redact what they need, while credentials and server-level powers stay protected. Make not over-engineering a code principle, and ask for pull requests that each deliver one unit of value end to end. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Richard Lundeen (richlundeen)
added a commit
to varunj-msft/PyRIT
that referenced
this pull request
Nov 10, 2026
Treating everything before the last "@" as a user name and password rewrote a URL such as https://graph.example/users/alex@contoso.com/mail into one that names the wrong host. Operators need the real URL (microsoft#4175: don't redact what operators need), and the case it covered, an unescaped "/", "?" or "#" in a password, is a malformed URL. The linear-time match, which also finds a URL right after "_", stays. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Nov 10, 2026
Ask contributors to match existing patterns, but to do what's best when a better way exists and converge on it instead of adding variants. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
List the patterns components should share instead of local variants: requirements and capabilities, identifiers and references, defaults, typed fields and metadata value objects, shared-instance state, prompt seeds, structured output, media storage, and UTC timestamps. Fold the Models bullets they replace into the new section. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drop the conventions bullet about referring to registered components by name and hash. Memory refers to components by identifier hash while the registry resolves them by name, and the registry guide already covers pinned references and recreate_instance. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Scope rules that a literal reader could misapply: identifiers leave out runtime settings and credentials, typed fields are for queried data, environment reads and file writes are scoped to components and results, attacks may forward replies, tags select technique subsets, invalid arguments stay ValueError, and declared skip and adapt policies aren't fallbacks. Also name the optional scenario hooks and the shared compatibility checks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add a convention that attacks, scorers, converters, and targets receive the components they use as objects, and leave construction to callers, initializers, the registry, technique factories, and scenario defaults. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reframe the pass-in-components convention: a component may build its own default collaborator, as long as callers can pass an object that replaces it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Rebuilding has legitimate uses, such as duplicating a component or building a variant through the registry, and the identifiers convention already records what actually ran. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Resolve the framework.md Datasets conflict with the objective decomposition work from microsoft#4110: keep the concise guidance, add an ownership rule (the dataset defines the tree; attacks execute its paths), and add a Framework Plan for executing decomposition paths. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Contributors and coding agents use
framework.mdto decide where code belongs. This rewrite turns it into a concise, prescriptive north star built on principles and core responsibilities, instead of feature-level implementation detail.pyrit.modelsfirst, and scenarios let theScenariobase class run themChatScoringService, the backend Word finding export, and the frontend conversation export🤖 Generated with Claude Code