[Adrian Gavrila (adrian-gavrila)]

Description

This is the follow-up promised in #3066 (Operations, findings, evidence). #3066 pinned a finding's severity, harm type and description with CHECK constraints. This PR drops those constraints and lets each deployment define its own finding fields in Python. A deployment that configures nothing keeps the #3066 fields (Critical through Informational, the HarmCategory list, a description), and its findings migrate in place.

It needs to ship in 1.2.0 together with #3066. #3066 isn't released yet, so removing FindingSeverity, /api/operations/finding-options and #3066's request shape needs no deprecation path. If 1.2.0 were cut with #3066 but without this PR, those removals would need one.

What changes

• Fields are code. A deployment subclasses FindingsInitializer (pyrit/setup/initializers/findings.py) and returns a FindingFields from get_finding_fields().
  • The severity slot is always a required choice field, and its option order is the ranking.
  • Other fields are choices (with optional cascade levels and Other text), single-line text or multi-line text, and can carry hint, section and internal.
  • DEFAULT_FINDING_FIELDS reproduces FEAT: Findings dashboard with persistent Operations in the GUI #3066.
  • Keys are permanent; labels can change. Changing a key retires the old field, and saved values under it stay visible and read-only.

• Storage. One migration, d43b1bcf9c56, after d6de1045cca4.
  • It drops the three CHECK constraints.
  • It stores Other severity text as the severity itself, and moves harm type and description into a JSON field_values column.
  • Finding IDs, Operation links, creation times and evidence links don't change.
  • severity becomes unbounded Unicode, with a binary collation on SQL Server so sorting and counting match SQLite.
  • Downgrade refuses to run offline or while any finding exists.
  • Memory never reads FindingFields: the service passes it the severity order.
• API. GET /api/operations/finding-fields replaces /finding-options.
  • Create and update check values against the active fields and answer 422 with a message naming the field.
  • Unchanged saved values that no longer match an option are kept.
  • Changed Other text that matches an option's key or label is rejected, with a 422 naming the option.
  • Updates keep retired values unless the request lists them in remove_retired_keys.
  • Request fields follow the existing request limits (TextStr, IdentifierStr, MAX_ITEMS).
• Setup. The live-apply preflight rejects invalid finding fields, or two FindingsInitializers, before changing anything.
• Export. The Word and Markdown export follows the fields: multi-line fields go in the Finding column, choice and text fields in Assessment. Internal fields and retired values are left out, and the export dialog warns when selected findings have retired values.

• CoPyRIT. The finding dialog, cards, Operations counts, the Chat evidence picker and the export dialog all render from /finding-fields.
  • Retired values show read-only, with Remove (after a confirmation) and Undo.

• The fields reload when the runtime generation changes.
• An open edit dialog keeps the fields it opened with and sends kept retired values back, so a live apply during an edit never clears a value.

• Docs. A new doc/code/setup/finding_fields.md (in doc/myst.yml) explains how to define fields, and doc/gui/0_gui.md describes deployment-defined finding fields.

[Copilot]

🟡 Changes recommended

Unresolved defects can drop custom-field data, repeat or skip findings after severity changes, misvalidate replacement environments, and conflate distinct severities in SQL Server.

5 open issues

• Reload findings when severity options change · New
• Invalidate findings query on severity order changes · New
• Handle proto keys with prototype-free dictionaries · New
• Use length-aware SQL Server severity comparisons · New
• Evaluate field definitions against proposed environment · New

What changed in this PR

This follow-up to #3066 makes finding fields deployment-configurable across PyRIT’s storage, API, setup, and CoPyRIT interface, while retaining the existing fields by default.

Changes:

• Adds initializer-based field configuration, validation, and live-apply preflight checks.
• Migrates finding values to JSON storage and adapts ordering, updates, and exports.
• Replaces fixed frontend fields with configurable controls, retired-value handling, and supporting tests and documentation.

File	Description
tests/​unit/​setup/​test_reinitialization.py	Tests finding-field preflight failures.
tests/​unit/​setup/​test_initialization.py	Tests field resets and setup validation.
tests/​unit/​setup/​test_findings_initializer.py	Tests initializer execution and discovery.
tests/​unit/​setup/​test_finding_fields.py	Tests active-field state management.
tests/​unit/​setup/​conftest.py	Adds custom-field fixtures.
tests/​unit/​models/​test_finding.py	Tests flexible finding values.
tests/​unit/​memory/​test_operations_and_findings_migration.py	Preserves earlier migration tests.
tests/​unit/​memory/​test_findings.py	Tests persistence and configurable ordering.
tests/​unit/​memory/​test_finding_field_values_migration.py	Tests migration data and database schema changes.
tests/​unit/​conftest.py	Adds field-state cleanup fixture.
tests/​unit/​backend/​test_operations.py	Tests field-aware routes and updates.
tests/​unit/​backend/​test_operation_service_findings.py	Tests retired-value preservation and removal.
tests/​unit/​backend/​test_finding_export.py	Tests configurable export projection.
tests/​unit/​backend/​test_common_models.py	Tests finding request limits.
tests/​unit/​backend/​mocks.py	Adds persisted-finding read helper.
tests/​unit/​backend/​conftest.py	Adds configured and legacy finding fixtures.
pyrit/​setup/​initializers/​findings.py	Introduces deployment field initializer.
pyrit/​setup/​initializers/​__init__.py	Exports the initializer lazily.
pyrit/​setup/​initialization.py	Resets fields during setup passes.
pyrit/​setup/​finding_fields.py	Holds active deployment fields.
pyrit/​setup/​configuration_loader.py	Adds field-definition preflight checks.
pyrit/​models/​finding.py	Replaces fixed classifications with field values.
pyrit/​models/​__init__.py	Updates public finding exports.
pyrit/​memory/​memory_models.py	Stores severity and JSON field values.
pyrit/​memory/​memory_interface.py	Accepts caller-defined severity ordering.
pyrit/​memory/​alembic/​versions/​d43b1bcf9c56_store_finding_field_values.py	Migrates existing finding values and constraints.
pyrit/​backend/​services/​operation_service.py	Validates values and preserves retired fields.
pyrit/​backend/​services/​finding_export.py	Projects configured, non-internal fields.
pyrit/​backend/​routes/​operations.py	Exposes fields and validation responses.
pyrit/​backend/​models/​operations.py	Defines bounded configurable-field requests.
frontend/​src/​utils/​findingSeverity.ts	Removes fixed severity helpers.
frontend/​src/​utils/​findingSeverity.test.ts	Removes fixed severity tests.
frontend/​src/​utils/​findingFields.ts	Adds field, cascade, and badge helpers.
frontend/​src/​utils/​findingFields.test.ts	Tests configurable-field helpers.
frontend/​src/​types/​index.ts	Defines configurable finding types.
frontend/​src/​test-utils/​findingFieldsFixtures.ts	Adds default and custom field fixtures.
frontend/​src/​test-utils/​findingDialog.ts	Adds shared dialog test helpers.
frontend/​src/​services/​api.ts	Updates finding API methods.
frontend/​src/​services/​api.test.ts	Tests revised API requests.
frontend/​src/​hooks/​useFindingFields.ts	Loads fields and handles runtime refreshes.
frontend/​src/​hooks/​useFindingFields.test.tsx	Tests loading, retries, and refreshes.
frontend/​src/​components/​Operations/​RetiredFieldValues.tsx	Adds removal confirmation and undo controls.
frontend/​src/​components/​Operations/​RetiredFieldValues.test.tsx	Tests retired-value controls and focus.
frontend/​src/​components/​Operations/​RetiredFieldValues.styles.ts	Styles retired-value panels.
frontend/​src/​components/​Operations/​OperationsPage.tsx	Renders configured severity counts.
frontend/​src/​components/​Operations/​OperationsPage.test.tsx	Tests counts and field-loading failures.
frontend/​src/​components/​Operations/​Operations.styles.ts	Styles configurable and retired values.
frontend/​src/​components/​Operations/​OperationDetailPage.tsx	Integrates configurable finding displays and dialogs.
frontend/​src/​components/​Operations/​FindingSeverityBadge.tsx	Uses configured severity labels and colors.
frontend/​src/​components/​Operations/​FindingFieldInput.tsx	Adds text, choice, and cascade controls.
frontend/​src/​components/​Operations/​FindingFieldInput.styles.ts	Styles configurable inputs.
frontend/​src/​components/​Operations/​FindingExportDialog.tsx	Warns about excluded retired values.
frontend/​src/​components/​Operations/​FindingExportDialog.test.tsx	Tests retired-value export warnings.
frontend/​src/​components/​Operations/​FindingDialog.tsx	Builds drafts from configured fields.
frontend/​src/​components/​Operations/​FindingDialog.styles.ts	Adds field-section styling.
frontend/​src/​components/​Operations/​FindingCardValues.tsx	Displays configured and retired values.
frontend/​src/​components/​Chat/​FindingEvidenceDialog.tsx	Uses configurable fields in evidence selection.
frontend/​src/​components/​Chat/​FindingEvidenceDialog.test.tsx	Tests field-aware evidence workflows.
frontend/​e2e/​operations.spec.ts	Updates requests and tests retired-value interactions.
doc/​myst.yml	Adds finding-field documentation to navigation.
doc/​gui/​0_gui.md	Describes deployment-defined finding fields.
doc/​code/​setup/​finding_fields.md	Documents configuration, rules, and migration.

_{Review effort: Balanced}

[Richard Lundeen (richlundeen)]

Source review at d01f275. Rendered UI coverage is focused: required error/narrow, chat-picker and live-transition evidence was not supplied. See the supported inline findings.

[Richard Lundeen (richlundeen)]

I found two issues that need fixes. Source review is complete; rendered UI coverage is still partial.

[Richard Lundeen (richlundeen)]

[P3] Cap field and level key length at setup

FindingField accepts a key of any length, and FindingFields any number of fields and levels. A save request carries the values as field_values: dict[IdentifierStr, TextStr], which allows keys of at most 256 characters (MAX_IDENTIFIER_LENGTH) and 100 entries (MAX_ITEMS). A deployment that defines a 300-character key, or more than 100 fields and levels together, passes setup and the live-apply preflight, and then every save is rejected with a 422 about the request limit rather than the configuration. Reject keys longer than MAX_IDENTIFIER_LENGTH here, and more than MAX_ITEMS keys in FindingFields._validate_keys, so a bad configuration fails at startup or preflight. Both limits are in pyrit.models.request_limits.

[Richard Lundeen (richlundeen)]

[P2] Have the backend provide retired values and required checks instead of recomputing them here

This file re-implements finding-field rules that FindingFields already owns in Python. firstMissingRequired repeats the required-value rules of check_values (its comment says "as the backend checks them"), splitRetiredValues and describeFindingValues repeat describe_values, and levelOptions repeats the cascade walk. The proposed framework guidance in microsoft/PyRIT#4175 says the frontend should "Present; don't decide": don't compute "validation rules", and "if the UI needs a derived domain value, have the backend provide it from the library". Two implementations also have to be kept in step by hand: a rule change in check_values or describe_values silently diverges from what the dialog, cards and export warning show.

The backend already checks every save and answers 422 with a message that names the field, so the required check here is a convenience that can go. The retired values matter more, because the edit dialog uses them to decide what to resend and what to offer to remove. Return them from the API next to each finding (for example the retired list and labelled rows from describe_values), and have the dialog, cards and export dialog render that. Keep the pure display state that is derived while rendering, such as severity badge colors and the Other count grouping.