Repository navigation
FEAT: Scenario-aware dataset compatibility (-8) - #4180
Draft
Richard Lundeen (richlundeen) wants to merge 79 commits into
Draft
Richard Lundeen (richlundeen) wants to merge 79 commits into
Richard Lundeen (richlundeen) wants to merge 79 commits into
Conversation
Adds ASSERT judge dimension/preset models, ships ASSERT's 11 judge presets, judges custom boolean and ordinal dimensions (with not-applicable) through build_assert_policy_scorer, aggregates repeated judge samples as ASSERT does, and exposes judge_presets, disabled_judge_dimensions, judge_samples, and objective_dimension on foundry.assert_policy. Default identities are unchanged. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ge presets to built-in names - objective_dimension and other non-default judge settings are recorded in scenario identity params, so resume detects changes and saved GUI/API launches restore them. Default identities are unchanged. - Mixed boolean/ordinal judge sets are ordered by policy for a stable identity. - judge_presets accepts only shipped preset names; preset files move to the Python-only judge_preset_files (list[Path]) parameter, which the API guard rejects. Preset load errors no longer echo file contents. - Adds a Scenario._get_identity_params hook. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…converters - Custom judge dimensions are boolean bad-event flags judged by SelfAskTrueFalseScorer with an ASSERT dimension template; ASSERT's ordinal scale and allow_not_applicable options are rejected. - Repeated samples use TrueFalseCompositeScorer over the repeated judge with an ASSERT vote (ties count as failures, undetermined samples excluded, category re-vote with override warning), replacing the bespoke sampled scorers. - Public exports trimmed to AssertJudgeDimension, AssertJudgePreset, AssertSampleMetadata, and the preset list/load functions. - foundry.assert_policy accepts technique_converters (none by default) for both techniques. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- AssertTechnique is removed. The technique class is built from the AttackTechniqueRegistry, with red_teaming as the default (ASSERT's scenario case). - The baseline, now on by default, sends each group's stored system and user prompts (ASSERT's prompt case). - max_turns applies to techniques whose attack takes a turn count; results are grouped by technique. - AssertPolicy.VERSION is 2, so runs from version 1 do not resume. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- allow_not_applicable: a not-applicable answer is undetermined with assert_not_applicable metadata; repeated samples exclude it and keep the marker only when every sample is not applicable. - ASSERT's ordinal scale is accepted with a PyRIT violation_at grade. The judge's grade position is thresholded by FloatScaleThresholdScorer, so every dimension still yields a true/false verdict, and the grade is kept in assert_grade metadata. Graded dimensions can be the objective. - AssertDimensionMetadata is exported from pyrit.score. Default identities are unchanged. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Adds a scenario-local assert_package technique to foundry.assert_policy: PyRIT's RedTeamingAttack whose tester uses ASSERT's own tester system prompt (pinned 04b713fe) filled with each group's stored tester instructions. It is the default technique alongside the baseline; red_teaming stays selectable. Groups without tester instructions are skipped or rejected per incompatible_technique_policy. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Resolve judge presets in a worker thread instead of the synchronous validator. Drop a judge sample whose judgment stays invalid after retries (scoring fails only when every sample does), as ASSERT does. Require native multi-turn targets for turn-taking techniques in foundry.assert_policy, keep turn settings a registered technique sets itself, pass the adversarial chat only to techniques that use one, and warn once per even sample count. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Add AssertDimensionResult, AssertCategoryJudgment (with AssertNodeJudgment) and AssertSampleVotes in pyrit.models as the only readers and writers of the ASSERT score metadata keys, keeping the stored key format. Missing keys are unset and malformed present data raises ValueError. The scenario judge wrapper now records the assert_dimension marker on judged and skipped scores. The custom-dimension judge renders its request from a YAML SeedPrompt, and both ASSERT response handlers validate and convert the decoded response through a new JsonSchemaResponseHandler._convert_response hook instead of parsing twice. AssertDimensionMetadata and AssertSampleMetadata are removed. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ence ASSERT judges can read what the application did, not only what it said: tool calls with arguments and results, model calls, and agent steps captured with OpenTelemetry, rendered after the turns they produced with citable step IDs. - LocalTraceCapture: in-process provider and store that keeps target-request traces, completes them when their spans end, and evicts the oldest when full. - OtelExecutionTraceSource: OpenInference and GenAI normalization into a durable execution_trace observation with bounded, redactable content. - ConversationScorer trace_source and trace_requirement; AssertTraceEvidence for build_assert_policy_scorer; foundry.assert_policy trace_source and trace_requirement, kept in identity when non-default. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Withdraw completion when a span starts after it; export and completion share the capture lock; evict any trace when full with explicit loss markers and bounded bookkeeping; render steps in parent cycles; remove the unreachable unattributed block; require a trace source for trace_requirement; document strict snapshot matching and citation behavior. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…mplate additional_instructions is an explicit dimension_system_prompt.yaml parameter rendered in the sandbox instead of appended after rendering; the prompt is unchanged when it is empty. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Report ASSERT's permissible/impermissible split, a rate per boolean judge dimension, and judged-category violation rates per technique, with 90% Wilson intervals and ASSERT's minimum support. Expose them through a backend route and the GUI run page. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
- Match results to their stored group by full seed identity, so groups sharing an objective but not their prompts resolve; keep the objective seed ID suffix only for exact duplicates and match it without parsing technique names. - Read judge dimensions independently: an undetermined objective no longer drops the policy split or other verdicts; count blocked/failed judges per dimension as no verdict, separate from not applicable. - Order tied score timestamps by score ID. - Return 409 only for unfinished runs and a sanitized 500 for unreadable saved results. - Rename BinaryRate to AssertBinaryRate and show 90% intervals in the run-page panel. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
ASSERT metrics now read the dimension, skip reason, not-applicable answer, grade and scale through AssertDimensionResult, and per-category judgments through AssertCategoryJudgment, instead of decoding the metadata keys locally. Missing judgment metadata stays unset, as on older scores, and malformed metadata now raises instead of being dropped. The scenario's judge wrapper in the layer below writes the dimension marker on judged and skipped scores, so this layer no longer edits it. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…RT behavior library AssertGenerator gains a checkpointed staged API (advance_async, build_result, estimate, validate_config): stop after a stage, apply a reviewed taxonomy and dimensions (recorded in each group's provenance), and resume after interruption without repeating model calls. TargetObjectiveGenerator.estimate and the provider-free PromptGenerationEstimate report remaining calls and a token range. The ASSERT v0.3.0 behavior (52) and suite (3) presets are vendored with loaders, and StorageIO gains list_files_async. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Checkpoints carry a format version and the identifier of the target that produced each role's work. Resuming with a different target is rejected before any model call, build_result attributes work to the recorded producers, and stored seed-group assignments must match the planned batches. Estimates count the dimensions and target system prompt sent with every seed-group call. Objective datasets record the request that generated them, the objective estimate reads its prompt once, and Azure Blob listings percent-encode names so they round-trip. AssertGenerationStage is a StrEnum. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
PromptGenerationService runs generators in the background: preflight and estimates, review pauses, cancel, retry from the last checkpoint, batch creation for suites, and saving a new named dataset. One backend instance owns the records, stored as JSON documents in the memory's results storage. Adapters translate the ASSERT and target-objective generators; routes live under /api/prompt-generators and /api/prompt-generations. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
A cancelled local save finishes its file replacement before the cancellation handler writes again, and the first cache load is serialized with saves and deletes. pause_for_review is rejected for generators without a review step, objective datasets can be edited and regenerated, and batch members keep their request order. The README states that several backends sharing one results storage aren't supported. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Datasets > Generate dataset estimates, runs, reviews, and retries generations, with an ASSERT form (bundled behavior and suite templates), a taxonomy review editor, and a target-objective form. Suites run as a batch, grouped in the generation list. Completed ASSERT datasets open foundry.assert_policy with repeatable ?dataset= preselection, and the Datasets catalog gains an Origins facet (for example, Generated). Includes Jest tests, an isolated Playwright journey over an offline generation target, CI wiring, and GUI docs. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The launch page's dataset table hid the dataset picker until a source was selected, so a scenario with no default datasets (foundry.assert_policy) said it had no named datasets and could never select one. Selection now follows the scenario's dataset options alone: with overrides allowed and nothing selected, the table says No dataset selected and offers Add dataset. Scenarios that don't take datasets, or that fix them, are unchanged. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
New generations preselect the default adversarial target for every role, including the research target, and apply it once a slow target registry loads, without replacing a user's or source's choices. Polling waits for the runtime and never shows another record's or runtime's data, a failed preflight can be checked again, generator metadata refreshes keep unsaved input, and regenerated names keep their version suffix at the maximum length. Suite dataset names follow the label, batch members list in request order, and the regenerate link retries transient failures. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Add an in-process agent example to the custom targets guide and replace the HTTPTarget + MockTransport workaround in the tool-call scorer guide with a local custom target. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ss agents Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Share scenario requirements and technique preparation across read-only assessment, selected-data validation, and launch. Add scanner coverage and incompatibility reasons, and recommend tester inputs for new ASSERT GUI generations while preserving SDK defaults. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Use existing JSON key iteration with pre-aggregate projections for literal scalar/null metadata keys on SQLite and SQL Server. Require named keys before schema parsing, and leave filtered/custom populations unverified until resolved validation. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Use the scenario-local ASSERT package factory and tester-instruction check, preserve native multi-turn requirements and existing target diagnostics, and validate trace references before input selection. Preserve the final lower-layer picker and research-target changes during rebase. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep dataset option names stable with accessible compatibility descriptions. Exercise generation, filtering, first selection, and a completed default ASSERT scan using deterministic schema-aware offline targets for every role. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Restore the generic pre-PR8 provider-inclusive metadata path, explicit missing-only loading and errors, Reset defaults, default-source choices, last-source guard, and reload triggers. Keep linked preselection mounted during background refresh and preserve preset metadata/reset behavior without inventing a preset load action. ASSERT retains its compatible-only one-load picker; existing top-layer scorer, dataset actions, and All available prototype changes are unchanged. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Reuse stored sources and load only missing selected providers through the existing preparation route. Keep compatible choices, surface Retry errors, guard stale work, and remove GUI fetch-policy and manual loading controls. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Use technique and stored generation-category display groups, qualifying policies without changing per-case execution identities. Preserve saved-plan grouping on resume and remove technical name lists from ASSERT summary headers; category-rate metrics and non-ASSERT presentation stay unchanged. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Display one category group across baseline and selected techniques, with readable row labels and unchanged per-technique metrics, execution identities and persisted grouping. Keep server totals independent of visible result pages. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Offer assert_package_spanish using the existing TranslationConverter and configured adversarial chat for target-bound native user requests, including subsequent conversation turns. Keep stored policy, tester controls, judges, baseline and original assert_package unconverted, with separate stable converter identities. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Reconstruct only the disjoint published layer8 contracts, bounded memory facts, read-only scenario assessment/listing routes and shared validation/sampling hooks on pinned predecessor e1bf06e. Native ASSERT binding and GUI integration remain separate owner8 work pending refreshed lower-layer APIs. No prototype scorer, category display or provider-preparation behavior is included. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ng in the launch form AssertPolicy(objective_scorer=...) uses a supplied true/false scorer unchanged and rejects mixing it with judge_target, a different scoring_target, or non-default legacy judge settings. Default per-policy judge construction moves into the scorer package as build_assert_judge_scorers. The ASSERT launch form hides the objective-scorer picker and ASSERT judge settings in the form, request and preview, and rejects saved custom scoring instead of replaying it as defaults. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…objective scorer Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…ts readiness Default ASSERT judges resolve their target through Scenario._resolve_scoring_target, so the explicit scoring_target run parameter takes precedence over judge_target, and AssertPolicy.get_readiness binds that target or reports it missing. A supplied objective scorer needs no default judge target. The ASSERT launch form also treats a carried scoring_target as scorer-owned, and compares saved judge settings to their display defaults by type: an empty list is the default only for list settings, and an empty or list value for target_system_prompt is rejected. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Satisfies main's check-keyword-only-params gate. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Merges the published trace evidence layer (7bf65f3) into the reshaped judge dimensions layer. Trace evidence now flows through build_assert_judge_scorers, a supplied objective scorer also owns trace configuration, and the launch form hides and rejects non-default trace settings with the other ASSERT scoring settings. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…only parameters Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…layer Bring the published layer-3 metrics (4848f91) onto the rebuilt layer 2 (06ccdf2). The merge applied without conflicts. The Foundry notebook keeps layer 2's outputs and is synced from foundry.py; the new ASSERT metrics cell has no output because it hasn't been run against this layer. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…distinct ASSERT metrics now let a saved registry name decide whether foundry.assert_policy produced a result, so another scanner's AssertPolicy class isn't mistaken for ASSERT. Older results without a registry name must come from PyRIT's AssertPolicy module. The metrics route answers other scenarios with a 400 problem of type /errors/not-assert-policy-run. The run page mounts the panel on a matching registry name, or tentatively on the class name for older runs, and hides it only for that problem type; compatibility, validation and other errors still show. A supplied objective scorer's uncategorized verdicts are reported as the objective dimension, not as ASSERT's policy judge. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep legacy NULL prompt roles as possible user inputs until hydrated validation resolves their sequence role. Preserve explicit non-user negative evidence and bounded SQLite/SQL Server query structure; compare real SQLite candidate hints with canonical ASSERT metadata and hydrated role validation. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Apply shared ANY, SUBSET and FIXED source rules before the empty-requirements fast path, keeping required ingredient metadata visible. Verify real Scam, FigStep and ApiKey services against isolated stored/provider metadata without fetching providers or hydrating seed bodies. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Group bounded compatibility facts by an explicit grouped/ungrouped discriminator as well as the UUID, matching canonical seed grouping without changing compressed profile output. Compare real SQLite UUID collisions with hydrated attack groups and retain portable SQL Server projections. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…sions A result's policy now comes only from the stored seed group with exactly the seeds it recorded, and only when that group's taxonomy matches the taxonomy the policy judge recorded in its saved scorer identifier, which survives sampling and trace wrappers. A replaced or edited dataset leaves the conversation without a matching stored policy instead of reporting another group's categories. Unmarked legacy scores count as ASSERT dimensions only with that judge provenance, so a supplied scorer that uses the policy_violation category is reported as the objective dimension. Category rates are kept per policy, with a policy_id, so policies that share a behavior or category name aren't pooled, and the panel names them apart. Results without a scenario module are no longer accepted as AssertPolicy by class name alone. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Do not apply assembled-population requirements to mandatory template-only or unloaded ingredient sources. Keep strict validation for selectable stored populations and exclude malformed, missing-input and unloaded populations without fetching or assembling providers. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
A stored seed group's policy is used only when the policy score's saved identifier comes from the scenario's built-in judge wrapper and records the same taxonomy; a policy score without that evidence leaves the conversation without a matching stored policy, while its dimension rates still count it. Only questions under that wrapper count as judge evidence, so a supplied SelfAskTrueFalseScorer whose question carries ASSERT-shaped metadata and the policy_violation category is reported as the objective dimension. Category rates are keyed by the judged policy (taxonomy, application context and target instructions), so policies that share a taxonomy but differ in context or instructions aren't pooled. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The metrics no longer read a stored group's assigned category. ASSERT's judges recompute policy_violation from their category judgments, so every violation verdict already names a violated category; ASSERT's attribution of category-less violations to the generated category isn't needed and would depend on seed metadata that can be edited after a run. A stored policy verdict that disagrees with its category judgments is now rejected as malformed instead of being attributed, and AssertCategoryMetrics drops attributed_by_fallback. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Integrates -3 d0b7c2a (typed judgments, provenance and verdict corrections) under the published -4 head; no conflicts. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Replaces the per-generator links on scanner launch pages with one Browse datasets / Generate datasets group in the Datasets header. Generate waits for the current runtime's generators, then opens the scenario's only follow-up generator (ASSERT for foundry.assert_policy), otherwise the generator chooser. Linked ?dataset= names stay explicit, exact selections for the server to validate instead of being checked against a second catalog fetch; only identical names are deduplicated. A blank name, or any name on a scanner that takes no datasets, blocks the estimate and launch with an error and a link that removes them. Regenerate links read the stored request by the dataset's exact selection key. foundry.assert_policy marks its datasets as required and asks for one at launch, using the rebuilt -3's ASSERT scenario check. Scenario.get_dataset_options reports inline seeds as fixed sources, so inline-only scenarios offer no dataset picker while their limits and filters stay editable. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
The rebuilt -3 launch-request tests build foundry.assert_policy requests without a dataset, which this layer now rejects with 'Select an ASSERT dataset.' They select one so they keep checking the default-scoring rules. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…docs Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Integrate checked compatibility infrastructure and conservative facts onto exact refreshed layer7 5431c38. Preserve relocated scorer/default and grouped-action ownership, add final native prompt/conversation dispatch and Spanish request conversion, and retain generic manual loading, fetch policies, Reset and metadata refresh. Keep category grouping and automatic provider preparation out of this layer. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Address the full pre-commit public-import gate without changing compatibility behavior or GUI source. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Integration-only publication merge. Keep the reconstructed layer8 tree exactly identical to 19f9684; original c58 publication and the preserved 3e demo history remain ancestors. Superseded category/autoload prototype hunks stay deferred to their ledger owners rather than reappearing in source. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part -8 of a 10-layer ASSERT stack. Layers -9 and -10 are Planned until their PRs exist. Design and how to use it: ASSERT in PyRIT.
Each PR targets
mainand contains the lower layers. Review the layer-only Files changed comparison, not only the commit list: retained prototype ancestry includes superseded category/loading commits, but their changes are absent from this layer's final tree.Description
Show datasets that satisfy a scenario's input contract without fetching providers while browsing. ASSERT uses one validated policy/case contract and its native prompt/conversation execution, including an explicit Spanish request variant.
PromptSendingAttackand conversation cases with ASSERT-framedRedTeamingAttack.assert_package_spanishcomposes the existingTranslationConverterwith the configured adversarial chat; it does not strip case inputs or adapt objective-only techniques.Own GUI changes
assert_packagesends stored prompt cases directly, including with comparison baseline off, or follows stored tester instructions for conversation cases.assert_package_spanishis separately selectable and translates target-bound user requests, including later turns. It requires a supplied/configured adversarial chat; baseline, policy/system context, tester controls and judges are not translated.Shared/non-ASSERT impact
The shared picker/preset metadata path now uses the read-only compatible-list API. Scenarios with no structural requirements retain their valid general catalog for
ANY;SUBSET/FIXEDrestrictions and mandatory ingredient metadata are honored. Ingredients are not certified as standalone attack populations.Other scanners retain their existing manual Load datasets action, loading-policy choices, Reset, configured-default fallback choices, final-source guard, source restrictions, metadata refresh triggers, limits and filters. Browsing or selecting an unloaded provider does not fetch it automatically. Preset editing remains read-only and retains its saved loading settings and Reset; no preset Load button is introduced. The general dataset catalog remains unfiltered.
Unchanged/deferred behavior
Default-only ASSERT scoring and hidden judge/trace controls are inherited from #4132/#4130; grouped Browse/Generate actions and explicit linked-source handoff are inherited from #4135. This layer does not change judge models, add the deferred Judge target picker, alter operation gating/preferences, numeric limits or target setup, or rewrite existing run identities.
Planned -9 (category-first result groups) and Planned -10 (nonblocking automatic dataset loading) are separate follow-up layers, not present in this PR's net source; they remain Planned until their PRs exist. Native conversion does not relax compatibility for
role_play_movie_script, TAP or other objective-only techniques. Import/export, ACS export, bootstrap/macros, run comparison and archive/evidence-retention work remain future scope.Ancestry disclosure: published head
fa20157a6a1fdd484bea1a5dc41d957bfc2cfad3has the exact tree of checked reconstruction19f96845b7c6e594477336f7f1dc22faf2178209. Its integration merge also retains second-parent prototype history from3e76d6e40aeea0cbfaab4fadc2c48454b0ac485e, including superseded category/loading commits. Their category-grouping and automatic-loading changes are absent from this tree. Review the layer-only Files changed comparison, not only the commit list: merged prototype commit titles do not describe the final net scope.Own-source demonstration
All images below are real isolated synthetic UI at exact published
fa20157a6a1fdd484bea1a5dc41d957bfc2cfad3, with cumulative lower layers present. No user configuration/data, provider download, model generation or scan was used. The former c58/7b images and URLs are archived as historical evidence, not relabeled as the reconstructed source.Compatible listing: one deliberately failed browser request exposes Retry; retrying uses the real endpoint.
Retry succeeds with validated prompt/conversation datasets only; the fixture's ordinary objective and unloaded provider are not offered.
Native default and explicit Spanish variant, with a selected policy dataset. Browse/Generate layout and default scoring are inherited lower-layer controls.
Fresh ASSERT GUI generation includes tester instructions; SDK defaults do not change.
Narrow layout and preserved ordinary scanner/preset behavior
Narrow compatible selection; grouped actions are inherited from #4135. Loading-policy controls remain until the separate loading change.
Actual non-ASSERT
airt.cyber, with an unloaded synthetic selected provider: manual Load, Reset and policies are retained; no automatic provider work occurs.Unsaved ordinary preset with an unloaded source: metadata/settings/Reset remain, with no manual or automatic loading.