[BREAKING] FEAT: Require a name when the API registers a target (part 16)

.pyrit_conf_example

@@ -161,6 +161,21 @@ allow_custom_initializers: false
 # as scenario_presets_source. Defaults to ~/.pyrit/scenario_suites.
 # scenario_suites_source: https://account.blob.core.windows.net/container/scenario_suites
 
+# Optional storage for saved targets, converters, scorers, and techniques, so they
+# come back when the backend restarts. Each kind has its own folder under the source.
+# This supports the same local directory and Azure Blob sources as
+# scenario_presets_source. If any saved entry can't be restored, the backend stays
+# not ready and lists the failures in its runtime status. Fix or delete the listed
+# files, then apply the configuration or restart the backend. Defaults to
+# ~/.pyrit/registry_recipes.
+#
+# ⚠️  WARNING: The backend rebuilds every saved entry in this source as an
+# administrator, at startup and live reinitialization. Anyone who can write to this
+# source can make the backend run a model's repository code or use its identity.
+# Restrict write access to trusted administrators, and use a dedicated folder or
+# blob prefix.
+# registry_recipes_source: https://account.blob.core.windows.net/container/registry_recipes
+
 # Upload Folder for API-Created Targets
 # -------------------------------------
 # Folder that targets created through the REST API may upload local files from

doc/getting_started/pyrit_conf.md

@@ -194,6 +194,20 @@ Suites store ordered references to scenario presets and resolve each preset's cu
 
 A SAS query string may be included; otherwise, PyRIT uses `DefaultAzureCredential`. The default is `~/.pyrit/scenario_suites`. Local writes use the same hidden lock-file and atomic replacement behavior as scenario presets. Startup scans the configured source so malformed suite documents are reported immediately without preventing the backend from starting.
 
+### `registry_recipes_source`
+
+Stores saved targets, converters, scorers, and techniques in a local directory or Azure Blob container, so they come back when the backend restarts. Targets created through `POST /api/targets` with `"persist": true` are saved here. An Azure URI may include a blob-name prefix:
+
+```yaml
+registry_recipes_source: https://account.blob.core.windows.net/pyrit-storage/registry-recipes
+```
+
+Each kind has its own folder under the source (`targets/`, `converters/`, `scorers/`, `techniques/`), with one JSON file per saved entry named after the entry. A file holds the create request the entry was made with, and restore builds the entry through the API's create for its kind. A file never holds a credential: restore refuses a file that stores a value for a parameter its class marks as a credential, such as `api_key`. A saved target also can't use a key from this server: it signs in with Microsoft Entra ID (`"auth_mode": "identity"`), which uses this server's Azure identity for everyone using the app, or needs no key. Targets from the server's configuration that it uses, such as a round-robin's members, keep their own sign-in. So restore refuses a target file that would read an empty key from an environment variable or leave an address to this server's settings. The addresses each target type declares (such as `endpoint`, `container_url` for Blob storage, or an A2A agent card path) can't hold a user name, password, SAS token, fragment, path parameters (`;`, `&`, `=`), or a query other than `api-version`, and a path can't have a query at all. An address the server would otherwise fill in must be a full http(s) or ws(s) address. In identity mode, an OpenAI target file sets `"headers": "{}"` in its `params` so that the server's `OPENAI_ADDITIONAL_REQUEST_HEADERS` aren't added. The OpenAI SDK adds the headers in `OPENAI_CUSTOM_HEADERS` to every request, so while the server sets it, OpenAI targets can't be saved and saved ones aren't restored. The SDK's `OPENAI_ORG_ID` and `OPENAI_PROJECT_ID` still apply to every OpenAI target on this server. GitHub Copilot, Hugging Face, HackAPrompt, LiteLLM, HTTP, and Copilot WebSocket targets can't be saved yet. A SAS query string may be included in the source; otherwise, PyRIT uses `DefaultAzureCredential`. The default is `~/.pyrit/registry_recipes`.
+
+Restore can't tell who wrote a saved file, even one that names the user who saved it, so the backend rebuilds every entry in this source as an administrator, at startup and after a live apply. A file here can, for example, make the backend run a model's repository code, or use the server's identity on a storage container that the file names. Write access to this source is administrator authority on the backend: grant it only to principals you already trust as backend administrators, and use a dedicated directory or blob prefix that less-trusted users and services cannot write to.
+
+The backend restores saved entries at startup and after a live apply, once the configured initializers have registered theirs. If any saved entry can't be restored, the backend doesn't become ready (state `failed`, outcome `restore-failed`) and lists each failure with its reason in `GET /api/config/runtime`, so a scan can't quietly run without an entry it expects. Until then, only the configuration and status routes answer; the rest return 503. Fix or delete the listed files, or, when a folder couldn't be listed, check that the backend can read this source. Then apply the configuration, if `enable_live_reinitialization` is on, or restart the backend. Files whose names aren't valid entry names (up to 64 lowercase letters, digits, and underscores, starting with a letter) are skipped without blocking startup; administrators can see them under `restore_ignored` in `GET /api/config/runtime`.
+
 ### `initialization_scripts`
 
 Local paths to custom Python scripts containing `PyRITInitializer` subclasses. Paths can be absolute or relative to the current working directory.

doc/gui/0_gui.md

@@ -209,8 +209,28 @@ Editing a preset preserves unchanged parameter values exactly, including commas
 and whitespace inside list entries, empty lists, duplicates, and their order.
 Editing a list field uses the comma-separated format shown in the editor.
 
-Targets, converters, and techniques added only to the live registry must be
-recreated after a backend restart or reinitialization. Use
+**New Target** saves the target, so it comes back after the backend restarts or
+is reinitialized. Give it a registry name of lowercase letters, digits, and
+underscores that starts with a letter; the dialog suggests one from the target
+type and model. Saved targets are kept in `registry_recipes_source`
+(`~/.pyrit/registry_recipes` unless set in your `.pyrit_conf`). To remove a saved
+target, delete its file (`targets/<name>.json`) there, and the file of any saved
+round robin that uses it, then restart the backend. If a startup initializer later
+registers a target under a saved target's name, the saved one isn't restored and
+the backend isn't ready until that file is renamed or removed and the backend
+restarted.
+
+Saved targets can't hold credentials yet, so choose Identity-based or No key
+authentication when the target supports it. A target that needs an API key or
+another credential value, a type that can't be saved, or a round robin that uses
+a Temporary target is created as **Temporary** instead, and the dialog says why.
+A Temporary target is gone after the backend restarts or is reinitialized.
+
+The Targets table labels each target **Saved**, **From configuration**
+(registered by an initializer at startup), or **Temporary**.
+
+Converters and techniques added only to the live registry must be recreated
+after a backend restart or reinitialization. Use
 [initializers](../code/setup/0_setup.md) for repeatable component setup.
 
 Use **Auto-detect capabilities** to select targets and run live checks. Results

frontend/e2e/api.spec.ts

@@ -1,3 +1,5 @@
+import { randomUUID } from "node:crypto";
+
 import { test, expect } from "./_fixtures";
 import { compatibilityHeaders, getCompatibilityId, mockVersion } from "./_compatibility";
 
@@ -72,6 +74,7 @@ test.describe("Targets API", () => {
   test("should create and retrieve a target @seeded", async ({ request }) => {
     test.setTimeout(90_000);
     const createPayload = {
+      name: `api-${randomUUID()}`,
       type: "OpenAIChatTarget",
       auth_mode: "api_key",
       params: {

frontend/e2e/chat-recovery.spec.ts

@@ -107,6 +107,7 @@ const test = base.extend<{ localTarget: LocalTarget; imageConverterId: string }>
       const created = await request.post("/api/targets", {
         headers: compatibilityHeaders(),
         data: {
+          name: `recovery-${randomUUID()}`,
           type: "OpenAIChatTarget",
           auth_mode: "api_key",
           params: {

frontend/e2e/config.spec.ts

@@ -302,7 +302,7 @@ test.describe("Target Registry Page", () => {
 
     // Dialog should open
     await expect(page.getByText("Create New Target")).toBeVisible();
-    await expect(page.getByText("Create Target")).toBeVisible();
+    await expect(page.getByRole("button", { name: "Save Target" })).toBeVisible();
   });
 
   test("should update targets after a visible minute without a Refresh control", async ({ page }) => {
@@ -406,12 +406,14 @@ test.describe("Create Target Dialog", () => {
     await expect(selectedDetails).toContainText("Supported authentication: API key");
   });
 
-  test("should create a target through the dialog", async ({ page }) => {
+  test("should save a target through the dialog", async ({ page }) => {
     let createdTarget: FlatTarget | null = null;
+    let createBody: Record<string, unknown> | null = null;
 
     await page.route(/\/api\/targets/, async (route) => {
       if (route.request().method() === "POST") {
         const body = JSON.parse(route.request().postData() ?? "{}");
+        createBody = body;
         createdTarget = {
           target_registry_name: "new-target-1",
           target_type: body.type,
@@ -429,6 +431,27 @@ test.describe("Create Target Dialog", () => {
         await route.fulfill(mockTargetsList(items));
       }
     });
+    // Registered last, so it answers the type list instead of the route above.
+    await page.route(/\/api\/targets\/types(?:\?.*)?$/, async (route) => {
+      await route.fulfill({
+        status: 200,
+        contentType: "application/json",
+        body: JSON.stringify({
+          items: [
+            {
+              target_type: "OpenAIChatTarget",
+              parameters: [
+                { name: "endpoint", type_name: "str", required: true, default: null },
+                { name: "model_name", type_name: "str", required: true, default: null },
+              ],
+              supported_auth_modes: ["api_key", "identity", "none"],
+              saveable_auth_modes: ["identity", "none"],
+              not_saveable_reason: null,
+            },
+          ],
+        }),
+      });
+    });
 
     await goToTargets(page);
 
@@ -453,11 +476,16 @@ test.describe("Create Target Dialog", () => {
     // Fill model name
     await dialog.getByPlaceholder("e.g. gpt-4o, my-deployment").fill("gpt-4o-test");
 
-    // Click Create Target
-    await dialog.getByRole("button", { name: "Create Target" }).click();
+    await dialog.getByRole("button", { name: "Save Target" }).click();
 
     // Dialog should close and target should appear in the list
     await expect(page.getByText("Create New Target")).not.toBeVisible({ timeout: 5_000 });
+    expect(createBody).toMatchObject({
+      name: "openai_chat_gpt_4o_test",
+      type: "OpenAIChatTarget",
+      auth_mode: "identity",
+      persist: true,
+    });
     await expect(page.getByText("gpt-4o-test", { exact: true })).toBeVisible({ timeout: 10_000 });
     await expect(page.getByText("OpenAIChatTarget")).toBeVisible();
   });
@@ -496,7 +524,7 @@ test.describe("Create Target Dialog", () => {
     await expect(page.getByText("Create New Target")).toBeVisible();
 
     const dialog = page.locator('[role="dialog"]');
-    const createBtn = dialog.getByRole("button", { name: "Create Target" });
+    const createBtn = dialog.getByRole("button", { name: "Save Target" });
     await expect(createBtn).toBeDisabled();
 
     await selectTargetType(

frontend/e2e/fixtures/manual_send_backend.py

@@ -4,6 +4,7 @@
 """Offline backend for manual-send browser tests, with no real configuration or providers."""
 
 import os
+import tempfile
 from collections.abc import AsyncGenerator
 from contextlib import asynccontextmanager
 from unittest.mock import AsyncMock, patch
@@ -27,19 +28,22 @@ async def _initialize_offline_async(self: ConfigurationLoader, *, raise_on_initi
 
 @asynccontextmanager
 async def _offline_lifespan_async(application: FastAPI) -> AsyncGenerator[None, None]:
-    config = ConfigurationLoader(memory_db_type="in_memory", env_files=[], env_akv_ref=[])
-    with (
-        patch.dict(os.environ, {"PYRIT_AUTH_MODE": "local", "PYRIT_DEV_MODE": "true"}, clear=True),
-        patch.object(
-            ConfigurationFileService,
-            "_read_source_async",
-            new=AsyncMock(return_value="memory_db_type: in_memory\nenv_files: []\ninitializers: []\n"),
-        ),
-        patch.object(ConfigurationLoader, "load_with_overrides", return_value=config),
-        patch.object(ConfigurationLoader, "initialize_pyrit_async", new=_initialize_offline_async),
-    ):
-        async with lifespan(application):
-            yield
+    with tempfile.TemporaryDirectory() as recipes:
+        config = ConfigurationLoader(
+            memory_db_type="in_memory", env_files=[], env_akv_ref=[], registry_recipes_source=recipes
+        )
+        with (
+            patch.dict(os.environ, {"PYRIT_AUTH_MODE": "local", "PYRIT_DEV_MODE": "true"}, clear=True),
+            patch.object(
+                ConfigurationFileService,
+                "_read_source_async",
+                new=AsyncMock(return_value="memory_db_type: in_memory\nenv_files: []\ninitializers: []\n"),
+            ),
+            patch.object(ConfigurationLoader, "load_with_overrides", return_value=config),
+            patch.object(ConfigurationLoader, "initialize_pyrit_async", new=_initialize_offline_async),
+        ):
+            async with lifespan(application):
+                yield
 
 
 app.router.lifespan_context = _offline_lifespan_async

frontend/e2e/flows.spec.ts

@@ -1,3 +1,4 @@
+import { randomUUID } from "node:crypto";
 import { readFileSync } from "node:fs";
 
 import {
@@ -71,7 +72,7 @@ async function createTarget(
 ): Promise<string> {
   const resp = await request.post("/api/targets", {
     headers: compatibilityHeaders(),
-    data: { type: targetType, params, auth_mode: authMode },
+    data: { name: `flow-${randomUUID()}`, type: targetType, params, auth_mode: authMode },
   });
   expect(resp.ok()).toBeTruthy();
   const body = await resp.json();

frontend/e2e/registry.spec.ts

@@ -142,6 +142,32 @@ test("creates and selects a runtime technique with the real backend @seeded", as
   expect(promptRequests).toEqual([]);
 });
 
+test("saves a target from the New Target dialog and still lists it as Saved after a reload @seeded", async ({ page, request }) => {
+  const name = `saved_chat_${Date.now()}`;
+  await page.goto("/registry/targets");
+  await page.getByRole("button", { name: "New Target", exact: true }).click();
+  const dialog = page.getByRole("dialog");
+  const picker = dialog.getByRole("combobox", { name: "Target Type" });
+  await expect(picker).toBeEnabled();
+  await picker.click();
+  await page.getByRole("option", { name: /Implementation: OpenAIChatTarget/ }).click();
+  await dialog.getByRole("radio", { name: /^No key/ }).check();
+  await dialog.getByPlaceholder("https://your-resource.openai.azure.com/").fill("http://127.0.0.1:9/v1");
+  await dialog.getByPlaceholder("e.g. gpt-4o, my-deployment").fill("local-test");
+  await dialog.getByRole("textbox", { name: /Registry Name/ }).fill(name);
+  await expect(dialog.getByText("This target can't be saved")).toHaveCount(0);
+  await dialog.getByRole("button", { name: "Save Target" }).click();
+  await expect(dialog).toHaveCount(0);
+
+  await expect(page.getByTestId(`target-row-${name}`).getByText("Saved", { exact: true })).toBeVisible();
+  const saved = await request.get(`/api/targets/${name}`, { headers: compatibilityHeaders() });
+  expect(saved.ok(), await saved.text()).toBe(true);
+  expect((await saved.json()).origin).toBe("saved");
+
+  await page.reload();
+  await expect(page.getByTestId(`target-row-${name}`).getByText("Saved", { exact: true })).toBeVisible();
+});
+
 interface RegisteredConverter {
   converter_id: string;
   identifier: {

frontend/playwright.config.ts

@@ -1,3 +1,7 @@
+import { mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs";
+import { tmpdir } from "node:os";
+import path from "node:path";
+
 import { defineConfig, devices } from "@playwright/test";
 
 import type { UserPreferences } from "./src/types";
@@ -24,6 +28,28 @@ const VITE_SERVER_OPTIONS =
 const E2E_BACKEND_PORT = process.env.PYRIT_E2E_BACKEND_PORT ?? "18000";
 const E2E_BACKEND_URL = `http://127.0.0.1:${E2E_BACKEND_PORT}`;
 
+/**
+ * Copies the seeded backend's configuration into a fresh folder that also holds
+ * its saved targets, converters, scorers, and techniques, so a run never reads
+ * or writes the ones saved on the machine running it.
+ */
+function createSeededBackendConfigFile(): string {
+  const folder = mkdtempSync(path.join(tmpdir(), "pyrit-seeded-backend-"));
+  // The runner stops the backend before it exits, so nothing still uses the folder by then.
+  process.once("exit", () => rmSync(folder, { recursive: true, force: true }));
+  const configFile = path.join(folder, "browser_config.yaml");
+  const config = readFileSync(new URL("../tests/end_to_end/browser_config.yaml", import.meta.url), "utf-8");
+  const recipesSource = JSON.stringify(path.join(folder, "registry_recipes"));
+  writeFileSync(configFile, `${config}\nregistry_recipes_source: ${recipesSource}\n`);
+  return configFile;
+}
+
+// Workers load this file too, but only the main process starts the backend.
+const SEEDED_BACKEND_CONFIG_FILE =
+  SEEDED_MODE && process.env.TEST_WORKER_INDEX === undefined
+    ? createSeededBackendConfigFile()
+    : "";
+
 export default defineConfig({
   testDir: "./e2e",
   testIgnore: ["**/analytics.spec.ts", "**/paired-outcomes.spec.ts", "**/scenario-hierarchy.spec.ts", "**/scenario-datasets.spec.ts", "**/scenario-techniques.spec.ts", "**/scan-launch.seeded.spec.ts"],
@@ -113,7 +139,7 @@ export default defineConfig({
           command:
             `cd .. && uv run --no-sync python -m pyrit.backend.pyrit_backend ` +
             `--auth-mode local --host 127.0.0.1 --port ${E2E_BACKEND_PORT} --log-level warning ` +
-            "--config-file tests/end_to_end/browser_config.yaml",
+            `--config-file "${SEEDED_BACKEND_CONFIG_FILE}"`,
           env: { PYRIT_DEV_MODE: "true" },
           url: `${E2E_BACKEND_URL}/api/health`,
           reuseExistingServer: false,