Skip to content

Consolidate managed storage configuration, references, and migration - #50

Merged
jay-m-dev merged 4 commits into
developfrom
feat/50-managed-storage
Nov 10, 2026
Merged

jay-m-dev merged 4 commits into
developfrom
feat/50-managed-storage

Conversation

@jay-m-dev

@jay-m-dev jay-m-dev commented Nov 10, 2026 •

Copy link
Copy Markdown
Collaborator

Closes EpistasisLab/ASAREE_Issues#50

Managed files previously used independent roots and deployment-specific paths. This change adds a shared data root with compatible directory overrides, propagates resolved configuration to workers/MCP subprocesses, and resolves stable namespace/key references in workspace state, managed knowledge commands, and Agent runtime metadata. Row snapshots now use workspace storage without depending on the dataset directory's parent.

An offline copy-first migration inventories both databases and files, verifies ready artifacts and pinned identities, migrates documented locator fields, and supports verification, interruption recovery, and rollback. Opaque row values, learned payloads, names, and arbitrary output JSON remain unchanged; explicitly unavailable historical artifacts retain their status and evidence. Sources and unknown files are retained.

Atomic publication protects dataset files, workspace generations, manifests, scripts, exports, and row snapshots. Retention protects publication/attempt references and retained workspace inputs. Knowledge unregister retains historical managed content and never deletes external files. Compose narrows new external bundle browsing; the operator guide covers configuration, migration, lifecycle, and coordinated PostgreSQL/files/Redis backup and restore. Existing input pins, workspace formats/slots, MCP scope checks, export expiry, and SQLite indexes remain supported; #69 replaces SQLite separately.

Validation:

  • Initial focused implementation checks: 115 passed, 5 skipped.
  • Review-fix checks: 74 passed, 1 PostgreSQL-dependent test skipped; includes relocation after removing old files, native Agent snapshot restore, exact payload preservation, missing history, interrupted row recovery, script readers, workspace compatibility, and protocol configuration.
  • Focused lint and whitespace checks pass.
  • Final PR CI passed: 1,989 backend tests, 14 statistical validation tests, and 643 frontend tests. PostgreSQL inventory/apply/verify/rollback checks ran in CI because local container operations are prohibited.
  • The repository's warn-only mypy check reports 693 errors in 57 files; required checks passed.

Storage migration is an explicit offline operator action. Both databases use separate guarded transactions, so writers must remain stopped through cutover/recovery. Reclamation remains conservative and operator-directed; legacy knowledge ownership requires explicit confirmation. No production migration or deployment is part of this PR.

@jay-m-dev
jay-m-dev marked this pull request as ready for review November 10, 2026 02:17
@jay-m-dev
jay-m-dev merged commit d14d995 into develop Nov 10, 2026
3 checks passed
@jay-m-dev
jay-m-dev deleted the feat/50-managed-storage branch November 10, 2026 02:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant