Skip to content

feat(sbx): trust host state for built-in kit heal; add MSB forged-marker and legacy-sandbox coverage - #637

Merged
mogul merged 71 commits into
mainfrom
feat/503-host-authority-followups
Oct 28, 2026
Merged

mogul merged 71 commits into
mainfrom
feat/503-host-authority-followups

Conversation

@mogul

@mogul mogul commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Stacked on Quickstart PR #611; it cannot merge before #611. Continues the host-authoritative lifecycle migration under #503 and #636.

  • SBX built-in kit heal trusts host state. The Zscaler CA, USAi, and playbook kits were skipped when a guest sbx exec probe reported their files present, which a sudo guest can forge. A host config builtin-kits record (kit ref) is now seeded at create and updated on heal success; unrecorded or stale entries are re-applied, and ACQ_FORCE_KIT_REAPPLY is unchanged. The probe is a debug-only diagnostic.
  • MSB forged-marker tests (test-only): forged guest values for agent, workspace, SSH socket, kit env, install gate, and agent-user-ready do not change behavior.
  • MSB read-only host-config decision is host-recorded. ro-host-config is written only after a create that passed the :ro volume flag and became exec-ready; a missing record takes the existing guest-copy path with its warning, and the guest probe is diagnostic only. The record does not prove the guest mount works, so when it disagrees with the probe the heal fails closed (documented in ADR-0035).
  • Accepted residual risks and decisions, documented not coded: the MSB passwd shell and the SBX startup barrier (known-failure-modes sections 46 and 47), SBX trusted-startup parity (ADR-0036, proposed), and the session-state acceptance mapping (ADR-0035).
  • Legacy-sandbox docs and tests, and a fix to the stale verify-backends OCI marker name (oci-engine-ready, as written by the pinned kit).

Plan / Context

From a read-only source audit of the post-#611 lifecycle (gaps G1, G2, G4, G5, G6, G8). The Patterns half of the root-startup-script finding is Patterns PR #543; after its release a pin-bump commit lands here.

Verification

  • bash -n on changed scripts: pass.
  • Focused bats (83, 79, 90, 85, 95, 30, 140, 72, 81, 82, 78, 115, 145; 291 tests): 0 failures, run in a scratch copy with vendored bats.
  • Mutation checks: reverting to guest-probe authority, dropping the create-time seed, dropping the builtin-kits clear, and seeding USAi regardless of provider selection each fail a test.
  • Independent adversarial reviews (two rounds on the later slices): no blockers or majors remaining; findings fixed. Not run locally: shellcheck, markdownlint, two env-dependent tests; CI to confirm.

Rollback

Revert this PR; the heal returns to guest-probe decisions.

Security impact

A sudo guest can no longer suppress or fake the built-in kit setup gates. Sandboxes created before the record exists re-apply on next heal; on sbx 0.38+ that is refused, so they show the recreate notice until recreated.

AI-assisted (OpenCode). Human review required.

Co-authored-by: OpenCode Agent bret.mogilefsky@gsa.gov

mogul and others added 24 commits October 24, 2026 20:50
Add a non-mutating source seam for production verifier functions, generic provider-facts regression coverage, and guarded live SBX/MSB validation procedures that record PASS, FAIL, or BLOCKED honestly.

Co-authored-by: OpenCode Agent <agent@gsa.gov>
On macOS the host-config dir often resolves through the /var ->
/private/var alias (e.g. when HOME or XDG_STATE_HOME sits under
/var/folders/...). msb cannot mount a symlinked host path and fails
with "mount ...: Not a directory (os error 20)".

The workspace mount already calls canonicalize_path for the same
reason (diagnosed at msb.sh lines 3574-3579); apply the identical
guard to the host-config dir mount source.

Falls back to the original path when canonicalize_path is unavailable.
The dir is acq-owned (not user-supplied) so there is no
arbitrary-symlink-traversal risk.

Closes #594

Tests: 71c-msb-canonical-host-config.bats covers the Darwin /var alias
simulation (case a) and an arbitrary symlink negative-control (case b).

Co-authored-by: OpenCode [claude_4_6_sonnet] <bret.amogilefsky@gsa.gov>
Pass the provider generation through SBX workspace mount syntax after the agent token so it is not resolved as a kit path.

Co-authored-by: OpenCode Agent <agent@gsa.gov>
Verify the provider root environment token independently of trailing kit and workspace arguments in the consolidated adoption branch.

Co-authored-by: OpenCode Agent <agent@gsa.gov>
Mount SBX provider inputs after the primary workspace, require trusted canonical MSB host-config paths, replace obsolete provider-owned config assertions, and preserve durable live failure evidence.

Closes #594
Closes #629

Co-authored-by: OpenCode Agent <agent@gsa.gov>
Record observed SBX/MSB results without overstating unexecuted rows and add an actionable opencode models check for the next live run.

Co-authored-by: OpenCode Agent <agent@gsa.gov>
Preserve observed SBX and MSB startup, config-generation, model-listing, and verifier results while keeping unexecuted destructive/negative rows explicitly blocked.

Co-authored-by: OpenCode Agent <agent@gsa.gov>
Keep the focused MSB signal regression self-contained until the trusted macOS canonicalizer lands on main.

Co-authored-by: OpenCode Agent <agent@gsa.gov>
Install the adapter projection boundary after sourcing the MSB adapter while preserving the narrow canonicalizer contract.

Co-authored-by: OpenCode Agent <agent@gsa.gov>
Preserve SBX positional grammar by placing the provider generation mount after the agent and primary workspace.

Co-authored-by: OpenCode Agent <agent@gsa.gov>
Update SBX create assertions for the corrected positional mount order.

Co-authored-by: OpenCode Agent <agent@gsa.gov>
Reflect provider mount placement and current interactive environment tokens.

Co-authored-by: OpenCode Agent <agent@gsa.gov>
Record post-fix SBX and MSB live validation totals and model-list evidence.

Co-authored-by: OpenCode Agent <agent@gsa.gov>
Record the observed interactive MSB run, USAi model listing, and successful model response.

Co-authored-by: OpenCode Agent <agent@gsa.gov>
Record successful interactive USAi model requests on both SBX and MSB without retaining secret material.

Co-authored-by: OpenCode Agent <agent@gsa.gov>
Convert malformed provider-input JSON into an actionable acq diagnostic before schema validation.

Co-authored-by: OpenCode Agent <agent@gsa.gov>
Record observed fail-closed malformed provider input rejection on SBX and MSB.

Co-authored-by: OpenCode Agent <agent@gsa.gov>
Allow an authoritative empty provider set without changing the default USAi selection.

Co-authored-by: OpenCode [gpt-5.6-terra] <bret.mogilefsky@gsa.gov>
Use Bash 3.2-safe empty-array expansion when no provider kits are selected.

Co-authored-by: OpenCode Agent <agent@gsa.gov>
Record successful no-provider SBX and MSB validation evidence.

Co-authored-by: OpenCode Agent <agent@gsa.gov>
Silence dynamic-global ShellCheck checks and tolerate current interactive environment tokens.

Co-authored-by: OpenCode Agent <agent@gsa.gov>
Use a per-test sandbox, lock, and PID path so parallel workers cannot share signal lifecycle state.

Co-authored-by: OpenCode Agent <agent@gsa.gov>
Replace guest-writable extra-kit markers with host-authoritative state for SBX create and heal paths.

Refs: #636

Co-authored-by: OpenCode [gpt-5.6-terra] <bret.mogilefsky@gsa.gov>
mogul added 6 commits October 28, 2026 17:00
…eep evidence in the repo

A live run reported FAIL only because `mount -o remount` exited zero, which the script
recorded as a write. A remount the guest kernel accepts changes the guest's own mount
flags and is not a data write, so it is now recorded as REMOUNT-ACCEPTED or
REMOUNT-DENIED (informational) and every kind of data write is repeated after it. The
verdict is computed from both the guest's reported writes and the host-side snapshot
comparison, which remains authoritative, and a PASS needs neither to show a change.

All output now goes to <repo>/tmp/qualify-msb-ro-mount-<stamp>/ (gitignored, override
ACQ_QUALIFY_OUT_DIR): results, guest log with /proc/mounts before and after, host
snapshots and diff, and the throwaway directories on FAIL or BLOCKED. Every run ends
with one pasteable RESULT line. Tests in 84 cover the accepted-remount PASS, a data
write FAIL, a host change under a forged-denied log, and the output layout.

Co-authored-by: OpenCode Agent <bret.mogilefsky@gsa.gov>
…date canary output location

Co-authored-by: OpenCode Agent <bret.mogilefsky@gsa.gov>
…y mount

The host-observed canary passed on msb 0.7.7 (macOS arm64, default shell-docker image,
fresh create): every data write was denied before and after the remount attempts, the
read-write control worked, and the real host directory was unchanged. Add 0.7.7 to
MSB_RO_MOUNT_QUALIFIED_VERSIONS. Any other version, a prerelease or an unparsable
version is still refused, a host ro-host-config record is still required, and the guest
probe still only adds refusals.

The test that asserted the list ships empty is replaced deliberately by one asserting
the list is exactly 0.7.7; new cases use the shipped list rather than an override.

Co-authored-by: OpenCode Agent <bret.mogilefsky@gsa.gov>
Add the per-version evidence to ADR-0035, including that the guest's own view of the
mount flag turned rw after an accepted remount while writes stayed denied and the host
directory was unchanged, so the guest's mount flags are not a reliable signal. State the
scope (fresh create, this version, image and host class) and what remains unverified.

Co-authored-by: OpenCode Agent <bret.mogilefsky@gsa.gov>
… not a version allowlist

Replace the per-version allowlist and canary-qualification gate with a check of what
msb itself recorded for the sandbox. Trusted staging and execution of readonly kit files
from the host-config mount now require: the host ro-host-config record; msb inspect (run
on the host) showing exactly one read-only Bind mount at the host-config guest path
backed by acq's own host config directory, with no overlapping writable mount; and a
final msb release at or above a version floor (0.7.7). A null active_config is retried
briefly; inspect failure, unparsable or unexpected output, a missing python3, or an
unreviewed platform (Windows) refuse. The guest probe stays a diagnostic that only adds
refusals. The gate runs lazily, at the first readonly file of a kit.

scripts/qualify-msb-ro-mount stays as an optional, non-gating corroboration tool.

Tests that pinned the allowlist are replaced deliberately by cases for the new gate; the
stub msb now serves the real observed inspect shape. Tests 79 and 83 give the stub a
floor-satisfying version and a read-only inspect record.

Co-authored-by: OpenCode Agent <bret.mogilefsky@gsa.gov>
… not prove

Co-authored-by: OpenCode Agent <bret.mogilefsky@gsa.gov>
@mogul

mogul commented Oct 28, 2026

Copy link
Copy Markdown
Contributor Author

🤖: AI agent (First Mate), posting on behalf of @mogul. Follow-up to the review at b92549d4 (acq.backends/msb.sh, read-only mount probe). This supersedes my earlier description of a per-version allowlist (d774e22, docs 978fa533), which is replaced by the gate below.

Current head 84827f17 (code 75528a03, docs 84827f17). Trusted staging and execution of readonly: true kit files now depend on what msb itself records for the sandbox, not on a guest answer or a version list. All of these must hold:

  1. acq's host-recorded ro-host-config record exists (unchanged).
  2. msb inspect <name> --format json, run on the host, shows in .active_config.mounts[] exactly one mount at /var/lib/acq/host that is a Bind, has options.readonly == true, and is backed by this sandbox's acq host config directory (symlinks resolved), with no other writable mount at, above or below that guest path. A null active_config is retried briefly. inspect failing, unparsable or unexpected output, or a missing python3 refuses. The parse is done in python3, not with grep or sed.
  3. The installed msb version is a final X.Y.Z at or above 0.7.7. Older, prerelease and unparsable versions are refused. On Windows (MINGW/MSYS/CYGWIN) the gate refuses, because that platform's msb filesystem backend was not reviewed.

The guest probe remains a diagnostic that can only add refusals. The gate runs lazily, at the first readonly file of a kit, so kits without such files pay nothing.

Basis, recorded in ADR-0035: in msb v0.7.7 the host-side virtiofs server rejects mutating operations when the mount is read-only, independent of the guest, and the flag it enforces is the same ro option that msb inspect reports. A live canary on msb 0.7.7 (macOS arm64) agreed: every write was denied after accepted root remounts and the host directory was unchanged. scripts/qualify-msb-ro-mount is kept as an optional, non-gating corroboration check.

What this does not prove: inspect shows what msb was asked to enforce, not independent proof of enforcement, so an upstream regression would not be caught by the gate, only by the optional canary. The gate has not yet been exercised against a real sandbox with a readonly: true kit (no shipped kit uses one), and .config.mounts and non-0.7.7 inspect shapes are unverified. The tests that pinned the allowlist were replaced deliberately by cases for the new gate, using a stub msb inspect in the shape observed live.

Also in this push range: c87c8d0a renumbers the sbx trusted-startup parity ADR to 0037, and 9d3ef1b5 (docs 09ee6967) stops gating msb setup on host records of guest-reported success.

mogul added 9 commits October 28, 2026 18:11
…line

Co-authored-by: OpenCode Agent <bret.mogilefsky@gsa.gov>
## Summary

Generalizes acq's provider-kit model so provider kits can come from
**any trusted repository** — not just the pinned
`agentic-coding-patterns` — and removes the special-casing of
`usai-provider`, which is now just the default provider kit expressed as
data.

Stacked on #611.

## What changed

- **Trusted kit-source registry** — `acq kit-source add|rm|ls`,
persisted as `kit_sources:` (default
`github.com/GSA-TTS/agentic-coding-patterns`), overridable via
`ACQ_KIT_SOURCES`. The kit fetch allowlist derives from this registry,
and provider kits from trusted sources are auto-listed (`kit-source ls`,
`acq configure`).
- **`provider_kit:` as data** — the default provider is configurable
(default `agentic-coding-patterns/usai-provider`) instead of the
hardcoded `USAI_KIT`.
- **usai de-specialized** — `usai-provider` flows through the same
generic provider path as any provider (facts.json → credential-bindings
registry → generic gate/injection/key-check). `usai-rotate-api-key` is
retained.
- **Generic credential injection** — provider credentials bind via a
registry that pins `providerId → keyEnv → host(s)`, with a fail-closed
gate and host pinning so a kit cannot redirect a stored credential to a
foreign host.

## Trust model

- Trusting a repo grants **kit fetch** and **credential bindings** only;
the provider-input validator + schemas stay pinned to the
`agentic-coding-patterns` control plane.
- The default provider (`usai`) is seeded with its known host
(`api.gsa.usai.gov`) as a trust anchor.

## Security

Multiple adversarial review rounds hardened this: host pinning
(credential redirection), positional URL passing (command injection), a
shared host-service predicate (`github`/`gitlab` token exfiltration),
removal of an `eval` sink, and fail-closed provider selection.

## Usage

```bash
acq kit-source add github.com/mogul/agentic-coding-kits
acq provider-credential set opencode-go OPENCODE_GO_API_KEY --host opencode.ai
ACQ_PROVIDER_KITS=opencode-go-provider acq run opencode .
```

## Tests

355+ bats tests green across the provider/secret/kit/provenance/dispatch
suites; `bash -n` clean. (`shellcheck` and the full suite were not
runnable in this environment.)
Remove a trailing space in common.sh, end 50-kit-list-completeness.bats with a single
newline, and annotate the two functions shellcheck 0.9 flags as SC2120: one takes an
optional argument that sbx.sh (another file) passes, the other documents an optional
explicit-ref form that no in-tree caller uses.

Co-authored-by: OpenCode Agent <bret.mogilefsky@gsa.gov>
…g table

The provider loop in the msb secret binding took each provider's host and env from the
registry and recorded facts only, so an explicit `acq secret set <id> --host H --env E`
written by the user was ignored at create while rotate and `secret rm` still acted on
it. Route the loop through _acq_msb_service_binding, the single source used by
provision, rotate and unbind (user-written mapping first, then the registry/facts
binding). The values still pass the same env and host validation before they reach
argv, and the credential gate is unchanged.

Co-authored-by: OpenCode Agent <bret.mogilefsky@gsa.gov>
…rovider

The default provider kit is now resolved from provider_kit config (or the default
constant) rather than $USAI_KIT, so the startup-staging and merged-env fixtures set the
default provider ref alongside USAI_KIT, using a local ref the way the dispatch seam
does. The key-check test drove check_fresh_sandbox_key, which the provider-key
refactor removed; it now drives acq_backend_provision with the no-ports flag directly
and also refutes the overridden host ports.

Co-authored-by: OpenCode Agent <bret.mogilefsky@gsa.gov>
The provider-kit generalization made KIT_SOURCE_PREFIXES start empty and be filled
only by _build_kit_list. Two sbx allowlist helpers still expanded it as a bare
"${arr[@]}" inside a command substitution; on bash 3.2 that is an unbound-variable
error under set -u, which set -e then turned into an abort at the very start of
provisioning, before the projection root was created. Use the ${arr[@]+...} guard at
both sites (the third already had it), which is identical on bash 4+. An empty list
still leaves the allowlist unchanged because the helper only ever unions.

Adds a behavioral smoke test with an empty array under set -euo pipefail and a static
guard that fails if a bare expansion is reintroduced; the dynamic test cannot
reproduce the bash 3.2 failure on a newer bash.

Co-authored-by: OpenCode Agent <bret.mogilefsky@gsa.gov>
@mogul
mogul enabled stacked auto-merge October 28, 2026 22:41
@mogul
mogul disabled stacked auto-merge October 28, 2026 22:43
wz-gsa
wz-gsa previously approved these changes Oct 28, 2026

@wz-gsa wz-gsa left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI-assisted review (OpenCode) performed under maintainer authorization.

Fixed-head review of 90788fc84d513e499f66f79428fd27238db8d468 against feat/601-provider-neutral-adoption.

Verdict: APPROVE

  1. Host-Authoritative Built-In Kits (acq.backends/sbx.sh): Built-in kit records (builtin-kits) and extra kits are now recorded based on host-observed create completion before the startup barrier is consulted, preventing guest marker forgery.
  2. Fail-Closed MSB Read-Only Staging (acq.backends/msb.sh):
    • Staging failures on sandboxes with a recorded ro-host-config now fail closed rather than degrading to a guest copy.
    • Read-only kit execution validates backend virtiofs mount options via msb inspect on the host and requires qualified MSB versions (>= 0.7.7) rather than trusting a guest-returned probe.
  3. Comprehensive Regression Coverage: Extensive coverage added in test/bats/81-msb-forged-guest-markers.bats, 83-msb-ro-host-config-record.bats, 84-msb-ro-mount-qualification.bats, and 90-sbx-startup-kit.bats.
  4. CI & Hygiene: pre-commit, markdown lint, and conventional commit title checks are green.

Base automatically changed from feat/601-provider-neutral-adoption to main October 28, 2026 22:48
wz-gsa pushed a commit that referenced this pull request Oct 28, 2026
## Summary

Adopt provider-neutral OpenCode configuration on both sandbox backends,
validate provider inputs on the host, and record real SBX/MSB acceptance
evidence.

## Issues

- [Quickstart
#601](#601):
provider-neutral OpenCode adoption and backend verification.
- [Quickstart
#594](#594):
canonical MSB host-config mount path.
- [Quickstart
#629](#629):
SBX provider-generation mount ordering.
- [Quickstart
#634](#634):
SBX boot stall caused by invalid positional order.
- [Quickstart
#635](#635):
explicit provider omission and authoritative empty provider sets.

## Before / After

**Before:** USAi owned OpenCode configuration, provider inputs were not
safely projected to agent kits, default provider behavior could not be
omitted, and live backend proof did not exist.

**After:** Quickstart pins Patterns v1.17.0, validates and projects
immutable provider generations, OpenCode owns generated configuration,
and `--no-provider-kits` creates an authoritative empty provider set
without a USAi credential gate.

## Verification

- Focused provider-selection/provenance/backend tests: 137 passed; 2
expected CI-only skips.
- Focused SBX routing/image/migration tests: 107 passed.
- SBX live verifier: 13 passed, 0 failed, 1 skipped.
- MSB live verifier: 18 passed, 0 failed, 1 skipped.
- Literal `acq run opencode .` with real USAi credential: PASS on SBX
and MSB; USAi models were listed and a model request succeeded.
- Malformed provider input: PASS on SBX and MSB before backend creation.
- Explicit empty provider selection: PASS on SBX and MSB; no USAi kit,
credential gate, or model entry remained.
- See `docs/VERIFY_PROVIDER_ADOPTION.md` for durable evidence and
intentionally retained historical failures.

## Additional host-authority changes

This PR also moves SBX extra-kit "already applied" state from the guest
marker `~/.acq-extra-kits` to the host config key `extra-kits` (commit
`d9499e5`, folded issue #636), so a
sudo guest can no longer suppress extra-kit re-application. The same
treatment for the built-in kits follows in the stacked [Quickstart PR
#637](#637).

## Rollback

Revert this PR to restore the prior provider bundle, configuration
ownership, and validation behavior.

## Security Impact

Provider input is validated before backend creation, mounted read-only
as immutable generations, and fails closed on malformed bundles. Empty
provider selection is host-side only and never mutates guest projection
data.

**Security impact (addition):** extra-kit applied state is
host-authoritative; legacy sandboxes without the host record re-apply on
the next heal (on sbx 0.38+ that is refused and shows the recreate
notice until the sandbox is recreated).

## Merge Guard

**Merge gate: live validation done at `8a594932`.** On 2026-10-27 the
maintainer re-ran the live checks on a real host at head
`8a594932e47745ed8f670a477f9da974121f3506`, after the merge of `main`
that changed `acq`, `acq.backends/kit-translate.sh` and
`acq.backends/msb.sh`. On SBX v0.48.0 and MSB 0.7.7, the identity-bound
check in `scripts/verify-backends` passed (SBX 13 passed, 0 failed, 1
skipped; MSB 18 passed, 0 failed, 1 skipped; both reported `rendered
config and opencode models match the projected usai snapshot (24
models)`), and the `--no-provider-kits` removal procedure passed all
three omission assertions on both backends. The skip on both is the
playbook fetch, because no GitHub token was available. Offline Bats
tests are separate and cover decision logic with stubbed backends only.
Results are in `docs/VERIFY_PROVIDER_ADOPTION.md` (Live Validation
Record and Merge Gate); raw results are in a gitignored host directory
and not committed. The gate is satisfied for `8a594932` only: a
docs-only commit does not change the tested code, but changes to `acq`,
`acq.backends/`, `scripts/verify-backends`, the pinned kits, or
provider-input handling need their effect judged and, where relevant, a
re-run. The earlier `ec8d5e17` rows are kept in the doc as history.

---------

Co-authored-by: OpenCode Agent <agent@gsa.gov>
Co-authored-by: OpenCode [claude_4_6_sonnet] <bret.amogilefsky@gsa.gov>
Co-authored-by: OpenCode Agent <user@gsa.gov>
…ty-followups

# Conflicts:
#	acq.backends/common.sh
#	acq.backends/sbx.sh
#	test/bats/90-sbx-startup-kit.bats
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants