From d47d4d02dc5e6e111c677887853f6de173935626 Mon Sep 17 00:00:00 2001 From: Jaana Dogan Date: Mon, 26 Oct 2026 22:14:52 -0700 Subject: [PATCH] README: trim the Quickstart, build the worker image with make images `make images` now also builds and pushes Substrate's ateworker-gvisor worker image to $(ATE_ENV_IMAGE_REPO). It builds the Substrate version pinned in go.mod, from that module's own directory so its .ko.yaml and go.sum apply and this repo's go.mod and go.sum stay unchanged. Image naming (--base-import-paths) and the version stamp match Substrate's `ate-setup publish worker-images`. The stamp goes through GOFLAGS because older ko releases do not have --ldflags. The target prints the ref to pass as --worker-image next to --api-image and --guest-image. README: - Add a "Build and Push the Images" step (`make images`) and use $ATE_ENV_IMAGE_REPO in the --worker-image example. - Cut the prerequisites down to the minimum Substrate version, the version ate-env is built against, and a GCS snapshots bucket. Remove the commit-by-commit compatibility notes and the paragraph on re-registering templates after an upgrade. - Remove the separate Images section, the --wake-budget paragraph, the "from a source build" comments, and all --rootfs-image material (also in docs/release.md). - Use for --snapshots-bucket. - List only the MCP tools internal/mcp defines (read_file, write_file, shell), and drop the edit_file, glob and grep entries for tools that do not exist. Signed-off-by: Jaana Dogan --- Makefile | 15 +++++++-- README.md | 90 +++++++++++++------------------------------------ docs/release.md | 3 +- 3 files changed, 37 insertions(+), 71 deletions(-) diff --git a/Makefile b/Makefile index 4ceee7d3..adf3c582 100644 --- a/Makefile +++ b/Makefile @@ -60,13 +60,24 @@ python-protos: python-test: cd clients/python && python3 -m pytest +# The worker image is built from the Substrate module go.mod pins, in its own +# module directory so its .ko.yaml and go.sum apply. --base-import-paths and +# the version stamp match what Substrate's `ate-setup publish worker-images` +# does, so the image lands at $(ATE_ENV_IMAGE_REPO)/ateworker-gvisor. +SUBSTRATE_DIR = $(shell go list -m -f '{{.Dir}}' go.ate.dev/substrate) +SUBSTRATE_VERSION = $(shell go list -m -f '{{.Version}}' go.ate.dev/substrate) + images: @echo "Building and pushing container images to $(ATE_ENV_IMAGE_REPO)..." @guest_img=$$(KO_DOCKER_REPO=$(ATE_ENV_IMAGE_REPO)/ate-env-guest ko build --bare ./cmd/ate-env-guest | tail -n 1); \ api_img=$$(KO_DOCKER_REPO=$(ATE_ENV_IMAGE_REPO)/ate-env-api ko build --bare ./cmd/ate-env-api | tail -n 1); \ + worker_img=$$(cd $(SUBSTRATE_DIR) && KO_DOCKER_REPO=$(ATE_ENV_IMAGE_REPO) \ + GOFLAGS=-ldflags=-X=go.ate.dev/substrate/internal/version.Version=$(SUBSTRATE_VERSION) \ + ko build --base-import-paths ./cmd/ateworker-gvisor | tail -n 1); \ echo "Published:" && \ - echo " --guest-image $$guest_img" && \ - echo " --api-image $$api_img" + echo " --guest-image $$guest_img" && \ + echo " --api-image $$api_img" && \ + echo " --worker-image $$worker_img" clean: rm -rf bin diff --git a/README.md b/README.md index 45e85add..fdac1395 100644 --- a/README.md +++ b/README.md @@ -43,69 +43,40 @@ this repo's images. Building from source works too, but then `--api-image` and go install github.com/agent-substrate/env/cmd/ate-env@latest ``` -## Images +## Quickstart -Each release publishes two images to GitHub Container Registry, multi-platform, -signed, and meant to be used by digest: +Prerequisites: a cluster running [Agent Substrate](https://github.com/agent-substrate/substrate) +at or after [`63a17a92`](https://github.com/agent-substrate/substrate/commit/63a17a92), and a GCS +bucket for snapshots. ate-env is built and tested against +[`3a2ca623`](https://github.com/agent-substrate/substrate/commit/3a2ca623). -| image | runs | -|---|---| -| `ghcr.io/agent-substrate/env/ate-env-api` | the API service, in the cluster | -| `ghcr.io/agent-substrate/env/ate-env-guest` | inside every environment actor | +### 1. Build and Push the Images -The digests are in the release notes and in the `images_.txt` asset; the -release's `ate-env` binary defaults to them. The worker image -(`--worker-image`, `ateworker-gvisor`) comes from the -[Substrate repo](https://github.com/agent-substrate/substrate) and must match the -Substrate version on your cluster. See [docs/release.md](docs/release.md) for -verification, pinning and how releases are cut. +Build and push the `ate-env-api`, `ate-env-guest` and Substrate `ateworker-gvisor` images with +[ko](https://ko.build): -## Quickstart +```bash +export ATE_ENV_IMAGE_REPO=registry.example.com/substrate-env +make images +``` + +It prints the digest-pinned refs to pass as `--api-image`, `--guest-image` and `--worker-image`. The +worker image is built from the Substrate version in `go.mod`; if your cluster runs a different one, +build it from that Substrate checkout with `ate-setup publish worker-images` instead. -Prerequisites: a cluster with [Agent Substrate](https://github.com/agent-substrate/substrate) -installed and a snapshots bucket. ate-env is built against Substrate -[`3a2ca623`](https://github.com/agent-substrate/substrate/commit/3a2ca623) (2026-10-16, Go module -`go.ate.dev/substrate`). The cluster needs a Substrate recent enough to accept what ate-env writes: -the WorkerPool declares `spec.sandboxClasses`, the ActorTemplate uses `snapshotConfig` and -`wakeupProbe`, names its sandbox class as the string `gvisor`, and declares CPU and memory limits. -That means Substrate at or after -[`63a17a92`](https://github.com/agent-substrate/substrate/commit/63a17a92) (2026-10-15), which also -renamed the worker image: `--worker-image` must be the `ateworker-gvisor` image published by that -Substrate install. An older Substrate rejects the string sandbox class (from -[`9ec49759`](https://github.com/agent-substrate/substrate/commit/9ec49759)) or the limits (from -[`c58f374a`](https://github.com/agent-substrate/substrate/commit/c58f374a)); use an ate-env from before -this bump with it. - -When you upgrade an existing cluster across that boundary, register your templates again. -Substrate cannot update an ActorTemplate in place, and a template registered earlier has no CPU and -memory limits. Delete it with `kubectl-ate delete actor-template`, then register it again with -`ate-env manifest template`. Templates that older `ate-env-api` versions derived per rootfs image -(named `-<12 hex digits>`) are no longer used and can be deleted the same way. - -### 1. Deploy the Kubernetes Resources +### 2. Deploy the Kubernetes Resources Deploy the namespace, worker pool, and API service: ```bash -# With a release binary the API image defaults to the release's. The worker -# image is ateworker-gvisor from the Substrate repo at your cluster's version; in -# that checkout, `go run ./cmd/ate-setup publish worker-images` pushes it and -# prints the digest-pinned ref. Use the digest: a :latest tag is whatever was -# built last, and a stale worker image registers workers with no capacity. ate-env manifest \ - --worker-image /ateworker-gvisor@sha256: | kubectl apply -f - -# From a source build, add: --api-image ghcr.io/agent-substrate/env/ate-env-api@sha256: + --worker-image $ATE_ENV_IMAGE_REPO/ateworker-gvisor@sha256: | kubectl apply -f - # Ensure that the pods are running: kubectl get pods -n ate-env ``` -`ate-env-api` waits for a suspended environment to wake before forwarding a guest call, up to -`--wake-budget` (default 2m; cold image pulls and cross-region snapshot reads exceed the router's own -parking budget) and, once the control plane reports the environment RUNNING, up to -`--wake-running-budget` (default 20s). Both are flags on the API deployment. - -### 2. Register the ActorTemplate in Substrate +### 3. Register the ActorTemplate in Substrate Substrate manages ActorTemplates directly in its control plane rather than Kubernetes CRDs. Use `ate-env manifest template` to generate the Substrate ActorTemplate manifest: @@ -114,24 +85,12 @@ Templates live in the `ate-env` atespace, which `ate-env-api` creates when it st ```bash # With a release binary the guest image defaults to the release's. ate-env manifest template \ - --snapshots-bucket | kubectl-ate create actor-template -f - -# From a source build, add: --guest-image ghcr.io/agent-substrate/env/ate-env-guest@sha256: + --snapshots-bucket | kubectl-ate create actor-template -f - ``` Each actor gets 2 CPUs and 4 GiB unless you pass `--cpu` and `--memory` (Kubernetes quantities such as `500m` or `8Gi`; CPU at most `999.999`, as Substrate requires). Substrate requires both limits and books them against worker capacity. -That template's container is the guest image itself. To run another image unmodified, name -it with `--rootfs-image`: the guest is then mounted into it as a read-only OCI image volume at -`/ate` and started from there, so the rootfs image is never rebuilt. - -```bash -ate-env manifest template --template py312 \ - --rootfs-image docker.io/library/python@sha256: \ - --guest-image /ate-env-guest@sha256: \ - --snapshots-bucket | kubectl-ate create actor-template -f - -``` - Then create and use an environment: ```bash @@ -259,12 +218,9 @@ The API exposes a streamable MCP endpoint at `POST /v1alpha/envs/{id}/mcp` servi | Tool | Category | Description | | ---- | -------- | ----------- | -| `read_file` | Filesystem | Read a text file with optional line numbers or line ranges | -| `write_file` | Filesystem | Create or overwrite a file | -| `edit_file` | Filesystem | Replace exact text matching target content in a file | -| `glob` | Filesystem | Search for files matching glob patterns | -| `grep` | Filesystem | Search for text or regular expressions across files | -| `shell` | Shell | Run a shell command line inside the environment | +| `read_file` | Filesystem | Read a file | +| `write_file` | Filesystem | Create or overwrite a file, with an optional POSIX mode | +| `shell` | Shell | Run a shell command line inside the environment, with optional cwd, env, stdin and timeout | ### JSON-RPC Over HTTP diff --git a/docs/release.md b/docs/release.md index 2e87d802..6757f6f3 100644 --- a/docs/release.md +++ b/docs/release.md @@ -88,8 +88,7 @@ workflow's `GITHUB_TOKEN`, under the repository. Once: - The guest's base image is pinned by digest in [`.ko.yaml`](../.ko.yaml). It is a `bash` image so that templates whose container is the guest itself have - a shell for `shell()`; for rootfs-image templates only the static binary is - used. Bump the pin deliberately and mention it in the release notes. + a shell for `shell()`. Bump the pin deliberately and mention it in the release notes. - The API image uses ko's default distroless base. - The actions both workflows use are pinned by commit SHA, with the version in a trailing comment; the organization's workflow security scan (zizmor)