From ee01b700d6115d219e1220ec1521d0656702f87d Mon Sep 17 00:00:00 2001 From: gregoryfoster Date: Sun, 15 Nov 2026 20:07:46 +0000 Subject: [PATCH 1/4] #130 docs: retire Watcher's co-gcs-blob-reader grant on co-gcs-processor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CannObserv/watcher#420 removed the binding and deleted the account (read back 2026-11-15). Spec: Amended 2026-11-15 entry; Grants row deleted; D4 restated with an inline note; §1 and the prefix note name Archiver and Provisioner; the paragraph under the table notes the removal (no etag recorded). PROVISIONING: the row retired in place; GCP step 4 reduced to a retired line, so a re-run binds no deleted account. Co-Authored-By: Claude Opus 5.5 --- docs/PROVISIONING.md | 11 ++------ .../2026-09-29-processor-service-design.md | 28 +++++++++++-------- 2 files changed, 20 insertions(+), 19 deletions(-) diff --git a/docs/PROVISIONING.md b/docs/PROVISIONING.md index 5a5a996..1e3381d 100644 --- a/docs/PROVISIONING.md +++ b/docs/PROVISIONING.md @@ -21,7 +21,7 @@ What the operator set up once, on `co-processor` and around it: grants and crede | Tailnet `tag:processor` → `tag:status:9000` | tailnet policy | #35 gate 2, done 2026-10-07: `http://status:9000/health` answered `production` from `co-processor` at 14:03Z | | Status tenant `co-processor`, its production key, and the monitor `co-processor-drift` (`01M4BHMGGTXWQ16J3G5WWDQRYQ`) | `co-status` | CannObserv/status#24 (#35 gate 1), done 2026-10-07 16:02Z | | Status key at `/etc/processor/status-checkin.key` (root 600; `processor-drift.service`'s `LoadCredential=`) | this VM | #35 gate 3, [The drift check](MONITORING.md#the-drift-check) | -| Watcher's reader: `objectViewer` on `co-gcs-processor`, bucket level, for `co-gcs-blob-reader`, the identity Watcher already reads `gs://` blobs with | GCP | done 2026-10-02 (in the bucket's IAM policy); Watcher confirmed that identity on watcher#325 the same day ([its answers](https://github.com/CannObserv/watcher/issues/325#issuecomment-5958475061)) | +| Watcher's reader: `objectViewer` on `co-gcs-processor`, bucket level, for `co-gcs-blob-reader`, the identity Watcher already reads `gs://` blobs with | GCP | done 2026-10-02 (in the bucket's IAM policy); Watcher confirmed that identity on watcher#325 the same day ([its answers](https://github.com/CannObserv/watcher/issues/325#issuecomment-5958475061)). **Retired 2026-11-15:** binding removed, account deleted (CannObserv/watcher#420) | ### Broker credential handoff (hash-only, broker#75 as of 2026-09-30) @@ -74,13 +74,8 @@ gcloud storage buckets add-iam-policy-binding gs://co-gcs-processor \ gcloud storage buckets add-iam-policy-binding gs://co-gcs-blobs \ --member="serviceAccount:${SA}" --role=roles/storage.objectViewer -# 4. Watcher's reader. Proposed: co-gcs-blob-reader, which Watcher already reads -# gs:// blobs with (GCS_BLOB_CREDENTIALS). It gains nothing new, since the text -# is derived from blobs that identity can read. Run 2026-10-02, ahead of -# watcher#325's answer; if Watcher names another identity, grant that one too. -gcloud storage buckets add-iam-policy-binding gs://co-gcs-processor \ - --member="serviceAccount:co-gcs-blob-reader@${PROJECT}.iam.gserviceaccount.com" \ - --role=roles/storage.objectViewer +# 4. Retired 2026-11-15: Watcher's reader, co-gcs-blob-reader (granted 2026-10-02, +# removed with the account on CannObserv/watcher#420). Re-run nothing here. # 5. The key, straight onto co-processor over SSH stdin, never left on disk here. ( diff --git a/docs/specs/2026-09-29-processor-service-design.md b/docs/specs/2026-09-29-processor-service-design.md index 6229f37..0c87bb1 100644 --- a/docs/specs/2026-09-29-processor-service-design.md +++ b/docs/specs/2026-09-29-processor-service-design.md @@ -137,6 +137,12 @@ on `co-processor`. stays `0.38.0+3`. No behaviour changes: the same wheel bytes, which the lock holds to their sha256. +**Amended 2026-11-15 (Watcher's reader retired, processor#130):** +- Watcher no longer reads `gs://co-gcs-processor` (watcher#379). Its + `co-gcs-blob-reader` binding is removed and the account deleted + (CannObserv/watcher#420, read back 2026-11-15). Its Grants row is gone; D4, + Section 1 and the prefix note name only Archiver and Provisioner (Section 2). + --- ## Why a new service @@ -178,7 +184,7 @@ state and UI it should not have). | D1 | The service is **Processor**: repo `CannObserv/processor`, VM `co-processor`. `command.processor` names a transform *within* it (`"extract"` in v1). | | D2 | **Headless and stateless in v1**: no UI, no database. Idempotency is the write-if-absent store; correlation and dedupe are the issuer's (by `command_id`). | | D3 | **Own identities**: broker ACL user, GCS service account, bucket, tailnet tag. Observo's broker#62 `observo` user is stripped. | -| D4 | **Own bucket** `gs://co-gcs-processor`, derived outputs only, so Watcher's read grant is a plain bucket-level `objectViewer`. | +| D4 | **Own bucket** `gs://co-gcs-processor`, derived outputs only, so each reader's grant is a plain bucket-level `objectViewer`. *Amended 2026-11-15 (processor#130): was "Watcher's read grant"; Watcher's reader retired on watcher#420.* | | D5 | **co-core pinned exactly; Processor owns the pin** (0.38.0 today, processor#37). Bumps are deliberate (Section 5). Processor coordinates with Watcher only when a bump changes the contract: the `content.process` / `content.derived` models or `canonical_text` semantics. It **gives Watcher notice whenever `processor_version` moves**, because Watcher re-baselines on it. Watcher still pins co-core for contracts and the bus (`==0.19.7` on 2026-10-10), and that pin no longer has to equal Processor's. *Amended 2026-10-10 (processor#47, after watcher#350). Until then: matched to Watcher, in lockstep, since both sides extracted during shadow. Watcher agreed to that on 2026-10-02 (watcher#325), and neither side deployed a bump until the parity corpus passed on it unchanged. Amended 2026-10-21 (processor#57): essence resolution left the contract, so a change to it is notice (Section 5).* | | D6 | **Extraction runs in a killable child process** with a timeout and an address-space limit, never a thread. | | D7 | Broker `NOPERM` and `OOM` are **transient**; a reclaim **re-runs** the extraction (no dedupe key). | @@ -202,8 +208,8 @@ adapters later, selected by `command.processor` + spec); its bucket; draining **Does not own:** spec authoring (issuer), interpretation (issuers), fetching bytes (Replicator), change detection and diffing (Watcher), the durable registry -(Archiver), what readers do with its output. Watcher, Archiver and Provisioner -each read `blobs/` with their own `objectViewer` (Section 2). +(Archiver), what readers do with its output. Archiver and Provisioner each read +`blobs/` with their own `objectViewer` (Section 2). **v1:** Watcher's text extraction — #629's requirements 1–7 as amended here. **Later, each its own spec:** agenda adapters; Observo as an issuer; migrating @@ -238,14 +244,15 @@ processor-version pin on the command; no job API — the bus is the interface. | `co-gcs-processor-writer` | `gs://co-gcs-processor` | `objectCreator` + `objectViewer`; **no delete** — append-only, never deleted | | `co-gcs-processor-writer` | `gs://co-gcs-blobs` | `objectViewer` (Replicator's raw blobs — the input) | | Status tenant `co-processor` (production key, `/etc/processor/status-checkin.key`) | Status monitors `co-processor-drift` and `co-processor-live` | check-in only (`POST /api/v1/monitors/{id}/checkin`); Status's operator owns the monitors and their channels (CannObserv/status#24, #35, #39) | -| Watcher's service account, `co-gcs-blob-reader` (its `GCS_BLOB_CREDENTIALS`; confirmed on watcher#325, granted 2026-10-02) | `gs://co-gcs-processor` | `objectViewer`, bucket-level | | Archiver's `archiver-processor-text-reader@co-gcs.iam.gserviceaccount.com` (its notification composer: a size check, a ranged read and a hash check of `blobs/.bin`; CannObserv/archiver#367 D4). No binding on `gs://co-gcs-blobs`, read 2026-10-31 | `gs://co-gcs-processor` | `objectViewer`, bucket-level | | Provisioner's `co-gcs-provisioner-reader@co-gcs.iam.gserviceaccount.com` (`provisioner-consumer`, by `LoadCredential=`: the current text by `extracted_fingerprint`; CannObserv/provisioner's spec D6). Bound since Provisioner's go-live, recorded here 2026-10-31 (processor#106) | `gs://co-gcs-processor` | `objectViewer`, bucket-level | The GCP rows match the buckets' IAM policies as the operator read them on 2026-10-31 (`gs://co-gcs-processor` etag `CAY=`, `gs://co-gcs-blobs` etag -`CAk=`; processor#106). `gs://co-gcs-processor` also carries the defaults GCS -gives a new bucket, for project `co-gcs`: Owner and Editor hold +`CAk=`; processor#106), less Watcher's `co-gcs-blob-reader` binding, removed +2026-11-15 (read back on CannObserv/watcher#420, no etag recorded). +`gs://co-gcs-processor` also carries the defaults GCS gives a new bucket, for +project `co-gcs`: Owner and Editor hold `legacyBucketOwner` and `legacyObjectOwner`, Viewer holds `legacyBucketReader` and `legacyObjectReader`. So anyone with Viewer on `co-gcs` can read our objects. @@ -257,11 +264,10 @@ and the broker widens grants live. Bucket: uniform bucket-level access, public access prevention, no lifecycle rules, key scheme `blobs/.bin` (co-core's cohort scheme, -`DEFAULT_PREFIX = "blobs"`). Every reader's grant is bucket-level, so Watcher, -Archiver and Provisioner read every prefix, and all three read `blobs/` today. A -later processor whose output any of them must not read gets its own prefix then, -and the split names, for each of the three, whether its grant narrows to -`blobs/`. +`DEFAULT_PREFIX = "blobs"`). Every reader's grant is bucket-level, so Archiver +and Provisioner read every prefix, and both read `blobs/` today. A later +processor whose output either must not read gets its own prefix then, and the +split names, for each of the two, whether its grant narrows to `blobs/`. ## Section 3 — runtime and code structure From a352c28eca8d6a17e89c8eee3577d92e798c6db6 Mon Sep 17 00:00:00 2001 From: gregoryfoster Date: Sun, 15 Nov 2026 20:08:29 +0000 Subject: [PATCH 2/4] =?UTF-8?q?#130=20fix:=20CR=201=20=E2=80=94=20the=20Am?= =?UTF-8?q?ended=20entry=20describes=20D4=20and=20the=20sections=20it=20to?= =?UTF-8?q?uched?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 5.5 --- docs/specs/2026-09-29-processor-service-design.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/specs/2026-09-29-processor-service-design.md b/docs/specs/2026-09-29-processor-service-design.md index 0c87bb1..bdb42f7 100644 --- a/docs/specs/2026-09-29-processor-service-design.md +++ b/docs/specs/2026-09-29-processor-service-design.md @@ -140,8 +140,9 @@ on `co-processor`. **Amended 2026-11-15 (Watcher's reader retired, processor#130):** - Watcher no longer reads `gs://co-gcs-processor` (watcher#379). Its `co-gcs-blob-reader` binding is removed and the account deleted - (CannObserv/watcher#420, read back 2026-11-15). Its Grants row is gone; D4, - Section 1 and the prefix note name only Archiver and Provisioner (Section 2). + (CannObserv/watcher#420, read back 2026-11-15). Its Grants row is gone; D4 + names no reader; Section 1 and the prefix note name only Archiver and + Provisioner (Decisions, Sections 1 and 2). --- From 3d27a8c5363a0494c1a6e7e6dcaeeba684d1db40 Mon Sep 17 00:00:00 2001 From: gregoryfoster Date: Sun, 15 Nov 2026 20:08:29 +0000 Subject: [PATCH 3/4] =?UTF-8?q?#130=20fix:=20CR=202=20=E2=80=94=20one=20re?= =?UTF-8?q?ference=20style=20in=20the=20Amended=20entry?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 5.5 --- docs/specs/2026-09-29-processor-service-design.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/specs/2026-09-29-processor-service-design.md b/docs/specs/2026-09-29-processor-service-design.md index bdb42f7..2083de2 100644 --- a/docs/specs/2026-09-29-processor-service-design.md +++ b/docs/specs/2026-09-29-processor-service-design.md @@ -138,7 +138,7 @@ on `co-processor`. holds to their sha256. **Amended 2026-11-15 (Watcher's reader retired, processor#130):** -- Watcher no longer reads `gs://co-gcs-processor` (watcher#379). Its +- Watcher no longer reads `gs://co-gcs-processor` (CannObserv/watcher#379). Its `co-gcs-blob-reader` binding is removed and the account deleted (CannObserv/watcher#420, read back 2026-11-15). Its Grants row is gone; D4 names no reader; Section 1 and the prefix note name only Archiver and From 263f3ae3ffa6eb368fb470b5d2dd3badf3775c9b Mon Sep 17 00:00:00 2001 From: gregoryfoster Date: Sun, 15 Nov 2026 20:08:29 +0000 Subject: [PATCH 4/4] =?UTF-8?q?#130=20fix:=20CR=203=20=E2=80=94=20the=20re?= =?UTF-8?q?tired=20row's=20identity=20reads=20as=20history?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 5.5 --- docs/PROVISIONING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/PROVISIONING.md b/docs/PROVISIONING.md index 1e3381d..4533f96 100644 --- a/docs/PROVISIONING.md +++ b/docs/PROVISIONING.md @@ -21,7 +21,7 @@ What the operator set up once, on `co-processor` and around it: grants and crede | Tailnet `tag:processor` → `tag:status:9000` | tailnet policy | #35 gate 2, done 2026-10-07: `http://status:9000/health` answered `production` from `co-processor` at 14:03Z | | Status tenant `co-processor`, its production key, and the monitor `co-processor-drift` (`01M4BHMGGTXWQ16J3G5WWDQRYQ`) | `co-status` | CannObserv/status#24 (#35 gate 1), done 2026-10-07 16:02Z | | Status key at `/etc/processor/status-checkin.key` (root 600; `processor-drift.service`'s `LoadCredential=`) | this VM | #35 gate 3, [The drift check](MONITORING.md#the-drift-check) | -| Watcher's reader: `objectViewer` on `co-gcs-processor`, bucket level, for `co-gcs-blob-reader`, the identity Watcher already reads `gs://` blobs with | GCP | done 2026-10-02 (in the bucket's IAM policy); Watcher confirmed that identity on watcher#325 the same day ([its answers](https://github.com/CannObserv/watcher/issues/325#issuecomment-5958475061)). **Retired 2026-11-15:** binding removed, account deleted (CannObserv/watcher#420) | +| Watcher's reader: `objectViewer` on `co-gcs-processor`, bucket level, for `co-gcs-blob-reader`, the identity Watcher then read `gs://` blobs with | GCP | done 2026-10-02 (in the bucket's IAM policy); Watcher confirmed that identity on watcher#325 the same day ([its answers](https://github.com/CannObserv/watcher/issues/325#issuecomment-5958475061)). **Retired 2026-11-15:** binding removed, account deleted (CannObserv/watcher#420) | ### Broker credential handoff (hash-only, broker#75 as of 2026-09-30)