Skip to content

Curate the agent context: AGENTS.md (6,663 / 6,000) and DEPLOYMENT.md (44,066 / 10,000) over budget #126

Closed

Description

@gregoryfoster

Problem

Two of the four live agent-context files are over budget. Every session pays for AGENTS.md, and DEPLOYMENT.md is the runbook most issues send an agent to.

Measured 2026-11-07 on e81eb9f with curating-context's measure-context.sh --exact --file AGENTS.md --docs-dir docs. Counts come from count_tokens, not estimates. The repo has no .skills/context-* knobs, so the skill's defaults apply: 6,000 tokens for the policy file and 10,000 per reference doc.

File Tokens Budget Verdict
AGENTS.md 6,663 6,000 over by 663 (11%)
docs/DEPLOYMENT.md 44,066 10,000 over by 34,066 (4.4×)
docs/SKILLS.md 5,574 10,000 ok
docs/SOCRATICODE.md 4,477 10,000 ok
  • tokens_live is 60,780.
  • There are no orphaned or unindexed docs.
  • 31 archival docs (docs/plans/, docs/specs/, …) are excluded by the skill's default.
  • The offline estimator undercounts this repo: it measured 2.46 bytes/token against its 2.70 default, and so read AGENTS.md as 5,852 tokens, under budget. Gate on --exact.

AGENTS.md. ## Infrastructure is 2,781 tokens, 41% of the file. The skill treats any section over about 30% as the finding in itself. The section is a 28-row table whose cells carry runbook detail (patching times, CI pins, drift and liveness thresholds) that DEPLOYMENT.md already holds. The next largest sections are Conventions (598), Environment & Tooling (587) and Development Methodology (492).

DEPLOYMENT.md. It has 24 ## sections. These are the largest, using byte shares of the exact total:

Section ~Tokens Share
Patching 6,540 14.8%
Deploy a change: scripts/deploy.sh 4,526 10.2%
Prerequisites (operator) 4,020 9.1%
Tailscale DNS 3,439 7.8%
CI pins and Dependabot 3,139 7.1%
Tailnet flaps 2,253 5.1%
The drift check 2,243 5.1%

No single trim fixes a document at 4.4× its budget. Getting under budget probably means splitting it into several reference docs (for example deploy and rollback; CI and dependencies; host operations covering patching, disk and memory; and the network, covering Tailscale DNS and flaps), each under 10,000, all indexed from AGENTS.md.

Approach

Run curating-context as written, in a worktree on a branch: measure with --baseline, verify facts, keep/demote/tighten/delete, plan to the budget, then record the after-row.

  • Relocate, don't delete. Most of this text records why something is the way it is (issue refs, measured values, operator decisions). The skill's rubric decides what moves where.
  • Cross-links. The anchors other files point at (docs/DEPLOYMENT.md#bumping-co-core, #patching, #the-lock-audit, #tailscale-dns, …) must keep resolving after a split, or be rewritten wherever they're cited.
    • 15 test files mention DEPLOYMENT.md (tests/test_deploy.py, test_patching_knob.py, test_lock_audit.py, test_pin.py, …).
    • tests/test_shipping_knobs.py fails on any .skills/doc-sections heading that no longer exists.
    • Run the full suite. check-seams.sh and verify-facts.sh cover the rest.
  • Budget knobs. The skill reads .skills/context-budget, .skills/context-doc-budget and the guard (install-guard.sh), and can install a weekly cadence (install-cadence.sh). Decide with the operator whether to adopt the guard so that drift back over budget fails early.

Hazards

  • Don't touch § Patching before Patching: a missed window goes unnoticed — solicit approval before each window, or automate unless overridden #113's first real window (Tue 2026-11-10, 05:00–10:00Z) closes. The attended run follows that section. Start with AGENTS.md and the other sections, or start after the window.
  • Docs only, so no deploy. docs/ and AGENTS.md are not processor.drift.RUNTIME_* paths. Moving a deploy/ or scripts/ comment along with the text would make it one.
  • Line by line. Some sentences in DEPLOYMENT.md are the only record of a measurement (for example the CI image, the Redis pin and the WIF chain). A sentence that verify-facts.sh can't check gets moved, not dropped.

Done when

  • measure-context.sh --exact reports AGENTS.md ≤ 6,000 and every live doc ≤ 10,000, with no orphans or unindexed docs.
  • The baseline and after-rows are both recorded (record-telemetry.sh).
  • Every link into a moved section resolves.
  • CI is green by job (lint, test).

Refs #84. Found in #121's closing comment.

Activity

  1. added 2 commits that reference this issue on Nov 8, 2026
  2. gregoryfoster commented on Nov 8, 2026

    @gregoryfoster
    MemberAuthor

    Part 1 merged: PR #128, fast-forwarded to main at 25467d4 (CI green). Docs and tooling only, so no deploy.

    Left for part 2, after the window:

    • Move § Patching and § Node.js to docs/PATCHING.md behind stubs.
    • Repoint the runtime citations in that change's deploy: pyproject.toml:51 (Shell lint) and the tailscaled drop-in's "Install: docs/DEPLOYMENT.md".
  3. added 3 commits that reference this issue on Nov 10, 2026
  4. gregoryfoster commented on Nov 10, 2026

    @gregoryfoster
    MemberAuthor

    Part 2 merged and deployed. PR #132 was fast-forwarded to main at 176acf9 (CI run 42850230431 green), and live is now 176acf9b4564.

    Done-when, against measure-context.sh --exact:

    Live (2026-11-10):

    • scripts/deploy.sh: processor restarted on 176acf9 (starting, consuming, smoke run passed, NRestarts 0). No warnings since, and the liveness check-in returned 202 at 16:03:30Z.
    • Five hand-installed host configs, each differing by one comment line, were reinstalled from the release (root:root 644), followed by daemon-reload:
      • processor-periodic
      • the apt-daily.timer.d drop-in
      • needrestart processor.conf
      • disk-gc.conf
      • the tailscaled drop-in. tailscaled was not restarted: same PID, oom_score_adj still -950, broker resolves.
    • All 9 HOST_CONFIGS now match their installed copies, so the next deploy prints no differs from note.
    • On the node, test_apt_automatic, test_disk_gc, test_needrestart, test_tailscaled and test_nodesource give 73 passed.
    • The disk sensor's first tick on the new config (16:10:18Z) read 56% and checked in ok, HTTP 202.

    Not stubbed: the ### anchors #the-patch-due-check, #patching-node, #moving-the-major-line and #removing-it. As in part 1, only ## sections have stubs, and nothing in the repo links those four.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentation

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions