Skip to content

Compress large complete Turn sources on the local bridge - #8391

Open
loopx-agent wants to merge 1 commit into
mainfrom
codex/pack-complete-turn-source-20261109
Open

loopx-agent wants to merge 1 commit into
mainfrom
codex/pack-complete-turn-source-20261109

Conversation

@loopx-agent

Copy link
Copy Markdown
Collaborator

The Turn CLI can send a large complete work source from Python to the local TypeScript runtime just to compose its contract. This extracts C1's bounded compression for that request. The existing composition and integrity checks receive the same decoded source.

Goal And Delivered Outcome

  • Independent transport slice of C1: unify work sources and validate recovery #8266, targeting main.
  • The Python adapter packs sources between 16 KiB and 1 MiB when the encoded data saves enough bytes. Small, oversized and poorly compressible sources use the existing raw transport. The TypeScript adapter bounds both inputs and inflation, and rejects mixed or invalid carriers.
  • On File and SQLite, the turn plan composition request went from about 26 KB to 10 KB on the normal fixture and 139 KB to 17 KB with long Goal material. Across the entire invocation, request and response bytes fell 6.8% and 10.8% respectively. The plan output size and RPC count matched.

Author Declaration

  • Written by model_agent — OpenAI Codex.

Implemented against

Criterion Disposition Owner Evidence
Preserve the complete source, original numeric tokens and signed composition implemented Existing compose_complete TypeScript handler Raw/packed request parity and numeric integrity tests
Bound optional transport and reject damaged or mixed carriers implemented Turn bridge codec Invalid UTF-8, corrupt data, inflated-size cap, and small/oversized raw path tests
Verify the actual Turn caller implemented turn plan and turn run-once Isolated File/SQLite plans, real settlement fixture, and the built wheel

Scope And Continuation

  • Both encodings go through the same complete-source owner. Saved captures and the public CLI keep their existing format. The compression changes travel with one PR so C1 can absorb them from main.
  • Python CPU for the composition adapter increased by about 0.18–0.27 ms per measured request. Whole-plan wall-time medians varied by workload, from 8.4 ms faster to 8.9 ms slower. This comparison qualifies transfer bytes on the local bridge. The original single-session model and outcome comparison remains in C1: unify work sources and validate recovery #8266.

Validation

  • Tested revision a7e9ae10275312e9de59e65764fa3ca42b8875fd against aa475b29f1976a4fde901deca84672ddb3415152.
  • Run state — finished. Input classes — synthetic.
Check kind Result Public-safe evidence
regression_parity passed Same File/SQLite work and material at base and head. On the head runtime, 15 raw and 15 packed plans per fixture were interleaved using the baseline Python adapter for raw requests. A separate full comparison of two plans per arm found variation only in observation timing and its dependent hashes. With identical source inputs, the raw and packed compositions return identical signed edits.
real_entrypoint passed 125 source tests cover complete source, Host input, capture and numeric recovery, and actual turn run-once completion through a synthetic host.
real_backend passed File and SQLite with disposable state. Verified current plan ownership, selected work and complete source delivery.
integration passed 82 TypeScript Turn/frontier tests. The built wheel passed 9 focused tests and exercised 10 CLI subprocesses pinned to the wheel.
static passed TypeScript typecheck, Ruff, public-boundary scan, semantic vocabulary advisory and local diff-scoped premerge.
  • Coverage and gaps — The timing sample uses local CLI runs with synthetic tasks and observes bridge bytes and CPU. Model window, token usage and actual task outcomes remain separate qualification.

Frontend / Visual Evidence

  • UI impact — none.

Type of Change

  • Refactoring (no functional changes)
  • Test update

LoopX Area

  • Control plane (goals, todos, quota, scheduler, registry, runtime)

Shared-authority RFC fixture impact

N/A. This changes the local Turn bridge encoding.

Boundary Checklist

  • Diff and review materials contain public-safe evidence.
  • Scoped to the linked Turn transport slice.
  • No paired RFC changed.
  • UI impact marked none.
  • DCO sign-off present.

Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com>

@loopx-agent loopx-agent left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer: model_agent, gpt-6.1-sol, OpenAI, runtime_reported, reasoning_effort=max

Request changes conclusion (author-owned PR; GitHub blocks formal self-review)

动机 / Motivation

Turn CLI 需要把完整来源交给本地 TypeScript owner,同时保留原始数字、绑定与结算语义。原桥接把完整 JSON 作为 RPC 参数,长材料在本地重复传输。候选让合适的来源以较小的压缩体进入同一组合规则,但接收端没有检查流尾是否还有字节。当前评审范围是本地桥接编码、真实计划入口及原执行和结算。C1 正式切换、真实模型采用和完整成本留待原验收。

改动思路 / Approach

复用现有 RPC JSON 序列化和 TypeScript 完整来源 owner。当前边界是一次 compose_complete 请求的载体。普通旧调用仍传对象。只有满足 16 KiB 至 1 MiB 范围并实际节省字节时才传压缩体,原对象继续用于回放 edits。新增入口的最小拒绝边界是单个完整的 zlib stream,保留现有压缩前后大小上限与来源校验。

具体改动 / Changes

评审 a7e9ae1 对 aa475b2 的 3 个路径,+80/−3。complete-source-parity、raw-fallback 和 real-turn-caller 有独立实际入口证据。bounded-carrier-refusal 的尾随输入仍能通过,c1-outcome-cost 另行验收。候选和当前目标主干可无冲突合成。 C1 切换要求

关键代码讲解

  • Python 的 _turn_envelope_result 保留原 payload 和精确数字 token。符合阈值时,局部 RPC 参数改用 source_transport。短输入、超限及收益不足仍发普通对象,原快照回退留在 Effect runtime。 发送端
  • TypeScript 的 decodeCompositionSource 检查载体、拒绝 raw/packed 并存,限制编码和膨胀大小并校验 UTF-8。解压对象继续交给既有 composeCompleteTurnDecision 计算和验证签名。inflateSync 当前只返回首个流解压后的内容。 接收端 · 完整组合

对主干的风险 / Risks

P2 · 带尾随字节或第二个压缩流的载体被当作有效。 把正常发送器产生的字节交给实际本地 quota.turn_envelope.evaluate。截断、损坏头部、无效 UTF-8、超膨胀和 raw/packed 并存均被拒绝。将任意尾随字节或含不同 Goal 的第二个 zlib stream 拼在首流之后并重新编码,则两次都返回与首流相同的签名 edits。第二流未改变已验证的来源,却暴露出接收端未执行声明的无效载体拒绝。请在同一上限内核验解压器消耗了完整输入,保留合法单流正例,并加入这两个实际 RPC 反例。 可定位分支

用相同 File/SQLite Goal、任务和 turn plan 参数对照,普通组合参数从 27,714/27,740 B 降到 9,849/9,873 B,长材料从 153,772/153,798 B 降到 12,201/12,213 B。把候选的同一来源以 raw 送入同一 owner 后签名 edits 一致,原任务、计划输出字符数、来源和扣额结果相同。核验了大整数、负零和指数小数。1,403 B 小来源、超过 1 MiB 的来源及低收益样本都走 raw。现有显式关闭入口在两版保持 v0,JSON 9,449 字符和 Markdown 2,427 字符相同。基线 34 项与候选 35 项源测试通过,候选 standard premerge 的 6 项直接和 13 项选中检查通过。字节统计仅指组合请求参数。

我的整体评价 / Verdict

这项抽取实现了可量化的本地传输缩减,需补齐载体拒绝后再批准。当前证据覆盖精确源码的真实计划、两种 provider 和隔离执行结算。长期效果、模型 token、完整成本及独立安装采用未在此次验证。

English verdict: REQUEST_CHANGES for a7e9ae1. The real local bridge accepts a valid compressed source followed by junk or a second stream. Keep the verified raw/packed parity and require the decoder to consume the whole carrier.

}
let source: unknown;
try {
const decoded = inflateSync(Buffer.from(data, "base64"), {maxOutputLength: maxSourceBytes});

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 · 解压未核验完整载体

实际 quota.turn_envelope.evaluate 对单流、单流加尾随垃圾和单流加第二个不同 Goal 的 zlib 流返回相同的签名 edits。截断、损坏头部、无效 UTF-8 和超膨胀都正确拒绝。请保留这些边界,再校验解压器消耗的输入字节数等于整个 carrier。用两个尾随反例和现有合法单流正例回归。

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant