Repository navigation
fix(ai): investigations finish inside their clock and close out inside the context limit - #2719
Conversation
…e the context limit - A model call past its 90s attempt cap keeps running while the low-effort retry starts, and whichever answers first wins. Cutting it threw away 90s, and the retry took 83s more at the median. - Investigation passes get 15 minutes. At 10, 17% of passes ran out of clock after 9-17 model calls at about 45 tokens a second. - A close-out runs on its own 5-minute clock, so pass plus close-out stays under the turn watchdog. - The close-out's evidence drops to 120k characters. At 200k it opened at 123k of 128k tokens and the round after submit_diagnosis no longer fit.
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 15 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
This comment has been minimized.
This comment has been minimized.
| Effect.race( | ||
| Fiber.join(first), | ||
| Stream.runCollect(attempts.retry(attempt(options, 2))), |
There was a problem hiding this comment.
🟡 Failed responses cancel healthy retries
After the cap, Effect.race accepts an original response ending with finish.reason === "error" and cancels the retry. These provider failures complete Stream.runCollect successfully, so a retry that can recover the call never finishes.
Learn more
Provider failures can arrive as response data rather than Effect failures. The existing provider-failure fixture in model-call tests exercises a normally completed stream with an error finish reason. Collecting that stream succeeds, so the race treats it as a winning answer and interrupts the other request. Previously, the original request stopped at the cap and could not cancel its replacement this way.
Example: The original request streams reasoning for 100 seconds, then returns a finish part with reason error. Its retry starts at 90 seconds and would return a valid answer at 110 seconds. The original request wins at 100 seconds and cancels that retry.
Recommended fix: Validate each collected response before admitting it as a successful race result. Treat error parts and error finish reasons as failed attempts, allowing the other attempt to continue within the existing call timeout. Keep this response-specific validation outside the generic helper or pass it an explicit validation callback. Add a regression test with a late error finish on the original attempt and a later valid retry.
Was this helpful? React with 👍 or 👎 to provide feedback.
There was a problem hiding this comment.
Fixed: past the cap, a first attempt whose finish reason is error no longer wins the race; it waits for the retry and is only replayed if the retry fails too. Test added (fails without the fix).
Maple review🟢 Confidence 10/10 · safe to merge. No test covers a first attempt that fails with a typed error past the cap; every other path the change adds is pinned. Past its attempt cap the first model call is no longer cut: the low-effort retry races it and the first answer wins, while investigation passes move to 15 minutes and the close-out runs on its own 5-minute clock with 120k characters of evidence. Safe to merge; the one earlier note on
Still open from earlier reviews
|
Fixes two production error issues in maple-chat, both from investigations:
AgentPolicyError: Agent exceeded its 10m duration limit(322 events in 30 days, still firing today)ContextBudgetError: Compaction could not fit the next model prompt inside the 128000 token context targetWhat the traces show (2026-11-11..16)
ContextBudgetErrorspans in 14 days come from the close-out that runs after a stopped pass. Its first prompt was 123k of 128k tokens (39k system prompt plus 200k characters of evidence).submit_diagnosislanded, then the next round no longer fit (Run failed after its submission landed).Changes
genai-spans.ts: a call past its attempt cap keeps going while the retry starts, and whichever answers first is used. Failing before the cap still fails at once, and the 4-minute call timeout still applies.budgets.ts: investigation passes get 15 minutes. The warning then comes at 12, which leaves room for two slow calls. A close-out runs on its own clock of at most 5 minutes (closeOutAgent), so pass plus close-out plus margin stays withinTURN_STALE_MS(25 minutes). Pull request reviews and replies keep their budgets.close-out.ts: the close-out evidence drops from 200k to 120k characters (about 50k tokens), which leaves room for the round after submission.This stays one agent turn ending in
submit_diagnosis.Verification
bunx vitest run src/platform/model-call-span.test.ts src/chat/budgets.test.ts src/chat/autonomous-phases.test.ts src/chat/close-out.test.ts src/chat/investigation.test.ts: 50 passed.keeps the first attempt past the cap and delivers it when it beats the retry: fails on main, passes here.runs an investigation's close-out on its own short clock, not the pass'sleaves a close-out room for the round after it submits: fails at 200k.TURN_STALE_MSbound in budgets.test now uses pass plus the close-out clock.oxlint -c .oxlintrc.effect.jsonpasses on the changed files.MODEL_ATTEMPT_CAP's comment inLlm.tsstill says "cut". I leftLlm.tsalone because #2715 edits it.🤖 Generated with Claude Code
Open in [code]smith · Need help on this PR? Tag
@codesmith-botwith what you need.