fix(trace): prevent Codex token overcounting and separate cache usage

app/cli/internal/trace/README.md

@@ -16,7 +16,7 @@ Codex was tested with CLI 0.163.0 and `openrouter/openai/gpt-6-luna`: all 25 tes
 
 Milestone 2 live checks passed with the same CLI and model: skills in 3/3 sessions, spec pointers in 3/3 sessions, and pasted images in 1/1 session. The image scenario attaches the same image twice and requires one stored material. Transcript rendering and media redaction checks also passed. Provider and assembly regressions cover malformed images, storage failures, and reuse of an existing image material.
 
-Codex token usage comes from cumulative rollout counters. Cost is an estimate from the embedded standard API prices, with a warning for unknown models. Subscription billing and third-party provider prices can differ. Trace reads rollouts from `$CODEX_HOME/sessions` (default `~/.codex/sessions`) or `archived_sessions`. Their JSONL format is not a stable Codex interface.
+Codex token usage comes from unique response records, with cumulative rollout counters as a fallback for older transcripts. Codex evidence with `schema_version: v2` keeps cache reads and writes in separate fields; input and total tokens exclude cache, as they do for Claude Code. Legacy Codex `v1` counts include cache and must not be aggregated with `v2` counts without normalization. Cost includes all token categories and is an estimate from the embedded standard API prices, with a warning for unknown models. Subscription billing and third-party provider prices can differ. See [token accounting](codex/TOKEN_ACCOUNTING.md) for source selection, normalization, and the provider audit. Trace reads rollouts from `$CODEX_HOME/sessions` (default `~/.codex/sessions`) or `archived_sessions`. Their JSONL format is not a stable Codex interface.
 
 In a linked Git worktree, Codex 0.163 reads project hooks from the primary checkout. Install the Codex hooks in that checkout too, then review and trust them in Codex.
 
@@ -45,6 +45,7 @@ What the session evidence holds about the conversation.
 |---------|---------------|-------------|--------|--------------|------------|---|---|
 | Transcript | The evidence holds the conversation (`raw_session`): the messages and the tool calls of the agent. | Yes | Yes | Partial | Partial | No | Yes |
 | Token usage and cost | The evidence holds the tokens that the session used and their cost in USD. | Yes | No | Yes | Yes | No | Yes |
+| Cache token accounting | Cache reads and writes stay in separate evidence fields. Base input and total exclude cache. The suite compares stored token counts with the raw usage records. | Unknown | No | Unknown | Unknown | No | Unknown |
 | Model detection | The evidence names the primary model and the other models that the session used. | Yes | Partial | Yes | Yes | No | Yes |
 | Subagents | The evidence holds the conversations of the subagents, and their token usage, as part of the session. | Yes | No | No | No | No | No |
 | Missed session start | A session whose session-start hook did not fire is still recorded, from its first tool hook. | Yes | Unknown | Unknown | Yes | No | Unknown |
@@ -54,12 +55,17 @@ Why not Yes:
 - **Missed session start, Codex:** the scenario produces evidence, but its check also needs hook messages that Codex does not report in headless mode.
 - **Transcript, OpenCode:** OpenCode has no transcript file. The provider runs `opencode export` (1.x) or `opencode session export` (2) and rebuilds the transcript from the export. It keeps the text and the inputs of the tool calls. It drops the tool outputs, the reasoning and the file parts. OpenCode 2 exports also have no agent version and no session slug.
 - **Token usage and cost, Cursor:** the Cursor transcript does not hold token usage. Usage and cost are zero, and the evidence has a warning that tells so. (For OpenCode, the cost is the value that OpenCode reports. For Claude Code, Chainloop computes it from its price table.)
+- **Cache token accounting, Cursor:** the collected transcript does not expose usage or cache counts.
+- **Cache token accounting, Claude Code and Codex:** the new check has regression coverage, but has not run in the live agent matrix. Claude's source comparison also sums repeated assistant records; it does not establish that those records are unique.
+- **Cache token accounting, OpenCode:** separate cache fields are collected, but comparison against the export has not been implemented in the end-to-end check.
 - **Model detection, Cursor:** the primary model comes from the session-start hook. The transcript does not tell the models, so the evidence does not list the other models. In auto mode, the hook names the model `default`, not the model that ran.
 - **Subagents, Codex:** the initial provider records only the root session transcript. Child-session transcripts are not collected.
 - **Subagents, Cursor:** the provider does not read subagent conversations.
 - **Subagents, OpenCode:** a subagent runs in a child session with its own export. It is not part of the evidence of its parent session.
 - **Missed session start, Cursor and OpenCode 1.x:** not verified yet. The end-to-end suite finds the values (spec issue-3635).
 
+The cache accounting check reuses the edit-and-commit session, so it adds no agent runs. It compares the stored evidence with source usage, including unique Codex responses and Claude subagents, and fails if input or total includes cache or if a cache category is lost. The Codex check also rejects inflated usage by comparing with deduplicated responses or accepted cumulative snapshots. The Claude check sums all assistant records, so matching duplicated records in both source and evidence will pass; Claude deduplication remains a gap. A session with no cache activity is reported as not covered. Regression fixtures cover the compaction overcount, duplicate response IDs, resumed sessions, cumulative fallbacks, reasoning, and cache pricing. The live agent matrix must be rerun to record the new check's result.
+
 ## Code attribution
 
 Which lines of a commit the AI wrote.

app/cli/internal/trace/codex/TOKEN_ACCOUNTING.md

@@ -0,0 +1,116 @@
+# Codex token accounting
+
+Codex and Claude Code evidence use disjoint token categories:
+
+- `input_tokens`: base input, excluding cache reads and cache writes.
+- `cache_read_input_tokens`: input served from cache.
+- `cache_creation_input_tokens`: input written to cache.
+- `output_tokens`: output, including reasoning once.
+- `total_tokens`: `input_tokens + output_tokens`, excluding both cache fields.
+
+To count all processed tokens, add the two cache fields to `total_tokens`.
+Cache hits are still processed input; they have a different price. The field
+called `total_tokens` follows the existing Claude evidence convention rather
+than representing all processed tokens.
+
+Codex's source `input_tokens` includes both cache categories. Trace subtracts
+them once when writing evidence. The original counters remain in the raw
+transcript. Cost uses the same disjoint counts as evidence, with separate
+prices for base input, cache reads, cache writes, and output. When the pricing
+table has no cache-write rate, writes use the base input rate. The estimate
+does not establish billed spend.
+
+## Selecting usage records
+
+Trace reads the entire copied rollout before selecting one source:
+
+1. If native `token_usage_record` records exist and every one has a
+   `response_id` and per-response `usage`, sum those response usages. Count
+   each response ID once; the first valid usage for an ID wins. The model is
+   the model in effect at that record. Mirrored events and cumulative
+   counters do not contribute.
+2. Otherwise, use native `thread_token_usage` counters. If no native usage
+   records exist, use `event_msg.token_count.info.total_token_usage` instead.
+   Never interleave the two cumulative streams or add their totals together.
+   Cumulative snapshots are not deduplicated by response ID: another snapshot
+   with the same ID may contain a new increment.
+3. For a cumulative source, count only increases from the previous accepted
+   snapshot. Repeated snapshots add zero. A decreasing counter leaves the
+   previous baseline unchanged and produces a warning; Trace cannot infer a
+   genuine reset from that decrease. A missing cumulative snapshot in the
+   selected source also produces a warning.
+
+Per-response usage remains valid across compaction, cumulative-counter
+resets, and resumed sessions with replayed response IDs. With only cumulative
+counters, a real reset may leave usage undercounted until the counters exceed
+the previous baseline. The warning makes this uncertainty visible rather than
+adding a full session total again. Resuming with the complete transcript and
+monotonic counters adds each increment once.
+
+With cumulative counters, each increment belongs to the model at the snapshot;
+the first snapshot belongs entirely to that model. Historical model attribution
+cannot be recovered if earlier snapshots are absent. Negative counts and cache
+categories exceeding source input are rejected with a warning. Reasoning is
+already part of source output, so `reasoning_output_tokens` is not added again.
+
+## Regression evidence
+
+The sanitized `testdata/compacted-session.jsonl` fixture retains all 275 native
+usage records and 275 mirrored events from issue
+[#3706](https://github.com/chainloop-dev/chainloop/issues/3706), including the
+compaction boundary. It removes conversation content, replaces response IDs,
+and uses a fixed timestamp. Its token counters are unchanged.
+
+The original inclusive response totals are 51,803,136 input and 168,426 output
+(51,971,562 combined). Normalized evidence contains:
+
+| Field | Tokens |
+| --- | ---: |
+| `input_tokens` | 20,425 |
+| `cache_read_input_tokens` | 51,113,536 |
+| `cache_creation_input_tokens` | 669,175 |
+| `output_tokens` | 168,426 |
+| `total_tokens` | 188,851 |
+
+The embedded model prices produce an estimated cost of $8.5094. Parsing this
+fixture must produce no warnings. The previous parser reported 5,260,816,481
+total tokens, $900.8926 estimated cost, and 147 false reset warnings.
+
+## Other provider parsers
+
+This table describes the current code. This fix changes Codex only.
+
+| Provider | Input/cache fields | Total and cost | Duplicate and reasoning handling |
+| --- | --- | --- | --- |
+| Claude Code | Base input, cache reads, and cache writes are separate. | Total is base input + output. Cost includes cache reads and both 5-minute and 1-hour writes. | Sums each assistant usage record, including subagents, without message-ID deduplication. No separate reasoning count is added. Duplicate usage records can inflate counts and cost. |
+| Cursor | The collected transcript supplies no usage. | Usage and cost are unavailable, with a warning. | No token accounting to deduplicate or normalize. |
+| Pi parser | Copies input, output, cache reads, and cache writes separately. | Uses the supplied `totalTokens` and `cost.total` without reconciling categories. Its fixture total includes cache. | Rejects duplicate entry IDs and reads only the selected branch. No separate reasoning count is added. |
+| OpenCode | Copies input, output, and cache read/write fields separately. | Total is input + output. Prefers session counters/cost over message sums. | Message sums have no ID deduplication. A separate source `reasoning` counter is not represented in the parser's token types. |
+
+Pi's selected-branch scope and source totals differ from a complete session
+spend report. OpenCode's separate reasoning counter needs a follow-up decision
+and normalization if it is to be included in output. Those behaviors and
+Claude's missing deduplication remain outside this Codex fix.
+
+## Existing evidence and display
+
+Codex evidence now sets `data.schema_version` to `v2`. The JSON structure and
+schema URL stay the same; the data version identifies the accounting semantics.
+Consumers must check both `data.agent.name` and `data.schema_version`:
+
+| Agent and data version | Meaning of input and total | Inclusive processed-token total |
+| --- | --- | --- |
+| Codex `v1` | Input includes cache; total includes input and output. | Use `total_tokens` without adding the cache fields. Counts may still contain the old overcount. |
+| Codex `v2` | Input and total exclude cache, matching the existing Claude convention. | Add both cache fields to `total_tokens` exactly once. |
+
+Other providers retain their existing versions and semantics. An unknown
+version requires an explicit compatibility decision; do not infer its meaning
+from whether cache fields are present. No new evidence fields are needed.
+Consumers must not treat inclusive input as base input for pricing, or aggregate
+Codex `v1` and `v2` input/total fields together without normalizing them first.
+
+Previously uploaded attestations are not rewritten. Their inflated counts and
+cost estimates remain historical evidence and must not be interpreted as billed
+spend. Re-parsing a complete stored rollout with the fixed CLI recomputes counts
+and cost; replacing an uploaded record requires the normal re-attestation path.
+An old Codex aggregate alone is insufficient to recover the correct counts.

app/cli/internal/trace/codex/parse.go

@@ -53,6 +53,8 @@ type rolloutPayload struct {
 	Role          string      `json:"role"`
 	Name          string      `json:"name"`
 	CallID        string      `json:"call_id"`
+	ResponseID    string      `json:"response_id"`
+	Usage         *tokenUsage `json:"usage"`
 	ThreadUsage   *tokenUsage `json:"thread_token_usage"`
 	Info          *struct {
 		Total *tokenUsage `json:"total_token_usage"`
@@ -75,7 +77,7 @@ func (p *Provider) ParseSession(ctx context.Context, opts *trace.ParseOpts) (*ai
 	}
 	defer f.Close()
 	d := aicodingsession.Data{
-		SchemaVersion: "v1",
+		SchemaVersion: "v2",
 		Agent:         aicodingsession.Agent{Name: Name, Version: opts.AgentVersion},
 		Session:       aicodingsession.Session{ID: opts.SessionID},
 		Model:         &aicodingsession.Model{Primary: opts.Model},
@@ -137,9 +139,22 @@ type rolloutParser struct {
 	model                 string
 	first, last           time.Time
 	previous              tokenUsage
+	usageRecords          []usageRecord
 	haveUsage, haveHeader bool
 }
 
+// Keep the model at the time of the record. Source selection happens after
+// reading the rollout so a legacy snapshot before the first native record
+// cannot be counted in addition to native usage.
+type usageRecord struct {
+	model      string
+	line       int
+	native     bool
+	responseID string
+	usage      *tokenUsage
+	total      *tokenUsage
+}
+
 func (p *rolloutParser) consume(e rolloutEntry, payload rolloutPayload, line int) error {
 	if at, err := time.Parse(time.RFC3339Nano, e.Timestamp); err == nil {
 		if p.first.IsZero() || at.Before(p.first) {
@@ -186,22 +201,73 @@ func (p *rolloutParser) consume(e rolloutEntry, payload rolloutPayload, line int
 		}
 	case "event_msg":
 		if payload.Type == "token_count" && payload.Info != nil && payload.Info.Total != nil {
-			p.consumeUsage(*payload.Info.Total, line)
+			p.usageRecords = append(p.usageRecords, usageRecord{model: p.model, line: line, total: payload.Info.Total})
 		}
 	case "token_usage_record":
-		if payload.ThreadUsage != nil {
-			p.consumeUsage(*payload.ThreadUsage, line)
+		if payload.Usage != nil || payload.ThreadUsage != nil {
+			p.usageRecords = append(p.usageRecords, usageRecord{
+				model: p.model, line: line, native: true,
+				responseID: payload.ResponseID, usage: payload.Usage, total: payload.ThreadUsage,
+			})
 		}
 	}
 
 	return nil
 }
 
-func (p *rolloutParser) consumeUsage(total tokenUsage, line int) {
+func (p *rolloutParser) consumeSelectedUsage() {
+	haveNative, useResponses := false, true
+	for _, r := range p.usageRecords {
+		if r.native {
+			haveNative = true
+			useResponses = useResponses && r.responseID != "" && r.usage != nil
+		}
+	}
+	seen := make(map[string]bool)
+	for _, r := range p.usageRecords {
+		if r.native != haveNative {
+			continue
+		}
+		if r.native && useResponses && seen[r.responseID] {
+			continue
+		}
+		var u *tokenUsage
+		if r.native && useResponses {
+			u = r.usage
+		} else {
+			u = r.total
+		}
+		if u == nil {
+			p.data.Warnings = append(p.data.Warnings, fmt.Sprintf("missing Codex cumulative token usage on line %d", r.line))
+			continue
+		}
+		if !p.validUsage(*u, r.line) {
+			continue
+		}
+		if r.native && useResponses {
+			seen[r.responseID] = true
+		}
+		if r.native && useResponses {
+			p.addUsage(r.model, *u)
+		} else {
+			p.consumeUsage(r.model, *u, r.line)
+		}
+	}
+}
+
+func (p *rolloutParser) validUsage(total tokenUsage, line int) bool {
 	if total.Input < 0 || total.Output < 0 || total.Cached < 0 || total.CacheWrite < 0 || total.Total < 0 {
 		p.data.Warnings = append(p.data.Warnings, fmt.Sprintf("negative Codex token count on line %d", line))
-		return
+		return false
 	}
+	if total.Cached > total.Input || total.CacheWrite > total.Input-total.Cached {
+		p.data.Warnings = append(p.data.Warnings, fmt.Sprintf("Codex cache tokens exceed input tokens on line %d", line))
+		return false
+	}
+	return true
+}
+
+func (p *rolloutParser) consumeUsage(model string, total tokenUsage, line int) {
 	// Counters are session totals, not increments. Repeated snapshots add zero.
 	delta := tokenUsage{
 		Input:      total.Input - p.previous.Input,
@@ -211,24 +277,35 @@ func (p *rolloutParser) consumeUsage(total tokenUsage, line int) {
 		Total:      total.Total - p.previous.Total,
 	}
 	if delta.Input < 0 || delta.Output < 0 || delta.Cached < 0 || delta.CacheWrite < 0 || delta.Total < 0 {
-		p.data.Warnings = append(p.data.Warnings, fmt.Sprintf("Codex token counters reset on line %d", line))
-		delta = total
+		// A lower snapshot alone cannot establish a reset. Keep the previous
+		// high-water mark instead of adding a whole session again. Native
+		// per-response records, when available, handle real counter resets.
+		p.data.Warnings = append(p.data.Warnings, fmt.Sprintf("Codex cumulative token counters decreased on line %d; ignoring snapshot", line))
+		return
+	}
+	if !p.validUsage(delta, line) {
+		return
 	}
-	u := p.usageByModel[p.model]
+	p.addUsage(model, delta)
+	p.previous = total
+}
+
+func (p *rolloutParser) addUsage(model string, delta tokenUsage) {
+	u := p.usageByModel[model]
 	u.Input += delta.Input
 	u.Cached += delta.Cached
 	u.CacheWrite += delta.CacheWrite
 	u.Output += delta.Output
 	u.Total += delta.Total
-	p.usageByModel[p.model] = u
-	p.previous = total
+	p.usageByModel[model] = u
 	p.haveUsage = true
 }
 
 func (p *rolloutParser) finish() error {
 	if !p.haveHeader {
 		return errors.New("parse Codex session: missing session metadata")
 	}
+	p.consumeSelectedUsage()
 	if !p.first.IsZero() {
 		p.data.Session.StartedAt = p.first.UTC().Format(time.RFC3339)
 		p.data.Session.EndedAt = p.last.UTC().Format(time.RFC3339)
@@ -270,6 +347,10 @@ func (p *rolloutParser) finish() error {
 		sort.Strings(names)
 		for _, name := range names {
 			u := p.usageByModel[name]
+			// Match Claude's evidence convention: input and total exclude
+			// cache, which remains in separate fields and in the cost estimate.
+			u.Input -= u.Cached + u.CacheWrite
+			u.Total = u.Input + u.Output
 			p.data.Usage.InputTokens += u.Input
 			p.data.Usage.OutputTokens += u.Output
 			p.data.Usage.TotalTokens += u.Total