[Afloat16]

Problem

Streaming refusal fallback currently replaces the serving hop's reported usage breakdown with only its last record. Earlier tool-loop and compaction records are then lost from usage.iterations.

Changes

• Preserve the serving hop's reported entries and their order when a sampling record is present; relabel only its last sampling entry as fallback_message.
• Backfill the serving model on every entry that lacks a model, preserving existing explicit model labels and other fields.
• When the hop reports no sampling record, use one aggregate serving entry. The aggregate already includes any compaction/advisor tokens, so retaining that partial breakdown alongside it would count them twice.
• Keep the public stream framing and aggregate usage unchanged. Add a changelog fragment under .changelog/.

Validation

The sync and async public-stream tests cover missing/empty breakdowns, multiple tool-loop turns, compaction before and after the final sampling record, explicit and missing model labels, and compaction/advisor-only breakdowns.

On Python 3.12.14, tests/lib/streaming/test_beta_messages.py and tests/lib/test_refusal_fallback_streaming.py pass 142 tests with Pydantic 2, and 141 tests with Pydantic 1 (one existing compatibility skip). The updated review cases fail on the previous implementation. Ruff lint and formatting checks pass for both changed files. Pyright reports 0 errors and 0 warnings when checking those two files.

Current revision: f5cfb8fcb70578d71d399388d3b1b0b34bcb64b6, based on upstream 4c7b598466b88559583f98216139be8e9b34c1fc. The latest upstream CI run awaits maintainer approval.

Related implementations: TypeScript #1260 and Go #477.

[mangeshraut712]

Keeping the serving hop's tool-loop and compaction records makes sense. Two edge cases in the new splice:

[mangeshraut712]

LGTM! The revisions in f5cfb8fc address both previous feedback points thoroughly:

1. Model Label Backfilling: Every entry from the serving hop now has splice.model backfilled when lacking a model (entry.get("model") or splice.model), preserving any explicit model labels while ensuring entries in usage.iterations can be identified by model.
2. Double-Counting Prevention on Non-Sampling Breakdowns: When no sampling record is present (e.g. compaction-only or advisor-only hops), the self-reported breakdown is replaced with a single aggregate fallback_message entry, preventing compaction/advisor tokens from being counted twice against the cumulative total.
3. Union Alignment & Test Coverage: test_serving_iteration_types_track_generated_union ensures SAMPLING_ITERATION_TYPES remains synchronized with the generated Pydantic union, and the parametrized sync/async test cases thoroughly cover tool-loop turns, compaction, and advisor records.

[Afloat16]

Thanks for the review and approval. The current head remains f5cfb8fc, and both review threads are resolved.

The CI run for this revision is still awaiting maintainer approval and has not executed. Could a maintainer approve that run and take the required code-owner review so the normal merge checks can finish?