[edwingao28]

Summary

An HTTP success does not establish that the complete video and audio were delivered, or that two outputs are comparable.

runner.py records request payloads, warmup/measurement identity, failures and client delivery timing. media.py decodes every video frame and audio sample; incompatible geometry or timestamps leave comparison metrics unavailable rather than resizing or trimming outputs to match. compare.py binds numerical comparisons to their provenance. The separate pinned Diffusers producer retains reference arrays before lossy encoding.

Check	Meaning
Complete decode and stream properties	Technical media validity
Aligned PSNR/audio differences	Numerical distance, not semantic quality

Testing: Complete H100 media decoding is recorded in #2916. Cross-engine quality remains unjudged.

Depends on the workload contract in #4657.

Validation details

CPU CI and H3 checks passed at d9353c82.

PR checklist and change type

Type of Change

• Bug fix
• New feature
• Configuration change
• Documentation update
• Other (please describe)

Checklist

• I have tested my changes locally
• I have updated documentation if necessary
• If python -m infx.matrix.changed --base origin/main lists affected config keys, I have appended one new entry covering them to the physical end of inferencex-e2e/perf-changelog.yaml; if it lists none, I have added none. I have not edited historical entries
• If this PR appends an inferencex-e2e/perf-changelog.yaml entry, it carries exactly one primary sweep label (a maintainer applies it on fork PRs): full-sweep-fail-fast (recommended), full-sweep-enabled, or non-canary-full-sweep-enabled. Optional modifiers all-evals, evals-only, and agentx-fast require a primary label; the last two block reuse while applied.
• Before merging via reuse, an authorized maintainer (OWNER/MEMBER/COLLABORATOR) has commented /use <run_id> (or the legacy /reuse-sweep-run) on this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the primary sweep label will no longer automatically kick off new sweeps. Remove and re-add the primary sweep label to force a new sweep.