Skip to content

[Klaud Cold] Update Qwen3.5 FP8 MI300X SGLang AgentX MTP image to v0.5.22-rocm720-mi30x-20261024 / [Klaud Cold] 将 Qwen3.5 FP8 MI300X SGLang AgentX MTP 镜像更新至 v0.5.22-rocm720-mi30x-20261024 - #4479

Open
Klaud-Cold wants to merge 16 commits into
mainfrom
klaud/auto-b3bf0afab25b0047-23c620ad18216e45
Open

Klaud-Cold wants to merge 16 commits into
mainfrom
klaud/auto-b3bf0afab25b0047-23c620ad18216e45

Conversation

@Klaud-Cold

@Klaud-Cold Klaud-Cold commented Oct 25, 2026 •

Copy link
Copy Markdown
Collaborator

Goal: Update SGLang image from lmsysorg/sglang-rocm:v0.5.16-rocm720-mi30x-20260730 to lmsysorg/sglang-rocm:v0.5.22-rocm720-mi30x-20261024.
Baseline: 2026-08-12 · lmsysorg/sglang-rocm:v0.5.16-rocm720-mi30x-20260730
Mean latency · Sources: API 1, API 2, API 3, API 4

Point Total tok/s/GPU ↑ Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
AgentX c4 TP8 EP1 8ceea7 710.95 7.7 3,978.9 58.96
AgentX c4 TP8 EP8 abdb09 712.63 7.71 3,853.24 58.92
AgentX c16 TP8 EP1 57b47e 2,603.01 20.71 5,056.79 84.08
AgentX c16 TP8 EP8 0485a0 2,585.75 20.71 5,146.21 85.07
AgentX c20 TP8 EP1 a09bcb 3,045.57 25.63 5,234.24 87.27
AgentX c20 TP8 EP8 333bae 3,050.84 25.78 5,210.14 86.63
AgentX c24 TP8 EP1 f7e6b8 3,325.48 27.95 6,168.8 97.57
AgentX c24 TP8 EP8 a8a14f 3,330.68 27.75 6,224.83 95.53
AgentX c32 TP8 EP1 2babfe 2,334.82 15.75 54,980.93 213
AgentX c32 TP8 EP8 17d6e9 2,320.39 15.66 55,798.37 214.74
Eval Score ↑ Samples
gsm8k/em_strict · c32 97.65% 1,319

Stop: Close this PR at any time to stop this candidate now; Klaud cancels its runs and deletes its branch.

中文

**目标:**将 SGLang 镜像从 lmsysorg/sglang-rocm:v0.5.16-rocm720-mi30x-20260730 更新至 lmsysorg/sglang-rocm:v0.5.22-rocm720-mi30x-20261024。
**基线:**2026-08-12 · lmsysorg/sglang-rocm:v0.5.16-rocm720-mi30x-20260730
平均延迟 · 来源: API 1, API 2, API 3, API 4;数值及异常说明见上表。
**停止:**随时关闭此 PR 即可立即停止该候选:Klaud 会取消其运行并删除其分支。

…rocm720-mi30x-20261024

Pin lmsysorg/sglang-rocm:v0.5.22-rocm720-mi30x-20261024 by digest and rename
--cuda-graph-max-bs to --cuda-graph-max-bs-decode; upstream removed the
deprecated alias (sgl-project/sglang#38375), which mapped to the same field.

将 Qwen3.5 MI300X FP8 SGLang AgentX MTP 镜像更新至
lmsysorg/sglang-rocm:v0.5.22-rocm720-mi30x-20261024(按 digest 固定),并将
--cuda-graph-max-bs 改为 --cuda-graph-max-bs-decode;上游已移除该弃用别名
(sgl-project/sglang#38375),两者对应同一字段。

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献!

  • **审阅:**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER,请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,再通过 Slack 联系核心维护者。必须严格遵循模板,并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have,才能触发签核验证。
  • **PR 验证:**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目,并且恰好带有一个主标签时,才会运行扫描:full-sweep-fail-fast(强烈推荐;canary 加逐矩阵 fail-fast)、full-sweep-enabled(有 canary;矩阵任务在失败后继续运行)或 non-canary-full-sweep-enabled(无 canary,也无 fail-fast)。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
  • **合并后:**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复;参见重新运行失败任务的说明。

@Klaud-Cold

Klaud-Cold commented Oct 25, 2026 •

Copy link
Copy Markdown
Collaborator Author

Initial attempt · Passed · Run 40150038697 / attempt 1 · 2026-10-25 03:34 UTC
lmsysorg/sglang-rocm:v0.5.22-rocm720-mi30x-20261024@sha256:c7a107055e6805497fc57b9bfc33f9deea1b7dc6fbb119d9ab493493bd7eca5e · 367d181c38c9 · Mean latency
Change: Bump the image to the v0.5.22 ROCm 7.2 MI30x nightly (bundled SGLang main 3ea7e210d2, AITER 285579b6, from 2e9c82b359/AITER d9e5ef7c) pinned by digest, and rename cuda-graph-max-bs to cuda-graph-max-bs-decode because sgl-project/sglang#38375 removed that deprecated alias, which already set the same field in the old image; SGLANG_ENABLE_SPEC_V2 is a no-op in both images. Provenance: the nightly clones main at build time, so its exact commit may be slightly after the recorded g3ea7e210d2 pretend-version hash.

Point Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
AgentX c4 TP8 EP1 8ceea7 6.89 (-10.4%) 3,705.26 (-6.9%) 62.78 (+6.5%)
AgentX c4 TP8 EP8 abdb09 6.91 (-10.3%) 3,674.69 (-4.6%) 63.16 (+7.2%)
Eval Score ↑ Samples
gsm8k/em_strict · c32 97.65% (+0.00 pp) 1,319 each

Next: Smoke benchmarks and the gsm8k eval passed with zero request errors; append the perf-changelog entry and dispatch the full-sweep-fail-fast final sweep.

中文

初次尝试 · 已通过 · Run 40150038697 / attempt 1 · 2026-10-25 03:34 UTC
lmsysorg/sglang-rocm:v0.5.22-rocm720-mi30x-20261024@sha256:c7a107055e6805497fc57b9bfc33f9deea1b7dc6fbb119d9ab493493bd7eca5e · 367d181c38c9 · 平均延迟
**变更:**将镜像更新为按 digest 固定的 v0.5.22 ROCm 7.2 MI30x nightly(内置 SGLang main 3ea7e210d2、AITER 285579b6,原为 2e9c82b359/AITER d9e5ef7c),并将 cuda-graph-max-bs 改为 cuda-graph-max-bs-decode,因为 sgl-project/sglang#38375 移除了该弃用别名,而旧镜像中它本就设置同一字段;SGLANG_ENABLE_SPEC_V2 在两个镜像中均无作用。来源说明:nightly 在构建时克隆 main,实际提交可能略晚于记录的 g3ea7e210d2 版本哈希。实测数值及异常说明见上表。
**下一步:**冒烟基准测试与 gsm8k 评测均已通过且无请求错误;接下来追加 perf-changelog 条目并通过 full-sweep-fail-fast 发起最终扫描。

@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

parked · Session 1 is waiting on 40150038697. Recovery reruns preempted runs and starts the next session once they end.

中文

parked · 第 1 个会话正在等待 40150038697。恢复流程会重跑被抢占的运行,并在运行结束后启动下一个会话。

@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

resumed · The session ended before finishing; session 2 continues it in a new run of its planner.

中文

resumed · 会话未完成即已结束;第 2 个会话将在其规划运行的新运行中继续。

@Klaud-Cold Klaud-Cold added the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Oct 25, 2026
@Klaud-Cold

Klaud-Cold commented Oct 25, 2026 •

Copy link
Copy Markdown
Collaborator Author

Final full sweep · Passed · Run 40158818912 / attempt 1 · 2026-10-25 06:40 UTC
lmsysorg/sglang-rocm:v0.5.22-rocm720-mi30x-20261024@sha256:c7a107055e6805497fc57b9bfc33f9deea1b7dc6fbb119d9ab493493bd7eca5e · d787acd41fe6 · Mean latency
Change: Validate the complete updated-image family at this head.

Point Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
AgentX c4 TP8 EP1 8ceea7 6.91 (-10.2%) 3,730.74 (-6.2%) 62.65 (+6.3%)
AgentX c4 TP8 EP8 abdb09 6.9 (-10.5%) 3,824.14 (-0.8%) 63.14 (+7.2%)
AgentX c16 TP8 EP1 57b47e 19.38 (-6.4%) 5,044.28 (-0.2%) 92.63 (+10.2%)
AgentX c16 TP8 EP8 0485a0 19.35 (-6.6%) 5,114.94 (-0.6%) 93.07 (+9.4%)
AgentX c20 TP8 EP1 a09bcb 24.03 (-6.2%) 5,169.14 (-1.2%) 95.01 (+8.9%)
AgentX c20 TP8 EP8 333bae 24.02 (-6.8%) 5,205.09 (-0.1%) 95.45 (+10.2%)
AgentX c24 TP8 EP1 f7e6b8 26.17 (-6.4%) 6,209.5 (+0.7%) 105 (+7.6%)
AgentX c24 TP8 EP8 a8a14f 26.3 (-5.2%) 6,169.71 (-0.9%) 106.49 (+11.5%)
AgentX c32 TP8 EP1 2babfe 13.82 (-12.3%) 61,105.21 (+11.1%) 239.46 (+12.4%)
AgentX c32 TP8 EP8 17d6e9 14.13 (-9.8%) 61,763.22 (+10.7%) 232.79 (+8.4%)
Eval Score ↑ Samples
gsm8k/em_strict · c32 97.65% (+0.00 pp) 1,319 each

Next: Mark ready for maintainer review.

中文

最终完整 sweep · 已通过 · Run 40158818912 / attempt 1 · 2026-10-25 06:40 UTC
lmsysorg/sglang-rocm:v0.5.22-rocm720-mi30x-20261024@sha256:c7a107055e6805497fc57b9bfc33f9deea1b7dc6fbb119d9ab493493bd7eca5e · d787acd41fe6 · 平均延迟
**变更:**验证此提交更新镜像后的完整配置族。实测数值及异常说明见上表。
**下一步:**标记为就绪,等待维护者审查。

@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

parked · Session 2 is waiting on 40158818912. Recovery reruns preempted runs and starts the next session once they end.

中文

parked · 第 2 个会话正在等待 40158818912。恢复流程会重跑被抢占的运行,并在运行结束后启动下一个会话。

@github-actions

Copy link
Copy Markdown
Contributor

@Klaud-Cold
Klaud-Cold marked this pull request as ready for review October 25, 2026 06:40
@Klaud-Cold
Klaud-Cold requested a review from a team October 25, 2026 06:40
@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

validated · Repairs: unknown · Runs: 40150038697, 40158785046, 40158818912
All owned runs ended. Full sweep verified; ready for review.

中文

validated · 修复次数:未知 · 运行:40150038697, 40158785046, 40158818912
所有自有运行均已结束。完整 sweep 已验证;已就绪,等待审查。

@claude

claude Bot commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @Klaud-Cold's task in 20s —— View job


LGTM - no blocking issues found

Note for maintainers: the final sweep is green, but every point regressed against the 2026-08-12 baseline. Output tok/s/GPU is down 5–12% and TPOT is up 6–12%. gsm8k accuracy is unchanged.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review found no issues

No high-confidence issues detected in this change.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended)

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant