Skip to content

[Klaud Cold] Update GLM-5.2 FP8 B200 SGLang AgentX MTP image to nightly-dev-20261026-8252e3af / [Klaud Cold] 将 GLM-5.2 FP8 B200 SGLang AgentX MTP 镜像更新至 nightly-dev-20261026-8252e3af - #4543

Open
Klaud-Cold wants to merge 13 commits into
mainfrom
klaud/auto-fb56d2a358d2743c-0dd16aeb42d3dd5b
Open

[Klaud Cold] Update GLM-5.2 FP8 B200 SGLang AgentX MTP image to nightly-dev-20261026-8252e3af / [Klaud Cold] 将 GLM-5.2 FP8 B200 SGLang AgentX MTP 镜像更新至 nightly-dev-20261026-8252e3af#4543
Klaud-Cold wants to merge 13 commits into
mainfrom
klaud/auto-fb56d2a358d2743c-0dd16aeb42d3dd5b

Conversation

@Klaud-Cold

@Klaud-Cold Klaud-Cold commented Oct 26, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Goal: Update SGLang image from lmsysorg/sglang:nightly-dev-20261020-ab580b2e to lmsysorg/sglang:nightly-dev-20261026-8252e3af.
Baseline: 2026-10-22 · lmsysorg/sglang:nightly-dev-20261020-ab580b2e@sha256:0af411bb320a95bb5e1c58b4c20c9852920a19aa9f12f148f69731b0dca3b576
AgentX · TP8/EP1 · Mean latency · Sources: API 1, API 2, API 3, API 4

Concurrency Total tok/s/GPU ↑ Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
1 2,433.44 17.84 611.6 3.51
4 3,737.18 26.21 422.47 4.12
8 6,566.69 48.42 511.95 5.13
12 7,674.6 60.12 934.38 7.13
16 10,166.66 72.68 1,301.32 10.43
17 10,945.66 78.82 1,440.64 10.94
18 12,172.31 85.75 1,448.3 12.69
24 13,884.74 97.23 1,818.06 16.46
Eval Score ↑ Samples
gsm8k/em_strict · c24 96.66% 1,319

Stop: Close this PR at any time to stop this candidate now; Klaud cancels its runs and deletes its branch.

中文

**目标：**将 SGLang 镜像从 lmsysorg/sglang:nightly-dev-20261020-ab580b2e 更新至 lmsysorg/sglang:nightly-dev-20261026-8252e3af。
**基线：**2026-10-22 · lmsysorg/sglang:nightly-dev-20261020-ab580b2e@sha256:0af411bb320a95bb5e1c58b4c20c9852920a19aa9f12f148f69731b0dca3b576
AgentX · TP8/EP1 · 平均延迟 · 来源： API 1, API 2, API 3, API 4；数值及异常说明见上表。
**停止：**随时关闭此 PR 即可立即停止该候选：Klaud 会取消其运行并删除其分支。

@github-actions @claude
chore(glm5.2): update GLM-5.2 FP8 B200 SGLang AgentX MTP image to nig…
26d6e5c 
…htly-dev-20261026-8252e3af

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 26, 2026

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献！

  • **审阅：**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER，请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，再通过 Slack 联系核心维护者。必须严格遵循模板，并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have，才能触发签核验证。
  • **PR 验证：**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目，并且恰好带有一个主标签时，才会运行扫描：full-sweep-fail-fast（强烈推荐；canary 加逐矩阵 fail-fast）、full-sweep-enabled（有 canary；矩阵任务在失败后继续运行）或 non-canary-full-sweep-enabled（无 canary，也无 fail-fast）。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
  • **合并后：**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复；参见重新运行失败任务的说明。

@Klaud-Cold

Klaud-Cold commented Oct 26, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator Author

Initial attempt · Failed · Run 40359902668 / attempt 1 · 2026-10-26 16:57 UTC
lmsysorg/sglang:nightly-dev-20261026-8252e3af@sha256:b4e4abb9e1dc038f7e14c85805be97e4f74abb46e541bd4edd31401d50a2e593 · 9f50076143ac · AgentX · TP8/EP1 · Mean latency
Change: Pin lmsysorg/sglang:nightly-dev-20261026-8252e3af by digest; recipe flags unchanged; measured ref 26d6e5c. Source diff ab580b2e..8252e3af (sgl-project/sglang@ab580b2...8252e3a, 494 commits) keeps every recipe flag; MLA prefill graph max_bs default rises from 2048 to min(8192, chunked-prefill-size) and NCCL graph buffer registration is now off for multi-rank capture, with unmeasured performance impact.

Concurrency Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
1 N/A N/A N/A
16 N/A N/A N/A

Note: c1: cancelled.
Note: All rows: request errors unavailable.
Note: All rows: Δ N/A: no matched baseline.
Note: c16: failed.

Eval Score ↑ Samples
gsm8k/em_strict · c24 91.43% (-5.23 pp) 1,319 each

Next: c16 (TP8+DCP8, mem-fraction-static 0.88) hit CUDA OOM in the first warmup prefill inside flashinfer decode_cp_a2a_alltoall (1 GiB alloc, 0.6 GiB free); c1 was fail-fast cancelled. c24 at 0.83 on the same DCP8 shape passed gsm8k at 91.43% vs 96.66% baseline. Repair 1 lowers c16 mem-fraction-static to 0.83, matching c17/c18/c24.

中文

初次尝试 · 失败 · Run 40359902668 / attempt 1 · 2026-10-26 16:57 UTC
lmsysorg/sglang:nightly-dev-20261026-8252e3af@sha256:b4e4abb9e1dc038f7e14c85805be97e4f74abb46e541bd4edd31401d50a2e593 · 9f50076143ac · AgentX · TP8/EP1 · 平均延迟
**变更：**按 digest 固定 lmsysorg/sglang:nightly-dev-20261026-8252e3af；recipe 参数不变；测量 ref 为 26d6e5c。源码对比 ab580b2e..8252e3af（sgl-project/sglang@ab580b2...8252e3af，494 个提交）保留所有 recipe 参数；MLA prefill 图 max_bs 默认值由 2048 提高到 min(8192, chunked-prefill-size)，多 rank 捕获时关闭 NCCL 图缓冲注册，性能影响尚未测量。实测数值及异常说明见上表。
**下一步：**c16（TP8+DCP8，mem-fraction-static 0.88）在首个 warmup prefill 中于 flashinfer decode_cp_a2a_alltoall 发生 CUDA OOM（申请 1 GiB，仅剩 0.6 GiB）；c1 因 fail-fast 被取消。相同 DCP8 形态、0.83 的 c24 gsm8k 通过，得分 91.43%（基线 96.66%）。修复 1 将 c16 的 mem-fraction-static 降至 0.83，与 c17/c18/c24 一致。

@github-actions @claude
fix(glm5.2): lower GLM-5.2 FP8 B200 c16 DCP8 mem-fraction-static to 0.83
ee959a5 
The nightly-dev-20261026-8252e3af image OOMs in the DCP all-to-all
during the first c16 prefill at 0.88.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Klaud-Cold

Klaud-Cold commented Oct 26, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator Author

Repair 1/5 · Passed · Run 40364160017 / attempt 1 · 2026-10-26 18:36 UTC
lmsysorg/sglang:nightly-dev-20261026-8252e3af@sha256:b4e4abb9e1dc038f7e14c85805be97e4f74abb46e541bd4edd31401d50a2e593 · 9f50076143ac · AgentX · TP8/EP1 · Mean latency
Change: Lower c16 (TP8+DCP8) mem-fraction-static from 0.88 to 0.83, matching c17/c18/c24, after the updated image OOMed in flashinfer decode_cp_a2a_alltoall; measured ref ee959a5.

Concurrency Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
1 17.99 (N/A) 646.08 (N/A) 3.45 (N/A)
16 74.32 (N/A) 1,247.62 (N/A) 9.55 (N/A)

Note: All rows: Δ N/A: no matched baseline.
Note: c16: 3 request errors.

Eval Score ↑ Samples
gsm8k/em_strict · c24 92.19% (-4.47 pp) 1,319 each

Next: Smoke c1 and c16 and the c24 gsm8k eval passed (c16 had 3 of 2403 InvalidInferenceResultError drops; gsm8k 92.19% vs 96.66%), so append the perf-changelog entry and run the final full sweep.

中文

修复 1/5 · 已通过 · Run 40364160017 / attempt 1 · 2026-10-26 18:36 UTC
lmsysorg/sglang:nightly-dev-20261026-8252e3af@sha256:b4e4abb9e1dc038f7e14c85805be97e4f74abb46e541bd4edd31401d50a2e593 · 9f50076143ac · AgentX · TP8/EP1 · 平均延迟
**变更：**更新镜像在 flashinfer decode_cp_a2a_alltoall 中 OOM 后，将 c16（TP8+DCP8）的 mem-fraction-static 从 0.88 降至 0.83，与 c17/c18/c24 一致；测量 ref 为 ee959a5。实测数值及异常说明见上表。
**下一步：**smoke c1、c16 与 c24 gsm8k 评测均通过（c16 在 2403 个请求中有 3 个 InvalidInferenceResultError 被丢弃；gsm8k 92.19%，基线 96.66%），因此追加 perf-changelog 条目并运行最终完整扫描。

@Klaud-Cold

Klaud-Cold commented Oct 26, 2026

Copy link
Copy Markdown
Collaborator Author

parked · Session 1 is waiting on 40364160017. Recovery reruns preempted runs and starts the next session once they end.

中文

parked · 第 1 个会话正在等待 40364160017。恢复流程会重跑被抢占的运行，并在运行结束后启动下一个会话。

@github-actions github-actions Bot added the klaud-parked Klaud parked until its queued or running benchmark runs end label Oct 26, 2026
@Klaud-Cold

Klaud-Cold commented Oct 26, 2026

Copy link
Copy Markdown
Collaborator Author

resumed · The session ended before finishing; session 2 continues it in a new run of its planner.

中文

resumed · 会话未完成即已结束；第 2 个会话将在其规划运行的新运行中继续。

@github-actions @claude
chore(glm5.2): add perf-changelog entry for GLM-5.2 FP8 B200 SGLang A…
717333f 
…gentX MTP image update

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Klaud-Cold Klaud-Cold added the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Oct 26, 2026
@Klaud-Cold

Klaud-Cold commented Oct 26, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator Author

Final full sweep · Passed · Run 40379455332 / attempt 1 · 2026-10-26 21:38 UTC
lmsysorg/sglang:nightly-dev-20261026-8252e3af@sha256:b4e4abb9e1dc038f7e14c85805be97e4f74abb46e541bd4edd31401d50a2e593 · 717333f62ad4 · AgentX · TP8/EP1 · Mean latency
Change: Validate the complete updated-image family at this head.

Concurrency Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
1 17.86 (+0.1%) 616.66 (+0.8%) 3.47 (-1.1%)
4 26.48 (+1%) 421.47 (-0.2%) 3.9 (-5.3%)
8 48.39 (-0%) 462.27 (-9.7%) 5.07 (-1.2%)
12 61.07 (+1.6%) 989.29 (+5.9%) 7.29 (+2.2%)
16 74.22 (+2.1%) 1,251.81 (-3.8%) 9.21 (-11.7%)
17 80.93 (+2.7%) 1,452.34 (+0.8%) 10.17 (-7%)
18 87.11 (+1.6%) 1,433.87 (-1%) 11.59 (-8.7%)
24 99.04 (+1.9%) 1,868.57 (+2.8%) 14.91 (-9.4%)

Note: c4, c16: 3 request errors.
Note: c8, c24: 1 request error.
Note: c12, c17: 2 request errors.
Note: c18: 6 request errors.

Eval Score ↑ Samples
gsm8k/em_strict · c24 92.34% (-4.32 pp) 1,319 each

Next: Mark ready for maintainer review.

中文

最终完整 sweep · 已通过 · Run 40379455332 / attempt 1 · 2026-10-26 21:38 UTC
lmsysorg/sglang:nightly-dev-20261026-8252e3af@sha256:b4e4abb9e1dc038f7e14c85805be97e4f74abb46e541bd4edd31401d50a2e593 · 717333f62ad4 · AgentX · TP8/EP1 · 平均延迟
**变更：**验证此提交更新镜像后的完整配置族。实测数值及异常说明见上表。
**下一步：**标记为就绪，等待维护者审查。

@Klaud-Cold

Klaud-Cold commented Oct 26, 2026

Copy link
Copy Markdown
Collaborator Author

parked · Session 2 is waiting on 40379455332. Recovery reruns preempted runs and starts the next session once they end.

中文

parked · 第 2 个会话正在等待 40379455332。恢复流程会重跑被抢占的运行，并在运行结束后启动下一个会话。

@github-actions

github-actions Bot commented Oct 26, 2026

Copy link
Copy Markdown
Contributor

View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=40379455332

View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=40379455332

@Klaud-Cold
Klaud-Cold marked this pull request as ready for review October 26, 2026 21:38
@Klaud-Cold
Klaud-Cold requested a review from a team October 26, 2026 21:38
@Klaud-Cold

Klaud-Cold commented Oct 26, 2026

Copy link
Copy Markdown
Collaborator Author

validated · Repairs: unknown · Runs: 40359902668, 40364160017, 40379365450, 40379455332
All owned runs ended. Full sweep verified; ready for review.

中文

validated · 修复次数：未知 · 运行：40359902668, 40364160017, 40379365450, 40379455332
所有自有运行均已结束。完整 sweep 已验证；已就绪，等待审查。

@github-actions github-actions Bot added klaud-finished Klaud verified the full sweep; ready for review and removed klaud-parked Klaud parked until its queued or running benchmark runs end labels Oct 26, 2026
claude[bot]
claude Bot reviewed Oct 26, 2026
View reviewed changes

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review found no issues

No high-confidence issues detected in this change.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@claude claude[bot] claude[bot] left review comments
@Ankur-singh Ankur-singh Awaiting requested review from Ankur-singh Ankur-singh is a code owner
@kedarpotdar-nv kedarpotdar-nv Awaiting requested review from kedarpotdar-nv kedarpotdar-nv is a code owner
@wzhao18 wzhao18 Awaiting requested review from wzhao18 wzhao18 is a code owner
@nvpohanh nvpohanh Awaiting requested review from nvpohanh nvpohanh is a code owner
@RohitNagraj RohitNagraj Awaiting requested review from RohitNagraj RohitNagraj is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) klaud-finished Klaud verified the full sweep; ready for review

Projects

InferenceX Board
Status: No status

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@Klaud-Cold