Skip to content

[Klaud Cold] Update Qwen3.5 FP8 B300 SGLang AgentX MTP image to v0.5.22-cu130 / [Klaud Cold] 将 Qwen3.5 FP8 B300 SGLang AgentX MTP 镜像更新至 v0.5.22-cu130 - #4482

Open
Klaud-Cold wants to merge 22 commits into
mainfrom
klaud/auto-c840d71da0ca7bde-555679de269db36f
Open

Klaud-Cold wants to merge 22 commits into
mainfrom
klaud/auto-c840d71da0ca7bde-555679de269db36f

Conversation

@Klaud-Cold

@Klaud-Cold Klaud-Cold commented Oct 25, 2026 •

Copy link
Copy Markdown
Collaborator

Goal: Update SGLang image from lmsysorg/sglang:v0.5.16-cu130 to lmsysorg/sglang:v0.5.22-cu130.
Baseline: 2026-08-07 · lmsysorg/sglang:v0.5.16-cu130
Mean latency · Sources: API 1, API 2, API 3, API 4

Point Total tok/s/GPU ↑ Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
AgentX c1 TP2 EP2 1b1efd 6,521.61 67.04 1,706.35 8.06
AgentX c1 TP4 EP1 365af6 3,544.83 37.76 1,371 6.39
AgentX c4 TP2 EP2 39e3f3 11,543.71 124.88 856.54 8.39
AgentX c4 TP4 EP1 8f5459 6,446.35 71.71 413.24 3.4
AgentX c8 TP2 EP2 09a92c 19,447.07 181.12 1,276.18 6.63
AgentX c8 TP4 EP1 814aa4 10,593.97 104.98 442.79 3.02
AgentX c12 TP2 EP2 64905a 24,761.95 237.82 2,048.25 11.69
AgentX c12 TP4 EP1 3ecb3c 18,270.73 161.17 519.64 3.64
AgentX c16 TP2 EP2 9f6509 13,020.66 117.54 17,826.21 50.45
AgentX c16 TP4 EP1 d139ae 23,354.98 202.14 585.19 4.51
AgentX c20 TP2 EP2 f4b7f4 10,968 101.38 36,228.99 65.67
AgentX c20 TP4 EP1 ca9ded 28,278.84 235.78 652.12 4.57
AgentX c24 TP2 EP2 024cc3 11,360.03 104.77 54,466.35 62.27
AgentX c24 TP4 EP1 aef8ff 33,652.89 278.99 671.34 5.64
AgentX c28 TP2 EP2 fb8b81 11,202.22 98.51 79,270.73 54.79
AgentX c28 TP4 EP1 966a4d 36,430.28 317.6 705.82 6.63
AgentX c32 TP2 EP2 92786e 55,593.45 460.74 10,861.89 11.64
AgentX c32 TP2 EP2 b5ae14 11,688.47 97.97 101,166.92 55.32
AgentX c32 TP4 EP1 eb1cfc 38,164.6 337.49 747.67 7.34
AgentX c34 TP2 EP2 324e27 54,985.67 467.86 12,857.81 12.19
AgentX c36 TP2 EP2 fabb7b 54,272.25 460.86 15,644.09 11.79
AgentX c38 TP2 EP2 67d2b2 53,103.36 484.58 18,908.69 11.48
AgentX c40 TP2 EP2 534e71 52,828.67 484.74 20,878.87 11.14
AgentX c40 TP4 EP1 855ad5 43,541.78 414.76 805.71 8.75
AgentX c44 TP2 EP2 3e4349 51,691.36 469.74 24,197.05 11.61
AgentX c48 TP2 EP2 e136fc 51,404.79 471.39 29,628.26 11.93
AgentX c48 TP4 EP1 12039d 45,376.32 453.76 1,042.83 12.98
AgentX c52 TP2 EP2 980634 49,621.92 479.96 36,385.37 12.74
AgentX c56 TP2 EP2 05008f 49,618.19 477.83 41,409.75 12.54
AgentX c56 TP4 EP1 73dee6 46,211.39 475.98 1,271.15 18.35
AgentX c60 TP4 EP1 d743be 45,804.97 479.18 1,447.1 22.09
AgentX c64 TP4 EP1 aa6d8a 43,570.02 457.23 1,578.06 28.69
AgentX c68 TP4 EP1 e800a7 40,130.06 428.86 1,763.95 39.59
AgentX c72 TP4 EP1 ed5057 36,861.16 402.37 2,233.66 45.26
Eval Score ↑ Samples
gsm8k/em_strict · c72 97.8% 1,319

Stop: Close this PR at any time to stop this candidate now; Klaud cancels its runs and deletes its branch.

中文

**目标:**将 SGLang 镜像从 lmsysorg/sglang:v0.5.16-cu130 更新至 lmsysorg/sglang:v0.5.22-cu130。
**基线:**2026-08-07 · lmsysorg/sglang:v0.5.16-cu130
平均延迟 · 来源: API 1, API 2, API 3, API 4;数值及异常说明见上表。
**停止:**随时关闭此 PR 即可立即停止该候选:Klaud 会取消其运行并删除其分支。

Pin lmsysorg/sglang:v0.5.22-cu130 by digest and rename the removed
--cuda-graph-max-bs alias to --cuda-graph-max-bs-decode.

将 Qwen3.5 FP8 B300 SGLang AgentX MTP 镜像更新至 v0.5.22-cu130(按 digest 固定),
并将已移除的 --cuda-graph-max-bs 别名改为 --cuda-graph-max-bs-decode。

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献!

  • **审阅:**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER,请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,再通过 Slack 联系核心维护者。必须严格遵循模板,并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have,才能触发签核验证。
  • **PR 验证:**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目,并且恰好带有一个主标签时,才会运行扫描:full-sweep-fail-fast(强烈推荐;canary 加逐矩阵 fail-fast)、full-sweep-enabled(有 canary;矩阵任务在失败后继续运行)或 non-canary-full-sweep-enabled(无 canary,也无 fail-fast)。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
  • **合并后:**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复;参见重新运行失败任务的说明。

@Klaud-Cold

Klaud-Cold commented Oct 25, 2026 •

Copy link
Copy Markdown
Collaborator Author

Initial attempt · Passed · Run 40159476208 / attempt 1 · 2026-10-25 08:19 UTC
lmsysorg/sglang:v0.5.22-cu130@sha256:e3a4f06001a29542fc4a8638536d9ffc5eb9c9942b08603c09b19a56eae4a1ae · cfb7bdd0c8d3 · Mean latency
Change: Measured ref ce857a3: Pin lmsysorg/sglang:v0.5.22-cu130 by digest and rename --cuda-graph-max-bs to --cuda-graph-max-bs-decode, because v0.5.22 (baa9befa) no longer registers the deprecated alias that v0.5.16 (fdebc938) mapped to cuda_graph_max_bs_decode; all other recipe flags, choices and env vars exist with unchanged semantics in sglang/srt/arg_groups/fields and environ.py.

Point Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
AgentX c1 TP4 EP1 365af6 28.51 (-24.5%) 2,577.49 (+88%) 13.83 (+116.4%)
AgentX c1 TP2 EP2 1b1efd 50.04 (-25.4%) 3,087.63 (+80.9%) 16.13 (+100.1%)
AgentX c32 TP2 EP2 92786e 142.48 (-69.1%) 72,515.64 (+567.6%) 37.2 (+219.6%)
Eval Score ↑ Samples
gsm8k/em_strict · c72 97.65% (-0.15 pp) 1,319 each

Next: Smoke and gsm8k passed with zero request errors; append the perf-changelog entry and run the full-sweep-fail-fast final sweep.

中文

初次尝试 · 已通过 · Run 40159476208 / attempt 1 · 2026-10-25 08:19 UTC
lmsysorg/sglang:v0.5.22-cu130@sha256:e3a4f06001a29542fc4a8638536d9ffc5eb9c9942b08603c09b19a56eae4a1ae · cfb7bdd0c8d3 · 平均延迟
**变更:**测量提交 ce857a3:按 digest 固定 lmsysorg/sglang:v0.5.22-cu130,并将 --cuda-graph-max-bs 改为 --cuda-graph-max-bs-decode:v0.5.22(baa9befa)不再注册 v0.5.16(fdebc938)中映射到 cuda_graph_max_bs_decode 的弃用别名;其余配方参数、取值和环境变量在 sglang/srt/arg_groups/fields 与 environ.py 中均存在且语义不变。实测数值及异常说明见上表。
**下一步:**冒烟与 gsm8k 均通过且无请求错误;追加 perf-changelog 条目并运行 full-sweep-fail-fast 最终扫描。

@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

parked · Session 1 is waiting on 40159476208. Recovery reruns preempted runs and starts the next session once they end.

中文

parked · 第 1 个会话正在等待 40159476208。恢复流程会重跑被抢占的运行,并在运行结束后启动下一个会话。

@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

resumed · The session ended before finishing; session 2 continues it in a new run of its planner.

中文

resumed · 会话未完成即已结束;第 2 个会话将在其规划运行的新运行中继续。

@Klaud-Cold Klaud-Cold added the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Oct 25, 2026
@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

Final full sweep · Running · Run 40176215702 / attempt 1 · 2026-10-25 08:21 UTC
lmsysorg/sglang:v0.5.22-cu130@sha256:e3a4f06001a29542fc4a8638536d9ffc5eb9c9942b08603c09b19a56eae4a1ae · 58a28d09b5f5
Change: Merge main and append the perf-changelog entry; recipe and image are unchanged from the measured ref ce857a3.

Next: Wait for the full-sweep-fail-fast final sweep covering all 34 baseline AgentX points and the c72 gsm8k eval.

中文

最终完整 sweep · 运行中 · Run 40176215702 / attempt 1 · 2026-10-25 08:21 UTC
lmsysorg/sglang:v0.5.22-cu130@sha256:e3a4f06001a29542fc4a8638536d9ffc5eb9c9942b08603c09b19a56eae4a1ae · 58a28d09b5f5
**变更:**合并 main 并追加 perf-changelog 条目;配方与镜像与测量提交 ce857a3 相同。
**下一步:**等待覆盖全部 34 个基线 AgentX 点与 c72 gsm8k 评测的 full-sweep-fail-fast 最终扫描完成。

@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

parked · Session 2 is waiting on 40176215702. Recovery reruns preempted runs and starts the next session once they end.

中文

parked · 第 2 个会话正在等待 40176215702。恢复流程会重跑被抢占的运行,并在运行结束后启动下一个会话。

@github-actions

Copy link
Copy Markdown
Contributor

@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

resumed · The session ended before finishing; session 3 continues it in a new run of its planner.

中文

resumed · 会话未完成即已结束;第 3 个会话将在其规划运行的新运行中继续。

@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

Final full sweep · Failed · Run 40176215702 / attempt 2 · 2026-10-25 20:15 UTC
lmsysorg/sglang:v0.5.22-cu130@sha256:e3a4f06001a29542fc4a8638536d9ffc5eb9c9942b08603c09b19a56eae4a1ae · 58a28d09b5f5
Change: Recovery rerun of the preempted final sweep at the same head; recipe and image are unchanged from the measured ref ce857a3.

Next: Rerun the failed and cancelled jobs on the same head as infrastructure retry 1, reusing the 26 passed benchmarks and the gsm8k eval.

中文

最终完整 sweep · 失败 · Run 40176215702 / attempt 2 · 2026-10-25 20:15 UTC
lmsysorg/sglang:v0.5.22-cu130@sha256:e3a4f06001a29542fc4a8638536d9ffc5eb9c9942b08603c09b19a56eae4a1ae · 58a28d09b5f5
**变更:**在同一提交上由恢复流程重跑被抢占的最终扫描;配方与镜像与测量提交 ce857a3 相同。
**下一步:**在同一提交上以第 1 次基础设施重试重跑失败与被取消的任务,复用已通过的 26 个基准与 gsm8k 评测。

@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

Infrastructure retry 1/2 · Queued · Run 40176215702 / attempt 3 · 2026-10-25 20:15 UTC
lmsysorg/sglang:v0.5.22-cu130@sha256:e3a4f06001a29542fc4a8638536d9ffc5eb9c9942b08603c09b19a56eae4a1ae · 58a28d09b5f5
Change: Rerun the failed TP2 HiCache c36 job and the 7 fail-fast-cancelled jobs on the same head with no recipe change.

Next: Wait for the 8 rerun jobs; a passing rerun completes all 34 baseline AgentX points and the gsm8k eval for finish.

中文

基础设施重试 1/2 · 排队中 · Run 40176215702 / attempt 3 · 2026-10-25 20:15 UTC
lmsysorg/sglang:v0.5.22-cu130@sha256:e3a4f06001a29542fc4a8638536d9ffc5eb9c9942b08603c09b19a56eae4a1ae · 58a28d09b5f5
**变更:**在同一提交上重跑失败的 TP2 HiCache c36 任务及 7 个被 fail-fast 取消的任务,配方不变。
**下一步:**等待 8 个重跑任务;重跑通过后即覆盖全部 34 个基线 AgentX 点与 gsm8k 评测,随后执行 finish。

@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

parked · Session 3 is waiting on 40176215702. Recovery reruns preempted runs and starts the next session once they end.

中文

parked · 第 3 个会话正在等待 40176215702。恢复流程会重跑被抢占的运行,并在运行结束后启动下一个会话。

@github-actions github-actions Bot added the klaud-parked Klaud parked until its queued or running benchmark runs end label Oct 26, 2026
@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

Mean latency

Point Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
AgentX c1 TP2 EP2 1b1efd 50.14 (-25.2%) 3,072.72 (+80.1%) 15.95 (+97.9%)
AgentX c1 TP4 EP1 365af6 30.06 (-20.4%) 2,516.64 (+83.6%) 13.8 (+116%)
AgentX c4 TP2 EP2 39e3f3 114.05 (-8.7%) 854.56 (-0.2%) 11.18 (+33.3%)
AgentX c4 TP4 EP1 8f5459 68.48 (-4.5%) 534.82 (+29.4%) 7.69 (+126.2%)
AgentX c8 TP2 EP2 09a92c 191.26 (+5.6%) 1,190.13 (-6.7%) 7.38 (+11.3%)
AgentX c8 TP4 EP1 814aa4 106.24 (+1.2%) 378.92 (-14.4%) 2.94 (-2.6%)
AgentX c12 TP2 EP2 64905a 234.27 (-1.5%) 1,987.65 (-3%) 15.61 (+33.5%)
AgentX c12 TP4 EP1 3ecb3c 161.66 (+0.3%) 418.39 (-19.5%) 3.82 (+4.9%)
AgentX c16 TP2 EP2 9f6509 149.94 (+27.6%) 11,681.67 (-34.5%) 47.75 (-5.4%)
AgentX c16 TP4 EP1 d139ae 203.04 (+0.4%) 475.92 (-18.7%) 4.69 (+4%)
AgentX c20 TP2 EP2 f4b7f4 80.84 (-20.3%) 44,015.23 (+21.5%) 99.18 (+51%)
AgentX c20 TP4 EP1 ca9ded 238.91 (+1.3%) 515.44 (-21%) 4.92 (+7.7%)
AgentX c24 TP2 EP2 024cc3 105.63 (+0.8%) 54,089.66 (-0.7%) 57.19 (-8.2%)
AgentX c24 TP4 EP1 aef8ff 281.09 (+0.7%) 551.24 (-17.9%) 5.77 (+2.3%)
AgentX c28 TP2 EP2 fb8b81 99.53 (+1%) 78,402.63 (-1.1%) 57.74 (+5.4%)
AgentX c28 TP4 EP1 966a4d 320.78 (+1%) 590.82 (-16.3%) 7.08 (+6.8%)
AgentX c32 TP2 EP2 92786e 128.6 (-72.1%) 80,031.1 (+636.8%) 40.07 (+244.2%)
AgentX c32 TP2 EP2 b5ae14 98.55 (+0.6%) 101,374.96 (+0.2%) 60.82 (+9.9%)
AgentX c32 TP4 EP1 eb1cfc 214.55 (-36.4%) 1,028.67 (+37.6%) 26.66 (+263.2%)
AgentX c34 TP2 EP2 324e27 144.58 (-69.1%) 75,682.05 (+488.6%) 36.52 (+199.6%)
Eval Score ↑ Samples
gsm8k/em_strict · c72 97.8% (+0.00 pp) 1,319 each

@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

Mean latency

Point Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
AgentX c36 TP2 EP2 fabb7b 107.66 (-76.6%) 109,455.4 (+599.7%) 53.75 (+355.9%)
AgentX c38 TP2 EP2 67d2b2 131.4 (-72.9%) 90,202.3 (+377%) 44.57 (+288.2%)
AgentX c40 TP2 EP2 534e71 146.93 (-69.7%) 92,667.94 (+343.8%) 41.22 (+270%)
AgentX c40 TP4 EP1 855ad5 417.59 (+0.7%) 679.57 (-15.7%) 8.87 (+1.4%)
AgentX c44 TP2 EP2 3e4349 132.34 (-71.8%) 118,118.12 (+388.2%) 55.32 (+376.5%)
AgentX c48 TP2 EP2 e136fc 129.38 (-72.6%) 127,069.41 (+328.9%) 48.61 (+307.5%)
AgentX c48 TP4 EP1 12039d 460.41 (+1.5%) 917.27 (-12%) 13.12 (+1.1%)
AgentX c52 TP2 EP2 980634 144.69 (-69.9%) 140,662.98 (+286.6%) 46.35 (+263.8%)
AgentX c56 TP2 EP2 05008f 134.12 (-71.9%) 162,181.55 (+291.7%) 51.71 (+312.4%)
AgentX c56 TP4 EP1 73dee6 468.95 (-1.5%) 1,098.18 (-13.6%) 20.66 (+12.6%)
AgentX c60 TP4 EP1 d743be 422.01 (-11.9%) 1,326.89 (-8.3%) 32.74 (+48.2%)
AgentX c64 TP4 EP1 aa6d8a 408.41 (-10.7%) 1,482.95 (-6%) 40.16 (+40%)
AgentX c68 TP4 EP1 e800a7 354.37 (-17.4%) 1,693.83 (-4%) 50.96 (+28.7%)
AgentX c72 TP4 EP1 ed5057 304.11 (-24.4%) 2,046.48 (-8.4%) 62 (+37%)

@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

Final full sweep · Passed · Run 40176215702 / attempt 4 · 2026-10-27 14:26 UTC
lmsysorg/sglang:v0.5.22-cu130@sha256:e3a4f06001a29542fc4a8638536d9ffc5eb9c9942b08603c09b19a56eae4a1ae · 58a28d09b5f5
Change: Validate the complete updated-image family at this head.

Next: Mark ready for maintainer review.

Results continue in numbered report comments.

中文

最终完整 sweep · 已通过 · Run 40176215702 / attempt 4 · 2026-10-27 14:26 UTC
lmsysorg/sglang:v0.5.22-cu130@sha256:e3a4f06001a29542fc4a8638536d9ffc5eb9c9942b08603c09b19a56eae4a1ae · 58a28d09b5f5
**变更:**验证此提交更新镜像后的完整配置族。
**下一步:**标记为就绪,等待维护者审查。
结果见后续编号的报告评论。

@Klaud-Cold
Klaud-Cold marked this pull request as ready for review October 27, 2026 14:26
@Klaud-Cold
Klaud-Cold requested a review from a team October 27, 2026 14:26
@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

validated · Repairs: unknown · Runs: 40159476208, 40176195205, 40176215702
All owned runs ended. Full sweep verified; ready for review.

中文

validated · 修复次数:未知 · 运行:40159476208, 40176195205, 40176215702
所有自有运行均已结束。完整 sweep 已验证;已就绪,等待审查。

@github-actions github-actions Bot added klaud-finished Klaud verified the full sweep; ready for review and removed klaud-parked Klaud parked until its queued or running benchmark runs end labels Oct 27, 2026

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed this PR and found no bugs. Both changed config paths have NVIDIA CODEOWNERS, so their sign-off is still needed. One more check for a human: the MTP draft precision under v0.5.22 can only be confirmed by running the image.

What was reviewed:

  • Master config: the image moves to the same digest-pinned v0.5.22-cu130 tag that other entries in nvidia-master.yaml already use.
  • Recipe: all 34 cuda-graph-max-bs keys were renamed to cuda-graph-max-bs-decode with the same values, and no old keys are left. The new key name matches existing SGLang recipes.
  • perf-changelog.yaml: the new entry is only appended at the end, and it covers the affected config key.
Extended reasoning...

The change touches a single NVIDIA master-config entry, the matching single-node B300 FP8 MTP agentic recipe and the perf changelog. It touches nothing security-sensitive. The edits are mechanical: an image bump to a digest already used in that file, a key rename with values unchanged, and an entry appended to the changelog. The deciding factor is ownership: CODEOWNERS assigns nvidia-master.yaml and b300-* srt-slurm recipes to NVIDIA owners. Also, the repo conventions require a reviewer to check the effective MTP draft precision whenever the image is bumped.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) klaud-finished Klaud verified the full sweep; ready for review

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

1 participant