[shyeh25]

Summary

Moves the GB300 Muse Spark Open Dynamo + SGLang AgentX lane to lmsysorg/sglang:nightly-dev-cu13-20261103-797af717 (multiarch digest pinned) and runs the ai-dynamo 1.7.0.dev20261102 that the image ships (dynamo.install: false). It replaces the aggregated points under a new -agg key, adds TP8 disaggregated points under a new -disagg key, and removes musespark-fp4-gb300-dynamo-sglang-agentic-dflash (#4344).

Config key	Topology	Conc	Nodes	Key settings
musespark-fp4-gb300-dynamo-sglang-agentic-dflash-agg (new)	TP4/EP4	46, 50, 56	1	HiCache 140 GB per rank (write_back, 0.80 DRAM utilization), fcfs, 32 running requests, prefill-decode interval 24 / 20 / 8, chunked prefill 16384, breakable prefill CUDA graph, draft attention trtllm_mha
	TP8/EP8	36	2	GPU-resident KV, 36 running requests, shortest-prefill-first, prefill-decode interval 128, draft attention fa4
	TP8/EP8	1	2	GPU-resident KV, 1 running request, prefill-decode interval 64, draft attention trtllm_mha
musespark-fp4-gb300-dynamo-sglang-agentic-dflash-disagg (new)	3x TP8 prefill + 4x TP8 decode	672	14	56 requests per decode worker
	2x TP8 prefill + 3x TP8 decode	352	10	40 requests per decode worker
	1x TP8 prefill + 2x TP8 decode	160, 224	6	24 / 36 requests per decode worker
	1x TP8 prefill + 3x TP8 decode	224	8	20 requests per decode worker
	1x TP8 prefill + 4x TP8 decode	32, 56, 112	10	3 / 3 / 6 requests per decode worker
	1x TP8 prefill + 5x TP8 decode	120	12	5 requests per decode worker

• Disaggregated points share one setup: Mooncake KV transfer; prefill HiCache 150 GB per rank (write_back, 0.80 DRAM utilization), chunked prefill 32768, shortest-prefill-first; decode without layernorm SP. The frontend runs on the head node without KV events; the 3P4D and 2P3D points use soft session affinity across prefill workers.
• Full-attention decode in trtllm_mha splits the KV into 16 chunks (SGLANG_TRTLLM_MHA_FULL_ATTN_KV_SPLITS=16) when the per-rank decode batch is at least 4 (aggregated) or 12 (disaggregated decode) (SGLANG_TRTLLM_MHA_FULL_ATTN_KV_SPLITS_MIN_BS).
• SGLANG_FLASHINFER_NVFP4_PER_TOKEN_ACTIVATION=0 on every role.
• Removes the nine config(musespark): add GB300 Dynamo SGLang AgentX aggregated points / config(musespark): 新增 GB300 Dynamo SGLang AgentX 聚合配置点 #4344 points (TP4 c8, c16, c20, c32, c40; TP8 c3, c4, c10, c14). Each removed point is covered by the new curve on throughput per GPU and mean interactivity: seven are exceeded on both axes by a new point, and TP8 c14 and c10 lie below the line between 1P5D c120 and 1P4D c56.

AI model disclosure

• Model/version: Claude Opus 5.5
• Role: tuning, config changes

PR checklist and change type

Type of Change

• Bug fix
• New feature
• Configuration change
• Documentation update
• Other (please describe)

Checklist

• I have completed the AI model disclosure and kept it current
• I have tested my changes locally
• I have updated documentation if necessary
• If python -m infx.matrix.changed --base origin/main lists affected config keys, I have appended one new entry covering them to the physical end of inferencex-e2e/perf-changelog.yaml; if it lists none, I have added none. I have not edited historical entries
• If this PR appends an inferencex-e2e/perf-changelog.yaml entry, it carries exactly one primary sweep label (a maintainer applies it on fork PRs): full-sweep-fail-fast (recommended), full-sweep-enabled, or non-canary-full-sweep-enabled. Optional modifiers all-evals, evals-only, and agentx-fast require a primary label; the last two block reuse while applied.
• Before merging via reuse, an authorized maintainer (OWNER/MEMBER/COLLABORATOR) has commented /use <run_id> (or the legacy /reuse-sweep-run) on this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the primary sweep label will no longer automatically kick off new sweeps. Remove and re-add the primary sweep label to force a new sweep.

中文

将 GB300 Muse Spark Open Dynamo + SGLang AgentX 配置迁移到 lmsysorg/sglang:nightly-dev-cu13-20261103-797af717（固定 multiarch digest），并直接使用镜像自带的 ai-dynamo 1.7.0.dev20261102（dynamo.install: false）。本 PR 在新的 -agg config key 下替换聚合配置点，在新的 -disagg config key 下新增 TP8 分离式配置点，并移除 musespark-fp4-gb300-dynamo-sglang-agentic-dflash（#4344）。

• musespark-fp4-gb300-dynamo-sglang-agentic-dflash-agg（新增）：
  • TP4/EP4 c46、c50、c56（1 个节点）：每 rank HiCache 140 GB（write_back，DRAM 利用率 0.80），fcfs 调度，32 个运行请求，prefill-decode interval 分别为 24 / 20 / 8，chunked prefill 16384，prefill CUDA graph 使用 breakable，draft attention 使用 trtllm_mha。
  • TP8/EP8 c36（2 个节点）：KV 留在 GPU，36 个运行请求，shortest-prefill-first 调度，prefill-decode interval 128，draft attention 使用 fa4。
  • TP8/EP8 c1（2 个节点）：KV 留在 GPU，1 个运行请求，prefill-decode interval 64，draft attention 使用 trtllm_mha。
• musespark-fp4-gb300-dynamo-sglang-agentic-dflash-disagg（新增），prefill 与 decode worker 均为 TP8：
  • 3P4D c672（14 个节点），每个 decode worker 56 个请求。
  • 2P3D c352（10 个节点），每个 decode worker 40 个请求。
  • 1P2D c160、c224（6 个节点），每个 decode worker 24 / 36 个请求。
  • 1P3D c224（8 个节点），每个 decode worker 20 个请求。
  • 1P4D c32、c56、c112（10 个节点），每个 decode worker 3 / 3 / 6 个请求。
  • 1P5D c120（12 个节点），每个 decode worker 5 个请求。
• 分离式配置点共用设置：Mooncake KV 传输；prefill 每 rank HiCache 150 GB（write_back，DRAM 利用率 0.80），chunked prefill 32768，shortest-prefill-first 调度；decode 关闭 layernorm SP。前端运行在 head 节点，不使用 KV event；3P4D 与 2P3D 在多个 prefill worker 之间使用 soft session affinity。
• 全注意力层在 trtllm_mha decode 时把 KV 切成 16 段（SGLANG_TRTLLM_MHA_FULL_ATTN_KV_SPLITS=16），仅在每 rank decode batch 至少为 4（聚合）或 12（分离式 decode）时启用（SGLANG_TRTLLM_MHA_FULL_ATTN_KV_SPLITS_MIN_BS）。
• 所有角色均设置 SGLANG_FLASHINFER_NVFP4_PER_TOKEN_ACTIVATION=0。
• 移除 config(musespark): add GB300 Dynamo SGLang AgentX aggregated points / config(musespark): 新增 GB300 Dynamo SGLang AgentX 聚合配置点 #4344 的九个配置点（TP4 c8、c16、c20、c32、c40；TP8 c3、c4、c10、c14）。每个移除的点在每 GPU 吞吐与平均交互性上都被新曲线覆盖：其中七个点在两个轴上都被新点超过，TP8 c14 与 c10 位于 1P5D c120 与 1P4D c56 连线下方。

AI 模型使用说明：Claude Opus 5.5；负责调优与配置修改。

🤖 Generated with Claude Code

[github-actions]

Thanks for the contribution!

• Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
• PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
• After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.

中文

感谢你的贡献！

• **审阅：**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER，请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，再通过 Slack 联系核心维护者。必须严格遵循模板，并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have，才能触发签核验证。
• **PR 验证：**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目，并且恰好带有一个主标签时，才会运行扫描：full-sweep-fail-fast（强烈推荐；canary 加逐矩阵 fail-fast）、full-sweep-enabled（有 canary；矩阵任务在失败后继续运行）或 non-canary-full-sweep-enabled（无 canary，也无 fail-fast）。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
• **合并后：**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复；参见重新运行失败任务的说明。

[claude]

🔴 Muse Spark agentic runs could get turns rejected with HTTP 529 once a worker fills up. The base branch's Dynamo 1.6 did not have this backend admission check. The image's Dynamo 1.7 admits only a limited number of requests per engine, and almost every new point runs above max-running-requests. Fix: set DYN_BACKEND_ADMISSION_ENGINE_REQUEST_LIMIT, DYN_BACKEND_ADMISSION_ENGINE_REQUEST_WAITING_LIMIT and DYN_BACKEND_ADMISSION_REQUEST_QUEUE_LIMIT to "10000" on every engine role, which covers the 3 sites listed. Same pattern at 3 sites (agg-variants.yaml:54, disagg-variants.yaml:55, disagg-variants.yaml:113).

Why this was flagged

The trigger is load above what one engine has room for, under the Dynamo 1.7.0.dev20261102 that the image ships. The base branch installed the 1.6.0.dev20261016 wheel, but dynamo.install: false at agg-variants.yaml:9 and disagg-variants.yaml:9 now uses the image's 1.7 build. Every other dynamo-sglang 1.7 key in the repo raises these three limits to "10000", in dsv41flash agg-variants.yaml:69-73 and minimaxm31flash sglang agg-variants.yaml:63-65. Their comment says "Dynamo 1.7 backend admission rejects turns with 529 once the engine is full". The new points routinely run beyond that capacity. TP4 c46/c50/c56 cap max-running-requests at 32. Disagg decode capacity is 4x56=224 at c672. Rejected turns would fail the AgentX run or skew its throughput and interactivity results compared with the 1.6 runs on the base branch. Neither recipe sets any DYN_BACKEND_ADMISSION_* variable.

Verification: The diff changes the base's wheel: "1.6.0.dev20261016" to dynamo.install: false (agg-variants.yaml:9; also disagg-variants.yaml:9). Neither musespark recipe sets DYN_BACKEND_ADMISSION_*. Agg c46/c50/c56 run against max-running-requests: 32 on a single TP4 engine.

[shyeh25]

Not applicable to this image. Dynamo 1.7.0.dev20261102 includes ai-dynamo/dynamo#17481 (merged 2026-10-31), which made backend admission opt-in: with no engine limit set, the gate is bypassed (no queueing, no 529). Every run of these recipes logs Backend admission configuration engine_request_limit=unlimited engine_request_waiting_limit=unlimited request_queue_limit=0, and no 529s occurred. The dsv41flash / minimaxm31flash recipes set 10000 because they targeted earlier 1.7 builds, where admission was on by default. Keeping the recipes as measured.