做什么
威震天桥内MOE性能优化的证据化工作流程. 涵盖测量合同,"三壁"框架,平行折叠,剖析,匹配A/B调和,并进行最终验证.
技能库 智客分类:Agent 工作流 nemo-mbridge-perf-moe-optimization-workflow
威震天桥内MOE性能优化的证据化工作流程. 涵盖测量合同,"三壁"框架,平行折叠,剖析,匹配A/B调和,并进行最终验证.
官方网址:skills.sh
先看中文介绍;官方 description 原文单独保留,不改写 SKILL.md。
威震天桥内MOE性能优化的证据化工作流程. 涵盖测量合同,"三壁"框架,平行折叠,剖析,匹配A/B调和,并进行最终验证.
官方 description 未单独写出 Use when。按规范,代理会在用户任务与这段 description 的关键词匹配时激活本技能。
按 Agent Skills 渐进披露:启动时只加载 name 与 description(约 100 token);任务匹配后才读入整份 SKILL.md 正文;scripts/、references/、assets/ 仅在需要时再读。 本文件正文结构:MoE Training Optimization Workflow、Quick Reference、First Answer Checklist、Phase 0: Freeze The Measurement Contract、Phase 1: Make The Run Memory-Feasible、Recompute guidance。 其中含规范建议的小节:分步指令。
文件分析:这是一份仅含 SKILL.md 的指令型技能,代理激活后整份正文进入上下文。
Evidence-gated workflow for MoE performance optimization in Megatron Bridge. Covers measurement contracts, the Three Walls framework, parallel folding, profiling, matched A/B tuning, and final validation.
MoE Training Optimization WorkflowQuick ReferenceFirst Answer ChecklistPhase 0: Freeze The Measurement ContractPhase 1: Make The Run Memory-FeasibleRecompute guidancePhase 2: Choose Parallelism For ScaleParallel FoldingPhase 3: Profile The Dominant BottleneckProfile overlap without misreading itPhase 4: Retune From EvidenceDispatcher And Overlap Guidance
· 许可:Apache-2.0
来源分类:skills.sh agent-skill
namenemo-mbridge-perf-moe-optimization-workflowdescription具体调用语法与可用工具以目标 Agent 客户端为准。 查看调用机制说明 ↗
先选择目标 Agent 和安装范围,保留技能包的附属文件,安装后检查客户端能否发现该技能。
复制安装指令给支持 Agent Skills 的代理,确认其中的目标目录与客户端匹配。
把 Agent Skill「nemo-mbridge-perf-moe-optimization-workflow」安装到我的项目:SKILL.md 原文与官方 description 见 https://zicq.com/zh/skills/skl-781bca8d8753d679-Nemo-Mbridge-Perf-Moe-%E4%BC%98%E5%8C%96%E5%B7%A5%E4%BD%9C%E6%B5%81%E7%A8%8B.html 请存为 .cursor/skills/nemo-mbridge-perf-moe-optimization-workflow/SKILL.md 或 .claude/skills/nemo-mbridge-perf-moe-optimization-workflow/SKILL.md,frontmatter 的 name 与 description 保持原样,不要改写。
需要 Node.js 与 npx。先查看仓库技能列表,确认实际名称。
npx skills add 'https://github.com/nvidia/skills' --list
npx skills add 'https://github.com/nvidia/skills' --skill 'nemo-mbridge-perf-moe-optimization-workflow'
CLI 会交互选择目标 Agent,默认安装到项目;用户级安装使用 -g。先通过查看命令核对仓库内容,再用 npx skills list 检查已安装技能。
Stable docs: @docs/training/moe-optimization.md Card: @skills/nemo-mbridge-perf-moe-optimization-workflow/card.yaml Source: Scalable Training of MoE Models with Megatron Core
Start with the paper's Three Walls:
For operational diagnosis, split the compute-efficiency wall into compute and host/launch bottlenecks. They need different evidence and different fixes. MoE tuning is iterative, so use this order:
freeze the measurement contract -> fit -> scale -> profile -> retune -> validate
For MoE optimization workflow prompts, present the response in this order:
--fake-init-process-group to sanity-check large layouts.Attention: TP x CP x DP x PP
and MoE: ETP x EP x EDP x PP.alltoall for safe bring-up, then A/B flex + deepep and flex +
hybridep when their packages and target topology support them. Start from
BF16 and eager execution; introduce lower precision or the narrowest useful
CUDA-graph scope only after profiling justifies it.A comparison is valid only when the following stay fixed unless they are the single variable under test:
Separate two acceptance classes:
Start with a configuration that fits reliably before chasing throughput.
Recommended order:
--fake-init-process-group to sanity-check large parallel layouts on a
single GPU before burning cluster time.Prefer selective recompute for MoE runs:
layernorm, core_attn, moe_act, mlp, or
model-specific modules (shared_experts, mla_up_proj)As a rule of thumb, fine-grained recompute often recovers most of the needed memory while keeping throughput much closer to the non-recompute baseline than full-layer recompute does.
Priority order:
Parallel Folding decouples attention and MoE parallelism so you do not have to pick a single compromise layout:
Attention: TP × CP × DP × PP
MoE: ETP × EP × EDP × PP
Key knobs:
--expert-model-parallel-size--expert-tensor-parallel-sizeUse it when attention prefers some TP or CP, but expert layers benefit from a larger EP degree than the dense layers can tolerate.
| Bottleneck | What it looks like | Primary fixes |
|---|---|---|
| Memory | Run fits only with aggressive full recompute or OOMs during warmup | selective recompute, FP8, offloading, better PP layout |
| Communication | Nsight shows large all-to-all or collective blocks | DeepEP or HybridEP, EP overlap, DP/TP overlap, better PP layout |
| Host overhead | GPU gaps, launch-bound traces, Python overhead | CUDA graphs, --manual-gc, higher MBS, CPU affinity tuning |
| Compute | Low SM utilization after comm and host issues are addressed | grouped GEMM, fusion work, FP8, dispatcher-specific kernel tuning |
Use unprofiled steady iterations for the acceptance metric and a matched profile for causal explanation:
On a controlled 16×H100 Qwen3 30B-A3B HybridEP run, plain EP overlap increased
communication hidden by GEMM/attention from 0.11% to 36.55%. The unprofiled
step fell from 24.7138s to 20.9920s and throughput rose from 244.039 to 287.305
model TFLOPS/GPU. delay_wgrad_compute remained disabled.
Choose the smallest candidate that targets the profiled bottleneck and change one variable at a time.
Use dispatcher choice as a bottleneck fix, not as a hardware lookup table.
moe_token_dispatcher_type="alltoall": safest bring-up path, fine for
smaller EP sizesmoe_token_dispatcher_type="flex" + moe_flex_dispatcher_backend="deepep":
candidate when DeepEP is installed and communication is exposedmoe_token_dispatcher_type="flex" + moe_flex_dispatcher_backend="hybridep":
topology-sensitive candidate on both NVL8 and NVL72 systems when HybridEP is
installedHybridEP plus plain EP overlap is the current measured winner for the canonical
16×H100 Qwen3 30B-A3B shape, while the canonical 256×H100 Qwen3 235B recipe
uses standard alltoall plus overlap. Benchmark backend compatibility and
throughput in the target container; neither GPU name nor EP degree determines
the winner by itself.
If the all-to-all path is visible in profiles, combine dispatcher tuning with:
--overlap-moe-expert-parallel-comm--overlap-grad-reduce--tp-comm-overlapTest plain EP overlap, shared-expert overlap, and delayed weight-gradient compute as separate candidates first. A combination can regress even when one component helped on another model.
Start with a verified BF16 baseline. Hardware capability only determines which lower-precision candidates are legal; it does not guarantee a speedup.
| Platform | Candidate after BF16 is stable | |---|---| | Hopper | per-tensor, current-scaling, or blockwise FP8 supported by the target stack | | Blackwell | MXFP8 or another supported FP8 recipe | | Blackwell, speed-first exploration | NVFP4 after the BF16/FP8 path is stable |
Keep the router in FP32. The largest wins usually come from expert GEMMs and other heavy matrix math, not from trying to quantize every small MoE component. Require logs or traces showing that the intended kernels ran, and judge the candidate by end-to-end steady step time rather than theoretical peak FLOPS.
Use CUDA graphs only after a profile shows meaningful host/launch gaps. For dropless MoE, start with the narrowest partial TE-scoped graph candidate:
moe_routermoe_preprocessAdd attn only if it is supported for the model and improves the same matched
stack. A successful capture is not evidence of a speedup, and a graph win can
disappear after dispatcher, overlap, or precision changes.
This path keeps dynamic expert work outside the graph. Budget extra memory, verify that shapes remain static, confirm replay rather than capture alone, and time only post-capture iterations.
Use full-iteration graphs only for graph-friendly workloads such as drop-and-pad or tightly controlled static-shape experiments.
Related references:
Use 6–12 post-warmup iterations for inexpensive screening when the workload allows it. For the selected candidate, run at least 50 steps and report a fixed steady window such as steps 41–50. The final evidence bundle should contain:
Do not attribute the total gain of a final multi-change winner to one earlier A/B. For example, the Qwen3 overlap experiment isolated a rise from 244.039 to 287.305 TFLOPS/GPU; the later canonical recipe reached 299.352 after additional HybridEP tuning. They answer different questions.
Do not optimize in the wrong order: fitting the model and selecting sane parallelism matter more than micro-optimizations.
Platform changes the limiting wall: H100-class runs often feel more communication-bound, while GB200 or GB300 runs often expose CPU or launch overhead earlier.
FP8 MFU can look misleadingly low: compare absolute throughput as well as MFU when switching precision modes.
CUDA graphs and recompute interact: TE-scoped graphs are usually paired with selective recompute, not blanket full recompute.
Parallel Folding is not optional at large scale: once attention and expert layers want clearly different layouts, a single shared TP or EP plan becomes a tax on both.
Summed kernel time is not exposed time: use interval unions and communication/compute intersection when validating overlap.
Benchmark-only semantics are not production acceptance: forced routing, synthetic data, or disabled optimizer/checkpoint paths must be disclosed and validated separately from training-equivalent results.
Feature activation needs evidence: a config dump is insufficient when a backend can fall back, a graph can capture without helping, or a lower- precision recipe can miss the intended kernels.
Last signature refresh: 2026-08-03.
Agent 工作流
创造有效技能指南。 当用户想创造出新的技能(或更新现有的技能),以专业知识,工作流程,或工具集成来扩展克洛德的能力时,应该使用这种技能.
Agent 工作流
使用ClawdHub CLI搜索,安装,更新并发布从taladhub.com的代理技能. 需要获取苍蝇上的新技能时使用,将安装的技能同步到最新版本或特定版本,或者发布 npm-instainddhub CLI 的新/更新的技能文件夹.
Agent 工作流
管弦乐团多代理团队,任务设定周期,交接协议,审查工作流程. 使用时间: (1)建立2+特派员队伍,具有不同专业,(2)确定任务路线和生命周期(收录框_ spec_建设_审查_完成),(3)在特派员之间制定交接协议,(4)建立审查和质量关口,(5)管理特派员之间的交流和文物共享.
Agent 工作流
Spec-first,TDD,子代理驱动的软件开发工作流程. 当:(1)构建任何新功能或应用——触发脑暴_计划_子代理执行回路,(2)调试出一个bug或测试失败——触发系统性的根起过程,(3)用户说"让我们构建","帮助我计划","我想添加X",或"这个被打破",(4)完成一个功…