跳到主内容
智客 ZICQ

技能库 智客分类:Agent 工作流 dflash-mlx-speculative-decoding

Dflash Mlx Speculative Decoding

苹果硅上MLX的无损 DFlash投机解码 — 1.7–4x 更快的LLM推论,使用块扩散起草并有目标模型验证.

429 安装量

官方网址:skills.sh

技能介绍

先看中文介绍;官方 description 原文单独保留,不改写 SKILL.md。

做什么

苹果硅上MLX的无损 DFlash投机解码 — 1.7–4x 更快的LLM推论,使用块扩散起草并有目标模型验证.

何时用

官方 description 未单独写出 Use when。按规范,代理会在用户任务与这段 description 的关键词匹配时激活本技能。

代理如何加载

按 Agent Skills 渐进披露:启动时只加载 name 与 description(约 100 token);任务匹配后才读入整份 SKILL.md 正文;scripts/、references/、assets/ 仅在需要时再读。 本文件正文结构:dflash-mlx Speculative Decoding、Installation、or isolated install、Key CLI Commands、Generate text、Auto-resolve draft model from registry。

文件分析

文件分析:这是一份仅含 SKILL.md 的指令型技能,代理激活后整份正文进入上下文。

官方 description(原文)

Lossless DFlash speculative decoding for MLX on Apple Silicon — 1.7–4x faster LLM inference using block diffusion drafting with target model verification.

dflash-mlx Speculative DecodingInstallationor isolated installKey CLI CommandsGenerate textAuto-resolve draft model from registryExplicit draft modelDisable EOS (useful for benchmarking fixed token counts)OpenAI-compatible serverBasic serverWith explicit draftDisable thinking/reasoning tokens (Qwen3.5 thinking models)

来源分类:skills.sh agent-skill

SKILL.md 与 Agent 调用

官方规范 ↗
name
dflash-mlx-speculative-decoding
description
Lossless DFlash speculative decoding for MLX on Apple Silicon — 1.7–4x faster LLM inference using block diffusion drafting with target model verification.
  1. 发现技能客户端向 Agent 提供名称与描述目录。
  2. 匹配与调用用户指定或任务匹配后,载入 SKILL.md 指令。
  3. 按需加载按步骤读取参考文档、使用脚本与素材。

具体调用语法与可用工具以目标 Agent 客户端为准。 查看调用机制说明 ↗

安装这个技能

Skills CLI ↗

先选择目标 Agent 和安装范围,保留技能包的附属文件,安装后检查客户端能否发现该技能。

交给 Agent 安装

复制安装指令给支持 Agent Skills 的代理,确认其中的目标目录与客户端匹配。

把 Agent Skill「dflash-mlx-speculative-decoding」安装到我的项目:SKILL.md 原文与官方 description 见 https://zicq.com/zh/skills/skl-50c5657382bb68c7-Dflash-Mlx-%E5%85%89%E8%B0%B1-%E8%A7%A3%E7%A0%81.html
请存为 .cursor/skills/dflash-mlx-speculative-decoding/SKILL.md 或 .claude/skills/dflash-mlx-speculative-decoding/SKILL.md,frontmatter 的 name 与 description 保持原样,不要改写。

GitHub 完整包 ↗

终端安装 · Skills CLI

需要 Node.js 与 npx。先查看仓库技能列表,确认实际名称。

npx skills add 'https://github.com/reason-machines/trending-skills' --list

npx skills add 'https://github.com/reason-machines/trending-skills' --skill 'dflash-mlx-speculative-decoding'

CLI 会交互选择目标 Agent,默认安装到项目;用户级安装使用 -g。先通过查看命令核对仓库内容,再用 npx skills list 检查已安装技能。

阅读排版

name: dflash-mlx-speculative-decoding description: Lossless DFlash speculative decoding for MLX on Apple Silicon — 1.7–4x faster LLM inference using block diffusion drafting with target model verification. triggers:

  • use dflash for faster inference
  • speed up mlx generation with speculative decoding
  • dflash speculative decoding apple silicon
  • run dflash-serve openai compatible server
  • benchmark dflash vs baseline mlx
  • set up dflash with qwen model
  • dflash draft model auto resolution
  • faster token generation on apple silicon mlx

dflash-mlx Speculative Decoding

Skill by ara.so — Daily 2026 Skills collection.

DFlash implements lossless speculative decoding for MLX on Apple Silicon. A small draft model (~1B params) generates 16 tokens in parallel using block diffusion; the target model verifies all 16 in a single forward pass. Tokens are only emitted after target verification — output is lossless (every token is the target model's greedy argmax).

Typical speedups: 1.7x–4.1x over baseline mlx_lm depending on model size and context length. Acceptance rates hover around 87–90% for Qwen3.5 models.

Installation

pip install dflash-mlx

# or isolated install
pipx install dflash-mlx

Requires Python 3.10+, MLX 0.31.1+, Apple Silicon Mac.

Key CLI Commands

Generate text

# Auto-resolve draft model from registry
dflash --model Qwen/Qwen3.5-9B --prompt "Explain backpropagation"

# Explicit draft model
dflash --model Qwen/Qwen3.5-9B \
       --draft z-lab/Qwen3.5-9B-DFlash \
       --prompt "Explain backpropagation"

# Disable EOS (useful for benchmarking fixed token counts)
dflash --model Qwen/Qwen3.5-9B --prompt "..." --max-tokens 1024 --no-eos

OpenAI-compatible server

# Basic server
dflash-serve --model Qwen/Qwen3.5-9B --port 8000

# With explicit draft
dflash-serve --model Qwen/Qwen3.5-9B \
             --draft z-lab/Qwen3.5-9B-DFlash \
             --port 8000

# Disable thinking/reasoning tokens (Qwen3.5 thinking models)
dflash-serve --model Qwen/Qwen3.5-9B --port 8000 \
  --chat-template-args '{"enable_thinking": false}'

# Raise fallback threshold for longer prompts (large models)
dflash-serve --model mlx-community/Qwen3.5-35B-A3B-4bit --port 8000 \
  --chat-template-args '{"enable_thinking": false}' \
  --dflash-max-ctx 16384

Benchmark

dflash-benchmark \
  --model Qwen/Qwen3.5-9B \
  --draft z-lab/Qwen3.5-9B-DFlash \
  --prompt "The function f satisfies..." \
  --max-tokens 1024 \
  --repeat 3 \
  --no-eos

Outputs per-run JSON reports with tok/s, acceptance rate, and speedup vs baseline.

Supported Model Pairs

| Target Model | Draft Model | |---|---| | Qwen/Qwen3.5-4B | z-lab/Qwen3.5-4B-DFlash | | Qwen/Qwen3.5-9B | z-lab/Qwen3.5-9B-DFlash | | mlx-community/Qwen3.5-27B-4bit | z-lab/Qwen3.5-27B-DFlash | | mlx-community/Qwen3.5-35B-A3B-4bit | z-lab/Qwen3.5-35B-A3B-DFlash |

Draft models are auto-resolved from a registry — no --draft flag needed for listed pairs. Models without a matching draft are rejected at startup.

Python API Usage

Streaming generation

from dflash_mlx import DFlashRuntime

runtime = DFlashRuntime.from_pretrained(
    model="Qwen/Qwen3.5-9B",
    draft="z-lab/Qwen3.5-9B-DFlash",  # optional, auto-resolved
)

prompt = "Explain the Pythagorean theorem step by step."

for token_text in runtime.stream_generate(
    prompt=prompt,
    max_tokens=512,
    use_chat_template=True,
):
    print(token_text, end="", flush=True)
print()

Full generation with stats

from dflash_mlx import DFlashRuntime

runtime = DFlashRuntime.from_pretrained(model="Qwen/Qwen3.5-9B")

result = runtime.generate(
    prompt="What is speculative decoding?",
    max_tokens=256,
    use_chat_template=True,
)

print(result.text)
print(f"Tokens/sec: {result.tokens_per_second:.2f}")
print(f"Acceptance rate: {result.acceptance_rate:.2%}")
print(f"Total tokens: {result.total_tokens}")

Custom draft block size and context

from dflash_mlx import DFlashRuntime, DFlashConfig

config = DFlashConfig(
    draft_block_size=16,      # tokens drafted per speculative step
    max_ctx=8192,             # max context length before fallback
    enable_tape_replay=True,  # GatedDeltaNet recurrent rollback
    jit_sdpa=True,            # custom Metal SDPA for long contexts
)

runtime = DFlashRuntime.from_pretrained(
    model="mlx-community/Qwen3.5-27B-4bit",
    config=config,
)

OpenAI client against dflash-serve

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="not-needed",  # dflash-serve does not require auth by default
)

# Non-streaming
response = client.chat.completions.create(
    model="Qwen/Qwen3.5-9B",
    messages=[
        {"role": "user", "content": "Explain gradient descent."}
    ],
    max_tokens=512,
)
print(response.choices[0].message.content)

# Streaming
stream = client.chat.completions.create(
    model="Qwen/Qwen3.5-9B",
    messages=[{"role": "user", "content": "Write a haiku about silicon."}],
    max_tokens=128,
    stream=True,
)
for chunk in stream:
    delta = chunk.choices[0].delta.content
    if delta:
        print(delta, end="", flush=True)
print()

Tool calling (via dflash-serve)

import json
from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="not-needed")

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get current weather for a city",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {"type": "string", "description": "City name"},
                },
                "required": ["city"],
            },
        },
    }
]

response = client.chat.completions.create(
    model="Qwen/Qwen3.5-9B",
    messages=[{"role": "user", "content": "What's the weather in Tokyo?"}],
    tools=tools,
    tool_choice="auto",
)

tool_call = response.choices[0].message.tool_calls[0]
print(f"Function: {tool_call.function.name}")
print(f"Args: {json.loads(tool_call.function.arguments)}")

Common Patterns

Side-by-side demo (baseline vs DFlash)

PYTHONPATH=. python3 -m examples.demo --mode dflash \
  --target-model Qwen/Qwen3.5-9B \
  --draft-model z-lab/Qwen3.5-9B-DFlash \
  --prompt "Solve: f(x) + f(y) = f(x+y) - xy - 1" \
  --max-tokens 2048 \
  --no-eos

Integrating with Open WebUI

  1. Start dflash-serve --model Qwen/Qwen3.5-9B --port 8000
  2. In Open WebUI settings → Connections → add OpenAI API with URL http://localhost:8000/v1
  3. Select model Qwen/Qwen3.5-9B in the chat UI

Works the same for Continue, aider, OpenCode, and any OpenAI-compatible client.

Override draft for unsupported models

# Force a custom draft — bypasses registry check
dflash --model my-org/MyCustomModel \
       --draft my-org/MyCustomModel-DFlash \
       --prompt "Hello"

Disable thinking tokens for Qwen3.5

# CLI
dflash --model Qwen/Qwen3.5-9B \
       --chat-template-args '{"enable_thinking": false}' \
       --prompt "What is 2+2?"

# Server
dflash-serve --model Qwen/Qwen3.5-9B \
             --chat-template-args '{"enable_thinking": false}' \
             --port 8000

Architecture Notes

  • Tape-replay rollback: For hybrid GatedDeltaNet + attention models (Qwen3.5), dflash records an innovation tape during verify and replays only accepted steps via a custom Metal kernel — avoids full state snapshots.
  • JIT SDPA 2-pass: For contexts ≥ 1024 tokens, a custom Metal attention kernel maintains numerical alignment with stock MLX attention.
  • Greedy acceptance: Keeps the longest correct prefix from the 16 drafted tokens, rejects the rest. No temperature/sampling on verification — strictly lossless.
  • Qwen3 (pure attention) models work but don't benefit from tape-replay rollback (that's GatedDeltaNet-specific).

Troubleshooting

Model rejected at startup

Error: No DFlash draft found for model 'org/ModelName'

→ Pass --draft org/ModelName-DFlash explicitly, or use a model from the supported pairs table.

Low acceptance rate (< 80%)

  • Usually caused by very long context (4096+). Try --dflash-max-ctx 8192 to extend the fallback threshold.
  • Qwen3 (non-3.5) models have lower acceptance than Qwen3.5 hybrid models.

Numerical divergence / output differs from pure AR

  • Expected behavior: "Output can still differ from pure AR because of MLX dispatch divergence, but no unverified token is ever emitted."
  • If outputs seem wrong (not just different), ensure MLX 0.31.1+ is installed: python -c "import mlx; print(mlx.__version__)"

Server not accepting connections

# Check port is not in use
lsof -i :8000

# Bind to all interfaces for network access
dflash-serve --model Qwen/Qwen3.5-9B --port 8000 --host 0.0.0.0

Out of memory with large models

  • Use 4-bit quantized variants: mlx-community/Qwen3.5-27B-4bit instead of the full model.
  • The draft model loads alongside the target — budget ~1–2GB extra for the draft.

Benchmark results JSON location

ls benchmark/results/
# Per-run JSON with tok/s, acceptance rate, repeat measurements

相关技能

Agent 工作流

Skill Creator

创造有效技能指南。 当用户想创造出新的技能(或更新现有的技能),以专业知识,工作流程,或工具集成来扩展克洛德的能力时,应该使用这种技能.

Agent 工作流

Clawdhub

使用ClawdHub CLI搜索,安装,更新并发布从taladhub.com的代理技能. 需要获取苍蝇上的新技能时使用,将安装的技能同步到最新版本或特定版本,或者发布 npm-instainddhub CLI 的新/更新的技能文件夹.

Agent 工作流

Agent Team Orchestration

管弦乐团多代理团队,任务设定周期,交接协议,审查工作流程. 使用时间: (1)建立2+特派员队伍,具有不同专业,(2)确定任务路线和生命周期(收录框_ spec_建设_审查_完成),(3)在特派员之间制定交接协议,(4)建立审查和质量关口,(5)管理特派员之间的交流和文物共享.

Agent 工作流

Superpowers

Spec-first,TDD,子代理驱动的软件开发工作流程. 当:(1)构建任何新功能或应用——触发脑暴_计划_子代理执行回路,(2)调试出一个bug或测试失败——触发系统性的根起过程,(3)用户说"让我们构建","帮助我计划","我想添加X",或"这个被打破",(4)完成一个功…