SMRC-SD Framework: Addressing State-Reference Mismatch in Privileged Distillation for Multi-Turn Agents
Summary:A new research paper on arXiv introduces the State-Matched Routing and Contextualized Self-Distillation (SMRC-SD) framework to address the state-reference mismatch problem in privileged distillation for multi-turn agents. By applying distillation only when the student's current execution state matches a supported state along the reference trajectory and constructing state-conditioned teacher context, SMRC-SD significantly improves task success rates in interactive environments. Experiments on AL
Background and Problem
In the training of multi-turn agents, privileged distillation is a common method that leverages a synchronized teacher model to re-score the student's responses, providing dense supervision signals. However, in interactive environments, the student's preceding actions continually change the execution state, causing the student's rollout to reach states not covered by the reference trajectory. This state-reference mismatch makes the guidance from privileged distillation unreliable and affects the training effectiveness.
SMRC-SD Method
To address this issue, researchers propose the State-Matched Routing and Contextualized Self-Distillation (SMRC-SD) framework. The core idea of this method is:
-
State-Matched Routing: At each turn, SMRC-SD verifies whether the student's current execution state matches a supported state along the reference trajectory. Distillation is applied only when a match is found, filtering out turns where the reference lacks locally compatible guidance.
-
Contextualized Self-Distillation: For each matched state, SMRC-SD constructs a state-conditioned teacher context from the successful trajectory, grounding the supervision in the state actually reached.
Experimental Results
SMRC-SD demonstrates strong performance in ALFWorld and WebShop benchmarks:
- On ALFWorld, using the Qwen3-1.7B model, the task success rate increases from 74.6% to 86.5%.
- On WebShop, the task success rate increases from 57.4% to 69.3%.
Additionally, ablation studies show that selecting locally supported turns and constructing state-compatible teacher context are key factors in improving performance.
Technical Highlights
- Innovativeness: SMRC-SD is the first to introduce state-matched routing and contextualized self-distillation in privileged distillation, addressing the traditional state-reference mismatch problem.
- Effectiveness: The method shows significant performance improvements across multiple benchmarks, proving its effectiveness and versatility.
- Scalability: The method can be seamlessly integrated into existing multi-turn agent training pipelines, offering good scalability.
Industry Impact and Developer Recommendations
SMRC-SD provides a more efficient and precise solution for multi-turn agent training, particularly in scenarios that require handling complex interactive tasks. Developers can consider the following recommendations:
- Integrate SMRC-SD: Integrate the SMRC-SD framework into multi-turn agent training pipelines to improve training efficiency and model performance.
- Optimize Reference Trajectories: Ensure the quality and coverage of reference trajectories to fully leverage the advantages of SMRC-SD.
- Explore More Applications: Explore the application of SMRC-SD in other scenarios that require privileged distillation, such as robotics control, automated decision-making, etc.
Conclusion
The SMRC-SD framework offers an innovative solution for multi-turn agent training, significantly improving training effectiveness and model performance by addressing the state-reference mismatch problem.
— END —Source: ArXiv AI (cs.AI) (2026-08-07)
Tags: #Multi-Turn Agents #Privileged Distillation #State-Matching #Contextualized Self-Distillation #SMRC-SD
Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.
Community Comments