ZICQ
中 Log in / Sign up
Newsroom Research & Papers #Selective On-Policy Distillation #Learning Rate #LoRA #AdamW #Model Training

Research Reveals Learning Rate Dependence in Selective On-Policy Distillation: Challenging Traditional Assumptions

Avatar of Mr.Xu

By Mr.Xu Compiled & Reviewed by Editorial

Published: · 6 views

中文阅读 (Chinese) English Version
Co-reading Select any sentence in the article to highlight, annotate, or copy a quote card. Join discussion ↓

Summary:A study published on arXiv reveals a significant impact of learning rates on model performance in selective on-policy distillation, challenging the traditional assumption that a single shared learning rate serves as a neutral control. The research demonstrates that different selection strategies exhibit substantial sensitivity to learning rate changes, leading to performance swings of up to 49.5 percentage points. The study recommends reporting the full arm x rate matrix rather than relying sole


Background and Problem

Selective on-policy distillation is a method that improves efficiency by selectively training a student model. Traditionally, it has been assumed that different selection strategies can be fairly compared under a single shared learning rate. However, this study published by arXiv challenges this assumption, revealing that learning rates have a significant impact on the performance of different selection strategies.

Key Findings

  1. Learning Rate Sensitivity: Experiments on the GSM8K dataset using LoRA technology show that different selection strategies exhibit high sensitivity to changes in learning rates. For example, the random 5% subset selection strategy shows a performance swing of 5.4 percentage points across an 8x learning rate grid, while the teachability selector swings by 17.7 percentage points.

  2. Selector-Rate Entanglement: The study attributes this sensitivity to the selection strategies themselves rather than the learning rate step size. The AdamW update magnitudes track the learning rate changes closely, even with a 15.5x difference in gradient norms across different strategies.

  3. Fine-Grained Analysis: In a finer learning rate range (1e-6 to 1e-5), the dense supervision performance swings by 19.8 percentage points, and the selective arm swings by 49.5 percentage points.

  4. Recommendations: The study recommends reporting the full arm x rate matrix rather than relying solely on a single shared learning rate when comparing selection strategies.

Technical Highlights

  • Application of LoRA: The study demonstrates the effectiveness of LoRA (Low-Rank Adaptation) in handling complex tasks.
  • Analysis of AdamW Optimizer: By analyzing the update magnitudes of the AdamW optimizer, the study reveals the mechanism by which learning rates affect model performance.
  • Frozen Scoring Ablation Experiment: The frozen scoring ablation experiment further validates the phenomenon of selector-rate entanglement.

Industry Impact and Developer Recommendations

  • Impact on AI Research: This study provides a new perspective on the selection of learning rates in AI model training, emphasizing the complex interplay between selection strategies and learning rates.

  • Recommendations for Developers: Developers should carefully choose learning rates and consider the impact of learning rates when comparing different selection strategies to avoid biases caused by the single learning rate assumption.

Future Research Directions

Future research could further explore the interaction between selection strategies and learning rates under different model architectures and task types, and develop more robust selection strategies to reduce sensitivity to learning rates.


Source: ArXiv Machine Learning (cs.LG) (2026-09-22)

— END —

Tags: #Selective On-Policy Distillation #Learning Rate #LoRA #AdamW #Model Training

Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.

What's your take on this story?
🚀 Explore & Submit Works

Community Comments

Loading live comments and annotations…