HDM Chooses Between Zero-Token and Full Autoregressive Reasoning

Summary:HDM introduces a zero-or-T thinking paradigm built around a single frozen backbone. The model can answer directly from the computation performed during one forward pass or invoke full autoregressive reasoning when a lightweight self-routing block predicts that the expected gain justifies the additional token cost. A request-level price, λ, controls this tradeoff. For episodic agent tasks without per-decision labels, Calibrated Decision Policy Optimisation trains the router from episode-level ret

---

Core Idea: Decide Whether to Reason, Not How Much to Write

Most adaptive-reasoning methods adjust either the length of an explicit reasoning trace or when to switch between modes, but the final answer is still decoded token by token. HDM takes a more fundamental approach: zero-or-T thinking. The model either answers directly from the computation performed in a single forward pass, with no generated reasoning tokens, or launches full autoregressive reasoning that may consume up to T tokens.

The central contribution is therefore not faster reasoning, but online economic control over reasoning itself. Batch applications can avoid unnecessary decoding, while agent systems can reserve expensive reasoning for high-value decisions.

Architecture: Frozen Backbone and a Lightweight Self-Router

HDM consists of a frozen shared backbone and a lightweight self-routing block. In fast mode, the backbone produces an answer directly from one forward pass. In slow mode, it generates an explicit reasoning trajectory through standard autoregressive decoding.

The router uses information already produced during the implicit reasoning stage to estimate the expected utility of thinking:

V(x) = E[U(y_T | x) - U(y_0 | x)]

Here, y_0 denotes the zero-token fast answer and y_T denotes the answer reached after T explicit reasoning tokens. Slow reasoning is invoked only when V(x) > λ, where λ is a request-level price that can be adjusted according to latency targets, token budgets, or business value.

Decoupling the router from the backbone provides several practical benefits:

• Freezing the backbone avoids retraining the main model for routing;
• The router adds only a small number of parameters and one decision step;
• λ can be changed at runtime, turning the model service into an adjustable latency–cost policy.

A one-pass fast path is not computationally free. The backbone still performs its full forward computation; what it avoids is additional answer decoding, KV-cache growth, and autoregressive coordination. HDM therefore reduces decoding overhead rather than eliminating the model’s underlying FLOPs.

CDPO: Learning When Reasoning Is Worth It

Open-ended agent tasks rarely provide labels indicating whether a particular slow-mode invocation was worthwhile. The paper addresses this with Calibrated Decision Policy Optimisation, or CDPO, which trains the self-routing block from episode-level environmental returns.

The router chooses between fast and slow modes according to the current task state, and the trajectory outcome supplies the policy-optimisation signal. This differs from ordinary classification-based routing because CDPO optimises whether an additional reasoning step improves future cumulative return, which is useful when the value of reasoning appears only much later.

Episode-level rewards introduce two challenges. Sparse rewards make it difficult to distinguish poor action selection from failed reasoning, while long-horizon trajectories allow early routing decisions to alter later states and complicate credit assignment. The supplied material does not detail CDPO’s reward-attribution mechanism, variance control, or controlled comparison with PPO- and GRPO-style objectives, all of which are important for reproducibility.

Results: Small Accuracy Change, Large Efficiency Gains

HDM is evaluated on MMLU-Pro, AGIEval, GPQA-Diamond, JevBench, BANKING77, and MiniWoB. Two results illustrate its practical value.

On JevBench, HDM-4B scores 0.857, slightly above Jev’s 0.851, an absolute gain of 0.006. Median latency falls from 0.652 seconds to 0.179 seconds, a 72.5% reduction. The routing mechanism keeps most requests on the fast path without a meaningful loss in benchmark accuracy.

On the MiniWoB agent environment, HDM-27B reports an 80.1% reduction in token cost. It is important to determine whether this refers to generated reasoning tokens or all prompt, completion, and tool-trajectory tokens. Different accounting conventions can materially change the comparability of that figure.

These results should be interpreted cautiously. JevBench and MiniWoB have different task distributions, success criteria, and sampling budgets, and the reported 0.7% relative accuracy improvement may fall within statistical variation. The defensible conclusion is that HDM demonstrates a viable accuracy–cost tradeoff, not that it has established a universal advantage.

Engineering Trade-offs: The Fast Path Is Not Free

HDM’s gains come from skipping explicit reasoning tokens, but the fast branch still executes the backbone’s forward pass. For long-context workloads, prefill computation, attention cost, and memory bandwidth may therefore remain dominant. If the router reads full hidden states, it can also introduce extra synchronization and memory-access overhead.

Routing errors are asymmetric:

• Sending a difficult problem to the fast path can reduce answer quality or cause invalid agent actions.
• Sending an easy problem to the slow path wastes tokens, latency, and reasoning-service capacity.

Production deployments should therefore track more than mean accuracy. Relevant metrics include fast-path accuracy, error-routing rates, slow-mode invocation share, P50 and P95 latency, and total cost per successful task. High-risk agent operations should retain a forced slow path or an automatic escalation rule for low-confidence states.

Deployment Guidance

HDM can be implemented as a model-agnostic layer around an existing inference stack. A team can keep the current model and reasoning template, add a lightweight classifier that estimates whether a request needs long-form reasoning, and route requests through a unified scheduler.

A practical rollout would:

1. Build a dataset of quality differences between fast and fully reasoned answers from historical requests.
2. Train a separate router without modifying the main model weights.
3. Run the router in shadow mode to calibrate the quality–cost curve.
4. Set λ per workload class, using low thresholds for routine summaries and higher thresholds for consequential decisions.
5. Add mandatory slow-path fallbacks for uncertain cases, misroutes, and high-risk tool calls.
6. Report accuracy, routing error, latency, and cost per successful task rather than token reduction alone.

If the environment provides verifiers or unit tests, a CDPO-like signal can optimise the router from complete trajectories. For static question-answering data, paired generations can first serve as distillation labels. Keeping the main model frozen also makes the approach easier to deploy in private or edge environments.

Conclusion

HDM moves adaptive reasoning from deciding how many reasoning tokens to emit to deciding whether explicit reasoning is economically justified. A lightweight router, a frozen backbone, and an adjustable λ jointly expose accuracy, latency, and inference cost as controllable variables. The reported results show substantial latency and token savings, but independent replication is still needed to establish generalisation, CDPO stability, and robustness across task families.

---

Source: Hugging Face Daily Papers (2026-11-12)

— END —

Tags: #HDM Follow #Adaptive Reasoning Follow #Model Routing Follow #Reinforcement Learning Follow #Inference Cost Optimization Follow