Hugging Face Research Reveals 'Thinking Inertia' in LLMs: Persistent Reasoning Despite Explicit Prohibition

Summary:Hugging Face's research introduces the concept of 'Thinking Inertia,' highlighting that Large Language Models (LLMs) continue to exhibit reasoning behavior even when explicitly instructed not to. The study evaluates this phenomenon across tasks and models using three metrics: Empty-Thinking Rate, Question-Pre-answer Relevance, and LLM-as-judge Explicit Inference Rate. The findings reveal that explicit controls fail to reliably eliminate reasoning, and open-ended answer spaces exacerbate persiste

---

Background and Motivation

Large Language Models (LLMs) are increasingly integrating explicit 'thinking modes,' but their counterpart, 'no-thinking,' has received far less attention. Existing evaluation methods typically rely on proxies such as disabling the thinking mode or the absence of long reasoning traces, but these proxies are unreliable: disabled thinking modes may still emit reasoning, and long traces may contain filler rather than genuine inference.

Methodology

The research team proposes a new evaluation framework that normalizes each response into a pre-answer trace and a final answer, and evaluates it at three levels:

1. Empty-Thinking Rate: Assessing whether the model strictly complies with outputting only the answer.
2. Question-Pre-answer Relevance: Evaluating the similarity between the question and the pre-answer trace.
3. LLM-as-judge Explicit Inference Rate: Evaluating whether the model exhibits explicit reasoning behavior.

These metrics distinguish between answer-only output, relevant but non-inferential text, and explicit inference.

Key Findings

1. Explicit Controls Cannot Eliminate Reasoning: Even under strict controls, models continue to exhibit reasoning behavior.
2. Thinking Inertia: Explicit inference is more prevalent in open-ended answer spaces, indicating that models tend to persist in reasoning in more complex questions.
3. Trade-off Between Task and Accuracy: Accuracy remains stable on Boolean and multiple-choice tasks, while open-ended tasks reveal a trade-off between answer-only compliance and task accuracy.
4. Impact of Providing Candidate Answers: Providing candidate answers makes it easier to produce answer-only responses.

Engineering Trade-offs and Recommendations

1. Optimizing Reasoning Control Mechanisms: Existing explicit control mechanisms are insufficient to eliminate reasoning behavior; more refined control strategies are needed.
2. Model Training and Fine-tuning: Introducing explicit constraints on reasoning behavior during training may help reduce thinking inertia.
3. Evaluation and Benchmarking: New benchmarks are needed to more accurately assess model performance in terms of stopping reasoning.

Developer Deployment Recommendations

1. Clarify Task Requirements: When designing applications, choose appropriate reasoning control strategies based on task requirements.
2. Combine Multiple Control Methods: Combine explicit controls, prompt engineering, and post-processing methods to achieve more reliable reasoning control.
3. Continuous Monitoring and Optimization: Regularly monitor model behavior and optimize based on actual application scenarios.

---

Source: Hugging Face Daily Papers (2026-10-08)

— END —

Tags: #Hugging Face Follow #LLMs & Foundation Models Follow #Reasoning Control Follow #Thinking Inertia Follow