arXiv Research Reveals Impact of Instruction Tuning on Number-Choice Heuristics in Language Models

Summary:A new arXiv study investigates the impact of instruction tuning on the number-choice behavior of language models. By precisely measuring the probability distribution of integers from 1 to N when models are prompted for a random number, the research reveals that instruction tuning makes the distribution less uniform and introduces or sharpens specific heuristics, such as avoiding integer boundaries and multiples of 10. The study also finds that while instruction tuning does not create a shared fa

---

Background and Motivation

In recent years, large language models (LLMs) have made significant advances in natural language processing tasks, but their behavior in generating random numbers has not been fully explored. This study aims to investigate the impact of instruction tuning on the number-choice heuristics of language models by precisely measuring their behavior in random number generation tasks.

Methodology

The research team conducted experiments through the following steps:

1. Data Collection and Processing: Testing 18 open-weight models across 5 different ranges (1 to N, N ≤ 100) under 29 decoding configurations and two inference stacks.
2. Instruction Tuning: Applying instruction tuning to 15 pairs of models and comparing their behavior before and after tuning.
3. Probability Calculation: Calculating the probability assigned to each integer by summing over each integer's scored spellings rather than estimating by sampling, thus eliminating noise.

Key Findings

1. Introduction and Strengthening of Heuristics: Instruction tuning makes the distribution less uniform in all 15 pairs and introduces or strengthens specific heuristics, such as avoiding integer boundaries and multiples of 10.
2. Consistency in Odd Number Preference: At N=100, instruction-tuned models show a consistent preference for odd numbers, while baseline models only show a weak preference.
3. No Shared Favorite Integer Across Models: Instruction tuning does not create a shared favorite integer across models, with variations in the preference for the number 17 across different models.
4. Impact of Temperature and Nucleus Truncation: At temperature 1.0, nucleus truncation (top-p 0.95) adds a median of 0.035 to the total variation distance.

Engineering Trade-offs and Practical Applications

1. Training Cost and Benefit: While instruction tuning increases training costs, it significantly improves model performance in specific tasks, such as applying heuristics in random number generation.
2. Model Interpretability: This study provides new insights into the impact of instruction tuning on model behavior, contributing to the development of more interpretable models.
3. Application Scenarios: The research results can be applied to tasks that require random number generation, such as game AI, simulations, and random sampling.

Developer Recommendations

1. Choice of Tuning Strategy: For tasks that require specific number-choice behavior, it is recommended to use instruction tuning to reinforce or introduce specific heuristics.
2. Model Evaluation: Before deploying the model, it is important to evaluate its behavior using precise probability calculation methods rather than relying on sampling estimates.
3. Continuous Optimization: Continuously optimize the instruction tuning strategy according to the specific application scenario to achieve the desired model behavior.

---

Source: ArXiv NLP/LLM (cs.CL) (2026-10-28)

— END —

Tags: #arXiv Follow #instruction tuning Follow #language models Follow #random number generation Follow #heuristics Follow