Behavioral History Significantly Enhances Prediction Accuracy of LLM Synthetic Personas

Summary:A new study published on arXiv examines the effectiveness of Large Language Models (LLMs) as synthetic personas in predicting individual decisions. The research demonstrates that LLM-based personas incorporating behavioral history outperform those relying solely on descriptive information such as demographics, personality traits, or cognitive scores. In the experiment, models with behavioral history achieved a 28% prediction accuracy at the individual level, compared to 7-12% for description-bas

---

Background and Motivation

Large Language Models (LLMs) have shown great potential in simulating human behavior and decision-making, particularly in creating synthetic personas to represent survey respondents. However, the validity of LLM-based personas depends on their ability to accurately reproduce individual decision processes. This study investigates the impact of different types of information on the predictive capabilities of LLM personas, with a specific focus on the role of behavioral history.

Methodology

The research utilized a two-wave panel dataset of 845 U.S. adults who completed measures of 14 behavioral biases (including risk preference, time preference, overconfidence, and reasoning ability). The study designed five conditions that progressively added richer information:

1. No personal information
2. Demographic information
3. Personality traits
4. Cognitive scores
5. Behavioral history (previous survey choices of the respondent)

In the behavioral history condition, all items scoring the target bias were retained, while other conditions relied on descriptive information.

Key Findings

1. Population-Level Analysis: The average number of biases per respondent across all conditions was close to the human average (7.1-8.1 biases vs. 7.1), but description-based personas recovered only 53-67% of the human between-person variation, whereas behavioral history restored it to approximately the human level.
2. Individual-Level Analysis: Description-based personas achieved only 7-12% of the informedness observed in human test-retest responses, while models with behavioral history raised this to 28%.
3. Demographic Group Differences: The condition including behavioral history had the highest estimated informedness in all 17 demographic groups, whereas description-based conditions provided little or no information for some groups.
4. Education and Income Differences: Synthetic responses exhibited stronger education- and income-related differences than human responses.

Technical Highlights

• Importance of Behavioral History: The study is the first to systematically demonstrate the critical role of behavioral history in enhancing the predictive capabilities of LLM personas.
• Multi-Group Applicability: Behavioral history performed well across all demographic groups, indicating its broad applicability.
• Data-Driven Approach: By leveraging a data-driven approach, LLMs can better capture individual decision patterns, thereby improving prediction accuracy.

Industry Impact and Developer Recommendations

1. Survey Analysis: In survey analysis, LLM personas can more accurately simulate respondent behavior and decision-making, enhancing the reliability of data analysis.
2. User Modeling: In user modeling, the application of behavioral history can improve model prediction capabilities, thereby optimizing personalized recommendations and user experience.
3. AI Ethics and Privacy: When applying behavioral history, it is crucial to prioritize privacy protection and data security to prevent misuse of user information.
4. Model Optimization: Developers should consider incorporating behavioral history data into LLM training to enhance model prediction performance.

---

Source: ArXiv AI (cs.AI) (2026-10-07)

— END —

Tags: #LLMs & Foundation Models Follow #Behavioral History Follow #Synthetic Personas Follow #Prediction Accuracy Follow #arXiv Follow