arXiv Research: Optimizing LLM Inference for Aggregate Predictions from Behavioral Data
Summary:This research investigates how to combine individual behavioral histories using Large Language Models (LLMs) to predict the fraction of users taking a particular action, particularly in e-commerce. The study focuses on two design choices: summarizing product sessions to reduce inference costs but potentially losing predictive details, and predicting outcomes for individual sessions before aggregating to preserve details but increasing inference costs. The findings reveal that predictions from in
Background and Motivation
In scenarios like e-commerce, using Large Language Models (LLMs) to predict user behavior has become an important approach. However, effectively combining individual behavioral histories to predict the proportion of users taking a particular action remains a challenging problem. This study aims to explore the impact of two design choices on prediction performance:
- Summarizing Product Sessions: Summarizing product sessions before inference to reduce costs but potentially losing predictive details and introducing bias.
- Predicting Individual Sessions and Aggregating: Preserving the details of each session but requiring multiple model calls to reduce variance.
Technical Mechanism Analysis
The study evaluates these choices through the following methods:
- Cross-Product Comparison: Evaluating multiple products in a single prompt helps the model infer typical scenarios, reducing bias but introducing noise as predictions depend on other products.
- Bayesian Model Interpretation: A Bayesian model explains the second tradeoff, suggesting that comparison helps more when the inputs in the prompt reveal more about differences across products.
Engineering Trade-offs and Performance
Experiments conducted across five LLMs and two datasets show that:
- When Evaluating Products Separately: Predictions from individual sessions outperform summaries.
- When Evaluating Products Together: Summaries lead, especially when cross-product comparison is involved. Comparative summaries achieve the highest Spearman correlation in 12 out of 15 settings.
- Inference Cost: The inference cost of summaries is only a small fraction of the session methods.
Developer Deployment and Recommendations
For developers, the findings of this study offer the following insights:
- Balancing Inference Cost and Prediction Accuracy: Summarized predictions are an effective choice in resource-constrained environments, especially when dealing with a large number of products.
- Leveraging Cross-Product Comparison: Introducing comparative context in prompts can significantly improve prediction accuracy.
- Combining Bayesian Models for Interpretation: Understanding the trade-offs of different design choices helps make more informed decisions in specific application scenarios.
Conclusion
This research provides new insights into the application of LLMs in aggregate predictions from behavioral data, emphasizing the importance of cross-product comparison in enhancing prediction performance, and offers guidance on the trade-off between inference cost and accuracy in practical applications.
— END —Source: ArXiv Machine Learning (cs.LG) (2026-10-21)
Tags: #LLMs & Foundation Models #Inference Optimization #Behavioral Prediction #E-commerce #Bayesian Model
Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.
Community Comments