arXiv Research: Optimizing LLM Inference for Aggregate User Behavior Predictions
Summary:arXiv has released a study on optimizing Large Language Model (LLM) inference for predicting aggregate user behavior. The research focuses on combining individual user behavioral histories to forecast the fraction of users taking specific actions, particularly in e-commerce settings. The study proposes two design choices: summarizing product sessions before inference and predicting outcomes for individual sessions before aggregation. Results indicate that individual session predictions outperfor
Background and Motivation
When using Large Language Models (LLMs) to simulate user population behavior, the key challenge lies in effectively combining individual user behavioral histories to predict the likelihood of specific actions. This study focuses on this problem within the e-commerce domain, aiming to predict product conversion rates based on browsing sessions.
Methodology and Design Choices
The study proposes two main design choices:
- Session Summarization: Summarizing product sessions before inference. This approach reduces inference costs but may discard predictive details, introducing bias.
- Individual Session Prediction: Predicting outcomes for individual sessions before aggregating. This preserves details but requires multiple model calls to reduce variance.
Additionally, the study explores the impact of evaluating multiple products in a single prompt. This can help the model infer typical patterns, reducing bias, but also makes each prediction dependent on the other products, adding noise.
Key Findings
- Individual Session Prediction vs. Session Summarization: Individual session predictions outperform summaries when products are evaluated separately, but summaries lead when evaluated together.
- Advantage of Comparative Summaries: Comparative summaries achieve the highest Spearman correlation with observed rates in 12 out of 15 settings, significantly outperforming other methods.
- Balancing Inference Cost and Accuracy: Comparative summaries significantly reduce inference costs while maintaining high prediction accuracy, costing only a fraction of the session methods.
Engineering Trade-offs and Practical Applications
- Trade-off Analysis: Session summarization reduces costs but may lose predictive details, increasing bias. Individual session prediction preserves details but incurs higher inference costs due to multiple model calls.
- Practical Recommendations: In resource-constrained environments, comparative summaries offer a balanced solution. For high-precision requirements, individual session predictions are preferable, albeit with higher costs.
Conclusion
This study provides empirical evidence on the performance differences of LLM inference under various design choices, offering valuable insights for practical applications. Comparative summaries emerge as a highly effective solution for multi-product evaluation in e-commerce, balancing cost and accuracy effectively.
— END —Source: ArXiv Machine Learning (cs.LG) (2026-10-20)
Tags: #LLMs & Foundation Models #Inference Optimization #User Behavior Prediction #E-commerce #arXiv
Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.
Community Comments