LLMs vs Naive Bayes: Benchmarking Classical and Modern Approaches in Text Classification
Summary:A new study published on arXiv benchmarks the performance of Complement Naive Bayes against zero-shot and few-shot LLMs across various text classification tasks. The results show that while LLMs dominate in zero-data scenarios, NB achieves comparable or superior accuracy when labeled data is available, with significantly higher throughput and lower energy consumption. The study highlights that the decision between LLMs and NB is task-dependent, and provides a Kubernetes Helm operator to automate
Background and Objectives
Large Language Models (LLMs) have revolutionized the field of natural language processing, but their high computational costs and energy consumption remain a concern. Meanwhile, classical machine learning algorithms like Naive Bayes (NB) continue to demonstrate strong performance in certain tasks. This study aims to benchmark the performance of LLMs against NB in various text classification tasks, and to evaluate their respective strengths and weaknesses in practical applications.
Key Findings
- Zero-shot Learning: LLMs excel in zero-shot learning scenarios. For example, in the Amazon Polarity sentiment analysis task, LLMs achieve an accuracy of 98.0%, compared to 88.2% for NB.
- Labeled Data Scenarios: As the amount of labeled data increases, NB's performance improves rapidly. In the AG News task, NB achieves an accuracy of 89.1%, comparable to the zero-shot 27B LLM (89.0%) and surpassing the 397B frontier model (84.8%).
- Computational Efficiency and Energy Consumption: NB demonstrates significant advantages in terms of computational efficiency and energy consumption. Small-scale LLM inference on GPUs is 40-486 times slower than NB CPU inference, with roughly two orders of magnitude higher energy consumption.
- Task Dependency: The choice of model depends on the specific task. For instance, NB reaches parity with LLMs at around 10^4 labels for topic classification, while zero-data sentiment analysis favors LLMs across all tested data sizes.
Technical Highlights
- Kubernetes Helm Operator: The research team developed a Kubernetes Helm operator to automate model selection based on task requirements. This tool leverages configurable thresholds and verifiable Prometheus metrics to simplify the model selection process.
- Balancing Performance and Efficiency: The study shows that NB achieves a good balance between accuracy and efficiency when processing labeled data, providing a more optimal solution for resource-constrained HPC practitioners.
Industry Impact and Developer Recommendations
- Resource Optimization: Developers working in resource-constrained environments may consider using NB as an alternative to LLMs to reduce computational costs and energy consumption.
- Model Selection Tool: The Kubernetes Helm operator offers a convenient tool for developers to automatically select the most suitable model based on specific task requirements.
- Future Research Directions: Future research could explore ways to combine LLMs with classical algorithms to achieve more efficient and accurate text classification solutions.
Conclusion
This study provides a comprehensive comparison of LLMs and NB in different scenarios, offering new insights and tools for resource-constrained applications.
— END —Source: ArXiv Machine Learning (cs.LG) (2026-09-15)
Tags: #LLMs #Naive Bayes #Text Classification #Kubernetes #Performance Optimization
Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.
Community Comments