arXiv Research: Complementary Query Rewriting Boosts Performance Under Strong RAG Baseline

Summary:A new arXiv study examines the impact of query rewriting strategies on Retrieval-Augmented Generation (RAG) performance under a strong baseline. The research shows that while individual rewriting strategies are at best competitive with the baseline, combining multiple strategies yields significant performance gains. For instance, a four-method union improves HIT@10 by 12.5 points on enterprise data. Additionally, the study introduces a confidence-gated router that dynamically decides whether to

---

Background and Motivation

In Retrieval-Augmented Generation (RAG) systems, query rewriting is a common technique to improve performance by rewriting the user's question into multiple variants and searching with all of them. However, it is unclear whether this approach remains effective when the underlying search system is already strong.

Methodology and Experiments

The research team compared four query-rewriting strategies (S1-S4) against two strong LLM baselines (HyDE and Query2Doc) under a fixed, competitive pipeline (BGE dense retrieval, cross-encoder reranking, and MMR diversification). The experiments were conducted on three datasets (HotpotQA, AmbigNQ, and the 512K-entry EnterpriseRAG-Bench) across three seeds, using paired-bootstrap significance tests.

Key Findings

1. Limitations of Individual Rewriting Strategies: Individual query-rewriting strategies are at best competitive with the strong baseline and do not significantly outperform it.
2. Advantages of Combined Strategies: Combining multiple strategies yields significant performance gains. For example, a four-method union improves HIT@10 by 12.5 points on enterprise data, and a five-method union reaches 52.98 (+13.8).
3. Complementarity Over Budget: Budget-matched controls capture only ~40% of the gain, confirming that the advantage comes from the complementarity of different strategies, not from increased retrieval budget.
4. Confidence-Gated Routing: The study introduces a confidence-gated router that dynamically decides whether to apply rewriting. This mechanism captures about half of the full combination gains while reducing rewriting costs to less than 40% of queries.

Industry Impact and Recommendations

1. Optimizing RAG Systems: Developers should treat query rewriting as a complementary coverage source rather than a standalone replacement for a strong baseline.
2. Dynamic Strategy Selection: By using a confidence-gated router, developers can achieve performance gains while controlling computational costs.
3. Dataset-Specific Considerations: Different datasets respond differently to query-rewriting strategies, so developers should choose appropriate strategy combinations based on the specific application scenario.

Technical Highlights

• Multi-Strategy Combination: The combination of multiple query-rewriting strategies significantly boosts RAG system performance.
• Confidence Routing: The introduction of a confidence-gated routing mechanism balances performance and cost.
• Experimental Validation: Experiments on multiple datasets and baselines validate the reliability and generalizability of the research findings.

---

Source: ArXiv NLP/LLM (cs.CL) (2026-09-09)

— END —

Tags: #RAG #Query Rewriting #LLMs & Foundation Models #arXiv #Retrieval-Augmented Generation