arXiv Introduces EBG Framework: Revolutionizing Evidence-Based Generation for Long-Context QA
Summary:Deboban Banerjee and colleagues introduce Evidence-Based Generation (EBG), a novel approach addressing the limitations of Retrieval-Augmented Generation (RAG) in long-context question answering. EBG mandates models to provide answers with line-level citations and introduces the Ans+Evid metric to simultaneously evaluate correctness and grounding. The researchers reformulated two corpora into evidence-grounded multi-document QA benchmarks—MultiHop-RAG and QASPER-MP—and paired them with controlled
Background and Motivation
Retrieval-Augmented Generation (RAG) models are currently the dominant paradigm for grounding large language models in external corpora. However, their similarity-driven retrieval mechanisms and chunked processing pipelines have structural blind spots: they cannot anticipate which passages a question requires and discard document-level context needed to resolve cross-passage references.
Overview of EBG
To address these limitations, the research team introduces Evidence-Based Generation (EBG), featuring the following key innovations:
- Line-Level Citations: EBG requires models to return answers with line-level citations, ensuring traceability and credibility of the answers.
- Ans+Evid Metric: The introduction of the Ans+Evid metric allows for simultaneous evaluation of answer correctness and evidence sufficiency, avoiding biases from answer-only evaluation.
Datasets and Experimental Design
The researchers developed two new benchmark datasets:
- MultiHop-RAG: A multi-hop reasoning RAG benchmark dataset containing complex multi-document question-answering tasks.
- QASPER-MP: A multi-paper adaptation of QASPER, simulating short paper cues in real academic papers.
Additionally, they designed a test environment that includes a controlled synthetic two-hop haystack and an extended NoLiMa dataset to comprehensively evaluate EBG's performance.
Key Findings
- RAG's Limitations: EBG exposes that RAG models, in handling multi-fact queries, can only reliably surface relevant chunks rather than all the necessary ones.
- Destruction of Document Metadata: Chunking destroys document metadata, preventing the resolution of identity-bearing queries.
- Evidence Insufficiency: EBG reveals evidence insufficiency issues that are invisible to answer-only evaluation, such as incomplete citations in two-needle settings and a widening gap between Ans and Ans+Evid when context challenges parametric knowledge.
Technical Value and Impact
The EBG framework provides a more rigorous evaluation method for long-context question-answering tasks, enabling a more comprehensive assessment of a model's reasoning capabilities and evidence usage. This not only helps improve the performance of existing RAG models but also paves the way for future AI systems to handle complex reasoning tasks more effectively.
Recommendations for Developers
For developers, the introduction of the EBG framework implies that building long-context QA systems requires greater attention to evidence sufficiency and answer traceability. It is recommended to incorporate the Ans+Evid metric into model training and evaluation processes and consider using the EBG framework for more thorough performance assessments.
— END —Source: ArXiv NLP/LLM (cs.CL) (2026-10-21)
Tags: #EBG #Long-Context QA #RAG #Evidence Generation #Multi-Hop Reasoning
Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.
Community Comments