ArXiv Introduces R2VC: A Modular Architecture for Evidence-Based Fact-Checking with Retrieval, Verification, and Calibra
Summary:ArXiv has introduced R2VC, a novel modular architecture for automated fact-checking that separates evidence retrieval, reasoning, and uncertainty estimation to enhance predictive accuracy and confidence reliability. In tests on the FEVER dataset, an 8B backbone with R2VC achieved a 13.74% higher accuracy compared to baseline methods. Ablation studies highlight the critical role of verifier-based candidate selection and confidence calibration, while retrieval failures, especially those involving
Background and Challenges
As large language models (LLMs) are increasingly applied to automated fact-checking, traditional end-to-end prompting methods often entangle evidence retrieval, reasoning, and uncertainty estimation, making it difficult to diagnose errors and trust confidence scores. R2VC aims to address this by introducing a modular architecture that separates these tasks, thereby enhancing overall performance and reliability.
Core Architecture of R2VC
R2VC's architecture consists of four key modules:
- Retrieval Module: Utilizes hybrid sparse+dense retrieval over Wikipedia to efficiently extract relevant evidence from vast amounts of text.
- Reasoning Module: Employs a supervised fine-tuned and DPO-aligned generator to produce diverse structured verdict candidates.
- Verification Module: Uses an external NLI cross-encoder to select candidates based on evidence.
- Calibration Module: Implements a lightweight sequence-level calibrator for confidence estimation and selective abstention.
Experimental Results and Performance
In tests on the FEVER dataset, R2VC's 8B backbone model achieved a 13.74% higher accuracy compared to baseline methods. Ablation studies indicate that candidate selection and calibration mechanisms are the main contributors to this performance improvement. Removing candidate selection drops the FEVER accuracy to 76.24%, while removing calibration nearly doubles the Brier score to 0.161.
Key Findings and Bottlenecks
A manual analysis of 250 errors reveals that retrieval failures, particularly those involving incorrect entities, remain the primary bottleneck. This suggests that while R2VC has made significant strides in overall performance, there is still room for improvement in the accuracy of evidence retrieval.
Industry Impact and Future Directions
The introduction of R2VC provides new insights and methods for the field of automated fact-checking. Its modular design not only enhances predictive accuracy but also improves confidence reliability, offering AI researchers and developers a more powerful tool. As retrieval technology continues to advance, R2VC is expected to play a significant role in a wider range of applications and scenarios.
— END —Source: ArXiv NLP/LLM (cs.CL) (2026-09-14)
Tags: #LLMs & Foundation Models #Fact-Checking #Modular Architecture #NLP #ArXiv
Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.
Community Comments