DS@GT ARC Generates Fact-Checking Articles with a Verdict-Anchored Pipeline
Summary:DS@GT ARC has developed a four-stage pipeline for CLEF 2026 CheckThat! Lab Task 3. It cleans noisy scraped evidence, retrieves and clusters relevant sources, conditions Llama-3.1-8B-Instruct on the original fact-checker verdict to generate the article, and restores inline citations through deterministic post-processing. The system ranked third among nine teams, achieving a mean score of 0.428 versus a 0.272 baseline. Its central design choice is to separate verdict determination from article gen
Core Idea: Let the Model Argue, Not Adjudicate
DS@GT ARC presented a four-stage fact-checking article generation pipeline for CLEF 2026 CheckThat! Lab Task 3. Its inputs are a claim, an original fact-checker verdict, and a set of scraped source documents. The output is a complete fact-checking article with inline citations.
The architecture separates automation from responsibility. The model does not independently decide whether a claim is true or false. Instead, it receives the established verdict and organizes the supporting evidence into a coherent, traceable explanation. This is more controllable than asking one model to perform verdict prediction, retrieval, and long-form writing in a single end-to-end operation, and it is better suited to newsroom workflows where a human retains editorial accountability.
How the Four-Stage Pipeline Works
The first stage cleans noisy web documents by separating substantive content from navigation, advertisements, templates, and unrelated fragments. This step matters because document boundaries and boilerplate directly affect both recall and the amount of irrelevant context sent to the generator.
The second stage retrieves and clusters relevant evidence. Clustering groups documents that discuss the same evidence or event. Beyond shortening the context window, it can prevent syndicated or near-duplicate articles from being treated as independent corroboration, which would create a misleading impression of source diversity.
The third stage conditions Llama-3.1-8B-Instruct on the original fact-checker verdict to generate the article. Verdict anchoring is the system's central constraint: the article's stance and argumentative direction must agree with the externally established verdict. The model is responsible for language organization and evidence explanation rather than truth classification.
The final stage restores inline citations through deterministic post-processing. This is more auditable than asking the language model to invent citation markers or URLs, provided that the implementation maintains consistent document identifiers, citation placeholders, and mappings between cleaned and original content. The cost is extra pipeline state and index management.
Results and Evaluation Boundaries
The system ranked third among nine teams in CLEF 2026 CheckThat! Lab Task 3, with a mean score of 0.428 compared with a 0.272 baseline. The roughly 57% relative increase indicates that the combination of evidence organization, verdict conditioning, and citation recovery improves performance on the official metric.
The result should not be interpreted as a 57% increase in overall factual accuracy. The task mean likely combines several evaluation components, and the official scoring implementation is not detailed in the abstract. With only nine teams, one benchmark, and a particular evidence distribution, external generalization remains uncertain. The summary also omits component scores, ablations, human preference evaluations, and failure-case analysis, making it difficult to isolate whether retrieval, verdict conditioning, or deterministic citation repair contributes most of the gain.
More importantly, the pipeline assumes that a valid original verdict already exists. It automates post-adjudication writing, not end-to-end fact discovery. An erroneous verdict can be expanded into a polished but incorrect article, while the model may overinterpret evidence or introduce unsupported implications. Production deployments should therefore retain human sign-off, enforce clickable citation verification, and run semantic consistency checks between claims, verdicts, and cited passages.
Engineering Trade-offs
Using Llama-3.1-8B-Instruct makes the system suitable for single-GPU or quantized deployments and avoids dependence on a large proprietary model. The tradeoff is that long-form consistency, cross-document synthesis, and nuanced argumentation are weaker than those of more capable frontier models. For an 8B model, separating adjudication from generation better matches its effective capability boundary.
The four-stage design sacrifices end-to-end simplicity in exchange for modularity and fault isolation. Cleaning errors can be inspected independently, retrieval can be replaced, and citation failures can be audited before publication. The engineering overhead includes intermediate representations, batch orchestration, and index maintenance. In an online system, cleaning and clustering can run in parallel before generation and citation recovery; in an offline workflow, cluster results can be cached to reduce repeated computation and request volume.
Practical Deployment Guidance
For newsroom or enterprise knowledge-verification deployments, teams should preserve the separation between verdict production and article generation. A rule engine, dedicated classifier, or human reviewer can first produce a structured verdict; the language model then only explains and organizes the evidence. The evidence store should retain stable document_id values, domains, publication timestamps, content hashes, and cleaning versions. Citation recovery should rely on those identifiers instead of model-generated URLs.
Before deployment, evaluate four dimensions independently: agreement between the article and the supplied verdict, entailment of generated statements by cited passages, citation resolvability, and total human-review time. Stress tests should include erroneous verdicts and conflicting sources, because performance on clean examples does not establish robustness in genuinely disputed cases. The public code provides a useful foundation for reproducing the work and implementing an internal fact-checking writing pipeline.
— END —Source: ArXiv NLP/LLM (cs.CL) (2026-11-05)
Tags: #Fact-Checking #Retrieval-Augmented Generation #Llama 3.1 #Citation Generation #Misinformation
Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.
Community Comments