DS@GT ARC Builds a Four-Stage Pipeline for Fact-Checking Article Generation

Summary:The DS@GT ARC team developed a four-stage fact-checking article generation pipeline for CLEF 2026 CheckThat! Lab Task 3. It cleans noisy web evidence, retrieves and clusters relevant sources, generates an article with Llama-3.1-8B-Instruct conditioned on the original fact-checker verdict, and restores inline citations through deterministic post-processing. The system achieved a mean score of 0.428 and ranked third among nine teams, compared with a 0.272 baseline. Its main engineering contributi

---

A Four-Stage Pipeline for Traceable Fact-Checking Articles

The DS@GT ARC team developed a system for automatically generating fact-checking articles for CLEF 2026 CheckThat! Lab Task 3. Its inputs include a claim, an original fact-checker verdict, and a collection of scraped source documents. Its output is a complete article with inline citations.

The pipeline has four stages:

1. Evidence cleaning: It removes noisy content from scraped documents and retains material that may support the argument.
2. Source retrieval and clustering: It retrieves documents related to the claim and verdict, then clusters similar sources so that redundant evidence can be consolidated.
3. Conditional article generation: It uses Llama-3.1-8B-Instruct to generate the article while conditioning on the original fact-checker verdict. This separates truth adjudication from explanatory writing and avoids asking the language model to invent a verdict from scratch.
4. Deterministic citation recovery: After generation, it restores inline citations through deterministic post-processing based on evidence clusters and source identifiers rather than relying entirely on the model to cite correctly while writing.

The main engineering contribution is not a larger model but a reduction in the language model's responsibility. The 8B model handles article organization, while the input supplies the verdict and a deterministic program handles citation mapping.

Engineering Value and Limitations

Conditioning generation on an existing verdict can reduce the risk that the model fabricates whether a claim is true or false. It does not, however, guarantee that the article faithfully expresses that verdict. A system may strengthen “insufficient evidence” into “mostly true” or convert a qualified conclusion into an absolute claim.

Using Llama-3.1-8B-Instruct also avoids dependence on a large proprietary API and may reduce deployment costs. However, the paper abstract does not disclose context length, quantization, GPU configuration, throughput, or per-article latency, making production cost estimates difficult.

The four-stage design also means that end-to-end latency is determined by cleaning, retrieval, clustering, inference, and citation recovery. Even if the 8B model is fast, web-document processing and retrieval may become the bottleneck. Production systems should profile each stage separately and cache cleaned and clustered evidence for repeated claims.

Why Deterministic Citation Recovery Is Useful

The important property of a fact-checking article is not merely that it contains links, but that each cited source supports the corresponding claim. Recovering citations after generation is easier to audit than asking the model to produce source references freely during decoding.

Citation presence, however, is not the same as citation correctness. At least three constraints remain:

• A cited document must exist in the candidate evidence set.
• The referenced passage must contain evidence for the claim rather than merely being topically related.
• Multiple citations must jointly support the conclusion without cherry-picking sources or overlooking conflicting evidence.

If post-processing relies only on paragraph positions or sentence similarity, it solves formatting and link mapping rather than entailment, source conflicts, or evidence strength. A stronger design could add a lightweight NLI model or evidence-quality classifier, producing a layered sequence of candidate retrieval, deterministic alignment, and entailment verification.

Interpreting the Competition Result

The system ranked third among nine teams with a mean score of 0.428, compared with a baseline mean score of 0.272. The absolute gain is 0.156, or roughly 57.4% relative to the baseline, indicating a substantial improvement over the baseline approach.

The 0.428 absolute score also shows that the system remains well below the standard required for direct publication without review. Ranking can be affected by the number of teams, scoring rules, test-set size, and baseline implementation. If teams used different retrieval systems, prompts, or compute budgets, the leaderboard should not be interpreted as a fully controlled comparison of methods.

Competition results also cannot replace error analysis. A more informative evaluation should separately measure verdict adherence, unsupported assertions, citation precision, citation recall, source diversity, and robustness under different languages and noise levels in scraped evidence.

Deployment Recommendations

The approach is suited to assisted writing when a professional verification system has already established the verdict. It is not a substitute for letting an LLM independently determine whether a claim is true.

For practical deployment:

• Represent the original verdict as a structured field that distinguishes true, false, misleading, unsupported, and insufficient evidence while preserving qualifiers.
• Preserve mappings among cleaned evidence, clusters, source URLs, and generated sentences for auditing and regeneration.
• Add citation-entailment checks for key claims, and surface conflicting evidence instead of silently selecting one source.
• Add human approval gates for high-risk domains such as medicine, elections, and public safety.
• Stress-test the system with varying web noise, duplicated sources, and cross-lingual claims.
• Measure costs stage by stage instead of attributing the entire pipeline's expense to LLM inference.

This work presents a pragmatic division of labor: a professional system adjudicates the facts, an LLM explains the evidence, and deterministic software maps citations. It can reduce the cost of writing fact-checking articles, but it cannot replace evidence review or editorial accountability.

---

Source: ArXiv NLP/LLM (cs.CL) (2026-11-06)

— END —

Tags: #Fact-Checking Follow #Llama-3.1-8B Follow #RAG Follow #Citation Recovery Follow #Information Retrieval Follow