IDIA Releases DialFork: First Benchmark for Evaluating Procedural Hallucination in Conversational Agents

Summary:IDIA has introduced DialFork, the first benchmark designed to evaluate procedural hallucination in conversational agents. Procedural hallucination occurs when an agent produces fluent and plausible utterances that deviate from the intended conversational process. DialFork injects four hallucination types into dialogues, yielding 36,278 samples across 24 tasks in 12 domains, validated by 30 human annotators. It introduces dialog-flow-aware metrics, FlowKL and FlowScore, which outperform baselines

---

Background and Motivation

In recent years, large language models (LLMs) have been widely deployed as conversational agents in various scenarios. However, existing evaluation methods primarily focus on factual hallucination, overlooking another critical issue: procedural hallucination. Procedural hallucination occurs when an agent produces utterances that are fluent and plausible but deviate from the intended conversational flow or logic.

Innovations of DialFork

1. First Procedural Hallucination Benchmark: DialFork is the first benchmark specifically designed to evaluate procedural hallucination in conversational agents, filling a gap in current evaluation methodologies.
2. Diverse Hallucination Types: DialFork injects four distinct hallucination types into dialogues, generating 36,278 samples across 24 tasks in 12 domains.
3. Dialogue-Flow-Aware Metrics: DialFork introduces metrics like FlowKL and FlowScore, which assess procedural deviation by comparing how dialogues unfold rather than their content. These metrics outperform traditional semantic similarity baselines across seven encoders, demonstrating that semantic similarity is not a substitute for procedural similarity.
4. Fine-Grained Evaluation Tasks: DialFork defines three evaluation tasks: the degree of procedural deviation in a set of dialogues (for monitoring deployed agents), the degree of procedural deviation in a single dialogue (for procedural retrieval), and the turn at which a dialogue first goes off-track (for real-time guardrails).

Engineering Trade-offs and Performance

• Strengths: DialFork leverages human annotations to ensure data reliability and provides fine-grained evaluation metrics that more accurately reflect an agent's performance in complex dialogue scenarios.
• Challenges: Building a large-scale, high-quality dialogue dataset requires significant human annotation effort, and the diversity of dialogue logic across domains may affect the generalizability of the evaluation.

Developer Deployment Recommendations

• Application Scenarios: DialFork is suitable for evaluating the procedural logic capabilities of conversational agents in complex tasks, especially in scenarios where strict adherence to a process is required, such as customer service bots, virtual assistants, etc.
• Deployment Recommendations: Developers are advised to use DialFork as a regular evaluation tool during the development of conversational agents to identify and fix procedural deviation issues promptly. Additionally, combining DialFork with other evaluation metrics (such as semantic accuracy, dialogue coherence) is recommended for comprehensive assessment.

Future Outlook

The release of DialFork provides new ideas and methods for evaluating the procedural logic of conversational agents. In the future, the scope of DialFork can be further expanded, such as introducing more types of hallucinations or cross-domain evaluation tasks to enhance its generalizability and practicality.

---

Source: Hugging Face Daily Papers (2026-10-15)

— END —

Tags: #Conversational Agents Follow #Procedural Hallucination Follow #Benchmark Follow #IDIA Follow #FlowScore Follow