Recursive Semantic Divergence: Enhancing Consistency in LLM Agent Behavior
Summary:This research introduces Recursive Semantic Divergence (RSD), a novel metric for evaluating the behavioral consistency of Large Language Model (LLM) agents during task execution. RSD measures the semantic disagreement between independent rollouts of the same agent from the same state, capturing logical inconsistencies and providing turn-by-turn consistency scores. By using RSD as an additional reward signal for fine-tuning, experiments demonstrate a significant reduction in task progress variati
1. Background and Motivation
Large Language Model (LLM) agents often exhibit inconsistent behavior when using tools, such as calling different tools, producing contradictory outputs, or pursuing divergent strategies across independent executions. This inconsistency accumulates over time, undermining the reliability of the agent in deployment. Existing evaluation metrics primarily focus on task completion rather than capturing behavioral consistency.
2. Recursive Semantic Divergence (RSD) Method
RSD is an unsupervised trajectory-level consistency metric adapted from bisimulation metrics in reinforcement learning. It measures the expected cumulative semantic disagreement between two independent rollouts of the same agent from the same state, using natural language inference to detect logical inconsistencies. RSD is defined recursively over future states and approximated by a neural network trained on independent rollouts. The metric estimates future divergence from the current state, providing a consistency score at each turn without requiring the full trajectory to complete.
3. Experiments and Results
In experiments on tau2-bench and toolsandbox benchmarks, RSD as a reward signal for fine-tuning reduced the variation in task progress from 132% to 91% and from 90% to 44%, respectively. Additionally, RSD improved performance across multiple benchmarks without requiring ground-truth labels during training.
4. Technical Mechanism Analysis
- Core Mechanism: RSD captures semantic divergence across different execution paths to quantify behavioral consistency.
- Implementation: Utilizes natural language inference models to detect logical inconsistencies and approximates future state divergence using neural networks.
- Advantages: Does not require ground-truth labels and can dynamically adjust consistency scores during training.
- Limitations: High computational overhead, especially for long trajectories.
5. Engineering Trade-offs and Deployment Recommendations
- Computational Overhead: RSD's computational complexity is high; it is recommended to use in resource-abundant training environments.
- Applicable Scenarios: Suitable for tasks requiring high behavioral consistency, such as dialogue systems and decision-making agents.
- Optimization Strategies: Pruning techniques or approximation methods can reduce computational overhead while maintaining evaluation accuracy.
6. Conclusion
RSD offers a new approach to evaluating agent behavioral consistency, demonstrating its potential to enhance the reliability of LLM agents in deployment by reducing task progress variation and improving model performance.
— END —Source: ArXiv NLP/LLM (cs.CL) (2026-10-27)
Tags: #LLMs & Foundation Models #Consistency Evaluation #Reinforcement Learning #Agentic #Behavioral Consistency
Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.
Community Comments