Closed-Loop Agent Evaluation: Layered Measurement and Self-Calibration

Summary:This paper proposes closed-loop evaluation for deployed AI agents, treating evaluation as performance estimation under biased raters and censored outcomes. It analyzes agents across span, trace, conversation, and engagement levels, measuring execution, substance, trajectory, and results while enforcing conduct as a cross-cutting constraint. The framework combines provenance, adjudication, versioned rules, and two feedback loops: an inner loop audits evidence, rules, and metrics, while an outer

---

Why Closed-Loop Evaluation Matters

For deployed agents, a one-time benchmark run, manual trace inspection, and a single aggregate score cannot establish whether the evaluation itself is trustworthy or whether the findings improve the next agent version. This paper frames closed-loop evaluation as a continuous process: every conclusion must trace back to atomic evidence, follow versioned rules, and feed into auditing and regression checks.

The statistical perspective is important. Failures in production systems are often only partially observed. Missing tool outputs, abandoned sessions, and truncated traces create censored outcomes, while human and model raters introduce systematic bias. An aggregate score should therefore be treated as an estimate under measurement error rather than a direct measurement of agent quality.

Four Analytical Levels

The framework analyzes agents across four levels:

• Span: evaluates an individual reasoning step or tool invocation.
• Trace: examines multi-step execution chains, dependencies, failure propagation, and path efficiency.
• Conversation: assesses goal retention, context use, clarification, and recovery across turns.
• Engagement: measures task completion, sustained participation, and real-world outcomes from the user's perspective.

Each level is evaluated along four quality dimensions. Execution asks whether the agent acted as required, Substance checks whether its content actually addresses the task, Trajectory evaluates the quality of the path, and Results measures the final outcome. Conduct is not another additive dimension; it is a cross-cutting constraint. An agent that completes a task while violating safety, permission, or behavioral rules should not be considered successful.

Provenance, Adjudication, and Versioned Rules

The method decomposes each conclusion into three auditable components:

1. Provenance records the atomic observations, timestamps, transformations, and data sources behind a conclusion, rather than storing only an opaque score.
2. Adjudication resolves conflicting evidence and evaluator disagreements. It specifies who can make the final decision and under which rules.
3. Versioned rules lock metric definitions, thresholds, and aggregation logic, allowing results to remain comparable over time and separating agent regressions from evaluation changes.

Evaluator agents generated by an Agent Factory combine domain knowledge packs with adapter harnesses. Knowledge packs encode task semantics, failure modes, and domain constraints; adapters connect evaluators to different agent frameworks and telemetry formats. Compared with prompt-only scoring, this design supports component-level regression analysis: after changing a rule or knowledge pack, teams can inspect its effects on individual metrics, adjudication outcomes, and final judgments.

The Two Feedback Loops

The framework contains two loops:

• The inner loop audits evidence, rules, metrics, and evaluators, detecting missing evidence, conflicting rules, metric drift, and reviewer bias.
• The outer loop performs component-level regression checks, converts evaluation findings into agent improvements, and verifies whether a change helps at one level without damaging another.

An automated pipeline links telemetry collection, evidence normalization, rule execution, adjudication, and regression testing. The critical engineering concern is avoiding an unverified black-box scorer supervising another model. If the evaluator drifts, the outer loop can amplify the error. Rule versions, evidence snapshots, adjudication records, and regression baselines should therefore be treated as first-class artifacts rather than optional debugging metadata.

Results and Engineering Trade-offs

Using the business-analysis agent SharpInsight as a case study and porting the method to public conversation data, the paper reports that evidence-and-rule judgment is more consistent than holistic scoring: ICC improves from 0.81 to 0.86, while Cohen's kappa rises from 0.63 to 0.81.

These figures should not be interpreted as proof of production superiority. ICC and kappa measure agreement, not accuracy, task success, or business value. The migration from a business-analysis agent to public conversations also introduces a domain shift. The abstract does not report full dataset sizes, evaluator models, confidence intervals, human-review cost, or rule-maintenance overhead, leaving cross-task generalization and long-term drift unresolved.

The approach also creates operational costs. Layered and traceable evaluation increases telemetry storage, rule execution, and adjudication overhead. Versioned rules reduce drift but require explicit lifecycle management. Component-level regression improves diagnosis, yet the number of rule, metric, and evaluator combinations can grow rapidly and make testing expensive.

Deployment Recommendations

Engineering teams should begin by constructing an atomic evidence layer from production traces, tool outputs, and user feedback. They can then define a small set of critical rules before attempting full automation. Every evaluation should preserve rule versions, agent and evaluator versions, evidence hashes, and adjudication records. Dashboards should expose layer-specific metrics rather than relying on a single Agent Score.

Regression suites should monitor missing-evidence rates, evaluator agreement, rule-conflict rates, contradictions across evaluation levels, and Conduct-constraint bypasses. The outer improvement loop is only trustworthy when the inner loop can detect evaluator degradation. High-stakes deployments also need human adjudication and rollback mechanisms because an automated evaluator does not replace accountability.

Limitations

Closed-loop evaluation is most useful for long-running, tool-using production agents that evolve continuously. It improves auditability and iterative engineering, but it cannot automatically solve ambiguous task definitions, reward misspecification, or insufficient domain knowledge. If the underlying objective is wrong, a closed loop may simply propagate that error more efficiently.

---

Source: ArXiv AI (cs.AI) (2026-11-06)

— END —

Tags: #Agent Evaluation Follow #Closed-Loop Evaluation Follow #Provenance Follow #Self-Calibration Follow #Production Monitoring Follow