MedGuard Hardens Clinical Agents Against Tool Failures
Summary:Clinical agents can solve routine tasks yet still make unsafe decisions when tools return stale, malformed, or misleading data. The paper introduces MedToolFault, a fault-injection benchmark that ties six classes of tool failures to answer correctness, required state changes, and forbidden mutations in public clinical-agent tasks. It also presents MedGuard, a tool-layer defense that validates read evidence and attributes ambiguous writes through public interfaces. Across 127,861 formal episode
Clinical agents fail when the tool layer becomes unreliable
The paper’s central finding is that clinical agents which solve routine tasks can still fail dramatically under injected tool faults. Across nine model configurations, failure rates on fault-injected episodes rose to 32.2%–67.8%, even though the same configurations had solved those tasks without faults. Errors crossing a tool boundary may therefore become incorrect evidence retrieval, missing record updates, or unauthorized mutations.
MedToolFault evaluates agents as stateful systems rather than as answer generators. Each task checks whether the final answer is correct, whether required state changes occur, and whether forbidden mutations remain absent. This closes two common evaluation gaps: treating tool failures as generic network errors, and ignoring whether an agent corrupts external state while producing a plausible answer.
Fault injection tied to safety constraints
The benchmark covers six classes of tool failure and links them to answer correctness, required state changes, and forbidden mutations. The evaluation also examines fault recovery, mechanism controls, task expansion, constrained tool budgets, and external question answering, covering 127,861 formal episode evaluations.
A particularly useful metric is paired safe success. For each configuration, the system first establishes that an agent can solve a task without faults and then injects a corresponding fault. An episode counts as safe only if the answer remains correct, all required state changes are completed, and no forbidden mutation occurs. This is more informative than averaging unrelated tasks because it directly measures whether a defense preserves behavior under degradation.
MedGuard places defense at the tool boundary
MedGuard does not ask the foundation model to relearn an elaborate cautious-reasoning procedure. Instead, it adds verification around tool use: read evidence is checked for consistency with the requested action, while ambiguous writes are intercepted or attributed through the available public interfaces.
This layer can be valuable operationally. It can protect an existing agent without modifying the base model, and it places detection where infrastructure errors are most visible. The tradeoff is additional latency, tool calls, validation logic, and audit storage. Whether those costs are justified depends on the risk profile of the workflow and on results not fully described in the abstract.
Improved benchmark scores are not clinical validation
A 13.7–46.6 percentage-point increase in paired safe success is substantial, but it is not equivalent to demonstrated clinical effectiveness. The benchmark relies on public clinical-agent tasks, and its injected faults may not capture interface timeouts, data drift, permission changes, or concurrent multi-tool failures found in hospital systems.
MedGuard also depends on information exposed through public tool interfaces. Production APIs must provide data versions, timestamps, patient identifiers, consistency state, idempotency guarantees, and sufficient write preconditions. Otherwise, evidence validation and write attribution will remain incomplete. A deployment should report safe success, ordinary task completion, false refusals, average tool-call count, and end-to-end latency rather than relying on a single safety metric.
Deployment guidance
Clinical teams seeking to reproduce the approach should first establish verifiable tool contracts. Read interfaces should return provenance and freshness metadata. Write interfaces should support preconditions, idempotency, change previews, and explicit rejection reasons. Every interaction should be logged with the requested action, validation decision, and final state delta.
Regression suites should separately cover normal execution, single faults, compound faults, cross-tool state conflicts, and permission failures. Low-risk operations may remain autonomous, while high-risk writes can require deterministic validation or human confirmation. This creates a more defensible balance between throughput and safety.
Why the work matters
MedToolFault and MedGuard show that model competence and tool reliability must be measured separately. A stronger model does not eliminate stale, conflicting, or ambiguous tool responses. For agents operating in healthcare, finance, and other critical infrastructure, fault injection, state-change verification, and auditable recovery should therefore become release criteria rather than optional resilience tests.
The results demonstrate a promising tool-layer defense, but independent replication, detailed per-fault analysis, real-world hospital validation, and complete cost and latency measurements remain necessary before production deployment.
— END —Source: ArXiv NLP/LLM (cs.CL) (2026-11-10)
Tags: #Clinical Agents #Fault Injection #Tool-Use Safety #State Verification #MedGuard
Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.
Community Comments