Actionable Feedback Filters Clinician Edits to Reduce LLM Hallucination
Summary:This work examines how clinician edits can be mined as supervision for clinical documentation models. In data from a deployed AI documentation product, more than 45% of edited outputs contain information not directly supported by the model input, making naive fine-tuning on raw edits a potential route to hallucination. The authors build a taxonomy of clinician-editing behaviors, generate semi-synthetic edit data for discharge-summary generation on MIMIC-III, and progressively filter for grounded
## The problem: clinician edits are not automatically usable supervision In clinical documentation workflows, an LLM generates a draft and a clinician reviews and edits it before deployment. These edits appear to offer a scalable source of supervision, but the authors' review of data from a deployed clinical documentation product found that more than **45%** of edited outputs contained user-added information not directly supported by the model input. Some edits may correct a wording error or reorganize a section. Others may introduce a clinical fact that was not present in the input. Fine-tuning directly on the final edited text teaches the model to reproduce both cases. In effect, the model can be rewarded for inventing unsupported content. The central distinction is therefore between: - **Actionable feedback**: a modification that can be derived from the input and the task constraints; - **Non-actionable feedback**: a modification that depends on external knowledge or newly supplied clinical facts. ## Method: taxonomy, synthetic edits, and grounded filtering ### 1. A taxonomy of clinician-editing behavior The authors first analyze edits collected from a deployed clinical documentation product and organize them into a taxonomy of clinician-editing behaviors. This taxonomy captures different reasons for editing and determines which edits can safely be converted into training targets. This is more reliable than treating every textual difference as equivalent. A one-phrase difference could represent an input-grounded correction, a formatting convention, or a fact supplied from outside the input. Edit distance alone cannot distinguish these cases. ### 2. Semi-synthetic edits on MIMIC-III Because raw clinician edits from commercial clinical tools are not publicly available, the study constructs a discharge-summary generation task on MIMIC-III and generates semi-synthetic clinician revisions according to the taxonomy. This design allows the authors to retain some distributional characteristics of real editing behavior without publishing patient-level records. Its main limitation is coverage: if the synthetic generator fails to reproduce omissions, wording preferences, or genuine mistakes made by clinicians, downstream filters will learn an incomplete notion of actionable feedback. ### 3. Progressively filter grounded edits The paper then applies relatively simple filtering methods to move from raw edits toward a subset of grounded edits. Training on this progressively filtered set avoids directly optimizing over the full edited-output distribution. This progressive design is important. A one-shot similarity filter can remove useful examples while leaving semantically unsupported ones. By tightening the data distribution over stages, the authors can compare the effects of raw supervision, conservative supervision, and increasingly grounded supervision. ## Technical mechanism: redefining the learning target Standard edit fine-tuning effectively asks the model to imitate the final document: \[ \mathcal{L}_{raw} = -\sum_t \log p(y_t^{edit} \mid x, y_{ **Source**: [ArXiv NLP/LLM (cs.CL)](https://arxiv.org/abs/2611.12904) (2026-11-11) — END —
Tags: #Medical LLMs #Clinical Documentation #Feedback Filtering #Hallucination Reduction #Supervised Fine-Tuning
Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.
Community Comments