ArXiv Releases GazeToSkill: Predicting Radiologist Expertise from 3D Gaze Patterns
Summary:The ArXiv team introduces GazeToSkill, a novel framework that integrates 3D gaze patterns of radiologists with the DINOv2 backbone to assess expertise in thoracic CT interpretation. By incorporating visual search behavior through learnable log-space biases and gaze-weighted pooling mechanisms, the model achieves high performance in ROC-AUC and F1 scores on a test set of 182 CT reading sessions, outperforming traditional methods. This research demonstrates the potential of combining visual behavi
Background and Motivation
In medical imaging analysis, the expertise of radiologists is crucial for accurate interpretation of 3D images. Traditional assessment methods rely on manual observation and subjective judgment, making it difficult to achieve objective and efficient skill evaluation.
Methodology and Innovation
The GazeToSkill framework proposed by the ArXiv team achieves the prediction of radiologist expertise through the following approaches:
- 3D Gaze Pattern Capture: Utilizes eye-tracking technology to capture the gaze points of radiologists while interpreting CT images.
- DINOv2 Backbone: Integrates 3D gaze patterns with the DINOv2 backbone to extract deep features of the images.
- Learnable Log-Space Bias: Introduces a learnable log-space bias into the self-attention mechanism to enhance the model's understanding of visual search behavior.
- Gaze-Weighted Pooling: Incorporates gaze point data into the feature aggregation process through a gaze-weighted pooling mechanism.
Experiments and Results
The research team conducted training and testing on a dataset of 182 CT reading sessions, and the results showed that:
- ROC-AUC Score: The GazeToSkill model achieved a ROC-AUC score of 0.91, significantly outperforming traditional methods.
- F1 Score: The model achieved an F1 score of 0.86, also demonstrating excellent performance.
Technical Highlights
- Integration of Visual Behavior with AI Models: For the first time, 3D gaze patterns are combined with the Transformer model, enabling more accurate prediction of expertise.
- Learnable Log-Space Bias: Enhances the model's understanding of visual search behavior, improving prediction accuracy.
- Gaze-Weighted Pooling: Effectively integrates gaze point data with image features, further enhancing model performance.
Industry Impact and Future Directions
The GazeToSkill framework provides a new method for skill assessment in medical imaging analysis, with the following potential applications:
- Doctor Training and Assessment: Provides an objective basis for the training and skill assessment of radiologists.
- AI-Assisted Diagnosis: Enhances the assistive capabilities of AI systems in medical imaging analysis.
- Interdisciplinary Research: Offers new research directions for the integration of visual behavior with AI models.
In the future, the research team plans to further optimize the model and explore its applications in other medical imaging fields.
— END —Source: ArXiv cs.AI (2026-08-24)
Tags: #ArXiv #Medical Imaging #Transformer #Eye-Tracking #Skill Assessment
Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.
Community Comments