ZICQ
中 Log in / Sign up
Newsroom Research & Papers #ArXiv #Medical Imaging #Transformer #Eye-Tracking #Skill Assessment

ArXiv Releases GazeToSkill: Predicting Radiologist Expertise from 3D Gaze Patterns

Avatar of Mr.Xu

By Mr.Xu Compiled & Reviewed by Editorial

Published:

中文阅读 (Chinese) English Version
Co-reading Select any sentence in the article to highlight, annotate, or copy a quote card. Join discussion ↓

Summary:The ArXiv team introduces GazeToSkill, a novel framework that integrates 3D gaze patterns of radiologists with the DINOv2 backbone to assess expertise in thoracic CT interpretation. By incorporating visual search behavior through learnable log-space biases and gaze-weighted pooling mechanisms, the model achieves high performance in ROC-AUC and F1 scores on a test set of 182 CT reading sessions, outperforming traditional methods. This research demonstrates the potential of combining visual behavi


Background and Motivation

In medical imaging analysis, the expertise of radiologists is crucial for accurate interpretation of 3D images. Traditional assessment methods rely on manual observation and subjective judgment, making it difficult to achieve objective and efficient skill evaluation.

Methodology and Innovation

The GazeToSkill framework proposed by the ArXiv team achieves the prediction of radiologist expertise through the following approaches:

  1. 3D Gaze Pattern Capture: Utilizes eye-tracking technology to capture the gaze points of radiologists while interpreting CT images.
  2. DINOv2 Backbone: Integrates 3D gaze patterns with the DINOv2 backbone to extract deep features of the images.
  3. Learnable Log-Space Bias: Introduces a learnable log-space bias into the self-attention mechanism to enhance the model's understanding of visual search behavior.
  4. Gaze-Weighted Pooling: Incorporates gaze point data into the feature aggregation process through a gaze-weighted pooling mechanism.

Experiments and Results

The research team conducted training and testing on a dataset of 182 CT reading sessions, and the results showed that:

  • ROC-AUC Score: The GazeToSkill model achieved a ROC-AUC score of 0.91, significantly outperforming traditional methods.
  • F1 Score: The model achieved an F1 score of 0.86, also demonstrating excellent performance.

Technical Highlights

  • Integration of Visual Behavior with AI Models: For the first time, 3D gaze patterns are combined with the Transformer model, enabling more accurate prediction of expertise.
  • Learnable Log-Space Bias: Enhances the model's understanding of visual search behavior, improving prediction accuracy.
  • Gaze-Weighted Pooling: Effectively integrates gaze point data with image features, further enhancing model performance.

Industry Impact and Future Directions

The GazeToSkill framework provides a new method for skill assessment in medical imaging analysis, with the following potential applications:

  • Doctor Training and Assessment: Provides an objective basis for the training and skill assessment of radiologists.
  • AI-Assisted Diagnosis: Enhances the assistive capabilities of AI systems in medical imaging analysis.
  • Interdisciplinary Research: Offers new research directions for the integration of visual behavior with AI models.

In the future, the research team plans to further optimize the model and explore its applications in other medical imaging fields.


Source: ArXiv cs.AI (2026-08-24)

— END —

Tags: #ArXiv #Medical Imaging #Transformer #Eye-Tracking #Skill Assessment

Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.

What's your take on this story?
🚀 Explore & Submit Works

Community Comments

Loading live comments and annotations…