Oncothresh: Open-source Python Library for Evaluating Oncology AI Models at Clinical Decision Thresholds

Summary:Oncothresh is an open-source, lightweight Python library designed to evaluate oncology AI models specifically at clinical decision thresholds. It addresses the limitations of existing benchmarks by providing metrics such as sensitivity, specificity, PPV, and NPV at exact cutoffs, along with advanced features like bootstrap confidence intervals, threshold-sensitivity curves, boundary-weighted calibration, decision-curve net benefit, and number-needed-to-test. Additionally, Oncothresh includes a c

---

Background and Motivation

In the evaluation of oncology AI models, existing classification metrics (such as AUC, ICC, MAE) typically focus on global agreement but fail to address the critical question at the point of care: how reliable is the model at the exact cutoff that determines whether a patient should be flagged for further examination or treatment? To address this gap, Omkar Adhali developed Oncothresh.

Key Features and Technical Highlights

1. Evaluation at Specific Clinical Thresholds: Oncothresh focuses on evaluating model performance at specific clinical decision thresholds, providing key metrics such as sensitivity, specificity, PPV, and NPV.
2. Advanced Analysis Features: It supports advanced analyses like bootstrap confidence intervals, threshold-sensitivity curves, boundary-weighted calibration, and decision-curve net benefit.
3. No-Code Web Dashboard: Users can quickly generate analysis reports by uploading CSV files without writing code.
4. Lightweight Dependencies: The library has minimal dependencies, primarily based on numpy, scipy, scikit-learn, and pydantic, making it suitable for rapid deployment.
5. Use Cases: It is applicable to tasks such as tumor cellularity, Ki-67, TMB, and PD-L1 scoring, where continuous model outputs are collapsed into binary clinical decisions at fixed cutoffs.

Industry Impact and Developer Recommendations

The release of Oncothresh provides a more reliable evaluation tool for oncology AI models in clinical settings, particularly for tasks requiring precise thresholds. Here are some recommendations:

• Clinicians: Can use Oncothresh's web dashboard to quickly assess AI model performance in specific clinical scenarios.
• AI Developers: Can integrate Oncothresh into existing model evaluation workflows to enhance clinical applicability.
• Researchers: Can build upon Oncothresh to develop new evaluation metrics or extend its functionality to accommodate more complex clinical needs.

Future Outlook

Although currently at version v0.1, the author welcomes community feedback to improve its features and expand its application scope. Future versions may include support for more clinical scenarios, more sophisticated analysis tools, and integration with other AI models.

Source

Oncothresh GitHub Repository Oncothresh-web GitHub Repository

---

Source: Reddit r/MachineLearning (2026-08-14)

— END —

Tags: #Oncothresh #Open-source Library #AI in Healthcare #Clinical Decision #Python