arXiv Releases MedJev: A Fast, Local Medical Decision Model Approaching GPT-4 Performance
Summary:arXiv has released MedJev, an open system model designed for medical decision support that can run locally on a laptop. MedJev demonstrates strong performance across multiple medical benchmarks, including MedQA (72.5%), USMLE (75.4%), and NEJM (80.1%), approaching GPT-4's capabilities while maintaining well-calibrated confidence estimates (ECE 0.028 to 0.063). In patient-level tasks, MedJev achieves an AUROC of 0.926 for in-hospital mortality prediction, outperforming XGBoost, but underperforms
Technical Mechanism Analysis
MedJev is a medical AI tool based on System 1 decision models, which answers structured questions in a single forward pass without generating text. This design offers significant computational efficiency advantages, making it suitable for clinical decision support scenarios that require rapid responses.
Engineering Trade-offs and Performance
MedJev demonstrates performance close to GPT-4 across multiple medical benchmarks:
- MedQA: 72.5% accuracy
- USMLE: 75.4% accuracy
- NEJM: 80.1% accuracy
In patient-level tasks, MedJev achieves an AUROC of 0.926 for in-hospital mortality prediction, outperforming XGBoost (0.902), but underperforms supervised models in operational outcomes like ED triage admission (0.745 vs. 0.809). Additionally, MedJev exhibits substantial miscalibration in patient-level predictions (In Hospital Mortality ECE 0.286), but a lightweight two-parameter recalibration method reduces ECE to 0.007 using 50 to 200 labeled examples.
Developer Implementation and Deployment Recommendations
MedJev's lightweight design allows it to run locally on a laptop without relying on cloud servers, providing higher privacy protection and lower latency for medical AI applications. For developers, the code and evaluation harness for MedJev are open-sourced, facilitating further research and optimization. The recalibration method also offers a new approach to handling model calibration issues, particularly in scenarios requiring localized adjustments.
Conclusion
MedJev demonstrates the potential of open system models in medical AI, combining competitive predictive performance with low computational cost and efficient local calibration. Although it underperforms supervised models in some operational outcomes, its overall performance approaches GPT-4, providing a new technical option for medical AI applications.
— END —Source: ArXiv NLP/LLM (cs.CL) (2026-10-28)
Tags: #MedJev #Medical AI #System 1 Decision Models #Local Execution #Open Source Model
Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.
Community Comments