NeurIPS Desk-Rejects 178 Papers Using AI Detector Pangram, Raising Concerns
By Mr.Xu Community Post
Published: · 7 views
Summary:The NeurIPS conference used a proprietary AI detector called Pangram to automatically desk-reject 18.4% of all submissions (178 papers) without human review or appeal, sparking significant controversy. The detector exhibited high false-positive rates, flagging papers by the track chairs themselves and disproportionately impacting non-native English speakers due to rigid structural patterns in their writing being misinterpreted as AI-generated.
Overview
The NeurIPS conference introduced a proprietary AI detector called Pangram to automatically review submissions for its Position Paper Track. This tool led to the desk-rejection of 18.4% of all submissions (178 papers) without any human review or appeal process, sparking significant controversy within the academic and AI communities.
Key Issues
-
High False-Positive Rate: Independent researchers tested Pangram on recent papers authored by the track chairs and found that the detector flagged them as AI-generated at rates ranging from 24% to 69%. Under the conference's enforcement rules, these papers could have also faced rejection.
-
Overly Strict Default Settings: Pangram's default settings flagged nearly half of all submissions as 90-100% AI-generated. To reduce the rejection rate, the conference organizers had to shrink the text windows, bringing the flag rate down to a more believable 12.7%.
-
The 'Circularity Trap': 22 papers were rejected because they scored above 0.5 on the detector, despite the authors explicitly denying AI use. The black-box score was used as evidence that the authors were lying.
-
Discrimination Against Non-Native English Speakers: A Stanford study showed that 61.22% of human-written TOEFL essays are falsely flagged as AI-generated due to the rigid structural patterns in non-native formal English. NeurIPS published no demographic calibration data, meaning non-native English speakers were essentially playing a game of chance.
Recommendations
For the authors of the 178 rejected papers, there is no blacklist and no misconduct mark on their record. They are advised to resubmit their papers to ICLR (by September 25) or ICML.
Technical and Ethical Reflections
This incident highlights the limitations of AI detectors in academic review, particularly when dealing with non-native English speakers or complex texts. Over-reliance on AI can lead to misjudgments and unfair outcomes, undermining the fairness and diversity of academic research. Future AI detectors need to be more transparent and accurate, and should be combined with human review to ensure fairness and accuracy in the review process.
— END —Source: Reddit r/MachineLearning (2026-09-08)
Tags: #NeurIPS #AI Detection #Academic Review #Pangram #AI Ethics
Editorial & Fact-Checking Note: This article is compiled from primary research, official release documentation, and source papers by the ZICQ Newsroom pipeline with automated entity verification and human editorial review. If you notice any technical inaccuracy, please submit a correction via our corrections policy or email our editorial desk directly.
Community Comments