AI
Study challenges ArXiv's review paper ban, finds more AI-generated content in non-review papers
Researchers quantify LLM-generated content across ArXiv papers, revealing the policy targeting review articles may miss a bigger problem.
ArXiv recently restricted publication of unpublished review papers in its Computer Science domain, citing concerns about widespread AI-generated content. Yet the platform offered limited data to back up this decision. A new study from researchers scrutinizes this policy by measuring the actual prevalence of LLM-generated material in both review and non-review papers.
What the analysis found#
Using two robust detection methods, the investigators documented a marked rise in AI-generated content across both paper types over recent years, with review papers showing higher rates of AI generation. However, the absolute numbers painted a different picture: non-review papers flagged as LLM-generated outnumbered their review counterparts by roughly six-fold.
The researchers also discovered that the ban's impact will vary dramatically by research area. Some Computer Science subdisciplines stand to lose substantially more papers than others. The Computers & Society domain faces potential cuts affecting up to 50% of its submissions—far more severe than the impact on other fields.
Why this matters#
The findings suggest ArXiv's approach may be solving a visible symptom while overlooking a more pervasive issue. While review papers do contain proportionally more AI-generated text, the sheer volume of compromised non-review work dwarfs the review problem. The policy's uneven harm across different CS subfields also raises questions about fairness and unintended consequences.
The researchers have released their code and analysis framework to enable further investigation, providing a template for how to evaluate such high-stakes content moderation decisions with empirical rigor.
