AI
Pangram's AI Detection Rise Sparks Trust Questions as Literary Scandals Mount
The Brooklyn startup's tool has become publishing's de facto arbiter of AI authorship—but its track record raises serious concerns about false positives and bias.
WIRED — AI
A small Brooklyn-based startup is reshaping how the publishing industry polices AI-generated writing. Pangram, a 24-person company founded in 2023, has grown from obscurity into what many view as the authoritative voice on detecting machine-written text—despite skepticism about whether that role should exist at all.
How Pangram Rose to Prominence#
The company burst into public awareness in January when CEO Max Spero flagged Shy Girl, a self-published novel acquired by Hachette, as 78 percent AI-generated. Author Mia Ballard denied the allegation, but the label stuck; Hachette later canceled the release. Since then, Pangram scores have ignited a cascade of high-profile accusations: the New York Times' Modern Love column (100 percent), a Commonwealth Short Story Prize winner (100 percent), and a thriller that sold for $2.4 million tagged as 97 percent machine-written.
Substack recently integrated Pangram into its platform, letting readers check suspected AI content with a click. Yet that integration reflects deeper anxieties: authors fear careers can be destroyed by a single algorithmic verdict, and publishing insiders express frustration with detection tools as much as with the AI companies themselves.
The Technology Behind the Tool#
Pangram employs a method called "synthetic mirroring"—feeding human writing to large language models and comparing their output to identify patterns. The company supplements this with "hard negative mining," hunting datasets for edge cases that might fool the detector and retraining on those mistakes.
Spero emphasizes that Pangram uses properly licensed data, a rarity in AI development. However, the tool has well-documented limitations. It performs poorly on texts under 100 words, can yield different verdicts on the same passage depending on context, and—according to independent research—sometimes misidentifies work by non-native English speakers as machine-generated. A 2024 study showed that after running AI-generated text through "humanizer" tools, Pangram caught less than 4 percent of it.
Mounting Concerns Over Accuracy and Bias#
Pangram claims a false-positive rate of just 0.0041 percent with its latest model, but the company also acknowledges it errs toward declaring borderline cases human-written—meaning AI-generated work slips through roughly 41 percent of the time. This conservative approach inflates the tool's reputation for precision without fully delivering it.
