New Dataset Enables Detection of AI-Written Text at Character Level

A newly developed dataset aims to strengthen the detection of text generated by large language models, addressing a growing challenge as AI-written content becomes increasingly difficult to distinguish from human writing.

The Problem with Current Detection Tools#

Existing systems for identifying AI-generated text suffer from significant limitations. Most rely on training data created with older models, are skewed heavily toward English, and fail to handle the increasingly realistic scenario where human and machine writing are blended together in a single document. Even rarer are datasets that can pinpoint the exact locations—down to individual characters—where AI contributions begin and end within a text.

Introducing LLMTrace#

Researchers have released LLMTrace, a large-scale dataset designed to overcome these gaps. The corpus spans two languages (English and Russian) and was constructed using both proprietary and open-source language models currently in use. This bilingual approach and reliance on recent models make it substantially more relevant to modern detection needs than earlier benchmarks.

What It Enables#

The dataset supports two critical detection tasks. The first is the traditional approach: classifying whether an entire piece of text was written by a human or a machine. More importantly, the second task allows models to perform AI-generated interval detection—identifying not just whether AI was involved, but precisely where within a document the machine-generated passages are located. This fine-grained capability relies on character-level annotations that mark the exact boundaries of AI-written segments.

Why It Matters#

As LLMs grow more capable and their use becomes widespread, the ability to detect and localize AI-generated content becomes increasingly vital. LLMTrace fills a critical gap by providing training material that reflects real-world conditions: multilingual text, contemporary models, and mixed authorship scenarios. The researchers believe the dataset will support the development of the next generation of AI detection systems that are both more accurate and more practically useful.