Researchers map 129 LLM prompt datasets to unlock linguistic patterns and boost AI efficiency

Researchers have assembled and analyzed a vast collection of 129 publicly available LLM prompt datasets, totaling over 1.22 terabytes and containing more than 673 million prompt instances. Their work establishes a unified framework for organizing this data and uncovers linguistic patterns that distinguish prompts from ordinary text.

Taxonomy and Dataset Scale#

The researchers created a structured taxonomy to classify and organize the enormous volume of prompt data available to the research community. This cataloguing effort provides a foundation for understanding how different types of prompts are constructed and used in practice.

Linguistic Features and Efficiency Gains#

Through detailed analysis of seven representative datasets, the team identified distinctive lexical, syntactic, and semantic patterns in prompts. Notably, they developed a 63-dimensional linguistic feature set that can run on standard CPU hardware, achieving over 91% of the performance of GPU-based sentence embeddings while reducing latency by nearly half per request. This efficiency gain is significant for production systems handling high-volume prompt processing.

Practical Applications#

The extracted features proved valuable across three key tasks:

• Prompt filtering — identifying and categorizing different prompt types
• Source domain routing — directing prompts to appropriate models based on origin
• Response quality analysis — predicting the caliber of model outputs

In cross-dataset routing experiments, sentence embeddings reached a macro-F1 score of 0.74, while a structural feature subset achieved 0.61, substantially outperforming the 0.20 chance baseline.

Response Quality Insights#

When analyzing what influences elicited response quality on the UltraFeedback dataset, structural routing features remained negatively associated with quality even after controlling for prompt length. Meanwhile, the apparent positive correlation between type-to-token ratio and response quality diminished considerably under the same statistical controls, suggesting more nuanced relationships than initially apparent.