New Method Maps LLM Weaknesses by Capturing Prompt Specificity, Not Just Similarity

Most techniques for analyzing large language model weaknesses rely on converting prompts into vector embeddings and clustering them. Yet this approach overlooks a crucial distinction: prompts on the same topic can vary widely in how specific—and therefore how difficult—they are to handle. A research team has now developed a new framework to address this gap.

The Problem with Vector Embeddings#

Standard vector embeddings excel at capturing whether prompts belong to related topics. However, they treat "write a story" and "write an adventure story set in medieval times with three main characters" as similarly difficult prompts, even though the second is far more constrained and specific. This similarity masks important variation in how language models actually perform, making it harder to pinpoint what they struggle with.

Introducing Prompt2Box#

The team proposes Prompt2Box, a method that places prompts into a box embedding space instead of traditional vector space. Rather than a single point representing each prompt, a box captures both semantic content and a specificity relation—an ordering that shows whether one prompt is more narrowly defined than another. The system uses a trained encoder developed on both real and synthetically generated prompt datasets to produce these richer representations.

To make these embeddings practical for exploration, the researchers also developed a new technique for reducing dimensionality in box embeddings, enabling researchers to visualize and compare whole datasets.

Measurable Improvements#

Experiments show the approach works. Prompt2Box reduced errors by roughly 45% on average when predicting how specific a prompt is, compared to simply using prompt length as a proxy. When applied to hierarchical clustering of 17 language models evaluated on the UltraFeedback dataset, the method identified 13.5% more distinctive model weaknesses than vector-based methods and achieved approximately 33% stronger correlation between how deep a prompt sits in the hierarchy and how specific that prompt actually is.

Why This Matters#

Understanding where and why language models fail is central to improving them. By distinguishing prompts not just by topic but by the degree of specification they require, this approach enables researchers to build more nuanced maps of model capabilities and limitations—ultimately supporting more targeted improvements.