Why Jailbreak Attacks on AI Models Transfer Between Systems—and How to Predict It

Attackers have long exploited a puzzling vulnerability in large language models: short, gibberish text strings appended to prompts can force AI systems to generate harmful content they're designed to refuse. Even more remarkably, these "adversarial suffixes" often work against different models and prompts than the ones they were originally crafted for—yet until now, scientists lacked a principled explanation for why.

New research from arXiv identifies three statistical markers that strongly predict whether such an attack will transfer from one model or prompt to another, providing the first rigorous framework for understanding this phenomenon.

How the Attack Works#

The approach relies on discrete optimization to generate brief, typically nonsensical strings that bypass a model's safety training. When appended to a user prompt, these suffixes manipulate the model's internal activations in ways that disable its refusal mechanisms, causing it to output text it normally wouldn't.

The Hidden Patterns of Transferability#

The researchers discovered that transfer success correlates most strongly with three internal properties:

• Baseline refusal strength: how much a model's internal "refusal direction"—the learned pattern that blocks harmful outputs—is activated by the original prompt alone
• Suffix push force: how powerfully the appended suffix nudges the model's representations away from that refusal direction
• Orthogonal shifts: the magnitude of changes the suffix induces in dimensions unrelated to the refusal mechanism

Interestingly, the semantic similarity between prompts proved to be only a weak predictor of transfer, suggesting that what matters is the low-level structure of model activations rather than what the prompts "mean."

Practical Implications#

Understanding these three properties opens a path to improving defense and attack strategies alike. In interventional experiments, the team showed how this statistical analysis can translate into concrete gains—making attacks more effective when applied to new targets, but also potentially informing better defenses that account for these vulnerabilities.

The findings represent a step toward demystifying why adversarial attacks generalize across AI systems, a critical question as models become more widely deployed.