Training language models to respect instruction hierarchies improves AI safety and control

When large language models operate in high-stakes contexts, they often receive competing directives—a user request might contradict safety guidelines, or a user query might clash with system-level constraints. A new research approach tackles this by teaching models to explicitly reason through instruction conflicts before generating responses.

The core problem#

LLM systems in production must reconcile instructions from multiple sources. Without a clear hierarchy, models can be tricked into ignoring safety rules or executing unintended actions through prompt injection or jailbreak techniques. The challenge lies in making these decisions predictable and reliable, especially when instructions pull in different directions.

How reasoning helps#

Rather than simply hardcoding priority rules, researchers reframed the challenge as a reasoning problem. Models first "think" through the relationship between a user request and higher-priority directives, then compose their response. This approach mirrors human decision-making: pause, reflect on constraints, and act accordingly.

To enable this capability, the team created VerIH, a training dataset of constraint-following tasks where answers can be objectively verified. The dataset includes scenarios where system instructions and user prompts align and cases where they conflict. Using lightweight reinforcement learning, the researchers showed this targeted training transfers reasoning abilities to the broader task of prioritizing instructions.

Performance gains and robustness#

The method delivered consistent improvements across multiple model families:

• ~20% absolute improvement in scenarios where instructions conflict
• Up to 20% reduction in jailbreak and prompt injection attack success rates
• Improved generalization to safety-critical scenarios beyond the training distribution

The benefits suggest that reasoning-based instruction resolution isn't simply memorizing specific conflicts—the models develop a more robust capacity to handle novel hierarchical decisions they haven't seen before.

Implications for AI reliability#

These findings establish reasoned instruction hierarchies as a practical lever for improving AI controllability. Rather than requiring extensive system prompt rewrites or architectural changes, targeted updates to how models think through competing directives produce predictable and robust behavioral shifts. This work points toward more reliable AI systems where decision-making remains transparent and aligned with intended priorities.