Agentic AI
New Training Method Teaches AI Agents to Critically Judge Their Own Actions
Researchers introduce Agentic Critical Training, a technique that trains language models to distinguish correct actions from plausible alternatives without relying on expert rationales.
A research team has proposed Agentic Critical Training (ACT), a novel approach to training language model agents that moves beyond traditional imitation learning by teaching models to actively evaluate and select correct actions from alternatives.
How Traditional Training Falls Short#
Most current approaches rely on imitation learning, which trains models to reproduce expert actions but doesn't equip them to recognize or avoid plausible mistakes. While self-reflection methods expose models to alternative possibilities, they typically still depend on supervised fine-tuning to mimic fixed explanations and actions—a limited and resource-intensive process.
The ACT Approach#
ACT employs reinforcement learning with verifiable rewards (RLVR) to train models to judge actions directly. The training process works by presenting models with pairs of actions—one from an expert and one sampled from an earlier model version—while randomizing their order. The model must generate its own reasoning and receives rewards only for correctly identifying the expert action.
Key advantages of this approach include:
- Efficient data reuse — pairs can be recycled across different model sizes
- No reference rationales needed — models learn without depending on pre-written explanations
- Simple inference — no candidate comparison required at test time
Demonstrated Results#
Testing on multiple models (Qwen3-8B and Olmo-3-7B-Instruct) across diverse environments—ALFWorld-ID, WebShop, and ScienceWorld—showed substantial improvements. ACT produced average gains of 5.85 points over standard imitation learning and 4.12 points over conventional imitation-learning-to-reinforcement-learning pipelines. On ScienceWorld with Olmo, the full pipeline delivered 15.36 points of improvement over chain-of-thought prompting.
The method also demonstrated surprising benefits beyond agent training: an ACT checkpoint that hadn't undergone reasoning-specific fine-tuning still achieved the highest average performance among compared models on MATH-500 and GPQA-Diamond, suggesting that learning to evaluate actions complements general reasoning capabilities.
Why It Matters#
By decoupling action judgment from explanation generation, ACT addresses a fundamental gap in how language models are trained to act as agents. The approach requires fewer resources than methods that rely on hand-crafted rationales while delivering stronger performance across multiple benchmarks and model sizes.

