Microsoft releases lightweight agent reinforcement learning framework that trains on real deployment harnesses

Microsoft Research Asia has unveiled a new approach to training AI agents that sidesteps a fundamental challenge in reinforcement learning systems: the need to rebuild agent logic inside training frameworks. The framework, called Agent Lightning v1.0, connects directly to the same agent orchestration layer used in production, allowing developers to improve agents without duplicating their code.

The core innovation: training on real agent systems#

Traditionally, reinforcement learning systems for AI agents have required developers to reconstruct the agent's full logic—context management, tool protocols, execution routines—inside the training framework itself. This duplication introduces both engineering overhead and a stubborn problem: the agent being trained no longer behaves identically to the one deployed in production.

Agent Lightning v1.0 inverts this pattern through what the researchers call "Harnessed Agentic RL." Rather than owning the interaction loop, the training system inserts an API proxy between the agent harness and the language model. The agent continues operating normally; developers simply redirect its model endpoint to point at Agent Lightning instead. The framework observes and records each model call, collects rewards from task execution, and drives training—all without touching the agent's internal logic.

Lightweight design and architecture#

Designed for clarity and extensibility, Agent Lightning v1.0 comprises approximately 3,500 lines of code organized around three core components:

• API Gateway: Acts as an OpenAI-compatible language model proxy, storing rollouts and linking each model call to its execution trace while recording prompts, responses, and log probabilities.
• Rollout Controller: Launches and monitors agent execution as Kubernetes jobs or local processes, keeping agent execution separate from the training pipeline.
• Customized Trainer: Gathers completed rollouts, prepares training samples through a configurable adapter, and orchestrates the learning process.

This modular separation allows the system to remain comprehensible and modifiable compared to monolithic training frameworks.

Solving practical training challenges#

Training with real harnesses surfaces four distinct technical obstacles. Sample boundaries shift when text is retokenized for training; advantage calculations must account for rollouts that spawn variable numbers of training samples; loss normalization risks being skewed by harness-dependent sample counts; and GPU scheduling must accommodate variable workload sizes that only become known after execution finishes.