Lightweight AI planner learns to break down tasks on edge devices

A new framework called Octo-planner separates task planning from execution, letting AI agents operate efficiently on edge devices without relying on heavyweight cloud models. Researchers built the system by pairing a compact language model with specialized agents that can call functions, achieving a 97% success rate on their test tasks.

The Architecture#

The approach splits agent responsibilities into two parts. The planner agent—built on Microsoft's Phi-3 Mini, a 3.8 billion parameter model designed for edge deployment—takes user queries and decomposes them into logical sub-steps. A separate action agent then executes those steps by invoking functions. By splitting planning from execution, the system avoids forcing a single model to juggle both reasoning and action, allowing each component to specialize.

Training for Efficiency#

Rather than relying on in-context learning (which demands more processing and memory), the researchers fine-tuned Phi-3 Mini on carefully curated training data. They used GPT-4 to generate diverse planning examples across multiple functions, then validated the data for quality. Fine-tuning reduced computational overhead and energy draw while speeding up response times—critical for devices with limited battery or processing power.

Handling Complex Queries#

To address multi-step requests across different domains—scheduling a meeting while booking a flight, for instance—the team developed a multi-LoRA approach. This method trains separate lightweight adapter modules (LoRAs) on distinct function categories, then merges their weights together. The result is a flexible system that tackles intricate, cross-domain queries without exploding memory requirements.

The researchers have open-sourced the model weights and released a demo to support further development in on-device agent systems.