New Method Enables Mobile LLMs to Continuously Merge Task-Specific Adapters On Device

Deploying large language models on mobile devices often requires using lightweight, task-specific modules called Low-Rank Adapters—or LoRAs—to keep storage and computational overhead manageable. But what happens when devices need to support new tasks over time without losing capabilities for older ones?

Researchers have introduced K-Merge, a new approach that addresses this challenge by intelligently selecting and combining adapters as they arrive on a device, all while respecting tight storage budgets and computational constraints.

The Core Problem#

Mobile and edge devices face a fundamental tension: they need to support multiple downstream language tasks (like translating new languages or solving novel problem types) but lack the storage to hold dozens of separate adapters. Current model-merging techniques focus on combining multiple LoRAs into a single adapter—but they assume all adapters are available upfront. In the real world, adapters arrive gradually as users request new functionality, creating an online continual merging scenario where the system must integrate new modules without degrading performance on previously supported tasks.

How K-Merge Works#

The approach is both data-free and computationally lightweight, making it practical for devices with limited resources. When a new LoRA arrives, the method determines which existing adapters to keep or discard, and how best to merge them with the incoming module. The strategy is guided by performance estimates across all supported tasks, ensuring that the device maintains capability across its full task portfolio even as storage fills up.

Real-World Validation#

Extensive testing on practical task sets shows that K-Merge outperforms simpler alternatives—such as replacing old adapters wholesale or averaging them naïvely—while still staying within the storage and computational budgets that on-device settings demand. The approach makes continual LLM deployment on mobile devices more feasible as language capabilities evolve and expand.