AI
G-Boost: New Framework Lets Weak Edge AI Models Tap Into Powerful Cloud AI to Boost Performance
Researchers propose an adaptive system where small language models on edge devices can collaborate with larger cloud-based models to improve reasoning without exposing private data or updating parameters.
A team of researchers has unveiled G-Boost, an edge-cloud architecture designed to enhance the capabilities of smaller, resource-constrained language models deployed on edge devices—without requiring any updates to the model's parameters.
How It Works#
The system enables a small language model (SLM) running on an edge device to intelligently collaborate with a larger language model (LLM) hosted in a private cloud. Rather than always relying on the smaller model or always fusing with the larger one, G-Boost makes step-by-step decisions about when and how to collaborate.
At each reasoning step, the framework chooses between two approaches:
- SLM-only inference: The edge device handles the computation independently
- SLM-LLM logit fusion: Domain knowledge flows from the adapted SLM to the cloud LLM through logit sharing, preserving privacy by avoiding direct data exposure
A process reward model running in the cloud guides this decision-making process, using Monte Carlo tree search to dynamically identify which collaboration steps will be most beneficial.
Architecture and Privacy#
The division of labor keeps computational overhead minimal on edge devices. The edge device runs the SLM and manages the search controller, while the cloud hosts the LLM and reward model. The two systems exchange only the current context needed for decision-making, reducing bandwidth and exposure risk.
Performance Results#
Testing on mathematical reasoning benchmarks—GSM8K and MATH-500—with Qwen2.5 and LLaMA2 models showed meaningful gains. G-Boost outperformed several baseline approaches:
- 8.6 percentage points above standard Monte Carlo tree search (MCTS)
- 10.7 percentage points above fine-tuned proxy methods (Proxy-Tuning)
- Stronger results than the SLM alone or simple static fusion strategies
Why It Matters#
The framework addresses a practical constraint in modern AI systems: resource-limited edge devices often need better reasoning capacity, but fine-tuning models in place is costly and impractical. By enabling dynamic, reward-guided collaboration between differently-sized models, G-Boost demonstrates how to extract greater utility from existing deployments while safeguarding sensitive data through careful control of what information is shared between edge and cloud.
