New technique enables efficient communication between AI models with different architectures

A team of researchers has developed Vision Wormhole, a novel framework that enables different AI models to communicate more effectively by encoding information into visual representations rather than relying on text-based exchanges.

The Problem with Heterogeneous Agent Systems#

When combining multiple AI models with varying architectures into a single system, they typically need a shared communication method. Current approaches require translating between the unique internal representations of each model—a process that adds computational overhead and information loss. Vision Wormhole sidesteps this challenge by leveraging an interface that most modern AI models already use: their visual input capability.

How Vision Wormhole Works#

The approach employs a Universal Visual Codec that operates in a hub-and-spoke architecture. For each model in the system, a dedicated codec learns to:

• Encode the model's internal states into a compact, fixed-size visual message
• Map that message through a shared reference space
• Decode incoming messages back into tokens the receiving model can process

Rather than requiring full retraining, each model learns its codec independently through self-distillation on anchor texts—allowing the system to reuse codecs across different communication partners. This design scales efficiently, requiring only O(N) components for N models.

Performance Gains#

Testing across four different vision-language model families, six team configurations, and nine reasoning benchmarks showed measurable improvements:

• 6.0 percentage point average accuracy improvement over text-based multi-agent systems
• 1.69× geometric-mean speedup in end-to-end runtime when accounting for batch normalization

The results suggest that visual encoding preserves more task-relevant information than standard text exchanges while reducing computational requirements during agent communication.