New framework speeds up multimodal AI training by 70% while boosting performance

Researchers have unveiled a novel approach to a persistent challenge in machine learning: selecting which training data actually matters. The method, called PRISM, addresses redundancy in visual instruction datasets used to adapt pre-trained multimodal large language models (MLLMs) to follow human commands.

The efficiency problem#

As vision-language datasets have grown, they've accumulated substantial redundancy that drives up computational costs during model tuning. Existing selection methods try to filter out this noise, but they typically rely on expensive inference or training-based metrics—essentially, they solve the efficiency problem by creating another one. This creates a catch-22: the process meant to save compute ends up consuming most of it.

How PRISM works differently#

The breakthrough lies in recognizing a previously overlooked structural issue: visual feature distributions exhibit anisotropy, an uneven directional quality that causes what researchers call global semantic drift. This drift corrupts the ability of current selection methods to identify truly useful training examples.

PRISM takes a different approach by modeling the core visual semantics through a technique called implicit re-centering. This surgically removes the distorting influence of background features—eliminating the noise source rather than trying to detect it after the fact.

Results and impact#

The efficiency gains are substantial: PRISM reduces the total end-to-end time for data selection and model tuning to just 30% of conventional pipelines—a 70% reduction. More striking is that this speed-up comes with a performance boost. Models trained on PRISM-selected datasets outperformed those fine-tuned on complete datasets across eight multimodal benchmarks and three language understanding tasks, achieving a 101.7% relative improvement over baseline approaches.

The code for PRISM has been made available for research use, opening the method to the broader community working on efficient multimodal AI development.