AI
New method lets AI models selectively ignore or focus on different input features via natural language
Researchers introduce FiLoRA, a parameter-efficient technique that uses instructions to control how multimodal AI systems rely on different internal feature pathways.
Controlling exactly how machine learning models process and weigh different inputs has long been a challenge. Researchers have typically resorted to analyzing models after the fact or modifying training data, neither of which directly shapes the internal decision-making process. A new approach, called FiLoRA (Focus-and-Ignore LoRA), aims to change that by enabling instruction-based control over which features a multimodal foundation model emphasizes.
How FiLoRA works#
The technique builds on low-rank adaptation—a parameter-efficient fine-tuning method—by decomposing it into modules that align with specific features. Crucially, it introduces instruction-conditioned gating, which treats natural language prompts as computational signals that modulate how the model uses different internal representations. This means a user can instruct the model to focus on certain information pathways or suppress others, all without changing the underlying task or objective.
What the research shows#
Tests across both controlled classification and generative multimodal tasks demonstrate that FiLoRA reliably shifts feature reliance in interpretable ways. When given different instructions—ranging from straightforward directives to compositional prompts—the model consistently amplifies or dampens different feature groups as instructed, while maintaining task accuracy. The shifts occur at the internal representation level, not merely by changing outputs.
Why it matters#
The ability to intervene directly on how models process information offers a new angle on model controllability and interpretability. Rather than remaining limited to output-level prompting or post hoc analysis, researchers and practitioners can now steer model behavior from within. This could help identify and mitigate spurious shortcuts or unwanted biases in multimodal systems, and provide a practical mechanism for understanding and controlling foundation model behavior at scale.
