Researchers Introduce SafeCoDe to Make AI Vision Models Better at Context-Aware Safety Decisions

Multimodal large language models (MLLMs)—systems that process both text and images—are increasingly being used in real-world applications, but they struggle to make nuanced safety decisions based on context. Existing safeguards tend to fall into two traps: either refusing harmless requests out of abundance of caution, or missing actual risks that appear in images.

The Problem: False Positives and False Negatives#

Current safety alignment methods leave a gap between oversensitivity and undersensitivity. A model might reflexively decline a request to describe how to sharpen a kitchen knife (benign) while failing to recognize when an image contains genuinely dangerous content. This binary approach undermines both user experience and safety.

How SafeCoDe Works#

A research team has proposed Safety-aware Contrastive Decoding (SafeCoDe), a lightweight, model-agnostic decoding approach that adapts token generation based on multimodal context. The framework operates in two stages:

• Contrastive token highlighting: The method compares how the model generates text when shown the original image versus a noise-corrupted version. Tokens that change significantly reveal which parts of the response depend on visual context—surfacing safety-relevant decisions.
• Global-aware modulation: It layers in scene-level reasoning to evaluate the overall safety of a situation, then adjusts token-level refusals accordingly. This allows the model to tailor its response to what the image actually shows.

Results and Impact#

Testing across multiple MLLM architectures and safety benchmarks demonstrates that SafeCoDe consistently improves how models handle context-sensitive refusals while maintaining their ability to be helpful. The approach works across scenarios involving undersensitivity, oversensitivity, and general safety tasks, suggesting it could become a practical tool for deploying safer multimodal AI systems.