AI
New AI System Combines Images and Text to Automatically Fact-Check Claims
Researchers present DEFAME, a multimodal fact-checking pipeline that evaluates both textual and visual evidence to verify claims at scale.
As misinformation spreads across digital platforms, researchers have unveiled a modular artificial intelligence system designed to automatically verify claims by examining both text and images. The approach, called DEFAME (Dynamic Evidence-based FAct-checking with Multimodal Experts), combines multiple specialized AI tools to evaluate evidence across different media types and generate transparent, structured reports about its findings.
How It Works#
DEFAME processes fact-checking through a six-stage pipeline that dynamically adapts its approach based on the claim being investigated. Rather than applying a one-size-fits-all method, the system selectively deploys different tools and decides how deeply to search for supporting or contradicting evidence. This modular design allows it to handle both text-only claims and those that incorporate images, performing end-to-end verification that accounts for visual content in both the original claim and any evidence sources.
Performance and Benchmarks#
Testing on established benchmarks including VERITE, AVerITeC, and MOCHEG demonstrates that DEFAME outperforms all previously published fact-checking methods. The team also introduced a new evaluation dataset called ClaimReview2024+ containing claims made after GPT-4o's knowledge cutoff date, which helps assess whether systems can handle recent information without relying on data leakage from their training sets. On this benchmark, DEFAME significantly surpasses baseline models, suggesting it could support real-time fact-checking applications.
Key Advantages#
Unlike earlier fact-checking approaches that rely solely on text, depend entirely on information already in an AI model's parameters, or lack transparency in their reasoning, DEFAME offers structured explanations alongside its verdicts. The multimodal focus acknowledges that modern disinformation often combines misleading text with doctored or decontextualized images—a challenge single-modality systems cannot adequately address.
