Researchers Test Whether LLMs Can Truly Explain Malware Behavior—and Find They Often Fall Short

Security teams don't just need to know software is malicious—they need to understand how and why. A new study from researchers working in cryptography and security finds that large language models, despite their promise, often fail to deliver the rigorous explanations that malware analysis demands.

The Problem With Current Approaches#

Traditional malware detection systems excel at flagging dangerous code, but explaining the specific malicious behaviors and backing those claims with actual code evidence remains a stubborn challenge. Signature-based methods lack flexibility, while existing explainability techniques struggle to produce human-readable justifications that security analysts can trust and act upon. LLMs initially seemed like a solution, but their suitability for this specialized domain—particularly their ability to ground claims in actual code—has never been properly tested.

Introducing MalEval#

Researchers introduced MalEval, a diagnostic framework designed to measure what LLMs can and cannot do when auditing malware. Rather than working with sanitized datasets, the team paired real-world application source code with expert-written security audit reports. This ground truth allows for precise, behavior-level evaluation that goes beyond simple detection.

The framework tackles three major obstacles: the absence of reliable human-authored behavior annotations, the practical impossibility of feeding entire large codebases into LLM context windows, and the lack of systematic ways to verify whether model claims actually connect to real code evidence. MalEval addresses these by compressing codebases into behavior-relevant program contexts—an intermediate representation that preserves function call relationships—and then mapping both expert reports and model outputs into structured chains of evidence.

What The Testing Revealed#

The researchers evaluated seven different LLMs on four sequential auditing tasks. Their findings were sobering: the models consistently relied on surface-level pattern matching rather than building verifiable chains of logic, struggled to synthesize dispersed facts into coherent attack scenarios, and proved extremely sensitive to how the code context was formatted and presented.

These weaknesses suggest that isolated LLM outputs are insufficient for malware auditing—the field needs robust workflows, human oversight, and potentially agentic systems that can iteratively refine their reasoning.