Study reveals why AI struggles to detect small tumors in lymphoma scans, matching human radiologists' blind spots

Researchers have developed a comprehensive evaluation framework for assessing how well deep learning systems can identify and measure lymphoma lesions on PET/CT medical images, revealing that the networks' performance mirrors the limitations of human radiologists.

The Problem with Current AI Models#

Most existing deep learning approaches for automated cancer detection focus narrowly on overall segmentation accuracy and fail to test how well models generalize to patient populations and imaging conditions beyond their training data. Additionally, few studies compare AI performance directly against expert human annotators—meaning researchers lack a clear sense of whether the technology is actually solving meaningful clinical problems or simply optimizing metrics that don't matter for patient care.

A New Framework for Real-World Evaluation#

The research team tested four different neural networks—ResUNet, SegResNet, DynUNet, and SwinUNETR—across 611 cases from multiple institutions, encompassing various lymphoma subtypes. Rather than relying solely on the Dice similarity coefficient and other standard statistical measures, the framework emphasizes lesion-specific metrics that directly relate to clinical decision-making. The team also introduced new detection criteria, including a novel approach based on metabolic characteristics of tumors, and carefully analyzed how radiologists themselves disagree when marking the same lesions.

Where AI Excels—and Fails#

The networks performed well on large, metabolically active lesions with clear imaging signatures. However, all four models struggled with the same cases that confound human experts: small, faint tumors with low metabolic activity. By comparing network errors to intra- and inter-observer variability among physicians, the researchers demonstrated that AI and human performance failures were strikingly similar. This suggests the difficulty lies in the imaging characteristics themselves rather than a fundamental flaw in the neural network approach.

Why This Matters for Clinical Practice#

Understanding where AI models fail—and confirming those failures match what radiologists struggle with—helps clinicians deploy the technology appropriately. Rather than viewing automated lesion segmentation as a replacement for human judgment, these findings support using AI as a complementary tool that highlights lesions humans might otherwise miss, while acknowledging both systems' shared limitations with small or ambiguous tumors. The framework aims to improve how cancer detection algorithms are evaluated and applied in real clinical settings, ultimately supporting better treatment decisions for lymphoma patients.