AI matches doctors at spotting macular degeneration but struggles to predict patient outcomes

Researchers evaluated 35 machine-learning teams on their ability to monitor age-related macular degeneration (AMD) through automated analysis of eye scans, uncovering both the promise and limits of AI in ophthalmology.

The MARIO Challenge#

The challenge, held at MICCAI 2024 (a major medical imaging conference), tasked researchers with building AI systems to detect and track changes in AMD—a leading cause of vision loss in older adults. Teams had access to optical coherence tomography (OCT) images, infrared scans, and clinical records from patients in Brest, France. An independent dataset from Algeria was later used to test how well the winning approaches generalized to new equipment and populations.

Two distinct tasks#

Participating teams tackled two separate problems:

• Task 1: Classifying whether disease activity had worsened, remained stable, or improved by comparing two consecutive OCT scans
• Task 2: Predicting whether patients receiving anti-VEGF injections (the standard treatment) would show disease progression over the next three months

The top 12 finalist teams presented their methodologies, establishing a benchmark for how modern AI approaches OCT imaging analysis.

Mixed results#

The findings revealed a striking asymmetry. When it came to detecting immediate changes between sequential scans—essentially comparing before-and-after images—AI performed comparably to practicing physicians, validating machine learning's role in clinical screening and monitoring.

However, when tasked with forecasting future disease course, the AI systems fell short. None achieved the performance threshold needed for reliable clinical prediction, suggesting that the challenge of anticipating how individual patients will respond to therapy remains fundamentally harder than pattern recognition in static images.

Implications#

The results indicate that while AI excels at image interpretation, predicting patient trajectories requires deeper understanding of disease biology and individual variation that current algorithms have not yet captured. This distinction underscores where human expertise and longitudinal clinical judgment continue to add essential value.