AI System Revives Broken Machine Learning Notebooks by Modernizing Outdated Code

Machine learning engineers rely heavily on interactive Jupyter notebooks to develop, share, and document their work—from data loading through model evaluation. Yet a major problem is eroding the usefulness of these notebooks: as software libraries and hardware environments evolve rapidly, older published notebooks break and become impossible to run, undermining reproducibility and code reuse across the field.

The Scale of the Problem#

Researchers examined over 12,000 notebooks from 75 prominent Kaggle competitions to measure the damage. Their finding was sobering: only 26% of these notebooks still run successfully today. Even more surprising, attempting to fix the issue by downgrading dependencies to match the original submission time made matters worse, dropping reproducibility further to just 12% and introducing new failure modes.

A New Approach: Modernization Instead of Backporting#

Rather than trying to go backward, the team designed MLEModernizer, an AI-driven system that treats the current software environment as fixed and modernizes the notebook code instead. The approach works iteratively: it runs each notebook, analyzes what breaks, and applies three categories of targeted repairs:

• Error-repair — fixing code that no longer works with current library versions
• Runtime-reduction — optimizing slow computations
• Score-calibration — adjusting model outputs to maintain expected performance

Results and Real-World Impact#

Testing on over 8,200 previously non-reproducible notebooks, MLEModernizer successfully restored reproducibility for roughly 40–45% of them, depending on the underlying language model (GPT-5.2 or GPT-OSS-120b). The tool offers practitioners a practical way to salvage, validate, and maintain machine learning artifacts as technology stacks continue to shift.

The research is available on arXiv.