Agentic AI System Automates Crystal Structure Analysis from Microscopy Images

Researchers have developed AutoMat, an AI system that automatically reconstructs crystal structures from scanning transmission electron microscopy (STEM) images—a notoriously difficult inverse problem in materials science where multiple atomic arrangements can produce similar visual patterns.

The Challenge#

Converting a single noisy STEM projection into a physically realistic crystal structure has long been intractable for traditional machine learning approaches. Simple feed-forward models cannot verify whether their predictions obey physical laws or match real material behavior, leaving no way to detect when inferences are wrong. This has forced materials scientists to manually interpret microscopy data and laboriously validate results against simulation.

How AutoMat Works#

Rather than relying on a single neural network prediction, AutoMat deploys an agentic controller that conducts systematic hypothesis testing with verification loops. The system chains together multiple specialized modules: pattern-adaptive denoising to clean up noisy images, physics-guided template matching to retrieve candidate structures, symmetry-constrained reconstruction to enforce crystallographic rules, and machine-learning interatomic potentials (MLIPs) to relax and validate proposed atomic arrangements. When verification fails—indicating a prediction violates physical constraints—the system rolls back and retries with alternative hypotheses.

Benchmarking and Results#

To rigorously assess performance, the team created STEM2Mat-Bench, a curated dataset of 450+ annotated samples. Evaluation metrics include lattice root-mean-square deviation (RMSD), formation energy error, and structure matching accuracy. AutoMat exceeded the performance of existing approaches, including state-of-the-art deep learning models, specialized domain software, and closed-source multimodal large language models.

Significance#

This work bridges a critical gap between experimental characterization and computational modeling. By automating the conversion of microscopy observations into simulation-ready atomic structures, it streamlines downstream materials discovery and property prediction workflows—opening new possibilities for accelerated materials research and development.