Large language models spontaneously develop physics intuition without explicit training

Large language models can grasp physical reasoning on the fly, organizing abstract concepts without anyone teaching them explicitly. Researchers analyzing this ability have uncovered how LLMs build internal representations aligned with real-world physics when given example data within their prompts.

How the study worked#

The investigation used dynamics forecasting—predicting how physical systems evolve—as a controlled experimental task. The team fed LLMs historical snapshots of physical motion and watched whether the models could extrapolate forward. As the models received longer context windows with more history, their prediction accuracy improved, suggesting they were learning meaningful patterns from the examples alone.

The researchers then peered inside the models' computational layers, examining their internal activation patterns. They discovered that certain neurons and pathways within the residual stream (the flow of information through the network) correlated strongly with fundamental physical quantities such as energy—even though the models had never been trained on physics-specific labels or data.

Evidence of physics understanding#

A striking finding was that these energy-aligned signals grew stronger as more examples were provided during in-context learning. This gradual sharpening suggested the models were genuinely extracting and refining physical structure from raw data, not simply memorizing surface features.

To verify these representations actually helped with prediction, the team applied layer-wise gradient attribution, a technique that measures which internal features most influence the model's final numerical outputs. Crucially, residual directions correlated with energy received higher attribution weights for forecasting accuracy than features tied to directly visible quantities like position. This pattern indicates the energy signal was doing real computational work—it wasn't just numerical noise or a copy of visible input.

Why it matters#

This work opens a window into how large language models solve reasoning tasks through in-context learning, a core capability that lets them adapt to new domains without retraining. By studying physical systems as a testbed, researchers can now trace the mechanics of how LLMs spontaneously organize structured knowledge. The findings suggest that models discover abstract, scientifically grounded concepts on their own, hinting at a form of emergent reasoning that goes beyond pattern matching.