Researchers use AI to reconstruct why code was changed by mining scattered documentation

Understanding the motivations behind code changes is essential for maintaining software, reviewing code, and debugging—yet developers often struggle to piece together this information because it's spread unevenly across multiple places and documented inconsistently.

Researchers from an empirical study of nine rationale components examined how and where developers document the reasons for code modifications across 63 commits in five popular open-source Java projects. They found that explanations are fragmented: commit messages and pull requests typically contain the goal of a change, while necessity and alternative approaches tend to live in issue trackers and pull requests instead. Crucially, no single source captures all the pieces developers need to fully understand a modification.

The Tool: ARGUS#

To address this fragmentation, the team developed ARGUS, an LLM-based system that sifts through all artifacts linked to a commit—messages, pull requests, issues—and extracts sentences expressing the goal, necessity, and alternatives behind a code change. It then weaves these fragments into a coherent summary.

Performance and Validation#

In testing, ARGUS identified rationale-expressing sentences with 51.4% precision and 93.2% recall, and the summaries it generated were rated as accurate against reference examples. Performance varied depending on which large language model powered the extraction, but the quality of final summaries remained consistently high across different LLMs.

Real-World Impact#

When twelve Java developers used ARGUS summaries to understand unfamiliar code changes, they found the tool valuable for code review, writing documentation, debugging, and general maintenance work—suggesting the approach could help teams collaborate more effectively and maintain clearer change histories.