Machine Learning Model Detects Argument-Swap Bugs in Python Code Without Call Mapping

A team of researchers has developed a machine-learning system called BugProbe that can spot a common type of programming mistake—swapped function arguments—in Python code without needing to track which method calls correspond to which definitions.

The Challenge#

Detecting argument-ordering bugs typically requires establishing links between where a function is called and where it's defined. This works reasonably well in statically typed languages, but becomes difficult in dynamically typed languages like Python, where such mappings are harder to determine automatically. Previous approaches have relied on comparing argument names to formal parameter names to catch these errors, but only when the call-to-definition connection can be made.

How BugProbe Works#

Rather than depending on explicit call-to-definition resolution, BugProbe takes a different route. The system combines multiple layers of context—including how arguments appear in their immediate surroundings and how they're actually used in the code—alongside name-based similarity analysis. This contextual information feeds into a machine-learning model that builds rich, expressive representations of method arguments, allowing it to detect mismatched orderings even without knowing where a method is defined.

The Dataset and Validation#

To train and test their approach, the researchers curated a substantial benchmark from open-source Python projects. They extracted 132,739 Python files from the top-1,000 starred repositories on GitHub, generating 3,371,244 synthetic training examples. They also manually verified a curated set of 55 real-world argument-swap bugs discovered by examining commit histories, creating a ground-truth test set.

Results#

Evaluation showed that BugProbe achieves high accuracy and consistently surpasses existing state-of-the-art baselines across standard performance metrics. The findings suggest that detecting argument-ordering errors is feasible in dynamically typed language environments when the approach doesn't depend on call-to-definition mapping.