Researchers enhance Python test generation by inferring types during code execution

Automated test generation has long been a cornerstone of software engineering research, but its success has been largely confined to statically-typed languages where type information is readily available. Python and other dynamically-typed languages present a different challenge: without explicit type declarations, test generators struggle to determine what arguments to pass when building test cases.

The Type Problem in Python Testing#

Test generation tools typically depend on knowing parameter and return types to construct meaningful test cases. In compiled languages like Java or C++, this information is baked into the code structure. Python, however, treats types more loosely, forcing test generators to make educated guesses about what inputs will work. This fundamental mismatch has limited how effective automated testing can be for Python codebases.

Type Tracing: Learning Through Execution#

Researchers have developed a solution that turns Python's frequent test execution into an asset. By introducing type tracing, the approach observes how functions actually behave when they run, capturing three key pieces of information:

• Parameter types inferred from how arguments are used during runtime
• Return types recorded when function calls complete
• Accumulated type knowledge applied to subsequent test generation

The technique was implemented as an extension to Pynguin, an existing Python test-generation framework, allowing it to progressively refine its understanding of types as more tests execute.

Results and Impact#

Testing the approach on real codebases yielded substantial improvements:

• Up to 87.8% more branch coverage compared to baseline approaches
• Improved mutation scores, indicating higher quality test cases
• Type information quality comparable to standalone type-inference tools

This outcome suggests that leveraging runtime information during the test-generation process itself can effectively bridge the gap between Python's dynamic nature and the type-aware strategies that make test generation successful in statically-typed languages.