New Deterministic Framework Simplifies Test Suites Without Randomness

Software teams spend enormous resources running test suites to catch bugs before release. A new approach called DTM (Deterministic approaches for black-box Test suite Minimization) aims to shrink these suites—cutting testing time and cost—while delivering reliable, reproducible results.

The Problem with Current Methods#

Existing techniques for test suite minimization often rely on evolutionary algorithms, which introduce randomness into their decision-making. This means the same test code can produce different reduction outcomes across separate runs, creating unpredictability in automated testing pipelines where consistency matters.

The researchers set out to preserve effectiveness and speed while eliminating this variability, focusing on a "black-box" scenario where testers don't have access to the underlying production code.

How DTM Works#

The framework takes three key steps:

• Converts test cases into Abstract Syntax Trees, representing the structural form of each test
• Measures similarity between test pairs using four tree-based comparison techniques
• Selects subsets using three deterministic algorithms: Modified Minimum Spanning Tree, Spectral Clustering, and Dynamic Programming

By grounding the process in trees and deterministic selection logic, DTM removes the randomness inherent in evolutionary approaches.

Testing and Results#

The team evaluated DTM across 16 Java projects from Defects4J, a curated benchmark containing 661 buggy versions. The framework achieved an average accuracy of 0.74 and completed the analysis in just 0.98 minutes per project. Crucially, running DTM multiple times produced identical results, confirming full determinism—a feature absent in competing tools.

The research was submitted to arXiv on 16 May 2026.