Software Engineering
GitHub Study Reveals Narrow Adoption of Chaos Engineering Tools in Open-Source Projects
A large-scale analysis of 1,275 open-source repositories shows that chaos engineering adoption is dominated by three tools and rarely targets application-level faults.
Researchers have conducted the first comprehensive empirical study of how chaos engineering—a practice that deliberately induces system faults to expose vulnerabilities—is actually implemented across open-source projects. The findings reveal significant gaps between the available tools and how practitioners use them.
What the Study Found#
The research team examined 1,275 validated repositories from GitHub associated with ten major chaos engineering tools. Several key patterns emerged:
-
Concentrated tool adoption: Three platforms—Toxiproxy, Chaos Mesh, and Chaos Monkey—account for nearly 69% of all analyzed repositories, suggesting most practitioners rely on a narrow set of solutions.
-
Where and who uses them: Development-focused repositories represent 56.55% of adopters, while industry organizations own 34.82% of projects using chaos engineering, with personal projects representing another 33.49%.
-
Limited project activity: More than half of repositories show minimal engagement, with 51.42% containing at most 50 commits, raising questions about sustained maintenance and real-world adoption depth.
The Fault Coverage Problem#
A striking finding concerns what kinds of faults practitioners actually test. Network-level disruptions and instance termination dominate the landscape, collectively accounting for nearly 75% of fault injections observed across 2,410 instances. Application-level faults—which target software logic and behavior—represent only 2.57% of experiments, indicating that most teams focus on infrastructure resilience rather than application robustness.
Implications for Teams and Researchers#
The study suggests practitioners should evaluate both repository activity levels and the types of faults a tool can target when selecting a chaos engineering platform. For the research community, the concentrated adoption pattern and the stark absence of application-level testing point to opportunities for developing new tools and investigating barriers to broader fault coverage adoption. The full study is available on arXiv.