Software Engineering
New Framework Pinpoints Timing-Dependent Failures in Microservices Without Code Changes
SequenceFI enables precise temporal fault injection to catch order-sensitive bugs in distributed systems by tracking when failures occur during request execution.
Researchers have developed a technique for systematically exposing timing-related vulnerabilities in microservice architectures—failures that depend not just on what breaks, but precisely when. Current fault-injection methods struggle with these scenarios because they lack fine-grained control over the temporal sequence of failures across distributed service calls.
The Timing Problem#
When client requests flow through multiple interconnected services, failures don't occur in isolation. A service might fail after completing a state-changing operation, or race conditions might emerge from the order in which concurrent responses arrive. Similarly, partial failures among repeated downstream calls create complex timing dependencies that existing testing approaches miss. These occurrence-sensitive failures—where timing matters as much as location—are difficult to reproduce and diagnose.
How SequenceFI Works#
SequenceFI addresses this by tracking when faults activate during distributed execution. Rather than modifying application code or serialization libraries, the framework observes message-level send and receive events flowing between services. It propagates compact temporal evidence through request executions and only triggers injected faults when specific temporal conditions are met.
The system also learns from execution traces to automatically synthesize temporal guards—rules that capture the timing constraints necessary to trigger particular failure modes. This automation dramatically cuts down the trial-and-error needed to discover problematic configurations.
Experimental Results#
Testing on Kubernetes with four standard microservice benchmarks, SequenceFI demonstrated strong results:
- Achieved 100% temporal precision across nine failure scenarios and 450 valid trials
- Located effective fault-injection configurations on the first attempt, on average
- Reduced end-to-end search time by 95.91% versus a baseline random-search strategy
The non-intrusive design—requiring no changes to running applications—makes it practical to deploy in existing systems while hunting down order-dependent bugs that traditional injection techniques often miss.