AI
New System Bridges Gap Between Service Monitoring Data and Automatic Code Repairs
Researchers introduce ORCA, an automated repair pipeline that uses operational telemetry to both diagnose and fix microservice failures.
When microservices fail in production, engineers typically rely on operational telemetry—logs, metrics, traces—to understand what went wrong. Yet most automated program repair systems begin from a different starting point: test cases, issue reports, or localized code snippets. That disconnect creates friction in translating real-world system failures into actionable code fixes.
A new research paper from arXiv's Software Engineering track presents ORCA, a novel automated repair pipeline designed to bridge this gap for microservice incidents. The approach uniquely combines operational monitoring data with code generation to create and verify patches that actually resolve production problems.
How ORCA Works#
The system operates in several integrated stages:
- Fault signature extraction: ORCA analyzes paired sets of telemetry—one from a failing service and one from a working reference—to distill the key differences into a structured fault signature.
- Localization: The signature identifies candidate locations for repair, spanning both source code and deployment configuration files.
- Patch generation: Specialized repair graph agents work alongside an Exploration agent to synthesize unified-diff patch candidates directly from these identified locations.
- Telemetry-grounded verification: Patches are evaluated not just on test correctness but against live observability data, checking patch validity, syntactic/semantic soundness, test reliability, and telemetry replay fidelity.
Performance at Scale#
The researchers tested ORCA on a benchmark of 575 incident cases and found it outperformed all comparison baselines in cost-effectiveness. A key insight emerged: operational telemetry can serve dual roles—both as diagnostic evidence and as rich context for repair generation. Paired telemetry supports more precise code localization, while repair graph agents convert that localized evidence into structured constraints for the language model to generate patches.
Notably, telemetry-grounded verification uncovered repair outcomes that evaluation based solely on tests or issue reports would have missed, suggesting the approach captures aspects of production correctness that traditional metrics overlook.
