LLM-Powered Incident Diagnosis Increasingly Depends on Data Pipeline Design, Not Model Capacity

Observability teams building AI-assisted tools for incident investigation are shifting focus from model capability to context engineering—the art of selecting and presenting the right telemetry data to an LLM before it reasons over the problem.

The consensus emerging from practitioners and vendors is that LLM reasoning ability itself is no longer the limiting factor. Instead, the harder challenge lies in designing the pipeline that decides which signals the model sees and how they're formatted. For teams deploying these systems, this suggests that investing in better data preparation may yield stronger results than simply upgrading to a larger model.

Two Competing Approaches#

AI-assisted root cause analysis (RCA) strategies typically fall into two camps:

• Agent-based systems equip models with tool access and let them actively investigate, fetching telemetry as they reason through a problem. This approach offers flexibility to discover unexpected signals, but introduces operational complexity.
• Deterministic pipelines pre-correlate signals offline and feed a single, prepared context bundle to the model. This method sacrifices some adaptability for clarity and debuggability.

Major vendors are gravitating toward the deterministic camp. Dynatrace's Davis AI, for instance, relies on topology-based causal analysis that traverses real-time dependency maps rather than unleashing open-ended agent loops. The trade-off: agent-based systems can chase unexpected trails in novel incidents, while deterministic designs offer repeatability and cleaner error diagnosis.

Testing the Model vs. the Harness#

Observability vendor Coroot recently investigated this question through original research by engineer Nikolay Sivko. His core insight: separating two distinct jobs—the model's reasoning capability and the harness that shapes the data fed to it—is essential to understanding what actually fails.

Coroot's own deterministic pipeline correlated observability signals into findings, then passed a single focused context (~9,800 tokens) to multiple models, including misleading signals intentionally embedded to test reasoning robustness. The scenario involved a Chaos Mesh network delay experiment between a catalogue service and its database, causing frontend failures.

Results showed frontier models (Claude Opus 4.8, GPT-5.5, Gemini 3.1 Pro) and most larger open-weight models correctly identified the injected fault and suggested fixes. Notably, was the only self-hostable model to pinpoint the root cause, while comparable alternatives like Qwen3.6 35B stumbled.