AI Agents Interpret Data, But Query Engines Remain Essential

Artificial intelligence agents can now generate analytical code and transform raw data into structured reports, raising a natural question: do we still need data lakes, lakehouses, and SQL engines? A real-world disaster-recovery analysis project reveals the answer: AI changes the interface and reasoning layer, but not the underlying need for precise computation.

The team automated the analysis of a large disaster-recovery exercise by storing 30-second service metrics as Parquet files in object storage, using DuckDB for deterministic aggregation, and asking an AI agent to classify failover outcomes and explain the evidence. The prototype showed that an embedded engine could work without a shared lakehouse—not because the query engine disappeared, but because it became an embedded component within the agent workflow itself.

Understanding Context, Not Just Numbers#

Analyzing a disaster-recovery exercise requires more than retrieving time series data. The difficult part lies in interpreting what the numbers mean. When one region is isolated and traffic fails over to another, a healthy outcome should show:

• Queries per second (QPS) dropping in the isolated region
• QPS rising correspondingly in the surviving region
• Service availability staying above objectives
• Latency spikes being brief and transient rather than sustained

Each observation requires context. A QPS increase alone doesn't guarantee successful failover—it might be too small to account for lost traffic. A latency spike immediately after cutover may be harmless if the system recovers quickly. Some cases even contained conflicting signals: a service might absorb expected traffic yet remain at risk due to overloaded downstream dependencies, or a lower-tier service might show temporary backlog without violating availability objectives.

Separating Deterministic Facts from Contextual Reasoning#

The architecture divided responsibility into three layers: data preparation, deterministic computation, and agent-driven interpretation. This separation proved critical to building trustworthy analysis.

The deterministic layer used DuckDB to read Parquet files directly, filter time windows, join service metadata, and compute specific facts such as traffic absorption ratio, SLO violations, latency changes, and sustained degradation duration. The AI agent received a compact evidence packet containing these exact numbers—not raw metric rows—along with service context and exercise configuration.