Datadog's Test-Driven AI Migration Cut Database Costs 90% and Latency from Hundreds of Milliseconds to Single Digits

Datadog has shared a detailed case study in how it used Claude and Cursor to refactor a critical production system, Stream Router, moving it from a key-value database to a relational schema. The effort resolved severe performance bottlenecks while reducing infrastructure costs by 90%.

The Problem: Key-Value Limits#

Stream Router, which manages routing in Datadog's metrics pipeline, was originally built as an eventually consistent system using a key-value (KV) model. As traffic and data volumes grew, the KV store hit hard transaction size limits. Complex operations that involved tens of thousands of entries began taking 45 minutes to complete, as the application had to fetch data into memory and emulate relational database behavior—including enforcing relationships that foreign keys would normally handle automatically.

The AI-Assisted Migration Approach#

Rather than hand-code the entire refactor, Datadog engineers designed a test-driven workflow leveraging AI. For each function, they provided Claude with the original implementation, details of the new schema, and a failing test case. The model would generate an initial version, and the test suite would immediately signal whether it was correct.

According to Datadog engineer Arnold Wakim, three structural conditions made this approach work:

• Code modularity: Stream Router's API remained the same while its backend swapped from FoundationDB to PostgreSQL
• Comprehensive test coverage: Tests acted as an objective pass/fail gate for every change
• Parallel infrastructure: Two independent instances ran side by side, with a validator service continuously comparing their outputs to catch any divergence

Where AI Excelled—and Where It Fell Short#

The migration unfolded across three phases: first, using Claude to document the intent of key functions; second, generating targeted fixes by feeding the model failing tests and expected behavior; and finally, deploying via blue-green rollout while both systems ran in parallel.

However, AI assistance had clear limits. Higher-level prompts proved less effective, and Claude frequently generated queries that were correct but inefficient—issuing far more database round trips than necessary. Optimizations like batching, UNNEST tricks, and common table expressions required manual engineering. Once engineers wrote these patterns, the model could replicate them in similar code, but it did not discover them independently despite their prevalence in published literature.