LLMs Are Superhuman at Finding Signals in Chaos—But Can't Yet Replace On-Call Engineers

Can large language models handle the night-shift grind of on-call engineering? At the company that built Claude, the answer is a decisive no—but with a crucial asterisk. Alex Palcuie, who leads the AI reliability team at Anthropic, uses Claude constantly during production incidents yet insists the technology remains a tool, not a replacement, for human engineering judgment.

His nuanced take cuts through both hype and dismissal: LLMs are wildly uneven across the stages of incident response, excelling in some domains while posing real risks in others.

Where LLMs Shine: The Observe Phase#

Palcuie frames incident response using the OODA loop (Observe, Orient, Decide, Act)—a decision-making framework borrowed from Air Force fighter pilot training. LLMs achieve near-superhuman performance in the Observe phase: gathering signal from distributed systems, reading dashboards, querying metrics, parsing logs. This is work that humans can do, but not at scale or speed.

During a December 31st incident when HTTP errors spiked on Claude Opus, Palcuie pointed the LLM at the logs with minimal context. In seconds, Claude wrote SQL queries, identified an unhandled exception in image processing, and kept digging—discovering not a bug, but coordinated fraud. It found 4,000 accounts created in a narrow time window, 200 of them sending exactly 22 images each, with 3,800 more dormant and waiting. A human might have stopped at the error logs and filed a ticket for the API team. Claude escalated to account abuse on holiday.

"This was my move 37 moment," Palcuie said, referencing AlphaGo's famous unexpected play in the Lee Sedol match.

Where LLMs Falter: Judgment and Context#

The Orient phase—building accurate mental models of what's actually broken—is where LLMs become dangerous. This requires interpreting noisy, contradictory signals within a domain's real business logic and constraints. LLMs can hallucinate plausible-sounding explanations, confidently misdiagnosing a situation or missing critical context that an experienced SRE would immediately catch.

The Economics of AI-SRE Skepticism#

Palcuie is skeptical of the wave of "AI SRE" startups not out of cynicism about the goal, but realism about the timeline. On-call remains "a tax we levy on humans because our systems are not good enough to look after themselves." He genuinely wants these companies to succeed—not to automate his job away, but to reduce the cumulative human cost of always-on duty rotations. "It's never quite relaxing," he notes, whether you're paged at 3 a.m. or on a follow-the-sun rotation.