Agentic AI
Why better AI models won't fix your failing agents—infrastructure will
The real bottleneck in AI agent systems isn't model capability—it's the data plumbing, tool retrieval, and execution guardrails underneath.
The New Stack
Teams building AI agents often chase the same mirage: hit a reliability wall, upgrade to a "smarter" model, watch performance barely budge, hit the wall again in a different shape. The problem is that teams are diagnosing a systems failure as a model failure.
The real bottleneck isn't reasoning power—it's the infrastructure layer that feeds context to the model, routes tool calls safely, and keeps agents grounded in organizational reality rather than hallucinated versions of it.
The compilation gap#
Andrej Karpathy articulated the shift recently on X: he noticed his compute was increasingly "less into manipulating code, and more into manipulating knowledge." He wasn't running a bigger model—he was building better infrastructure. Raw data fed into a structured wiki compiled by an LLM, complete with summaries and backlinks, then agents operated over that curated version using CLI tools.
Most production systems skip this step. They wire models directly to raw sources—APIs, databases, document stores—and expect the model to make sense of the noise within a narrow context window under time pressure. The result is pattern-matched guesswork.
Organizations that cracked this built a structured representation of how they actually work: naming conventions, decision paths, previous outcomes. Not generic documentation, but an operational mirror that agents can reason over instead of inventing.
Tool selection and retrieval at scale#
Vector-based tool retrieval breaks when vocabulary diverges. A developer asks "why did the deploy fail?" but the relevant tool is get_pipeline_run_logs—semantic similarity between those phrases is poor, so the agent picks a plausible wrong tool instead.
Production teams discovered the fix: translate intent into a hypothetical action first. Given the query, write out what a correct tool call should look like, then match against that shape rather than the raw question. A translation layer from user intent to executable action turns out to be where most agent failures originate—and fixing it beats upgrading the model.
At scale, organizations with dozens of integrated systems (GitHub, Jira, Confluence, cloud providers, CI/CD platforms) hit retrieval walls hard because dropping everything into context is slow, expensive, and produces poor tool selection. Indexing and interface engineering become mandatory.
