Agentic AI
Moving Beyond Prompt Injection: How to Build LLM Agents Like Real Software
A Walmart engineer argues that treating agent tools as pasted prompts recreates 1970s programming problems—and proposes a composition layer to bring decades of software engineering best practices to AI.
InfoQ
As enterprises scale LLM-powered agents to thousands of teams, the way they're built mirrors the brittle, monolithic style of programming from five decades ago. Instead of reinventing software engineering from scratch, the industry needs abstraction, encapsulation, and proper composition—the fundamentals that have matured over 50 years.
The 1975 Problem: Why Agents Are Stuck in the Past#
Building agents today feels oddly familiar to writing BASIC in the early 1970s: everything lives in a single main method, logic flows non-deterministically like GOTO statements, and there's no separation of concerns. When a tool gets called, what really happens is the entire tool schema and description get copy-pasted directly into the system prompt—unversioned, uncontrolled dynamic injection disguised as shared libraries.
This creates immediate pain. If your Jira platform team publishes a tool with 10 optional parameters, the LLM reads all of them and struggles to pick the right combination. If an upstream tool's description doesn't match your internal terminology (calling things "stories" instead of "tickets"), the LLM must invent a mapping on the fly—and consistency breaks. There's no way to wrap, adapt, or evolve a shared tool without forking it entirely.
Modern software solved this fifty years ago with versioned libraries, interfaces, and encapsulation. Agents get none of it.
The Model Context Protocol: Good Intent, Incomplete Solution#
The Model Context Protocol (MCP) was a necessary step. Before it, every agent team built the same tools repeatedly—quadratic waste at scale. MCP let specialized teams (Slack, Stripe, Jira) own their tools once and agents reuse them, creating a cleaner division of labor.
But the implementation choice was to concatenate tool schemas directly into prompts. This is equivalent to a programming language where import numpy literally inserts the entire NumPy source code into your file. You have shared libraries in theory; in practice, you have prompt injection with extra steps.
The real problem: agent authors own the eval quality and control context tuning, but they have zero ability to customize tools they're borrowing. In large organizations, you're pasting code from unknown teams into your LLM's reasoning loop with no versioning, no contract, no abstraction boundaries.
