Three Rules for Building Reliable LLM-Based Selection Systems

Building decision-making systems with large language models sounds straightforward in theory—just ask the model to choose—but the reality is far messier. A developer working on procurement automation systems discovered this the hard way while trying to extract train data from screenshots, leading to a set of architectural principles for making LLMs reliably output structured decisions.

The Hidden Costs of Conversational AI#

The appeal of LLMs is undeniable: you can rapidly prototype solutions, the model often seems to understand context, and the interface feels intuitive. But when your goal is to transform text into a specific database ID—not to chat—a conversational approach breaks down quickly.

The fundamental problem is that conversation history becomes your data model, which doesn't play well with relational databases. You end up with all the text generation you need, but extracting clean signals for metrics and integrations becomes nearly impossible. You can't run an equality check on strings. The context window explodes when handling large volumes of options to choose from. And when something goes wrong, the system can behave erratically—worse, prompt injection vulnerabilities and unchecked API calls through tools make compliance and monitoring a nightmare.

Rule 0: Eliminate Noise First#

Before even querying the model, set temperature to zero and disable random seeds. This sounds obvious once stated, but it's easy to skip. Thinking budgets add nonlinearity that makes outputs unpredictable across runs. The developer found this out the hard way when regenerating slides: the same prompts returned different outputs. Removing randomness is the foundation everything else depends on.

Rule 1: Provide Explicit Context, Not Implicit Knowledge#

When you need an ID, give the model a constrained set of options rather than relying on its general knowledge. The train-delay example illustrates this perfectly: the model extracted an EVA station number from a screenshot, but when that number was queried against the Deutsche Bahn API, it mapped to a completely wrong station (Hoßkirch Königseggsee, which has no trains). The model had "chosen correctly" from its training data, but the training data was fiction.

The fix is structured output and explicit enumeration. Instead of asking "what station is this?", provide the model with a predefined set of valid stations and their IDs, then ask it to pick one. This shift from open-ended reasoning to constrained selection is critical—it's the difference between a chatbot and a reliable backend component.