AI Security
Skip the expensive AI models: How smart filtering cuts security costs to $1 a day
Security teams burning through AI budgets are running every case through frontier models—but smart funneling and tiered approaches let you handle routine work cheaply while reserving premium models for what really matters.
The New Stack
The prevailing assumption among security teams is that AI-powered threat detection demands ruinous costs. In practice, this expense stems less from model pricing than from poor workflow design. A security operations leader explains how restructuring the detection pipeline—filtering early, routing smartly, and escalating selectively—can reduce daily AI spending for trust and safety work to roughly $1.
The filtering funnel, not the model, drives cost#
Before large language models existed, security teams stacked layered filters to winnow high-volume event streams down to cases a human analyst could handle. That same principle applies to LLM expenses: the tighter and more precise your funnel, the fewer tokens flow through expensive models.
In trust and safety operations, most abuse is caught by rules-based deterministic checks before any model even runs. Signals like account age, email provider, and behavioral patterns feed automated gates that settle obvious cases immediately. Only events that pass those filters reach an LLM. That's where the low daily cost originates.
The analogy is straightforward: you don't need a camera surveying everyone who passes your house. You activate monitoring when someone crosses the threshold.
Two-tier models keep costs proportional to need#
Once cases survive the initial funnel, a lightweight model performs the first assessment, outputting a binary judgment—malicious or benign—with a confidence score. High-confidence verdicts resolve automatically. Uncertain cases escalate to a more capable model with richer context and deeper reasoning.
Testing revealed only a 1–2% accuracy gap between lightweight and frontier models for these specific use cases, yet frontier models command roughly five times the per-token cost. The math works only when cases genuinely require that additional capability.
Prompt design is equally critical. Complex agentic scenarios may demand 1,900-word prompts covering every edge case, when-to-escalate rules, and autonomous thresholds. Simpler abuse reports might need only two or three sentences. The scope of the task determines how detailed the prompt must be.
Accuracy hinges on context. Feed a model a user report alone, and it may accept the report at face value. Provide the actual flagged content alongside the report, and the model can assess independently whether the allegation holds water.
