Smaller AI models crack security query problem at fraction of GPT-4 cost

Security analysts spend significant time crafting specialized database queries in Kusto Query Language to hunt through massive volumes of system logs, but the technical barrier has long constrained how many people can do this work. A new approach using compact language models promises to democratize this bottleneck by letting security teams search their data using plain English.

Researchers investigated whether smaller, more efficient models could accurately convert natural language questions into properly formatted KQL queries—a task that normally demands deep expertise. Their framework tackles the problem through three angles: smarter prompting that highlights common mistakes, fine-tuning strategies, and a two-stage architecture that pairs a small drafter model with a lightweight judge.

How the approach works#

The team adapted existing techniques to work within the constraints of smaller models. Rather than overwhelming them with KQL's complete rule set, they introduced error-aware prompting—a technique that supplies only the handful of tips most likely to prevent parser failures. They also experimented with LoRA fine-tuning, where models learned to explain their reasoning step-by-step before generating queries.

The most effective solution combined two stages: a small model drafts an initial query, then a low-cost LLM judge refines it with schema awareness before the system executes it.

Benchmarks against expensive alternatives#

Testing on Microsoft's NL2KQL Defender dataset, the two-stage approach achieved 98.7% syntax accuracy and 90.6% semantic accuracy (meaning the query was both valid and correct). On independently authored queries using the same schema, it maintained 96.4% syntax and 83.1% semantic accuracy.

Critically, the cost difference is stark. Running 230 queries cost just $0.213 with the researchers' approach compared to $2.998 using GPT-4o or $2.018 with GPT-5—a 9.5- to 14-fold reduction while matching or exceeding accuracy of those far more expensive models.

Implications for security operations#

The results suggest that small language models are now practical for real-world security tasks where accuracy matters and costs accumulate quickly. By lowering the barrier to natural-language querying, the approach could help security teams scale without hiring specialists fluent in query languages, and let organizations deploy threat-hunting at a fraction of current expenditure.