New framework enables smaller LLMs to match GPT-4 performance on database query tasks

Converting natural language into executable SQL queries has long been a barrier for non-technical database users. Researchers have now unveiled LearNAT, a training framework designed to unlock this capability in smaller, publicly available language models without relying on expensive proprietary systems like GPT-4.

The Challenge#

While large private language models have recently dominated Natural Language to SQL (NL2SQL) translation benchmarks, they come with significant drawbacks. These systems lack transparency, are difficult to reproduce, and carry substantial computational costs. Meanwhile, smaller open-source models have struggled to decompose complex queries into manageable steps—a key skill needed for tackling intricate database operations.

How LearNAT Works#

The framework introduces two core innovations:

• Decomposition Synthesis: Researchers developed a procedure that uses Abstract Syntax Tree (AST) guidance with intelligent pruning to break down queries into verifiable, efficient sub-steps. This structured approach helps the model understand which decompositions actually work.
• Margin-Aware Reinforcement Learning: Rather than relying on standard preference optimization techniques, this method fine-tunes multi-step reasoning by providing nuanced feedback at each decision point in the reasoning chain.

Results#

Experiments on established benchmarks show that a 7-billion-parameter model enhanced with LearNAT achieves performance levels comparable to GPT-4, while consuming far fewer resources during inference. The work demonstrates that thoughtfully designed decomposition training and fine-grained learning preferences can substantially narrow the gap between smaller and larger models.

Researchers have released the code publicly, supporting efforts toward more transparent, reproducible, and economical NL2SQL systems.