RAG Retrieval Methods Outperform LLMs on Enterprise SQL and API Code Generation Tasks

Enterprise software often requires users to choose between querying databases with SQL or invoking REST APIs—and both demand deep knowledge of technical schemas and endpoints. A new research paper from arXiv evaluates how large language models augmented with retrieval strategies can bridge this gap by automatically converting natural language requests into properly formatted code.

The Problem: Knowledge Gaps in Enterprise Systems#

Large language models show genuine potential for generating structured code like SQL queries and API calls. However, they struggle with a fundamental limitation: they typically lack reliable access to enterprise-specific documentation, schema definitions, and endpoint specifications. This gap is especially acute in mixed environments where users need to route their requests to either a database or an API depending on what they're trying to accomplish.

RAG Retrieval as a Solution#

Retrieval-augmented generation (RAG) addresses this by grounding LLM outputs in external documentation. The researchers systematically compared three retrieval strategies:

• Standard RAG: straightforward context retrieval
• Self-RAG: iterative self-assessment and refinement
• CoRAG: a hybrid approach designed for complex routing decisions

Experimental Design and Results#

Using SAP Transactional Banking as their test case, the team built an execution-validated dataset and evaluated performance across three scenarios: database-only tasks, API-only tasks, and mixed scenarios requiring routing between both operation types.

Retrieval augmentation proved indispensable—models with access to external documentation substantially outperformed baselines without retrieval. CoRAG delivered the strongest results in the combined SQL and REST API setting, achieving statistically significant improvements over standard RAG, particularly when handling mixed-documentation retrieval conditions.

Key Takeaway#

Retrieval strategy plays an outsized role in structured generation performance, especially when documentation contains both database schemas and API specifications. This finding underscores the importance of thoughtful information retrieval design in enterprise code generation systems.