FlockMTL extends DuckDB with native language models and retrieval-augmented generation

Building analytical systems that blend structured databases with AI reasoning often means wrestling with multiple tools, moving data between systems, and managing context windows manually. A new extension called FlockMTL aims to simplify that friction by integrating large language models and retrieval-augmented generation (RAG) directly into DuckDB, a lightweight embedded database.

What FlockMTL does#

The extension introduces LLM and RAG capabilities as native SQL constructs. Rather than orchestrating separate services, developers can now call language models through scalar and aggregate functions within queries themselves. This allows chained predictions—where outputs feed into subsequent operations—through tuple-level mappings and reductions, all expressed in familiar SQL syntax.

Key design principles#

FlockMTL borrows from relational database design to make LLM integration efficient and maintainable:

• Cost-based optimization: The system applies traditional query optimization techniques to LLM operations, automatically handling batching and caching to reduce redundant calls.
• Schema-first abstraction: Two new DDL objects—PROMPT and MODEL—join TABLE as first-class schema elements, decoupling the specification of language model workflows from their physical execution.

Why it matters#

Knowledge-intensive analytics often require drawing context from both structured data (tables, indices) and unstructured sources (documents, text). By embedding this retrieval and reasoning pipeline into the database layer itself, FlockMTL reduces the burden of managing data movement, context windows, and orchestration between disparate systems. The result is faster prototyping and simpler operations for applications that blend SQL queries with LLM-powered decision-making.