Language Models Create Domain-Specific Languages Better When Embedded in Known Code

Language models excel at writing Python, JavaScript, and SQL—the languages plastered across their training data. Ask them to generate code in your team's custom notation, and they confidently produce something that looks right but isn't: fabricated keywords, guessed parameter names, and structures that never existed. The core issue isn't a knowledge gap; it's a frequency gap.

Why Mainstream Languages Win#

Large language models generate syntax reliably in proportion to how often that syntax appears in their training corpus. Popular languages benefit from millions of code examples, allowing models to internalize legitimate patterns. A brand-new domain-specific language (DSL), by contrast, exists zero times in the training data. When models encounter unfamiliar notation, they don't fail loudly—they interpolate, borrowing half-remembered syntax from related grammars and handing back fluent-looking but invented code.

The trap deepens with lenient parsing. A traditional DSL parser might forgive unrecognized keywords and skip invalid lines, rendering a best guess. Humans notice broken output; language models don't. A model generates a diagram with one invented relationship keyword, the parser silently skips it, and an engineer downstream sees a polished visualization missing exactly one connection—with no reason to suspect anything is wrong.

Grounding Syntax, Not Just Facts#

Today's go-to approach for anchoring model output is Retrieval-Augmented Generation (RAG)—pulling relevant documentation into context so the model's claims stay tethered to real sources. RAG grounds content (whether the model knows facts about your system), but it does nothing for notation (how the model expresses those facts in code). A model can have perfect knowledge and still emit syntactically invented DSL code.

Enter Typed Domain Grounding (TDG). Instead of grounding facts in retrieved data, TDG grounds the language itself in what the model already knows how to write. The strategy is simple: embed the domain as a typed internal DSL inside a mainstream host language—Kotlin, Python, or TypeScript. The model now writes code in a language dense in its training data, while the compiler enforces domain rules.

How the Approach Works#

TDG rests on five design principles:

• Embedded, not external: The domain becomes an ordinary library inside a popular language. No new grammar, no bespoke parser. IDE support (syntax highlighting, autocomplete, debugging) arrives on day one.