Agentic AI
CRAFT: Teaching AI Coding Agents to Learn Database Schemas Once, Not Every Query
A new training method for enterprise AI agents reduces schema documentation fed to models by 9x while improving code generation accuracy and consistency in multi-turn analytics work.
Enterprise systems increasingly rely on AI agents to translate business questions into executable code against proprietary databases and APIs. The standard approach—stuffing full schema documentation into every prompt—creates friction: it balloons token overhead, makes schemas harder to update, and degrades performance when tasks span multiple conversational turns.
Researchers have introduced CRAFT, a two-stage fine-tuning framework that teaches AI agents to internalize schema knowledge and tool-use patterns during training, rather than repeatedly retrieving them at inference time. The method was evaluated on real advertising-analytics workflows.
How CRAFT Works#
The framework operates in two stages:
- Schema-Stripped Planning (PLAN): Agents learn to construct domain-level action plans and executable behaviors using validated task trajectories, without requiring exhaustive schema injection at prompt time. This stage teaches what the agent should do.
- Execution-Shaped Reinforcement Learning: A second phase refines the agent's policy across tool selection, code quality, plan-code alignment, and recovery from execution failures. This stage teaches how well to do it.
Training data is curated via a Tri-Gate filter that validates execution correctness, checks data integrity, and applies LLM-based reasoning audits to ensure training signals are reliable.
Enterprise Gains#
The system was tested on advertising analytics tasks spanning campaign performance analysis, metric breakdowns, entity-level reporting, and multi-turn refinement loops. The environment included 25 core data entities and 30 agent workflows operating against beta APIs.
Compared to baseline schema-injection approaches, CRAFT achieved:
- +9.6 percentage points on composite Agent Score
- +4.1 pp improvement in consistency
- +4.2 pp gain in multi-turn coherence
- ~9x reduction in input tokens
- Up to 5x fewer schema-discovery retry loops
Deployment Realities#
The authors acknowledged challenges in real-world deployment: reward-shaping limitations in RL, infrastructure complexity for multi-turn tool-use training, and tradeoffs between compression and reliability in production analytics systems. Despite these hurdles, the efficiency and consistency gains suggest a path toward leaner, more maintainable AI coding agents.
