AI
New Method Trains LLMs to Master Private Code Libraries Through Synthetic Data
Researchers introduce PriCoder, a technique that helps large language models reliably use unfamiliar private libraries for code generation by automatically creating diverse, high-quality training examples.
While large language models have demonstrated impressive capabilities in generating code, they struggle when tasked with using proprietary or private libraries—even when given direct access to the necessary API documentation. A new approach called PriCoder addresses this gap by teaching models to work with unfamiliar code libraries through automatically synthesized training data.
The Problem with Current Approaches#
Existing methods for private-library code generation rely on a straightforward strategy: retrieve relevant API documentation and inject it into the model's context window at inference time. However, recent research demonstrates this approach falls short. LLMs still fail to invoke private-library APIs correctly, even when they have access to accurate information about how to use them.
How PriCoder Works#
The technique frames library data synthesis as a graph construction problem and employs two key operations:
- Progressive Graph Evolution: Generates increasingly diverse training samples by building upon simpler examples, expanding the variety of code patterns the model learns
- Multidimensional Graph Pruning: Filters synthesized examples through a rigorous quality control pipeline, removing low-quality training data
This dual approach ensures that models encounter both breadth and rigor in their learning material.
Testing and Results#
To validate their method, the researchers created two new evaluation benchmarks using recently released libraries that the tested models had never encountered during training. Across three major LLM families, PriCoder delivered substantial improvements—boosting performance by over 20% on pass@1 metrics in many scenarios. Importantly, these gains came without degrading the models' general code generation abilities.
The work suggests that simply exposing LLMs to library documentation isn't enough; they need structured training on realistic usage patterns to reliably leverage unfamiliar code.
