LLMs frequently hallucinate nonexistent Rust packages, creating supply-chain risks

Researchers have conducted the first large-scale investigation into how large language models invent fake software packages when generating Rust code—a phenomenon that poses genuine security threats to the broader software supply chain.

The hallucination problem#

Large language models excel at code generation but are prone to hallucinations: they produce plausible-looking outputs that are factually incorrect or simply made up. One particularly hazardous form occurs when an LLM suggests non-existent dependencies that developers might unknowingly try to install. While earlier research has documented this issue in languages like Python and JavaScript, Rust had received little attention until now.

Study scope and findings#

The study combined coding datasets from Stack Overflow, GitHub, and synthetically generated tasks to evaluate both commercial and open-source models under different decoding configurations. The results diverged sharply from earlier findings in other languages. Instead of varying hallucination rates depending on model architecture or size, Rust crate hallucinations showed a remarkable consistency: different models produced nearly identical error rates, and tweaking model parameters had minimal impact on hallucination frequency.

Exploring solutions#

Rather than simply documenting the problem, the researchers explored prompt engineering strategies—ways of phrasing requests to LLMs to reduce false outputs. These approaches aimed to lower hallucination without degrading the quality of the generated code itself.

Why it matters#

As developers increasingly rely on AI assistants for code generation, understanding model reliability in specific languages becomes critical. The distinct hallucination patterns in Rust suggest that language-specific characteristics or the size and quality of training data may play a deeper role than previously recognized. The findings offer concrete guidance for safer deployment of AI-assisted development tools and inform future research into model trustworthiness.