AI

Library Hallucinations in LLM-Generated Code: A Risk Analysis Grounded in Developer Queries

Large language models (LLMs) used in code generation often 'hallucinate', inventing non-existent libraries that can mislead developers and expose systems to security threats. A new study analyzed how user-level prompt variations affect library hallucinations in LLM-generated code, finding that even small mistakes like one-character misspellings or fabricated library names can trigger hallucinations in up to 99% of tasks. The researchers introduce a benchmark called LibHalluBe
Large language models (LLMs) used in code generation often 'hallucinate', inventing non-existent libraries that can mislead developers and expose systems to security threats. A new study analyzed how user-level prompt variations affect library hallucinations in LLM-generated code, finding that even small mistakes like one-character misspellings or fabricated library names can trigger hallucinations in up to 99% of tasks. The researchers introduce a benchmark called LibHalluBench to help evaluate and mitigate these risks. --- Why it matters: This study matters because it highlights the fragility of LLMs to natural prompt variation, which can lead to security vulnerabilities and downstream risks. Engineers working with LLM-generated code need to be aware of these systemic vulnerabilities and take steps to prevent library-related hallucinations. Source: https://arxiv.org/abs/2509.22202

This article was originally published at: https://arxiv.org/abs/2509.22202