Hallucination
Model and prompts · LLM hallucination, AI hallucination
An LLM hallucination is a confident but invented answer from a model: a nonexistent fact, function or link. It appears where part of the answer has no support in the data, and the model fills the empty spot with a plausible guess. The opposite case, where every part of the answer has support, is called an epimorphism. It is reduced by grounding in reliable sources and by checking the result.
Hallucination examples and how to reduce them
- An invented function · the agent calls a method the library does not have, or a parameter the function does not take. The code looks plausible and fails when run.
- A nonexistent package · the agent suggests installing a package that does not exist on npm or PyPI. Models often repeat the same invented names, and attackers register packages under them to slip in their code (slopsquatting).
- A dead link · a link to an article, doc or section that does not exist.
- “Done” with no changes · the agent reports work it did not do.
- How to reduce it · ground the model: give it sources and context and require a supporting reference for each claim (that is an epimorphism); allow “I don’t know” in the prompt.
- How to detect it · with checks: tests, builds and linters catch invented code, an allow-list of packages catches invented packages, following links catches invented sources.
Source: Wikipedia: AI hallucination
Where it appears · 3
- Requirements and meeting notes with AI · Harness
- MCP for an AI agent · Harness
- ChatGPT Enterprise usage report for a company · Research