It is not lying. It is filling a gap.
A model asked a question it cannot answer does not stop. It produces the most plausible-looking text — which is exactly what a correct answer looks like. Here is the machinery, and here is how to break it.
8 min · Beginner · Playable
A librarian who never says no
Imagine asking a librarian for a policy document. A good one finds the right page, or tells you it does not exist. This librarian has been trained to always produce something that reads like an answer — so when the shelf is empty, it writes the page itself, in the house style, with a section number that looks right.
1 · You ask
Your question goes to a search system first, not to the model.
2 · It searches
The system pulls a handful of passages it thinks are relevant.
3 · It answers
The model writes an answer using only those passages — if it got the right ones.
In plain words
Almost every hallucination people blame on “the AI” happens at step two. The model is doing its job faithfully with the wrong material.
Now break it yourself
Below is a small library of eight real passages and four questions. Start with “Built properly” and watch it behave. Then try the other three presets, and ask the paternity leave question — the one the library genuinely cannot answer.
Ask the library
What came back (5 of 8 passages)
Employees on a permanent contract must give 60 days written notice. Employees on a fixed-term contract give 30 days, or the remainder of the term, whichever is shorter.
keyword 100% · meaning 67% · reranked · this is the passage that actually answers it
The company may place a departing employee on garden leave for all or part of the notice period. Salary and benefits continue unchanged during garden leave.
keyword 20% · meaning 33% · reranked · below threshold
Remote work is available to all roles after six months of service, subject to manager approval. Fully remote employees must attend the office one week per quarter.
keyword 0% · meaning 0% · reranked · below threshold
Economy airfare is reimbursed in full. Business class requires director approval and is permitted only for flights longer than eight hours.
keyword 0% · meaning 0% · reranked · below threshold
Each employee has an annual equipment budget of 60,000 rupees. Unspent budget does not carry over to the following year.
keyword 0% · meaning 0% · reranked · below threshold
The answer
Thirty days, or the remainder of the term if that is shorter. Permanent contracts require sixty days.
Source: Employee Handbook 2026, § 4.1 Notice periods
Try breaking it
Hybrid retrieval, a reranker, and a confidence floor. This is the configuration that behaves.
How it searches
Eight passages, four questions, and a scoring function written by hand — there is no model behind this page. What it reproduces exactly is the failure shape: when retrieval misses and nothing stops it, a fluent answer with a real-looking citation appears anyway.
In plain words
Watch what happens to the confidence floor. Set it to zero and the system will always answer, because something is always the closest match — even when the closest match is about something else entirely.
The four ways it goes wrong
Every one of these is a retrieval problem, not a model problem. That is why “use a better model” so rarely fixes it.
The passage was cut in half
A document chopped every 500 characters splits a clause from its definition. Both halves retrieve badly and neither answers the question.
The words did not match
You asked about “time off”, the document says “annual leave”. Keyword search finds nothing; meaning-based search finds it. This is why hybrid retrieval exists.
Too much was retrieved
Twenty passages sound safe. But now the one correct passage is competing with nineteen near-misses, and the model blends them.
Nothing stopped it
With no confidence floor and no refusal path, an empty shelf still produces an answer. This single missing check causes most of the damage.
Next
The fix is engineering, not prompting
In Module IV you build this pipeline properly — structural chunking, hybrid retrieval, reranking, refusal thresholds, and an evaluation set that catches regressions before users do.