Lesson 23 :: Where AI Fails — Hallucinations & Drift
Before you begin using AI tools regularly, there is something important you need to understand.
AI does not always get things right.
In fact, it can be confidently wrong.
This is not a temporary limitation that will disappear with time.
It is a natural characteristic of how these systems operate.
Understanding this does not make AI less useful.
It makes you a safer, more effective, and more responsible user of it.
Why This Matters
Without this understanding, it is easy to:
- trust outputs too quickly
- overlook subtle errors
- use incorrect information confidently
And because AI outputs often sound professional and well-structured, the errors are not always obvious.
That is what makes them dangerous.
Two Key Failure Modes
There are two specific failure modes you need to understand:
- Hallucination
- Drift
These are not edge cases.
They occur regularly — especially when structure is weak.
Failure Mode 1 — Hallucination
Hallucination occurs when an AI system produces information that appears credible, complete, and well-presented—
but is factually incorrect.
Why It Happens
AI does not “know” in the way a human expert knows.
It does not verify facts.
Instead, it generates responses based on patterns it has learned from data.
Sometimes, those patterns lead it to produce outputs that sound right—
but are not.
Practical Examples
- An AI tool is asked to cite a research study.
It produces a full citation — author names, journal, year — but the study does not exist. - An AI tool is asked about a law in a specific country.
It gives a confident answer — but the law is incorrect or outdated. - An AI tool is asked for a statistic.
It produces a number and attributes it to a recognised institution — but the institution never published it.
Key Characteristic
In all these cases:
The AI does not know it is wrong.
It is not guessing randomly.
It is producing the most likely-sounding answer.
That is what makes hallucination dangerous.
Failure Mode 2 — Drift
Drift is more subtle.
It occurs when the output gradually moves away from your original intention.
How It Happens
This often occurs in longer interactions.
At the beginning:
- the response is aligned
- the tone is correct
- the objective is clear
But over time:
- small deviations appear
- the tone changes
- the focus shifts
And eventually, the output reflects the conversation path—
not the original goal.
Practical Example
You begin by asking AI to draft a professional client email.
Midway, you ask a follow-up question about tone.
The AI adjusts.
By the end:
The email becomes informal or conversational—
no longer appropriate for the original purpose.
Key Characteristic
Drift does not announce itself.
It builds gradually.
And must be actively detected.
What Causes Both Failure Modes
Both hallucination and drift become more likely when:
- instructions are vague
- context is incomplete
- structure is weak
- sessions continue too long without reset
The Central Protection Principle
This leads back to the central idea of this session:
Structure is your protection.
When instructions are:
- clear
- specific
- structured
AI has less room to:
- invent
- assume
- deviate
Operating Rule (Non-Negotiable)
There is one rule that must always be applied:
AI drafts. Humans verify.
Practical Implication
This means:
- never use AI-generated facts without checking
- never rely on citations without confirming
- always review full outputs, not just the beginning
- reset when output begins to drift
Understanding failure is the foundation.
But awareness alone is not enough.
The next step is discipline.
How to use AI correctly — every time.
