AI hallucination refers to the phenomenon where a large language model (LLM) generates output that is stated with apparent confidence but is factually incorrect, unsupported by any source, or internally inconsistent. The term is borrowed from psychology but describes a distinct mechanism: LLMs are trained to produce fluent, coherent text by predicting likely token sequences, and that objective does not guarantee factual accuracy. In a learning context, hallucination is a serious risk because a learner who asks an AI tutor a question about a medical procedure, a legal rule, or a technical specification expects the answer to be correct — a plausible but wrong answer can cause real harm. Retrieval-augmented generation (RAG) is the primary technical mitigation: grounding the model in actual course content substantially reduces the rate of invented facts. However, RAG does not eliminate hallucination because the model can still misinterpret or contradict the retrieved passage. Human review of AI-tutor response policies, fallback to authoritative sources, and confidence-gating (refusing to answer when the model is uncertain) are complementary safeguards. For assessment content specifically, every AI-generated question must pass human review before reaching learners. Transparency matters too: citing the course passage the answer is based on lets the learner verify independently, turning the potential liability into a learning moment.
