Teaching Students to Question AI

The foundational misconception is treating a chatbot as a search engine: a machine that looks up the answer somewhere authoritative and reports back. That is not what is happening. A language model generates text word by word, based on patterns learned from its training data. Nothing is looked up, and nothing is checked against a source unless the tool has been specifically built to do so. The single most useful idea a student can carry is that these systems produce plausible text, not verified fact, and that the two overlap often but not always. The second problem is that fluency reads as authority. Novices in any subject judge credibility by surface signals: confidence, polish, the absence of hesitation. AI output has all three in unlimited supply, which creates an uncomfortable asymmetry: the students with the least knowledge are both the most likely to trust an AI answer and the least able to spot its errors. Hallucination is the vivid failure, invented citations, wrong dates, a confident description of an event that never happened, but the subtler failures matter more in the classroom: an answer that smooths over the complexity, presents a live scholarly debate as settled, or gives a bland average of positions where the disagreement was the whole point. One property of these tools is worth demonstrating rather than describing: ask the same question twice and you may get materially different answers. For students, that variability is the tell. A system that was looking up the answer would give the same one each time; a system that generates will not. Five minutes of live demonstration, on a topic the class knows well enough to mark, teaches this faster than any amount of warning.