Session 10: Fact-Checking and Verifying AI Output

Here is one of the most important lessons in this entire course: AI can be wrong, confidently and convincingly. This is not a flaw that will be fixed someday — it's a fundamental characteristic of how large language models work. AI generates text by predicting what words should come next based on patterns in its training data. When it doesn't have reliable information, it can still generate fluent, authoritative-sounding text that is simply not true. This is called hallucination.

Hallucination can take many forms. An AI might cite a scholarly article that doesn't exist, with a convincing-sounding author name, journal title, and volume number — all fabricated. It might state a historical date that's off by a decade. It might attribute a quote to the wrong person. It might describe a study's findings in a way that subtly reverses the actual conclusions. In every case, the text reads confidently, grammatically, and plausibly. That's what makes it dangerous for academic use.

This isn't a reason to avoid AI — it's a reason to verify. Think of AI responses the way a good journalist thinks about tips: useful leads that require confirmation before you use them. A journalist doesn't publish a story based on a single anonymous source; a scholar doesn't cite a source they haven't read. The same standard should apply to AI output. Before you use any AI-generated fact, statistic, quote, citation, or claim in academic work, verify it independently.

Perplexity is your most powerful verification tool because it cites its sources. When Perplexity tells you something, you can click through to the source and check it yourself. This is enormously valuable for fact-checking. But even Perplexity's sources require a second look — the platform can occasionally misread or misrepresent the pages it cites. Final verification always goes back to the primary source: the original article, the government database, the official document.

Today you'll build a verification practice by deliberately generating a factual response from ChatGPT and then fact-checking it using Perplexity. You'll be looking for any discrepancies — places where the two platforms disagree, where Perplexity's sources contradict ChatGPT's claims, or where a claim turns out to be difficult to verify at all. Finding discrepancies isn't a failure; it's the exercise working exactly as intended.

Developing a healthy skepticism about AI output is one of the marks of an AI-literate scholar. Trust, but verify. Use AI freely for brainstorming, organizing, and drafting — but treat any specific factual claim as a hypothesis to be tested, not a fact to be accepted. This habit will protect your academic credibility and make you a more rigorous thinker.