How to verify AI answers and spot hallucinations

What you need

To follow this guide, you only need an AI tool that provides answers with citations or explanations. You will also need access to a web browser to search for primary sources and verify the information the AI provides.

Step by step

1. Check if the source actually exists Before trusting a claim, make sure the source is real. Search for the exact title of the article or paper, the publisher, the author, or the digital object identifier (DOI).

2. Verify the specific details within the source Even if a source is real, the AI might misuse it. Open the cited page and find the exact statistic, quote, or conclusion mentioned. Read the text surrounding that information to ensure the AI hasn’t taken it out of context or overstated the findings.

3. Break the claim into smaller parts To avoid being overwhelmed, separate a complex claim into individual pieces: the person or organization, the specific event or feature, and the date or version. Verify each of these parts one by one.

4. Inspect precise numbers and data Be extra careful with exact figures like percentages, prices, or survey results. Trace these numbers back to the original source to confirm the date, the group of people studied (the sample), and the method used to calculate them.

5. Check for current information Verify when the source was published or last updated. Some topics, like laws, prices, or product features, change quickly, so an answer based on old data might be incorrect even if the source was once reliable.

6. Test for consistency with a new prompt If an answer feels unreliable, try asking the same factual question again using neutral language. Compare the details of the new answer to the first one to see if the facts remain the same.

Example: If you are using AI to help plan a trip and it gives you a specific price for a hotel, do not just accept the number. Search for the hotel’s official website to see if that price is current and if it matches what the AI claimed.

Tips and common mistakes

• Don’t trust polished writing: AI can write very convincingly, but a smooth-sounding answer does not mean the facts are correct. A “hallucination” happens when the model fills in gaps with false or inconsistent information.
• Watch for “too exact” numbers: While a specific number like “36.4%” sounds researched, it is often a place where errors hide. Always trace these back to the original dataset.
• A broken link isn’t always a hallucination: If a URL doesn’t work, it might just be a temporary issue. However, if the publisher has no record of the article or the author has no connection to the work, treat it as a major red flag.
• Look for inconsistencies: If the AI provides an explanation that contradicts itself, pay closer attention to the sources provided.

Use it responsibly

Always check AI-generated answers against independent, primary sources before relying on them for important tasks. Protect your privacy by never pasting private or sensitive data into an AI tool. Finally, ensure you are following the rules of your workplace or school, as well as the service’s own terms of use, when using these tools.