How to check AI answers for accuracy with Amazon Bedrock checks

What you need

• An AI tool you already use or plan to try.
• A question where accuracy matters, such as a trip plan, an email to a friend or landlord, or a tip for organising your photos.
• A trusted place to check facts, such as an official website or a reference book you already rely on.
• The AWS post about Amazon Bedrock Automated Reasoning checks, if you want to read more about the feature.

This guide is based on a short AWS description. It explains the idea behind the feature, but it does not list menus, buttons or settings, so this guide does not give any. Where the source is silent, the steps below stick to what you can do yourself.

Step by step

1. Understand why answers can be wrong. Large language models (LLMs) generate text, and that text can contain inaccurate facts. AWS calls this problem hallucinations, and its post is about reducing them. Knowing this risk exists is the first defence.

2. Learn the name of the accuracy feature. AWS describes Amazon Bedrock Automated Reasoning checks as a way to improve the factual accuracy of LLM responses and to help prevent hallucinations. Note the full name so you can look it up later.

3. Check whether the feature is available to you. AWS announced these checks at its re:Invent 2024 conference and describes them as being in gated preview. A gated preview is not always open to everyone, so read the AWS post for the current status before you plan around it.

4. Read the full AWS post for how the checks are used. The post discusses how to help prevent generative AI hallucinations using these checks. Read it in full rather than relying on a summary, because the details matter.

5. Treat every answer as unverified until you check it. This step is a habit, not a feature of any tool. Example: an AI tells you a train on your trip route runs every 10 minutes. Before you rely on that, open the operator’s official timetable and confirm the time yourself.

6. Check the parts that matter most. Pick the dates, names, prices and numbers in the answer and compare each one with your trusted source. If a fact does not match, use the trusted source and ignore the AI’s version.

7. Ask the AI to show its uncertainty. You can ask the AI to flag the facts it is least sure of, then check those first. Copy this prompt into any AI chat:

Answer my question, then list the facts you are least sure of so I can check them.

Tips and common mistakes

• Do not stop at a confident tone. A clear, polished answer can still contain false facts. Confidence in the wording is not evidence that the content is correct.
• Check the source, not just the answer. If the AI names a website or document, open it yourself and confirm the claim is really there.
• Be careful with email drafts. If an AI writes an email with a date, address or amount, check each detail against your own records before you send it.
• Double-check photo or file advice. If an AI tells you how to organise or delete files, test the steps on a copy first so you do not lose original photos.
• Do not assume a feature exists. An AI may describe a button or setting that your app does not have. Look for it yourself before you follow the instructions.
• Do not treat one check as final. Even after you verify one fact, check related facts that depend on it.

Use it responsibly

When you check AI answers, keep your own judgement in charge. Do not rely on an AI answer for health, money, legal or safety decisions without checking it with a qualified source.

Protect your privacy. Never paste private data into an AI tool, such as passwords, bank details, medical records, or the personal details of other people. Use only the information you are comfortable sharing.

Follow the rules where you work or study. Your workplace or school may limit which AI tools you can use or what you can share with them. Also read the service’s own terms before you use it, and do not use it in ways those terms forbid.

Keep your account secure. Use a strong, unique password for any AI service, and sign out on shared computers.