Latest Insights

How to Test AI Hallucinations in Medical Note-Taking Applications

Introduction to testing Hallucinations in AI Medical Scribe Applications

A hallucination is when an AI writes something that sounds true but never actually happened. In a medical note, that could mean describing an exam that was never done, listing a medicine the doctor never confirmed, or stating a detail about the patient that was never mentioned at all.

The dangerous part is that these sentences read exactly like the real ones around them, nothing about them looks wrong, nothing about the writing style gives it away. This blog explains what counts as a hallucination in a medical note, why it’s especially risky in this setting, how to actually measure it, and what kinds of hallucinations tend to show up once you start testing for them. Drawing on real testing performed by Testiva on a live AI medical scribe, it also shows how healthcare AI testing and hallucination detection help uncover documentation risks before they affect patient records.

What Is an AI Hallucination in a Medical Note?

A hallucination is any part of the note that isn’t actually backed by what was said in the conversation. That includes a detail that was never mentioned at all, but it also includes something real that got twisted along the way, like a possible plan written as a confirmed one, or something the patient said about themselves written as something the doctor personally observed.

The simplest way to think about it: if you can’t point to the exact moment in the conversation that supports a sentence in the note, that sentence is a hallucination, whether it was invented from nothing or just reshaped from something real.

Why AI Hallucinations in Medical Notes Are a Patient Safety Risk

A hallucinated answer from a chatbot is annoying. A hallucinated finding in a medical note can shape an actual treatment decision, because the note often becomes the record other people act on without re-checking the original conversation. A doctor reading a referral letter, or a nurse reading a plan, generally trusts that what’s written reflects what was actually said.

What makes this harder to catch than most software bugs is that the writing itself gives no clue. A made-up exam result and a real one are written in the exact same calm, professional tone, using the same kind of clinical language, in the same part of the note.

In Testiva’s evaluations of AI medical scribes, some of the highest-risk hallucinations were also the hardest to detect because the generated content appeared completely reasonable when read in isolation. There is no way to spot the difference just by reading it casually. You only catch it by checking it against the source conversation directly.

How to Measure Hallucinations in AI Medical Notes

One practical way to evaluate a note is to go sentence by sentence and ask a single question: can this claim be traced back to something that was actually said in the conversation? If yes, it is grounded. If not, it is a hallucination.

You can then calculate a simple groundedness score by dividing the number of supported claims by the total number of claims in the note. A note where every claim is supported scores close to 1. A note containing mostly unsupported content scores closer to 0.

The value of this approach is not the score itself, but the consistency it provides. Instead of relying on a subjective impression that a note “looked fine,” you have a repeatable way to measure clinical note accuracy and whether generated notes remain grounded over time. The challenge is that sentence by sentence review works for a handful of notes but quickly becomes impractical once you’re testing hundreds of conversations.

Using LLM Judges for AI Hallucination Detection in Medical Notes

Reviewing every sentence manually does not scale beyond a small set of test cases. In modern healthcare AI testing, this process is often automated using an LLM as a judge. In practice, this process is often automated using an LLM as a judge. The model is given both the source conversation and the generated note and asked to identify any claims that are not supported by the conversation, producing the same groundedness score automatically across large numbers of notes.

This dramatically reduces the effort required, but the judge itself is not perfectly reliable. Running the same evaluation twice can sometimes produce different results, and the model may flag unsupported content that turns out to be valid when checked manually.

Because of this, the judge works best as a first pass rather than a final verdict. A reliable workflow uses the LLM to quickly identify potentially problematic notes, then has a person review the flagged cases against the original conversation before treating them as confirmed hallucinations. This combination of automated and manual review is a key part of AI hallucination testing in healthcare.

How to Build Tests That Catch AI Hallucinations

You don’t need to wait for a hallucination to show up by accident, you can build conversations that specifically test for it.

Step 1: Write or generate a conversation.

This can be a real recording or a generated script for any medical condition, with any number of speakers.

Step 2: Plant a hallucination risk on purpose.

Build in a moment that’s known to cause trouble, like a doctor saying they’re about to examine the patient but the recording ending before any result is described, a medicine being mentioned and then taken back, or simply never stating the patient’s gender at all.

Step 3: Run it through the system.

Generate the note the same way it would be generated for a real patient, ideally across every note format the system supports, since a hallucination doesn’t always show up in every format the same way.

Step 4: Compare the note to the conversation.

Run it through an LLM judge for a fast first pass across every claim, then manually check anything it flags against the real conversation. Don’t just skim for anything that “feels off,” since hallucinated sentences don’t feel off, they read perfectly normal, this is exactly why the sentence by sentence check matters more than a casual read.

Step 5: Record exactly what was hallucinated, not just that something was.

Note down the specific sentence and what it should have said instead. A vague “this note had a hallucination” tag is much less useful later than an exact record of the planted risk and what the system actually did with it.

Common AI Hallucinations Found in Medical Note-Taking Applications

As part of Testiva’s evaluation of a live AI medical scribe, we ran more than 100 targeted test cases designed to expose hallucination risks. The most common hallucination we found was the system inventing examination findings that were never supported by the conversation. This appeared in 13 separate test cases and showed up in several different ways.

Sometimes no examination happened at all, yet the note still documented normal findings such as “no abnormalities found.” In other cases, something the patient reported about themselves was rewritten as something the doctor had personally observed. We also saw diagnoses being supported by examination findings that never existed, and cases where the recording ended before an examination took place, yet the generated note still described examination results.

We also found cases where the system added information that had never been established in the conversation. In one pediatric encounter, for example, the patient’s gender was never mentioned, but the note consistently referred to the patient as “she” throughout. Another recurring pattern involved treatment plans. In one case, a medication that was explicitly withdrawn by the doctor still appeared in the final plan. In another, a referral that depended on pending test results was documented as if it had already been confirmed.

These patterns were observed repeatedly during Testiva’s testing and highlighted how hallucinations often appear as subtle distortions of real information rather than completely invented facts. What made these issues difficult to catch was that none of them looked unusual. The hallucinated content was written in the same professional tone and placed naturally within the note. In most cases, the only reliable way to find the problem was to compare the note directly against the source conversation. These findings demonstrate why effective hallucination detection requires comparing every generated claim against the source conversation rather than relying on how convincing the note appears.

Conclusion

AI hallucination testing in healthcare is essential because hallucinations in medical notes are difficult to detect and often look indistinguishable from legitimate clinical documentation. A strong QA framework helps ensure that generated notes remain grounded in the source conversation, identifies unsupported claims before they reach patient records, and reduces the risk of documentation errors that can impact clinical decision-making.

At Testiva, we apply these best practices through structured hallucination testing, targeted test-case generation, scalable evaluation workflows, and human-verified validation processes. This helps teams uncover hidden documentation risks, improve clinical note accuracy, and build greater confidence in their AI medical scribe applications through structured healthcare AI testing.

Ready to reduce hallucinations in your AI medical scribe? Contact Testiva today to learn how structured QA can help improve note accuracy, strengthen patient safety, and increase trust in AI-generated clinical documentation.