AI transcriber for use by Ontario doctors 'hallucinated,' generated errors, auditor finds
>During a provincial procurement process for AI medical systems that transcribe conversations between doctors and patients, government evaluators found serious errors in transcripts generated by 20 programs, she said. [...]
>The evaluators ran a pair of test conversations between hypothetical doctors and patients through the AI systems to test their capabilities, and uncovered the problems, the report said.
>Evaluators found that nine of the systems had so-called "hallucinations," fabricating information or making suggestions to patient treatment plans not made by the doctors. Those included referring a patient for therapy or ordering blood tests, the report said.
>Transcripts created by 12 of the 20 programs contained incorrect information like capturing a different drug than was prescribed by the doctor. Seventeen of the 20 systems missed key details about patients’ mental health issues. [...]
>[Ontario Auditor General Shelley Spence] said she personally saw AI scribe systems in use during a recent doctor’s visit.
>“I actually went to my doctor because you can hear that my voice isn't exactly what it normally is … and they were using AI scribe,” she said. “So, I kind of mentioned, ‘Please look at the transcript when you're done.’”
>The evaluators ran a pair of test conversations between hypothetical doctors and patients through the AI systems to test their capabilities, and uncovered the problems, the report said.
>Evaluators found that nine of the systems had so-called "hallucinations," fabricating information or making suggestions to patient treatment plans not made by the doctors. Those included referring a patient for therapy or ordering blood tests, the report said.
>Transcripts created by 12 of the 20 programs contained incorrect information like capturing a different drug than was prescribed by the doctor. Seventeen of the 20 systems missed key details about patients’ mental health issues. [...]
>[Ontario Auditor General Shelley Spence] said she personally saw AI scribe systems in use during a recent doctor’s visit.
>“I actually went to my doctor because you can hear that my voice isn't exactly what it normally is … and they were using AI scribe,” she said. “So, I kind of mentioned, ‘Please look at the transcript when you're done.’”