In the study, the researchers had two versions of Chat GPT draw conclusions based on doctors’ notes on 30 patients. It is about symptoms and results from physical examinations that have been entered into the systems. In addition, there was information on lab values as a result of blood and urine tests.
For each patient, the Chat GPT made suggestions for different diagnoses which were compared with the suggestions made by doctors and the correct diagnosis that was later made. The researchers compared in how many of the cases the correct answer was among the top five suggestions provided by doctors or Chat GPT.
According to the results, as published in the journal Annals of Emergency Medicine, doctors had the correct diagnosis among their top five suggestions 87 percent of the time. It was the same share as version 3.5 of Chat GPT. The later version 4.0, on the other hand, had the correct diagnosis in 97 percent of the cases.
The researchers point out that Chat GPT is not a medical device and that there are many privacy concerns when data is shared with it.
“But there is potential for us to use it in the future to save time and reduce waiting times in the emergency room. Artificial intelligence could also be a tool for less experienced doctors or to find rare diseases,” says Hidde ten Berg at the Jeroen Bosch Hospital in the Netherlands and one of the researchers behind the study in a press comment.