Published on
Updated
Reading 3 mins.
in collaboration with
Dr Gérald Kierzek (Medical Director of Doctissimo)
A study compared physicians’ written responses and ChatGPT responses to patient questions. Result: a panel of health professionals preferred ChatGPT answers in 79% of cases, finding them even more precise and more empathetic. Should you throw out your doctor? The opinion of Dr Gérald Kierzek, emergency doctor and medical director of Doctissimo.
There have been many speculations about how ChatGPT could be used in medicine. We have seen that according to a recent study, ChatGPT could be able to pass an exam to become a doctor. But can he directly answer patients’ questions with precision and empathy? That’s the question Dr. Davey Smith, physician-scientist, co-director of the UC San Diego Altman Clinical and Translational Research Institute and professor at the UC San Diego School, attempted to answer. of Medicine.
A methodology that questions…
To get a large and diverse sample of health questions and answers from doctors that did not contain personally identifiable information, the team turned to social media where millions of patients post medical questions answered by doctors. : RedditAskDocs. While anyone can answer a question, moderators check the credentials of healthcare professionals and the answers display the respondent’s level of credentials. The result is a large and diverse collection of patient medical questions and associated answers from licensed medical professionals. Do these exchanges reflect clinical experiences? Yes, according to the team members. A point that remains very open to criticism, however, a doctor does not address his patient whom he knows and face-to-face, in the same way that he will respond on social networks to a stranger.
The team randomly sampled 195 AskDocs exchanges where a doctor answered a public question. The team provided the original question to ChatGPT and asked them to write an answer. A panel of three licensed medical professionals evaluated each question and the corresponding answers and did not know if the answer came from a doctor or from ChatGPT. They compared responses based on quality of information and empathy, noting which they liked best.
ChatGPT more accurate and more empathetic?
The panel of healthcare professional evaluators preferred ChatGPT responses over physician responses 79% of the time. “ChatGPT answers provide nuanced and accurate information that often touched on more aspects of the patient’s questions than the doctor’s answers.“, said Jessica Kelley, nurse practitioner of the Human Longevity Society of San Diego and co-author of the study.
In addition, the quality of ChatGPT responses was significantly higher than that of doctors’ responses: good or very good quality responses were 3.6 times higher for ChatGPT than for doctors (doctors 22.1% versus ChatGPT 78.5 %). Responses were also more empathetic: empathetic or very empathetic responses were 9.8 times higher for ChatGPT than for doctors (doctors 4.6% vs. ChatGPT 45.1%).
“I never thought I would say that“, added Dr. Aaron Goodman, associate clinical professor at UC San Diego School of Medicine and co-author of the study, “but ChatGPT is a great helper that I would like to add to my inbox. The tool will transform the way I support my patients“.
Harnessing AI to respond to patient messages?
The authors deny wanting to compare doctors and AI, but rather want to insist on the help that AI could bring to doctors in their relationship with patients. According to Dr. Christopher Longhurst, chief medical officer and chief digital officer at UC San Diego Health. “These results suggest that tools like ChatGPT can efficiently write high-quality personalized medical advice before it is validated by clinicians, and we are beginning to implement it at UCSD Health.“.
According to the authors, in addition to improving workflow, implementing AI assistants in messaging could impact patient health and physician performance.
“Empathy is not just a form of politeness”
For Doctor Gérald Kierzek, emergency doctor and medical director of Doctissimo, the study does not compare ChatGPT with the empathy of a real doctor-patient relationship. According to him, “it would have been necessary to compare the answers of ChatGPT with what occurs during the singular colloquium which is the medical consultation and to ask the patients to evaluate the empathy and the precision of the returns. This study purports to measure the empathy of stereotypically worded, quick and terse physician responses due to the format of medical forums.“.
Beyond this questionable methodological comparison, the doctor returns to the very definition of empathy and emphasizes its capital importance during diagnoses or serious pathologies. “Empathy in the care relationship means greater attention to the patient, an attitude of listening and availability. It is not just a polite formula but a real adaptation in real time according to the emotions of the patient. No machine will replace the human relationship” concludes Dr Gérald Kierzek.