personalized education models, such as chatgpt (or chat generative pre-trained transformer), are growing in popularity because of their ability to respond to user prompts with human-like responses drawn from vast databases. the software has already demonstrated its value to the health-care industry through its ability to compose basic medical reports and correctly answer questions that commonly appear on medical school exams.
“chatgpt has shown to be able to provide professional, yet highly comprehensible responses,” said yee hui yeo, first author of the study and a clinical fellow in the karsh division of gastroenterology and hepatology at cedars-sinai. “however, this is one of the first studies to examine the ability of chatgpt to answer clinically oriented, disease-specific questions correctly and compare its performance to physicians and trainees.”
to test chatgpt’s knowledge of cirrhosis and liver cancer, the team presented the chatbot with 164 frequently asked questions that were divided into five categories: basic knowledge, diagnosis, treatment, lifestyle or preventive medicine. two liver transplant specialists graded the results.
according to investigators, chatgpt answered about 77 per cent of questions correctly, with high levels of accuracy in over 90 per cent of questions from different categories. however, the liver specialists who graded the answers found that 75 per cent of responses relating to basic knowledge, treatment and lifestyle categories were correct but inadequate. the proportion of answers that were “mixed with correct and incorrect data” was as follows: basic knowledge (22 per cent), diagnosis (33 per cent), treatment (25 per cent), lifestyle (18 per cent) and preventive medicine (50 per cent).
chatgpt also a tool for doctors