A research team at Yongin Severance Hospital has demonstrated the effectiveness of ChatGPT, specifically the GPT-4 model, in accurately predicting cardiovascular diseases.
GPT-4, the latest iteration of the ChatGPT series developed by OpenAI, has previously garnered attention for passing the United States Medical Licensing Examination (USMLE) with a score over 90 percent and diagnosing rare congenital diseases with remarkable accuracy.
However, as there were the challenges of applying AI in medical fields due to concerns over hallucination phenomena, accuracy, and bias, the team, led by Professors Bae Sung-A of the Department of Cardiology and Yoon Duk-yong of the Department of Biomedical Systems Informatics, set out to provide compelling evidence of its viability and usefulness.
The research team evaluated GPT-4's predictive capabilities using extensive patient cohort data from the U.K. Biobank, involving approximately 50,000 individuals, and the Korean Genome and Epidemiology Study (KoGES), which included around 6,000 participants.
By analyzing age, medical history, and blood test data, the team aimed to predict the occurrence of cardiovascular diseases over a decade.
Results indicated that GPT-4's performance in forecasting cardiovascular events is comparable to widely used models like the Framingham Risk Score and the risk scores from the American College of Cardiology and the American Heart Association (ACC/AHA).
The study utilized the Area Under the Receiver Operating Characteristic (AUROC) curve, a statistical measure, to assess the model's accuracy.
"Although GPT-4 was not specifically designed for medical purposes, it has acquired substantial medical knowledge from its vast training data,” Professor Yoon said. “This enables it to combine various cardiovascular risk factors effectively, leading to significant findings."
Professor Bae also highlighted the study's significance, noting, "This is the first time the accuracy and utility of cardiovascular disease predictions based on a large language model have been validated.”
The team anticipates that the GPT-4 model will become a promising tool in the medical field, he added.
The results of the research were published in iScience.