Universität zu Köln

Kaya, Kenan ORCID: 0009-0008-7625-3457, Gietzen, Carsten ORCID: 0000-0002-2354-3847, Hahnfeldt, Robert ORCID: 0000-0001-7997-3216, Zoubi, Maher, Emrich, Tilman, Halfmann, Moritz C., Sieren, Malte Maria, Elser, Yannic, Krumm, Patrick, Brendel, Jan M., Nikolaou, Konstantin, Haag, Nina, Borggrefe, Jan, Krüchten, Ricarda von, Müller-Peltzer, Katharina, Ehrengut, Constantin, Denecke, Timm, Hagendorff, Andreas, Goertz, Lukas ORCID: 0000-0002-2620-7611, Gertz, Roman J. ORCID: 0000-0002-6414-4105, Bunck, Alexander Christian ORCID: 0000-0003-0986-0042, Maintz, David ORCID: 0000-0002-8942-3776, Persigehl, Thorsten ORCID: 0000-0001-5928-4405, Lennartz, Simon ORCID: 0009-0000-7189-3764, Luetkens, Julian A., Jaiswal, Astha ORCID: 0000-0002-1502-1707, Iuga, Andra Iza ORCID: 0000-0002-3694-0235, Pennig, Lenhard ORCID: 0000-0002-6606-9313 and Kottlors, Jonathan ORCID: 0000-0001-5021-6895 (2024). Generative Pre-trained Transformer 4 analysis of cardiovascular magnetic resonance reports in suspected myocarditis: A multicenter study. Journal of Cardiovascular Magnetic Resonance, 26 (2). pp. 1-10. Elsevier. ISSN 1097-6647

PDF 1-s2.0-S1097664724010950-main.pdf Bereitstellung unter der CC-Lizenz: Creative Commons Attribution. Download (5MB)

Abstract

[Artikel-Nr.: 101068] Background: Diagnosing myocarditis relies on multimodal data, including cardiovascular magnetic resonance (CMR), clinical symptoms, and blood values. The correct interpretation and integration of CMR findings require radiological expertise and knowledge. We aimed to investigate the performance of Generative Pre-trained Transformer 4 (GPT-4), a large language model, for report-based medical decision-making in the context of cardiac MRI for suspected myocarditis. Methods: This retrospective study includes CMR reports from 396 patients with suspected myocarditis and eight centers, respectively. CMR reports and patient data including blood values, age, and further clinical information were provided to GPT-4 and radiologists with 1 (resident 1), 2 (resident 2), and 4 years (resident 3) of experience in CMR and knowledge of the 2018 Lake Louise Criteria. The final impression of the report regarding the radiological assessment of whether myocarditis is present or not was not provided. The performance of Generative pre-trained transformer 4 (GPT-4) and the human readers were compared to a consensus reading (two board-certified radiologists with 8 and 10 years of experience in CMR). Sensitivity, specificity, and accuracy were calculated. Results: GPT-4 yielded an accuracy of 83%, sensitivity of 90%, and specificity of 78%, which was comparable to the physician with 1 year of experience (R1: 86%, 90%, 84%, p = 0.14) and lower than that of more experienced physicians (R2: 89%, 86%, 91%, p = 0.007 and R3: 91%, 85%, 96%, p < 0.001). GPT-4 and human readers showed a higher diagnostic performance when results from T1- and T2-mapping sequences were part of the reports, for residents 1 and 3 with statistical significance (p = 0.004 and p = 0.02, respectively). Conclusion: GPT-4 yielded good accuracy for diagnosing myocarditis based on CMR reports in a large dataset from multiple centers and therefore holds the potential to serve as a diagnostic decision-supporting tool in this capacity, particularly for less experienced physicians. Further studies are required to explore the full potential and elucidate educational aspects of the integration of large language models in medical decision-making.

Item Type:	Article
Creators:	Creators Email ORCID ORCID Put Code Kaya, Kenan UNSPECIFIED https://orcid.org/0009-0008-7625-3457 UNSPECIFIED Gietzen, Carsten UNSPECIFIED https://orcid.org/0000-0002-2354-3847 UNSPECIFIED Hahnfeldt, Robert UNSPECIFIED https://orcid.org/0000-0001-7997-3216 UNSPECIFIED Zoubi, Maher UNSPECIFIED UNSPECIFIED UNSPECIFIED Emrich, Tilman UNSPECIFIED UNSPECIFIED UNSPECIFIED Halfmann, Moritz C. UNSPECIFIED UNSPECIFIED UNSPECIFIED Sieren, Malte Maria UNSPECIFIED UNSPECIFIED UNSPECIFIED Elser, Yannic UNSPECIFIED UNSPECIFIED UNSPECIFIED Krumm, Patrick UNSPECIFIED UNSPECIFIED UNSPECIFIED Brendel, Jan M. UNSPECIFIED UNSPECIFIED UNSPECIFIED Nikolaou, Konstantin UNSPECIFIED UNSPECIFIED UNSPECIFIED Haag, Nina UNSPECIFIED UNSPECIFIED UNSPECIFIED Borggrefe, Jan UNSPECIFIED UNSPECIFIED UNSPECIFIED Krüchten, Ricarda von UNSPECIFIED UNSPECIFIED UNSPECIFIED Müller-Peltzer, Katharina UNSPECIFIED UNSPECIFIED UNSPECIFIED Ehrengut, Constantin UNSPECIFIED UNSPECIFIED UNSPECIFIED Denecke, Timm UNSPECIFIED UNSPECIFIED UNSPECIFIED Hagendorff, Andreas UNSPECIFIED UNSPECIFIED UNSPECIFIED Goertz, Lukas UNSPECIFIED https://orcid.org/0000-0002-2620-7611 UNSPECIFIED Gertz, Roman J. UNSPECIFIED https://orcid.org/0000-0002-6414-4105 UNSPECIFIED Bunck, Alexander Christian UNSPECIFIED https://orcid.org/0000-0003-0986-0042 UNSPECIFIED Maintz, David UNSPECIFIED https://orcid.org/0000-0002-8942-3776 UNSPECIFIED Persigehl, Thorsten UNSPECIFIED https://orcid.org/0000-0001-5928-4405 UNSPECIFIED Lennartz, Simon UNSPECIFIED https://orcid.org/0009-0000-7189-3764 UNSPECIFIED Luetkens, Julian A. UNSPECIFIED UNSPECIFIED UNSPECIFIED Jaiswal, Astha UNSPECIFIED https://orcid.org/0000-0002-1502-1707 UNSPECIFIED Iuga, Andra Iza UNSPECIFIED https://orcid.org/0000-0002-3694-0235 UNSPECIFIED Pennig, Lenhard UNSPECIFIED https://orcid.org/0000-0002-6606-9313 UNSPECIFIED Kottlors, Jonathan UNSPECIFIED https://orcid.org/0000-0001-5021-6895 UNSPECIFIED
URN:	urn:nbn:de:hbz:38-814315
Identification Number:	10.1016/j.jocmr.2024.101068
Journal or Publication Title:	Journal of Cardiovascular Magnetic Resonance
Volume:	26
Number:	2
Page Range:	pp. 1-10
Number of Pages:	10
Date:	2024
Publisher:	Elsevier
ISSN:	1097-6647
Language:	English
Faculty:	Faculty of Medicine
Divisions:	Faculty of Medicine > Radiologische Diagnostik > Institut und Poliklinik für Radiologische Diagnostik
Subjects:	Medical sciences Medicine
Uncontrolled Keywords:	Keywords Language Cardiovascular magnetic resonance ; Generative Pre-trained Transformer 4 ; Artificial intelligence ; Large language models ; Myocarditis English
['eprint_fieldname_oa_funders' not defined]:	Publikationsfonds UzK
Refereed:	Yes
URI:	http://kups.ub.uni-koeln.de/id/eprint/81431