Follow up imaging recommendations are often made in radiology reports. Not adhering to these recommendations can result in treatment delays, negative patient outcomes, additional medical issues, excessive testing, financial loss, and potential legal repercussions. In this paper, we present a generative-discriminative deep learning approach to classify radiology reports based on the presence of follow up recommendations. The generative model aims to produce a distributed representation of words in radiology reports. The discriminative model aims to extract local features that can be used to identify follow up recommendations. Three radiology report datasets were collected: one (n = 41417) with examinations extracted from the information system by regular expression operations, one (n = 14993) with examinations performed between January 1, 2015 to January 10, 2015 and annotated by three radiologists, and one (n = 5093) annotated by our radiologists at the time of dictation. Classification performance of the proposed hybrid model was compared with four traditional classification algorithms (random forest, logistic regression, naive bayes and support vector machine (SVM)) and three neural network-based models (fastText, Convolutional neural network (CNN), Graph convolutional network (GCN)). A visualization algorithm was used to interpret the classification results of the hybrid model. Precision, recall, accuracy, and F1 scores were calculated for all models on the test set . Experimental results show that the hybrid model had a statistically significant higher F1 score (0.942 for Hybrid-random and 0.951 for Hybrid-report) than did four traditional machine learning algorithms (random forest: 0.914, logistic regression: 0.838, naive bayes: 0.808 and SVM: 0.890) and the best-performing neural network-based model (fastText: 0.918). Our model can accurately identify radiology reports that contain follow-up recommendations. It can also automate detection of follow-up recommendation, thus streamlining workflows in a clinical setting and ensuring timely and appropriate patient care.

Original languageEnglish
Title of host publicationMedical Imaging 2023
Subtitle of host publicationImaging Informatics for Healthcare, Research, and Applications
EditorsBrian J. Park, Hiroyuki Yoshida
ISBN (Electronic)9781510660434
StatePublished - 2023
EventMedical Imaging 2023: Imaging Informatics for Healthcare, Research, and Applications - San Diego, United States
Duration: Feb 19 2023Feb 21 2023

Publication series

NameProgress in Biomedical Optics and Imaging - Proceedings of SPIE
ISSN (Print)1605-7422


ConferenceMedical Imaging 2023: Imaging Informatics for Healthcare, Research, and Applications
Country/TerritoryUnited States
CitySan Diego


  • Deep learning
  • Follow up recommendation
  • Radiology report classification


Dive into the research topics of 'A Generative-Discriminative Deep Learning Approach to Classify Radiology Reports based on the Presence of Follow Up Recommendations'. Together they form a unique fingerprint.

Cite this