TY - JOUR
T1 - Prediction Models for Glaucoma in a Multicenter Electronic Health Records Consortium
T2 - The Sight Outcomes Research Collaborative
AU - SOURCE Consortium
AU - Wang, Sophia Y.
AU - Ravindranath, Rohith
AU - Stein, Joshua D.
AU - Amin, Sejal
AU - Edwards, Paul A.
AU - Srikumaran, Divya
AU - Woreta, Fasika
AU - Schultz, Jeffrey S.
AU - Shrivastava, Anurag
AU - Ahmad, Baseer
AU - Kim, Judy
AU - Bryar, Paul
AU - French, Dustin
AU - Vanderbeek, Brian L.
AU - Pershing, Suzann
AU - Wang, Sophia Y.
AU - Lynch, Anne M.
AU - Patnaik, Jenna
AU - Munir, Saleha
AU - Munir, Wuqaas
AU - Stein, Joshua
AU - DeLott, Lindsey
AU - Stagg, Brian C.
AU - Wirostko, Barbara
AU - McMillian, Brian
AU - Sheybani, Arsham
N1 - Publisher Copyright:
© 2023 American Academy of Ophthalmology
PY - 2024/5/1
Y1 - 2024/5/1
N2 - Purpose: Advances in artificial intelligence have enabled the development of predictive models for glaucoma. However, most work is single-center and uncertainty exists regarding the generalizability of such models. The purpose of this study was to build and evaluate machine learning (ML) approaches to predict glaucoma progression requiring surgery using data from a large multicenter consortium of electronic health records (EHR). Design: Cohort study. Participants: Thirty-six thousand five hundred forty-eight patients with glaucoma, as identified by International Classification of Diseases (ICD) codes from 6 academic eye centers participating in the Sight OUtcomes Research Collaborative (SOURCE). Methods: We developed ML models to predict whether patients with glaucoma would progress to glaucoma surgery in the coming year (identified by Current Procedural Terminology codes) using the following modeling approaches: (1) penalized logistic regression (lasso, ridge, and elastic net); (2) tree-based models (random forest, gradient boosted machines, and XGBoost), and (3) deep learning models. Model input features included demographics, diagnosis codes, medications, and clinical information (intraocular pressure, visual acuity, refractive status, and central corneal thickness) available from structured EHR data. One site was reserved as an “external site” test set (N = 1550); of the patients from the remaining sites, 10% each were randomly selected to be in development and test sets, with the remaining 27 999 reserved for model training. Main Outcome Measures: Evaluation metrics included area under the receiver operating characteristic curve (AUROC) on the test set and the external site. Results: Six thousand nineteen (16.5%) of 36 548 patients underwent glaucoma surgery. Overall, the AUROC ranged from 0.735 to 0.771 on the random test set and from 0.706 to 0.754 on the external test site, with the XGBoost and random forest model performing best, respectively. There was greatest performance decrease from the random test set to the external test site for the penalized regression models. Conclusions: Machine learning models developed using structured EHR data can reasonably predict whether glaucoma patients will need surgery, with reasonable generalizability to an external site. Additional research is needed to investigate the impact of protected class characteristics such as race or gender on model performance and fairness. Financial Disclosure(s): Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.
AB - Purpose: Advances in artificial intelligence have enabled the development of predictive models for glaucoma. However, most work is single-center and uncertainty exists regarding the generalizability of such models. The purpose of this study was to build and evaluate machine learning (ML) approaches to predict glaucoma progression requiring surgery using data from a large multicenter consortium of electronic health records (EHR). Design: Cohort study. Participants: Thirty-six thousand five hundred forty-eight patients with glaucoma, as identified by International Classification of Diseases (ICD) codes from 6 academic eye centers participating in the Sight OUtcomes Research Collaborative (SOURCE). Methods: We developed ML models to predict whether patients with glaucoma would progress to glaucoma surgery in the coming year (identified by Current Procedural Terminology codes) using the following modeling approaches: (1) penalized logistic regression (lasso, ridge, and elastic net); (2) tree-based models (random forest, gradient boosted machines, and XGBoost), and (3) deep learning models. Model input features included demographics, diagnosis codes, medications, and clinical information (intraocular pressure, visual acuity, refractive status, and central corneal thickness) available from structured EHR data. One site was reserved as an “external site” test set (N = 1550); of the patients from the remaining sites, 10% each were randomly selected to be in development and test sets, with the remaining 27 999 reserved for model training. Main Outcome Measures: Evaluation metrics included area under the receiver operating characteristic curve (AUROC) on the test set and the external site. Results: Six thousand nineteen (16.5%) of 36 548 patients underwent glaucoma surgery. Overall, the AUROC ranged from 0.735 to 0.771 on the random test set and from 0.706 to 0.754 on the external test site, with the XGBoost and random forest model performing best, respectively. There was greatest performance decrease from the random test set to the external test site for the penalized regression models. Conclusions: Machine learning models developed using structured EHR data can reasonably predict whether glaucoma patients will need surgery, with reasonable generalizability to an external site. Additional research is needed to investigate the impact of protected class characteristics such as race or gender on model performance and fairness. Financial Disclosure(s): Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.
KW - Deep learning
KW - Glaucoma
KW - Machine learning
KW - Multicenter study
UR - http://www.scopus.com/inward/record.url?scp=85185282707&partnerID=8YFLogxK
U2 - 10.1016/j.xops.2023.100445
DO - 10.1016/j.xops.2023.100445
M3 - Article
C2 - 38317869
AN - SCOPUS:85185282707
SN - 2666-9145
VL - 4
JO - Ophthalmology Science
JF - Ophthalmology Science
IS - 3
M1 - 100445
ER -