Comparison of State-of-the-Art Neural Network Survival Models with the Pooled Cohort Equations for Cardiovascular Disease Risk Prediction

Yu Deng, Lei Liu, Hongmei Jiang, Yifan Peng, Yishu Wei, Zhiyang Zhou, Yizhen Zhong, Yun Zhao, Xiaoyun Yang, Jingzhi Yu, Zhiyong Lu, Abel Kho, Hongyan Ning, Norrina B. Allen, John T. Wilkins, Kiang Liu, Donald M. Lloyd-Jones, Lihui Zhao

Research output: Contribution to journalArticlepeer-review

3 Scopus citations

Abstract

Background: The Pooled Cohort Equations (PCEs) are race- and sex-specific Cox proportional hazards (PH)-based models used for 10-year atherosclerotic cardiovascular disease (ASCVD) risk prediction with acceptable discrimination. In recent years, neural network models have gained increasing popularity with their success in image recognition and text classification. Various survival neural network models have been proposed by combining survival analysis and neural network architecture to take advantage of the strengths from both. However, the performance of these survival neural network models compared to each other and to PCEs in ASCVD prediction is unknown. Methods: In this study, we used 6 cohorts from the Lifetime Risk Pooling Project (with 5 cohorts as training/internal validation and one cohort as external validation) and compared the performance of the PCEs in 10-year ASCVD risk prediction with an all two-way interactions Cox PH model (Cox PH-TWI) and three state-of-the-art neural network survival models including Nnet-survival, Deepsurv, and Cox-nnet. For all the models, we used the same 7 covariates as used in the PCEs. We fitted each of the aforementioned models in white females, white males, black females, and black males, respectively. We evaluated models’ internal and external discrimination power and calibration. Results: The training/internal validation sample comprised 23216 individuals. The average age at baseline was 57.8 years old (SD = 9.6); 16% developed ASCVD during average follow-up of 10.50 (SD = 3.02) years. Based on 10 × 10 cross-validation, the method that had the highest C-statistics was Deepsurv (0.7371) for white males, Deepsurv and Cox PH-TWI (0.7972) for white females, PCE (0.6981) for black males, and Deepsurv (0.7886) for black females. In the external validation dataset, Deepsurv (0.7032), Cox-nnet (0.7282), PCE (0.6811), and Deepsurv (0.7316) had the highest C-statistics for white male, white female, black male, and black female population, respectively. Calibration plots showed that in 10 × 10 validation, all models had good calibration in all race and sex groups. In external validation, all models overestimated the risk for 10-year ASCVD. Conclusions: We demonstrated the use of the state-of-the-art neural network survival models in ASCVD risk prediction. Neural network survival models had similar if not superior discrimination and calibration compared to PCEs.

Original languageEnglish
Article number22
JournalBMC Medical Research Methodology
Volume23
Issue number1
DOIs
StatePublished - Dec 2023

Keywords

  • Artificial intelligence
  • Cardiovascular disease
  • Cox regression
  • Deep learning
  • Machine learning
  • Neural network
  • Pooled Cohort Equations
  • Predictive modeling
  • Survival analysis

Fingerprint

Dive into the research topics of 'Comparison of State-of-the-Art Neural Network Survival Models with the Pooled Cohort Equations for Cardiovascular Disease Risk Prediction'. Together they form a unique fingerprint.

Cite this