Transformer-based Multi-target Regression on Electronic Health Records for Primordial Prevention of Cardiovascular Disease

Raphael Poulain, Mehak Gupta, Randi Foraker, Rahmatollah Beheshti

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

4 Scopus citations

Abstract

Machine learning algorithms have been widely used to capture the static and temporal patterns within electronic health records (EHRs). While many studies focus on the (primary) prevention of diseases, primordial prevention (preventing the factors that are known to increase the risk of a disease occurring) is still widely under-investigated. In this study, we propose a multi-target regression model leveraging transformers to learn the bidirectional representations of EHR data and predict the future values of 11 major modifiable risk factors of cardiovascular disease (CVD). Inspired by the proven results of pre-training in natural language processing studies, we apply the same principles on EHR data, dividing the training of our model into two phases: pre-training and fine-tuning. We u se t he fine-tuned transformer model in a 'multi-target regression' theme. Following this theme, we combine the 11 disjoint prediction tasks by adding shared and target-specific l ayers t o t he m odel and jointly train the entire model. We evaluate the performance of our proposed method on a large publicly available EHR dataset. Through various experiments, we demonstrate that the proposed method obtains a significant improvement (12.6% MAE on average across all 11 different outputs) over the baselines.

Original languageEnglish
Title of host publicationProceedings - 2021 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2021
EditorsYufei Huang, Lukasz Kurgan, Feng Luo, Xiaohua Tony Hu, Yidong Chen, Edward Dougherty, Andrzej Kloczkowski, Yaohang Li
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages726-731
Number of pages6
ISBN (Electronic)9781665401265
DOIs
StatePublished - 2021
Event2021 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2021 - Virtual, Online, United States
Duration: Dec 9 2021Dec 12 2021

Publication series

NameProceedings - 2021 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2021

Conference

Conference2021 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2021
Country/TerritoryUnited States
CityVirtual, Online
Period12/9/2112/12/21

Keywords

  • cardiovascular disease
  • electronic health records
  • multi-target regression
  • prevention
  • transformers

Fingerprint

Dive into the research topics of 'Transformer-based Multi-target Regression on Electronic Health Records for Primordial Prevention of Cardiovascular Disease'. Together they form a unique fingerprint.

Cite this