TY - GEN
T1 - Attention-Based CNN-BiLSTM for Sleep State Classification of Spatiotemporal Wide-Field Calcium Imaging Data
AU - Zhang, Xiaohui
AU - Landsness, Eric C.
AU - Culver, Joseph P.
AU - Lee, Jin-Moo
AU - Anastasio, Mark A.
N1 - Publisher Copyright:
© 2023 SPIE.
PY - 2023
Y1 - 2023
N2 - Wide-field calcium imaging (WFCI) with genetically encoded calcium indicators allows spatiotemporal recordings of neuronal activity in preclinical models. When applied to the study of sleep, WFCI data are manually scored into sleep states of wakefulness, non-rapid eye movement (NREM) and REM by use of adjunct electroencephalogram (EEG) and electromyogram (EMG) recordings. However, this process is time-consuming, invasive and suffers from low inter- and intra-rater reliability. To overcome these limitations, an automated sleep state classification method that operates on spatiotemporal WFCI recordings is desired. Previous work that classifies sleep states from WFCI data by use of multiplex visibility graphs and deep learning only leverages shared information derived from average time series across parcellated brain regions, and thus fails to fully explore the spatiotemporal calcium dynamics recorded. In this work, a hybrid network architecture consisting of a convolutional neural network (CNN) to extract spatial features of image frames and a bidirectional long short-term memory network (BiLSTM) with attention mechanism to identify temporal dependencies among different time points was proposed to jointly learn spatial and temporal information from the WFCI sleep data. Nineteen transgenic mice expressing GCaMP6f in excitatory neurons were used for network training and testing. The CNN-BiLSTM achieved a weighted F1-score of 0.84 and Cohen's κ of 0.64, indicating substantial agreement with EEG/EMG-based human scoring. The gradient-weighted class activation maps were computed to provide deeper insights into the brain regions most relevant to the inference of individual sleep state. This work will enable further investigation of sleep neural activity using WFCI.
AB - Wide-field calcium imaging (WFCI) with genetically encoded calcium indicators allows spatiotemporal recordings of neuronal activity in preclinical models. When applied to the study of sleep, WFCI data are manually scored into sleep states of wakefulness, non-rapid eye movement (NREM) and REM by use of adjunct electroencephalogram (EEG) and electromyogram (EMG) recordings. However, this process is time-consuming, invasive and suffers from low inter- and intra-rater reliability. To overcome these limitations, an automated sleep state classification method that operates on spatiotemporal WFCI recordings is desired. Previous work that classifies sleep states from WFCI data by use of multiplex visibility graphs and deep learning only leverages shared information derived from average time series across parcellated brain regions, and thus fails to fully explore the spatiotemporal calcium dynamics recorded. In this work, a hybrid network architecture consisting of a convolutional neural network (CNN) to extract spatial features of image frames and a bidirectional long short-term memory network (BiLSTM) with attention mechanism to identify temporal dependencies among different time points was proposed to jointly learn spatial and temporal information from the WFCI sleep data. Nineteen transgenic mice expressing GCaMP6f in excitatory neurons were used for network training and testing. The CNN-BiLSTM achieved a weighted F1-score of 0.84 and Cohen's κ of 0.64, indicating substantial agreement with EEG/EMG-based human scoring. The gradient-weighted class activation maps were computed to provide deeper insights into the brain regions most relevant to the inference of individual sleep state. This work will enable further investigation of sleep neural activity using WFCI.
KW - Wide-field calcium imaging
KW - attention mechanism
KW - convolutional neural network
KW - long short-term memory
KW - sleep state classification
UR - http://www.scopus.com/inward/record.url?scp=85159779256&partnerID=8YFLogxK
U2 - 10.1117/12.2649375
DO - 10.1117/12.2649375
M3 - Conference contribution
AN - SCOPUS:85159779256
T3 - Progress in Biomedical Optics and Imaging - Proceedings of SPIE
BT - Neural Imaging and Sensing 2023
A2 - Luo, Qingming
A2 - Ding, Jun
A2 - Fu, Ling
PB - SPIE
T2 - Neural Imaging and Sensing 2023
Y2 - 30 January 2023 through 31 January 2023
ER -