TY - JOUR
T1 - Comparing stress prediction models using smartwatch physiological signals and participant self-reports
AU - Dai, Ruixuan
AU - Lu, Chenyang
AU - Yun, Linda
AU - Lenze, Eric
AU - Avidan, Michael
AU - Kannampallil, Thomas
N1 - Funding Information:
This research was supported in part by a grant-in-aid from the Division of Clinical and Translational Research of the Department of Anesthesiology, Washington University School of Medicine, St. Louis, Missouri; and in part, by the Healthcare Innovation Lab and the Institute for Informatics at BJC HealthCare and Washington University School of Medicine. The authors would also like to give special thanks to Dr. Simon Haroutounian for providing the physical stressor equipment.
Publisher Copyright:
© 2021 Elsevier B.V.
PY - 2021/9
Y1 - 2021/9
N2 - Recent advances in wearable technology have facilitated the non-obtrusive monitoring of physiological signals, creating opportunities to monitor and predict stress. Researchers have utilized machine learning methods using these physiological signals to develop stress prediction models. Many of these prediction models have utilized objective stressor tasks (e.g., a public speaking task or solving math problems). Alternatively, the subjective user responses with self-reports have also been used for measuring stress. In this paper, we describe a methodological approach (a) to compare the prediction performance of models developed using objective markers of stress using participant-reported subjective markers of stress from self-reports; and (b) to develop personalized stress models by accounting for inter-individual differences. Towards this end, we conducted a laboratory-based study with 32 healthy volunteers. Participants completed a series of stressor tasks—social, cognitive and physical—wearing an instrumented commercial smartwatch that collected physiological signals and participant responses using timed self-reports. After extensive data preprocessing using a combination of signal processing techniques, we developed two types of models: objective stress models using the stressor tasks as labels; and subjective stress models using participant responses to each task as the label for that stress task. We trained and tested several machine learning algorithms—support vector machine (SVM), random forest (RF), gradient boosted trees (GBT), AdaBoost, and Logistic Regression (LR)—and evaluated their performance. SVM had the best performance for the models using the objective stressor (i.e., stressor tasks) with an AUROC of 0.790 and an F-1 score of 0.623. SVM also had the highest performance for the models using the subjective stress (i.e., participant self-reports) with an AUROC of 0.719 and an F-1 score of 0.520. Model performance improved with a personalized threshold model to an AUROC of 0.751 and an F-1 score of 0.599. The performance of the stress models using an instrumented commercial smartwatch was comparable to similar models from other state-of-the-art laboratory-based studies. However, the subjective stress models had a lower performance, indicating the need for further research on the use of self-reports for stress-related studies. The improvement in performance with the personalized threshold-based models provide new directions for building stress prediction models.
AB - Recent advances in wearable technology have facilitated the non-obtrusive monitoring of physiological signals, creating opportunities to monitor and predict stress. Researchers have utilized machine learning methods using these physiological signals to develop stress prediction models. Many of these prediction models have utilized objective stressor tasks (e.g., a public speaking task or solving math problems). Alternatively, the subjective user responses with self-reports have also been used for measuring stress. In this paper, we describe a methodological approach (a) to compare the prediction performance of models developed using objective markers of stress using participant-reported subjective markers of stress from self-reports; and (b) to develop personalized stress models by accounting for inter-individual differences. Towards this end, we conducted a laboratory-based study with 32 healthy volunteers. Participants completed a series of stressor tasks—social, cognitive and physical—wearing an instrumented commercial smartwatch that collected physiological signals and participant responses using timed self-reports. After extensive data preprocessing using a combination of signal processing techniques, we developed two types of models: objective stress models using the stressor tasks as labels; and subjective stress models using participant responses to each task as the label for that stress task. We trained and tested several machine learning algorithms—support vector machine (SVM), random forest (RF), gradient boosted trees (GBT), AdaBoost, and Logistic Regression (LR)—and evaluated their performance. SVM had the best performance for the models using the objective stressor (i.e., stressor tasks) with an AUROC of 0.790 and an F-1 score of 0.623. SVM also had the highest performance for the models using the subjective stress (i.e., participant self-reports) with an AUROC of 0.719 and an F-1 score of 0.520. Model performance improved with a personalized threshold model to an AUROC of 0.751 and an F-1 score of 0.599. The performance of the stress models using an instrumented commercial smartwatch was comparable to similar models from other state-of-the-art laboratory-based studies. However, the subjective stress models had a lower performance, indicating the need for further research on the use of self-reports for stress-related studies. The improvement in performance with the personalized threshold-based models provide new directions for building stress prediction models.
KW - Objective stress
KW - Personalized threshold
KW - Smartwatch
KW - Subjective stress
UR - http://www.scopus.com/inward/record.url?scp=85109533823&partnerID=8YFLogxK
U2 - 10.1016/j.cmpb.2021.106207
DO - 10.1016/j.cmpb.2021.106207
M3 - Article
C2 - 34161847
AN - SCOPUS:85109533823
SN - 0169-2607
VL - 208
JO - Computer Methods and Programs in Biomedicine
JF - Computer Methods and Programs in Biomedicine
M1 - 106207
ER -