Natural Language Processing Identifies Underdocumentation of Symptoms in Patients on Hemodialysis

Background Patients on hemodialysis have a high burden of emotional and physical symptoms. These symptoms are often under-recognized. Natural language processing (NLP) can be used to identify patient symptoms from electronic health records (EHRs). However, whether symptom documentation matches patient-reported burden is unclear. Methods We conducted a prospective study of patients seen at an ambulatory nephrology practice from September 2020 to April 2021. We collected symptom surveys from patients, nurses, and physicians. We then developed an NLP algorithm to identify symptoms from the patients' EHRs and validated the performance of this algorithm using manual chart review and patient surveys as a reference standard. Using patient surveys as the reference standard, we compared symptom identification by (1) physicians, (2) nurses, (3) physicians or nurses, and (4) NLP. Results We enrolled 97 patients into our study, 63% were female, 49% were non-Hispanic Black, and 41% were Hispanic. The most common symptoms reported by patients were fatigue (61%), cramping (59%), dry skin (53%), muscle soreness (43%), and itching (41%). Physicians and nurses significantly under-recognized patients' symptoms (sensitivity 0.51 [95% confidence interval (CI), 0.40 to 0.61] and 0.63 [95% CI, 0.52 to 0.72], respectively). Nurses were better at identifying symptoms when patients reported more severe symptoms. There was no difference in results by patients' sex or ethnicity. NLP had a sensitivity of 0.92, specificity of 0.95, positive predictive value of 0.75, and negative predictive value of 0.99 with manual EHR review as the reference standard and a sensitivity of 0.58 (95% CI, 0.47 to 0.68), specificity of 0.73 (95% CI, 0.48 to 0.89), positive predictive value of 0.92 (95% CI, 0.82 to 0.97), and negative predictive value of 0.24 (95% CI, 0.14 to 0.38) compared with patient surveys. Conclusions Although patients on hemodialysis report high prevalence of symptoms, symptoms are under-recognized and underdocumented. NLP was accurate at identifying symptoms when they were documented. Larger studies in representative populations are needed to assess the generalizability of the results of the study.