Validating the Physician Documentation Quality Instrument for Intensive Care Unit–Ward Transfer Notes

Background: Physician communication failures during transfers of patients from the intensive care unit (ICU) to the general ward are common and can lead to adverse events. Efforts to improve written handoffs during these transfers are increasingly prominent, but no instruments have been developed to assess the quality of physician ICU–ward transfer notes. Objective: To collect validity evidence for the modified nine-item Physician Documentation Quality Instrument (mPDQI-9) for assessing ICU–ward transfer note usefulness across several hospitals. Methods: Twenty-four physician raters independently used the mPDQI-9 to grade 12 notes collected from three academic hospitals. A priori, we excluded the “up-to-date” and “accurate” domains, because these could not be assessed without giving the rater access to the complete patient chart. Assessments therefore used the domains “thorough,” “useful,” “organized,” “comprehensible,” “succinct,” “synthesized,” and “consistent.” Raters scored each domain on a Likert scale ranging from 1 (low) to 5 (high). The total mPDQI-9 was the sum of these domain scores. The primary outcome was the raters’ perceived clinical utility of the notes, and the primary measures of interest were criterion validity (Spearman’s r) and interrater reliability (intraclass correlation [ICC]). Results: Mean mPDQI-9 scores by note ranged from 19 (SD = 5.5) to 30 (SD = 4.2). Mean note ratings did not systematically differ by rater expertise (for interaction, P = 0.15). The proportion of raters perceiving each note as independently sufficient for patient care (the primary outcome) ranged from 33% to 100% across the set of notes. We found a moderately positive correlation between mPDQI-9 ratings and raters’ overall assessments of each note’s clinical utility (r = 0.48, P, 0.001). Interrater reliability was strong; the overall ICC was 0.89 (95% confidence interval [CI], 0.80–0.85), and ICCs were similar among reviewer groups. Finally, Cronbach’s a was 0.87 (95% CI, 0.84–0.89), indicating good internal consistency. Conclusions: We report moderate validity evidence for the mPDQI-9 to assess the usefulness of ICU–ward transfer notes written by internal medicine residents.