TY - JOUR
T1 - Simulation-Based Assessment of Critical Care “Front-Line” Providers
AU - Boyle, Walter A.
AU - Murray, David J.
AU - Beyatte, Mary Beth
AU - Knittel, Justin G.
AU - Kerby, Paul W.
AU - Woodhouse, Julie
AU - Boulet, John R.
N1 - Funding Information:
1Department of Anesthesiology, Washington University School of Medicine, St Louis, MO. 2Division of Anesthesiology Critical Care Medicine, Department of Anesthesiology, Washington University School of Medicine, St. Louis, MO. 3Wood Simulation Center, Washington University School of Medicine, St. Louis, MO. 4Foundation for Advancement of International Medical Education and Several million people in the United States admitted Research, Philadelphia, PA. annually to the ICU rely on the skill of ICU clinicians for University School of Medicine, St. Louis, MO.This study was conducted in the Wood Simulation Center, Washington life-saving interventions during their ICU stay (1–4). As Supplemental digital content is available for this article. Direct URL citations ICU admissions continue to rise (2), there has been a paral-appear in the printed text and are provided in the HTML and PDF versions lel increase in the demand for qualified ICU providers (3–6). of this article on the journal’s website (http://journals.lww.com/ccmjournal). For physician specialists (“intensivists”), this demand has been Supported, in part, by the Agency for Healthcare Research and Quality Grant: R18 HS022265-01 Critical Care Management: A Simulation-Based Assessment of Decision-Making Skills. Drs. Boyle’s, Murray’s, and Woodhouse’s institutions received funding from the Agency for Healthcare Research and Quality (AHRQ). Drs. Boyle and Murray received support for article research from the National Institutes of Health. Dr. Beyatte received funding from the AHRQ. The remaining authors have disclosed that they do not have any potential conflicts of interest. For information regarding this article, E-mail: [email protected] Copyright © 2018 by the Society of Critical Care Medicine and Wolters Kluwer Health, Inc. All Rights Reserved.
Publisher Copyright:
Copyright © 2018 by the Society of Critical Care Medicine and Wolters Kluwer Health, Inc. All Rights Reserved.
PY - 2018/6
Y1 - 2018/6
N2 - Objectives: Develop a standardized simulation method to assess clinical skills of ICU providers. Design: Simulation assessment. Setting: Simulation laboratory. Subjects: Residents, Critical Care Medicine fellows, acute care nurse practitioner students. Interventions: Performance scoring in scenarios from multiple Critical Care Medicine competency domains. Measurements and Main Results: Three-hundred eighty-four performances by 48 participants were scored using checklists (% correct) and holistic “global” ratings (1 [unprepared] to 9 [expert]). One-hundred eighty were scored by two raters. Mean checklist and global scores (± sd) ranged from 65.0% (± 16.3%) to 84.5% (± 17.3%) and 4.7 (± 1.4) to 7.2 (± 1.2). Checklist and global scores for Critical Care Medicine fellows and senior acute care nurse practitioner students (Experienced group, n = 26) were significantly higher than those for the Novice acute care nurse practitioner students (Novice group, n = 14) (75.6% ± 15.6% vs 68.8% ± 21.0% and 6.1 ± 1.6 vs 5.4 ± 1.5, respectively; p < 0.05). Residents (Intermediate group, n = 8) scored between the two (75.4% ± 18.3% and 5.7 ± 1.7). 38.5% of the Experienced group scored in the top quartile for mean global score, compared with 12.5% of the Intermediate and 7.1% of the Novice groups. Conversely, 50% of the Novice group scored in the lower quartile (< 5.3), compared with 37.5% of the Intermediate and 11.5% of the Experienced groups. Psychometric analyses yielded discrimination values greater than 0.3 for most scenarios and reliability for the eight-scenario assessments of 0.51 and 0.60, with interrater reliability of 0.71 and 0.75, for checklist and global scoring, respectively. Conclusions: The simulation assessments yielded reasonably reliable measures of Critical Care Medicine decision-making skills. Despite a wide range of performance, those with more ICU training and experience performed better, providing evidence to support the validity of the scores. Simulation-based assessments may ultimately prove useful to determine readiness to assume decision-making roles in the ICU.
AB - Objectives: Develop a standardized simulation method to assess clinical skills of ICU providers. Design: Simulation assessment. Setting: Simulation laboratory. Subjects: Residents, Critical Care Medicine fellows, acute care nurse practitioner students. Interventions: Performance scoring in scenarios from multiple Critical Care Medicine competency domains. Measurements and Main Results: Three-hundred eighty-four performances by 48 participants were scored using checklists (% correct) and holistic “global” ratings (1 [unprepared] to 9 [expert]). One-hundred eighty were scored by two raters. Mean checklist and global scores (± sd) ranged from 65.0% (± 16.3%) to 84.5% (± 17.3%) and 4.7 (± 1.4) to 7.2 (± 1.2). Checklist and global scores for Critical Care Medicine fellows and senior acute care nurse practitioner students (Experienced group, n = 26) were significantly higher than those for the Novice acute care nurse practitioner students (Novice group, n = 14) (75.6% ± 15.6% vs 68.8% ± 21.0% and 6.1 ± 1.6 vs 5.4 ± 1.5, respectively; p < 0.05). Residents (Intermediate group, n = 8) scored between the two (75.4% ± 18.3% and 5.7 ± 1.7). 38.5% of the Experienced group scored in the top quartile for mean global score, compared with 12.5% of the Intermediate and 7.1% of the Novice groups. Conversely, 50% of the Novice group scored in the lower quartile (< 5.3), compared with 37.5% of the Intermediate and 11.5% of the Experienced groups. Psychometric analyses yielded discrimination values greater than 0.3 for most scenarios and reliability for the eight-scenario assessments of 0.51 and 0.60, with interrater reliability of 0.71 and 0.75, for checklist and global scoring, respectively. Conclusions: The simulation assessments yielded reasonably reliable measures of Critical Care Medicine decision-making skills. Despite a wide range of performance, those with more ICU training and experience performed better, providing evidence to support the validity of the scores. Simulation-based assessments may ultimately prove useful to determine readiness to assume decision-making roles in the ICU.
KW - clinical competence
KW - critical care
KW - educational measurement
KW - intensive care units
KW - nurse practitioners
KW - patient simulation
UR - http://www.scopus.com/inward/record.url?scp=85049309118&partnerID=8YFLogxK
U2 - 10.1097/CCM.0000000000003073
DO - 10.1097/CCM.0000000000003073
M3 - Article
C2 - 29521715
AN - SCOPUS:85049309118
SN - 0090-3493
VL - 46
SP - E516-E522
JO - Critical care medicine
JF - Critical care medicine
IS - 6
ER -