Developing a Tool to Assess Placement of Central Venous Catheters in Pediatrics Patients

Geoffrey M. Fleming; Richard B. Mink; Christoph Hornik; Amanda R. Emke; Michael L. Green; Katherine Mason; Toni Petrillo; Jennifer Schuette; M. Hossein Tcharmtchi; Margaret Winkler; David A. Turner

doi:10.4300/JGME-D-15-00365.1

Developing a Tool to Assess Placement of Central Venous Catheters in Pediatrics Patients

Geoffrey M. Fleming, Richard B. Mink, Christoph Hornik, Amanda R. Emke, Michael L. Green, Katherine Mason, Toni Petrillo, Jennifer Schuette, M. Hossein Tcharmtchi, Margaret Winkler, David A. Turner

Research output: Contribution to journal › Article › peer-review

10 Scopus citations

Abstract

METHODS: Ten experts used the modified Delphi technique to create a 15-item direct observation tool to assess 5 scripted and filmed simulated scenarios of CVC placement. The scenarios were hosted on a dedicated website from March to May 2013, and respondents recruited by e-mail completed the observation tool in real time while watching the scenarios. The goal was to obtain 50 respondents and a total of 250 scenario ratings.

RESULTS: A total of 49 pediatrics intensive care faculty physicians (6.3% of 780 potential subjects) responded and generated 188 scenario observations. Of these, 150 (79.8%) were recorded from participants who scored 4 or more on the 5 scenarios. The tool correctly identified the expected reference standard in 96.8% of assessments with an interrater agreement kappa (standard error) = 0.94 (0.07) and receiver operating characteristic = 0.97 (95% CI 0.94-0.99).

CONCLUSIONS: This direct observation assessment tool for central venous catheterization demonstrates excellent performance in identifying the reference standard with a high degree of interrater reliability. These assessments support a validity construct for a pediatric critical care medicine faculty member to assess a provider placing a CVC in a pediatrics patient.

BACKGROUND: Pediatric critical care medicine requires the acquisition of procedural skills, but to date no criteria exist for assessing trainee competence in central venous catheter (CVC) insertion.

OBJECTIVE: The goal of this study was to create and demonstrate validity evidence for a direct observation tool for assessing CVC insertion.

Original language	English
Pages (from-to)	346-352
Number of pages	7
Journal	Journal of graduate medical education
Volume	8
Issue number	3
DOIs	https://doi.org/10.4300/JGME-D-15-00365.1
State	Published - Jul 1 2016

Access to Document

10.4300/JGME-D-15-00365.1

Cite this

@article{92843e273ec24f1a9d6c1d5f67dce618,

title = "Developing a Tool to Assess Placement of Central Venous Catheters in Pediatrics Patients",

abstract = "METHODS: Ten experts used the modified Delphi technique to create a 15-item direct observation tool to assess 5 scripted and filmed simulated scenarios of CVC placement. The scenarios were hosted on a dedicated website from March to May 2013, and respondents recruited by e-mail completed the observation tool in real time while watching the scenarios. The goal was to obtain 50 respondents and a total of 250 scenario ratings.RESULTS: A total of 49 pediatrics intensive care faculty physicians (6.3% of 780 potential subjects) responded and generated 188 scenario observations. Of these, 150 (79.8%) were recorded from participants who scored 4 or more on the 5 scenarios. The tool correctly identified the expected reference standard in 96.8% of assessments with an interrater agreement kappa (standard error) = 0.94 (0.07) and receiver operating characteristic = 0.97 (95% CI 0.94-0.99).CONCLUSIONS: This direct observation assessment tool for central venous catheterization demonstrates excellent performance in identifying the reference standard with a high degree of interrater reliability. These assessments support a validity construct for a pediatric critical care medicine faculty member to assess a provider placing a CVC in a pediatrics patient.BACKGROUND: Pediatric critical care medicine requires the acquisition of procedural skills, but to date no criteria exist for assessing trainee competence in central venous catheter (CVC) insertion.OBJECTIVE: The goal of this study was to create and demonstrate validity evidence for a direct observation tool for assessing CVC insertion.",

author = "Fleming, {Geoffrey M.} and Mink, {Richard B.} and Christoph Hornik and Emke, {Amanda R.} and Green, {Michael L.} and Katherine Mason and Toni Petrillo and Jennifer Schuette and Tcharmtchi, {M. Hossein} and Margaret Winkler and Turner, {David A.}",

year = "2016",

month = jul,

day = "1",

doi = "10.4300/JGME-D-15-00365.1",

language = "English",

volume = "8",

pages = "346--352",

journal = "Journal of graduate medical education",

issn = "1949-8357",

number = "3",

}

TY - JOUR

T1 - Developing a Tool to Assess Placement of Central Venous Catheters in Pediatrics Patients

AU - Fleming, Geoffrey M.

AU - Mink, Richard B.

AU - Hornik, Christoph

AU - Emke, Amanda R.

AU - Green, Michael L.

AU - Mason, Katherine

AU - Petrillo, Toni

AU - Schuette, Jennifer

AU - Tcharmtchi, M. Hossein

AU - Winkler, Margaret

AU - Turner, David A.

PY - 2016/7/1

Y1 - 2016/7/1

N2 - METHODS: Ten experts used the modified Delphi technique to create a 15-item direct observation tool to assess 5 scripted and filmed simulated scenarios of CVC placement. The scenarios were hosted on a dedicated website from March to May 2013, and respondents recruited by e-mail completed the observation tool in real time while watching the scenarios. The goal was to obtain 50 respondents and a total of 250 scenario ratings.RESULTS: A total of 49 pediatrics intensive care faculty physicians (6.3% of 780 potential subjects) responded and generated 188 scenario observations. Of these, 150 (79.8%) were recorded from participants who scored 4 or more on the 5 scenarios. The tool correctly identified the expected reference standard in 96.8% of assessments with an interrater agreement kappa (standard error) = 0.94 (0.07) and receiver operating characteristic = 0.97 (95% CI 0.94-0.99).CONCLUSIONS: This direct observation assessment tool for central venous catheterization demonstrates excellent performance in identifying the reference standard with a high degree of interrater reliability. These assessments support a validity construct for a pediatric critical care medicine faculty member to assess a provider placing a CVC in a pediatrics patient.BACKGROUND: Pediatric critical care medicine requires the acquisition of procedural skills, but to date no criteria exist for assessing trainee competence in central venous catheter (CVC) insertion.OBJECTIVE: The goal of this study was to create and demonstrate validity evidence for a direct observation tool for assessing CVC insertion.

AB - METHODS: Ten experts used the modified Delphi technique to create a 15-item direct observation tool to assess 5 scripted and filmed simulated scenarios of CVC placement. The scenarios were hosted on a dedicated website from March to May 2013, and respondents recruited by e-mail completed the observation tool in real time while watching the scenarios. The goal was to obtain 50 respondents and a total of 250 scenario ratings.RESULTS: A total of 49 pediatrics intensive care faculty physicians (6.3% of 780 potential subjects) responded and generated 188 scenario observations. Of these, 150 (79.8%) were recorded from participants who scored 4 or more on the 5 scenarios. The tool correctly identified the expected reference standard in 96.8% of assessments with an interrater agreement kappa (standard error) = 0.94 (0.07) and receiver operating characteristic = 0.97 (95% CI 0.94-0.99).CONCLUSIONS: This direct observation assessment tool for central venous catheterization demonstrates excellent performance in identifying the reference standard with a high degree of interrater reliability. These assessments support a validity construct for a pediatric critical care medicine faculty member to assess a provider placing a CVC in a pediatrics patient.BACKGROUND: Pediatric critical care medicine requires the acquisition of procedural skills, but to date no criteria exist for assessing trainee competence in central venous catheter (CVC) insertion.OBJECTIVE: The goal of this study was to create and demonstrate validity evidence for a direct observation tool for assessing CVC insertion.

UR - http://www.scopus.com/inward/record.url?scp=85015596558&partnerID=8YFLogxK

U2 - 10.4300/JGME-D-15-00365.1

DO - 10.4300/JGME-D-15-00365.1

M3 - Article

C2 - 27413436

AN - SCOPUS:85015596558

SN - 1949-8357

VL - 8

SP - 346

EP - 352

JO - Journal of graduate medical education

JF - Journal of graduate medical education

IS - 3

ER -

Developing a Tool to Assess Placement of Central Venous Catheters in Pediatrics Patients

Abstract

Access to Document

Other files and links

Fingerprint

Cite this