Reliability of evaluating hospital quality by colorectal surgical site infection type

Elise H. Lawson, Clifford Y. Ko, John L. Adams, Warren B. Chow, Bruce Lee Hall

Research output: Contribution to journalArticlepeer-review

37 Scopus citations


OBJECTIVE:: To determine whether risk-adjusted colorectal SSI rates are statistically reliable as hospital quality measures. BACKGROUND:: Policymakers use surgical site infections (SSI) for public reporting of hospital quality and pay-for-performance because they are a relatively common and costly cause of patient morbidity. METHODS:: Patients who underwent a colorectal procedure in 2009 were identified from the American College of Surgeons National Surgical Quality Improvement Program. We developed hierarchical multivariate logistic models for (1) superficial SSI, (2) deep/organ-space SSI, and (3) "any SSI" and compared how each model ranked hospital-level risk-adjusted performance. Statistical reliability of hospital quality measurements was estimated on a scale from 0 to 1; with 0 indicating that apparent variation between a hospital's quality measurement and the average hospital is statistically unreliable, and 1 indicating that any observed variation is due to a real difference in performance. RESULTS:: Mean reliability of hospital-level quality measurements was 0.650 for superficial, 0.404 for deep/organ-space, and 0.586 for "any SSI." Lower reliability was accounted for by relatively little variation in risk-adjusted SSI rates between hospitals and insufficient numbers of colorectal cases submitted by individual hospitals. In 2009, we estimate that 22.1% of all US hospitals performed a sufficient number of colorectal cases to report superficial SSI rates at a high standard of statistical reliability and 1.0% did for deep/organ-space SSI. CONCLUSIONS:: As currently constructed, colorectal SSI quality measures might not meet a high standard of statistical reliability for most hospitals, limiting their ability to confidently differentiate high and low performance. Despite an expectation of improving statistical power, combining superficial and deep/organ-space SSI into an "any SSI" measure worsens reliability.

Original languageEnglish
Pages (from-to)994-1000
Number of pages7
JournalAnnals of surgery
Issue number6
StatePublished - Dec 1 2013


  • colorectal
  • quality measure
  • reliability
  • surgical site infection


Dive into the research topics of 'Reliability of evaluating hospital quality by colorectal surgical site infection type'. Together they form a unique fingerprint.

Cite this