Identifying Worsening Surgical Site Infection Performance: Control Charts Versus Risk-Adjusted Rate Outlier Status

Control charts are used in industry to monitor performance and are being used increasingly in hospitals as a quality improvement tool. The authors' objective was to determine if control charts using surgical site infection (SSI) rates predict changes in outlier status for risk-adjusted SSI rates using data from a surgical registry, the American College of Surgeons National Surgical Quality Improvement Program. Control charts of monthly SSI rates for 100 hospitals were analyzed for indicators of a performance change in 2009 (vs 2008) using standard rules. Hospitals also were classified as having better, worse, or no change in outlier status for risk-adjusted SSI rates in 2009 (vs 2008). There was moderate agreement between these methods (weighted κ = 0.401). Control charts predicted nonworsening performance well (specificity = 92.9%) and identified changes in SSI performance sooner; however, they failed to identify 31.2% of hospitals with worsened outlier status. This study demonstrates that these quality measurement tools have unique strengths and weaknesses and are complementary uses of the same clinical data source.