Are we making an impact with incident learning systems? Analysis of quality improvement interventions using total body irradiation as a model system

Purpose Despite increasing interest in incident learning systems (ILS) to improve safety and quality in radiation oncology, little is known about interventions developed in response to safety data. We used total body irradiation (TBI) as a model system to study the effectiveness of interventions from our institutional ILS. Methods and materials Near-miss event reports specific to TBI were identified from a departmental ILS from March 2012 to December 2015. The near-miss risk index was rated at multidisciplinary review from 0 (no potential harm) to 4 (critical potential harm). Interventions were analyzed for effectiveness with a schema adapted from The Joint Commission and other agencies: “most reliable” (eg, forcing functions, automation), “somewhat reliable” (eg, checklists, standardization), and “least reliable” (eg, training, rules, procedures). Causal factors of each event were drawn from the casual factor schema used in radiation oncology ILS. Results Of 4007 safety-related reports, 266 reports pertained to TBI. TBI reports had a somewhat higher proportion of high-risk events (near-miss risk index 3-4) compared with non-TBI reports (25% vs 17%, P = .0045). A total of 117 interventions were implemented. The reliability indicators for the interventions were: most reliable (11% of interventions), somewhat reliable (17%), and least reliable (72%). Interventions were more likely to be applied to high-risk events (54% vs 41%, P = .03). There was a pattern of high-reliability interventions with increased risk score of events. Events involving human error (eg, slips) and equipment/information technology lent themselves more often to high-reliability interventions. Events related to communication, standardization, and training were associated with low-reliability interventions. Conclusions Over a 3.5-year period, 117 quality improvement strategies were developed for TBI based on ILS. Interventions were more likely to be applied to high-risk events and high-risk events were more likely to be associated with high-quality interventions. These results may be useful to institutions seeking to develop interventions based on ILS data.