Utility of the IFCN criteria for identifying interictal epileptiform discharges by experts: A decision hygiene approach to improve inter-rater reliability

Objective: To determine if implementing the IFCN criteria to define interictal epileptiform discharges (IEDs) improves expert inter-rater reliability (IRR) and diagnostic performance. Methods: Nine EEG experts rated the same 200 candidate IEDs (100 expert-consensus, 100 epilepsy monitoring unit [EMU]-validated) as epileptiform or not, in random order, in two rounds separated by at least 30 days. During the second round, raters additionally selected the applicable IFCN criteria for each candidate IED. Results: Overall, there were no major differences in performance (AUC; 0.90 vs. 0.91) or IRR (AC1; 0.48 vs. 0.47) between both Parts; nor was there a major difference in calibration within the expert-consensus dataset (median absolute calibration index; 35.5 vs. 30.0). Similarly, there were no major differences in performance or IRR within either dataset. IRR was substantial within the EMU-validated dataset and only fair within the expert-consensus dataset. IRR was fair for criteria 2, 3, 5 and 6, and moderate for criteria 1 and 4. Conclusions: Our findings suggest that the IFCN criteria to define IEDs may not significantly improve IRR, performance, or overall calibration among experts. Significance: Increasing expert IRR for each criterion may enhance the utility of the IFCN criteria in clinical practice.