Purpose: To evaluate the performance of contrast-enhanced spectral mammography (CESM) compared to MRI in the assessment of tumor response in breast cancer patients undergoing neoadjuvant systemic therapy (NST). Methods: The institutional review board approved this study. From September 2014 to June 2017, we identified patients with pathologically confirmed invasive breast cancer who underwent NST. All patients had both CESM and MRI performed pre- and post-NST with pathological assessment after surgical management. Size of residual malignancy on post-NST CESM and MRI was compared with surgical pathology. Lin concordance and Pearson correlation coefficient were used to assess agreement. Bland–Altman plots were used to visualize the differences between tumor size on imaging and pathology. Results: Sixty-five patients were identified. Mean age was 52.7 (range 30–76) years. Type of NST included chemotherapy in 53 (82%) and endocrine therapy in 12 (18%). Mean tumor size after NST was 14.6 (range 0–105) mm for CESM and 14.2 mm (range 0–75 mm) for MRI compared with 19.6 (range 0–100) mm on final surgical pathology. Equivalence tests demonstrated that mean tumor size measured by CESM (p = 0.009) or by MRI (p = 0.01) was equivalent to the mean tumor size measured by pathology within − 1 and 1-cm range. Comparing CESM versus MRI for assessment of complete response, the sensitivity was 95% versus 95%, specificity 66.7% versus 68.9%, positive predictive value 55.9% versus 57.6%, and negative predictive value 96.7% versus 96.9% respectively. Conclusions: CESM was comparable to MRI in assessing residual malignancy after completion of NST.