Deep Learning-Based Automatic Detection of Brain Metastases in Heterogenous Multi-Institutional Magnetic Resonance Imaging Sets: An Exploratory Analysis of NRG-CC001

Purpose: Deep learning-based algorithms have been shown to be able to automatically detect and segment brain metastases (BMs) in magnetic resonance imaging, mostly based on single-institutional data sets. This work aimed to investigate the use of deep convolutional neural networks (DCNN) for BM detection and segmentation on a highly heterogeneous multi-institutional data set. Methods and Materials: A total of 407 patients from 98 institutions were randomly split into 326 patients from 78 institutions for training/validation and 81 patients from 20 institutions for unbiased testing. The data set contained T1-weighted gadolinium and T2-weighted fluid-attenuated inversion recovery magnetic resonance imaging acquired on diverse scanners using different pulse sequences and various acquisition parameters. Several variants of 3-dimensional U-Net based DCNN models were trained and tuned using 5-fold cross validation on the training set. Performances of different models were compared based on Dice similarity coefficient for segmentation and sensitivity and false positive rate (FPR) for detection. The best performing model was evaluated on the test set. Results: A DCNN with an input size of 64 × 64 × 64 and an equal number of 128 kernels for all convolutional layers using instance normalization was identified as the best performing model (Dice similarity coefficient 0.73, sensitivity 0.86, and FPR 1.9) in the 5-fold cross validation experiments. The best performing model demonstrated consistent behavior on the test set (Dice similarity coefficient 0.73, sensitivity 0.91, and FPR 1.7) and successfully detected 7 BMs (out of 327) that were missed during manual delineation. For large BMs with diameters greater than 12 mm, the sensitivity and FPR improved to 0.98 and 0.3, respectively. Conclusions: The DCNN model developed can automatically detect and segment brain metastases with reasonable accuracy, high sensitivity, and low FPR on a multi-institutional data set with nonprespecified and highly variable magnetic resonance imaging sequences. For large BMs, the model achieved clinically relevant results. The model is robust and may be potentially used in real-world situations.