First Report On Physician Assessment and Clinical Acceptability of Custom-Retrained Artificial Intelligence Models for Clinical Target Volume and Organs-at-Risk Auto-Delineation for Postprostatectomy Patients

Purpose: To assess the clinical acceptability of a commercial deep-learning-based auto-segmentation (DLAS) prostate model that was retrained using institutional data for delineation of the clinical target volume (CTV) and organs-at-risk (OARs) for postprostatectomy patients, accounting for clinical and imaging protocol variations. Methods and Materials: CTV and OARs of 109 prostate-bed patients were used to evaluate the performance of the vendor-trained model and custom retrained DLAS models using different training quantities. Two new models for OAR structures were retrained (n = 30, 60 data sets), while separate models were trained for a new CTV structure (n = 30, 60, 90 data sets), with the remaining data sets used for testing (n = 49, 19). The dice similarity coefficient (DSC), Hausdorff distance, and mean surface distance were evaluated. Six radiation oncologists performed a qualitative evaluation scoring both preference and clinical utility for blinded structure sets. Physician consensus data sets identified from the qualitative evaluation were used toward a separate CTV model. Results: Both the 30- and 60-case retrained OAR models had median DSC values between 0.91 to 0.97, improving significantly over the vendor-trained model for all OARs except the penile bulb. The brand new 60-case CTV model had a median DSC of 0.70 improving significantly over the 30-case model. DLAS (60-case model) and manual contours were blinded and evaluated by physicians with contours deemed acceptable or precise for 87% and 94% of cases for DLAS and manual delineations, respectively. DLAS-generated CTVs were scored precise or acceptable in 54% of cases, compared with the manual delineation value of 73%. The 30-case physician consensus CTV model did not show a significant difference compared with the randomly selected models. Conclusions: Custom retraining using institutional data leads to performance improvement in the clinical utility and accuracy of DLAS for postprostatectomy patients. A small number of data sets are sufficient for building an institutional site-specific DLAS OAR model, as well as for training new structures. Data indicates the workload for identifying training data sets could be shared among groups for the male pelvic region, making it accessible to clinics of all sizes.