First Report On Physician Assessment and Clinical Acceptability of Custom-Retrained Artificial Intelligence Models for Clinical Target Volume and Organs-at-Risk Auto-Delineation for Postprostatectomy Patients

Dean Hobbis, Nathan Y. Yu, Karl W. Mund, Jingwei Duan, Jean Claude M. Rwigema, William W. Wong, Steven E. Schild, Sameer R. Keole, Xue Feng, Quan Chen, Carlos E. Vargas, Yi Rong

Research output: Contribution to journalArticlepeer-review

3 Scopus citations

Abstract

Purpose: To assess the clinical acceptability of a commercial deep-learning-based auto-segmentation (DLAS) prostate model that was retrained using institutional data for delineation of the clinical target volume (CTV) and organs-at-risk (OARs) for postprostatectomy patients, accounting for clinical and imaging protocol variations. Methods and Materials: CTV and OARs of 109 prostate-bed patients were used to evaluate the performance of the vendor-trained model and custom retrained DLAS models using different training quantities. Two new models for OAR structures were retrained (n = 30, 60 data sets), while separate models were trained for a new CTV structure (n = 30, 60, 90 data sets), with the remaining data sets used for testing (n = 49, 19). The dice similarity coefficient (DSC), Hausdorff distance, and mean surface distance were evaluated. Six radiation oncologists performed a qualitative evaluation scoring both preference and clinical utility for blinded structure sets. Physician consensus data sets identified from the qualitative evaluation were used toward a separate CTV model. Results: Both the 30- and 60-case retrained OAR models had median DSC values between 0.91 to 0.97, improving significantly over the vendor-trained model for all OARs except the penile bulb. The brand new 60-case CTV model had a median DSC of 0.70 improving significantly over the 30-case model. DLAS (60-case model) and manual contours were blinded and evaluated by physicians with contours deemed acceptable or precise for 87% and 94% of cases for DLAS and manual delineations, respectively. DLAS-generated CTVs were scored precise or acceptable in 54% of cases, compared with the manual delineation value of 73%. The 30-case physician consensus CTV model did not show a significant difference compared with the randomly selected models. Conclusions: Custom retraining using institutional data leads to performance improvement in the clinical utility and accuracy of DLAS for postprostatectomy patients. A small number of data sets are sufficient for building an institutional site-specific DLAS OAR model, as well as for training new structures. Data indicates the workload for identifying training data sets could be shared among groups for the male pelvic region, making it accessible to clinics of all sizes.

Original languageEnglish
Pages (from-to)351-362
Number of pages12
JournalPractical Radiation Oncology
Volume13
Issue number4
DOIs
StatePublished - Jul 1 2023

Fingerprint

Dive into the research topics of 'First Report On Physician Assessment and Clinical Acceptability of Custom-Retrained Artificial Intelligence Models for Clinical Target Volume and Organs-at-Risk Auto-Delineation for Postprostatectomy Patients'. Together they form a unique fingerprint.

Cite this