TY - GEN
T1 - 3D-2D GAN Based Brain Metastasis Synthesis with Configurable Parameters for Fully 3D Data Augmentation
AU - Zhao, Gengyan
AU - Yoo, Youngjin
AU - Re, Thomas J.
AU - Das, Jyotipriya
AU - Hesheng, Wang
AU - Kim, Michelle M.
AU - Shen, Colette
AU - Lee, Yueh
AU - Kondziolka, Douglas
AU - Ibrahim, Mohannad
AU - Lian, Jun
AU - Jain, Rajan
AU - Zhu, Tong
AU - Parmar, Hemant
AU - Balter, James M.
AU - Cao, Yue
AU - Gibson, Eli
AU - Comaniciu, Dorin
N1 - Publisher Copyright:
© 2023 SPIE.
PY - 2023
Y1 - 2023
N2 - Recent technological advances in deep learning (DL) have led to more accurate brain metastasis (BM) detection. As a data driven approach, DL’s performance highly relies on the size and quality of the training data. However, collecting large amount of medical data is costly, and it’s difficult to include BMs with various locations, sizes, and structures etc. Thus, we propose a 3D-2D GAN for fully 3D BM synthesis with configurable parameters. First, two 3D networks are used to synthesize the mask and quantized intensity map of a lesion from 3 concentric spheres, which are used to control the lesion’s location, size and structure. Then, a 2D network is used to synthesize the final lesion with proper appearance from the quantized intensity map and the background MR image. With this 3D-2D design, the 3D networks enable the synthetic metastasis to be spatially continuous in all 3 dimensions through the guidance of the 3D intermediate presentation of the lesion, while the 2D network enables the use of 2D perceptual loss to make the final synthesized lesion look realistic. In addition, different network up-sampling strategies and postprocessing are used to control the heterogeneity and contrast of the synthetic lesion. All the synthesized images were reviewed by a radiologist. The indistinguishability rate of the synthesized lesion is above 70%. The configurable parameters for the lesion’s location, size, and structure, heterogeneity and contrast were reviewed to be effective. Our work demonstrates the feasibility of synthesizing configurable 3D BM lesions for fully 3D data augmentation.
AB - Recent technological advances in deep learning (DL) have led to more accurate brain metastasis (BM) detection. As a data driven approach, DL’s performance highly relies on the size and quality of the training data. However, collecting large amount of medical data is costly, and it’s difficult to include BMs with various locations, sizes, and structures etc. Thus, we propose a 3D-2D GAN for fully 3D BM synthesis with configurable parameters. First, two 3D networks are used to synthesize the mask and quantized intensity map of a lesion from 3 concentric spheres, which are used to control the lesion’s location, size and structure. Then, a 2D network is used to synthesize the final lesion with proper appearance from the quantized intensity map and the background MR image. With this 3D-2D design, the 3D networks enable the synthetic metastasis to be spatially continuous in all 3 dimensions through the guidance of the 3D intermediate presentation of the lesion, while the 2D network enables the use of 2D perceptual loss to make the final synthesized lesion look realistic. In addition, different network up-sampling strategies and postprocessing are used to control the heterogeneity and contrast of the synthetic lesion. All the synthesized images were reviewed by a radiologist. The indistinguishability rate of the synthesized lesion is above 70%. The configurable parameters for the lesion’s location, size, and structure, heterogeneity and contrast were reviewed to be effective. Our work demonstrates the feasibility of synthesizing configurable 3D BM lesions for fully 3D data augmentation.
KW - Brain Metastasis
KW - Generative Adversarial Network
KW - Image Synthesis
KW - MRI
UR - http://www.scopus.com/inward/record.url?scp=85159713602&partnerID=8YFLogxK
U2 - 10.1117/12.2654281
DO - 10.1117/12.2654281
M3 - Conference contribution
AN - SCOPUS:85159713602
T3 - Progress in Biomedical Optics and Imaging - Proceedings of SPIE
BT - Medical Imaging 2023
A2 - Colliot, Olivier
A2 - Isgum, Ivana
PB - SPIE
T2 - Medical Imaging 2023: Image Processing
Y2 - 19 February 2023 through 23 February 2023
ER -