3D-2D GAN Based Brain Metastasis Synthesis with Configurable Parameters for Fully 3D Data Augmentation

Gengyan Zhao, Youngjin Yoo, Thomas J. Re, Jyotipriya Das, Wang Hesheng, Michelle M. Kim, Colette Shen, Yueh Lee, Douglas Kondziolka, Mohannad Ibrahim, Jun Lian, Rajan Jain, Tong Zhu, Hemant Parmar, James M. Balter, Yue Cao, Eli Gibson, Dorin Comaniciu

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review


Recent technological advances in deep learning (DL) have led to more accurate brain metastasis (BM) detection. As a data driven approach, DL’s performance highly relies on the size and quality of the training data. However, collecting large amount of medical data is costly, and it’s difficult to include BMs with various locations, sizes, and structures etc. Thus, we propose a 3D-2D GAN for fully 3D BM synthesis with configurable parameters. First, two 3D networks are used to synthesize the mask and quantized intensity map of a lesion from 3 concentric spheres, which are used to control the lesion’s location, size and structure. Then, a 2D network is used to synthesize the final lesion with proper appearance from the quantized intensity map and the background MR image. With this 3D-2D design, the 3D networks enable the synthetic metastasis to be spatially continuous in all 3 dimensions through the guidance of the 3D intermediate presentation of the lesion, while the 2D network enables the use of 2D perceptual loss to make the final synthesized lesion look realistic. In addition, different network up-sampling strategies and postprocessing are used to control the heterogeneity and contrast of the synthetic lesion. All the synthesized images were reviewed by a radiologist. The indistinguishability rate of the synthesized lesion is above 70%. The configurable parameters for the lesion’s location, size, and structure, heterogeneity and contrast were reviewed to be effective. Our work demonstrates the feasibility of synthesizing configurable 3D BM lesions for fully 3D data augmentation.

Original languageEnglish
Title of host publicationMedical Imaging 2023
Subtitle of host publicationImage Processing
EditorsOlivier Colliot, Ivana Isgum
ISBN (Electronic)9781510660335
StatePublished - 2023
EventMedical Imaging 2023: Image Processing - San Diego, United States
Duration: Feb 19 2023Feb 23 2023

Publication series

NameProgress in Biomedical Optics and Imaging - Proceedings of SPIE
ISSN (Print)1605-7422


ConferenceMedical Imaging 2023: Image Processing
Country/TerritoryUnited States
CitySan Diego


  • Brain Metastasis
  • Generative Adversarial Network
  • Image Synthesis
  • MRI


Dive into the research topics of '3D-2D GAN Based Brain Metastasis Synthesis with Configurable Parameters for Fully 3D Data Augmentation'. Together they form a unique fingerprint.

Cite this