3D-2D GAN based brain metastasis synthesis with configurable parameters for fully 3D data augmentation

Gengyan Zhao; Youngjin Yoo; Thomas J. Re; Jyotipriya Das; Hesheng Wang; Michelle Kim; Colette Shen; Yueh Lee; Douglas Kondziolka; Mohannad Ibrahim; Jun Lian; Rajan Jain; Tong Zhu; Hemant Parmar; James M. Balter; Yue Cao; Eli Gibson; Dorin Comaniciu

doi:10.1117/12.2654281

3 April 2023 3D-2D GAN based brain metastasis synthesis with configurable parameters for fully 3D data augmentation

Gengyan Zhao, Youngjin Yoo, Thomas J. Re, Jyotipriya Das, Hesheng Wang, Michelle Kim, Colette Shen, Yueh Lee, Douglas Kondziolka, Mohannad Ibrahim, Jun Lian, Rajan Jain, Tong Zhu, Hemant Parmar, James M. Balter, Yue Cao, Eli Gibson, Dorin Comaniciu

Author Affiliations +

Proceedings Volume 12464, Medical Imaging 2023: Image Processing; 124640J (2023) https://doi.org/10.1117/12.2654281
Event: SPIE Medical Imaging, 2023, San Diego, California, United States

Abstract

Recent technological advances in deep learning (DL) have led to more accurate brain metastasis (BM) detection. As a data driven approach, DL’s performance highly relies on the size and quality of the training data. However, collecting large amount of medical data is costly, and it’s difficult to include BMs with various locations, sizes, and structures etc. Thus, we propose a 3D-2D GAN for fully 3D BM synthesis with configurable parameters. First, two 3D networks are used to synthesize the mask and quantized intensity map of a lesion from 3 concentric spheres, which are used to control the lesion’s location, size and structure. Then, a 2D network is used to synthesize the final lesion with proper appearance from the quantized intensity map and the background MR image. With this 3D-2D design, the 3D networks enable the synthetic metastasis to be spatially continuous in all 3 dimensions through the guidance of the 3D intermediate presentation of the lesion, while the 2D network enables the use of 2D perceptual loss to make the final synthesized lesion look realistic. In addition, different network up-sampling strategies and postprocessing are used to control the heterogeneity and contrast of the synthetic lesion. All the synthesized images were reviewed by a radiologist. The indistinguishability rate of the synthesized lesion is above 70%. The configurable parameters for the lesion’s location, size, and structure, heterogeneity and contrast were reviewed to be effective. Our work demonstrates the feasibility of synthesizing configurable 3D BM lesions for fully 3D data augmentation.

Conference Presentation

Citation Download Citation

Gengyan Zhao, Youngjin Yoo, Thomas J. Re, Jyotipriya Das, Hesheng Wang, Michelle Kim, Colette Shen, Yueh Lee, Douglas Kondziolka, Mohannad Ibrahim, Jun Lian, Rajan Jain, Tong Zhu, Hemant Parmar, James M. Balter, Yue Cao, Eli Gibson, and Dorin Comaniciu "3D-2D GAN based brain metastasis synthesis with configurable parameters for fully 3D data augmentation", Proc. SPIE 12464, Medical Imaging 2023: Image Processing, 124640J (3 April 2023); https://doi.org/10.1117/12.2654281

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available