Development of a GAN architecture based on integrating global and local information for paired and unpaired medical image translation

摘要

The purpose of this study is to develop and evaluate a novel end-to-end generative adversarial network for image-to-image translation of medical paired and unpaired data. The proposed deep framework is based on generative adversarial networks, in which blocks are designed and used to improve the estimated images and achieve more accurate results. A joint attention block is designed to capture and fuse the global and local information during the training process. A dual discriminator is employed to accurately determine whether the generated images are real or fake. A global guidance up-sampling block is proposed to reconstruct the local details of the estimated images with guidance of the low-level information of the shallow layers. Moreover, an edge detection network is utilized to make the tissue edges sharper. The loss function is composed of different criteria considering the geometric characteristics for obtaining more accurate results. Different translation tasks including PD/T2, CT/MR, and PET/CT are considered to evaluate the performance of the proposed deep framework. The evaluation results indicated averagely 28.28, 10.87, 31.30, 0.90, 0.92, 0.85, 24.80, and 0.042 for root-mean-squared error (RMSE), mean absolute error (MAE), peak signal to noise ratio (PSNR), dice similarity coefficient (DSC), normalized cross-correlation (NCC), structural similarity index measure (SSIM), learned perceptual image patch similarity (LPIPS) and frechet inception distance (FID) respectively. Evaluation of the results showed improvement of 106, 66, 12, 0.22, 0.17, 0.15, 58, and 0.39 for RMSE, MAE, PSNR, DSC, NCC, SSIM, LPIPS, and FID, respectively when the proposed blocks were used. Also, comparing the performance of the proposed network against other methods in the two modes of using paired and unpaired data showed the better performance of the proposed network.