skip to main content
research-article

Multi-Cycle-Consistent Adversarial Networks for Edge Denoising of Computed Tomography Images

Published: 21 July 2021 Publication History

Abstract

As one of the most commonly ordered imaging tests, the computed tomography (CT) scan comes with inevitable radiation exposure that increases cancer risk to patients. However, CT image quality is directly related to radiation dose, and thus it is desirable to obtain high-quality CT images with as little dose as possible. CT image denoising tries to obtain high-dose-like high-quality CT images (domain Y) from low dose low-quality CT images (domain X), which can be treated as an image-to-image translation task where the goal is to learn the transform between a source domain X (noisy images) and a target domain Y (clean images). Recently, the cycle-consistent adversarial denoising network (CCADN) has achieved state-of-the-art results by enforcing cycle-consistent loss without the need of paired training data, since the paired data is hard to collect due to patients’ interests and cardiac motion. However, out of concerns on patients’ privacy and data security, protocols typically require clinics to perform medical image processing tasks including CT image denoising locally (i.e., edge denoising). Therefore, the network models need to achieve high performance under various computation resource constraints including memory and performance. Our detailed analysis of CCADN raises a number of interesting questions that point to potential ways to further improve its performance using the same or even fewer computation resources. For example, if the noise is large leading to a significant difference between domain X and domain Y, can we bridge X and Y with a intermediate domain Z such that both the denoising process between X and Z and that between Z and Y are easier to learn? As such intermediate domains lead to multiple cycles, how do we best enforce cycle- consistency? Driven by these questions, we propose a multi-cycle-consistent adversarial network (MCCAN) that builds intermediate domains and enforces both local and global cycle-consistency for edge denoising of CT images. The global cycle-consistency couples all generators together to model the whole denoising process, whereas the local cycle-consistency imposes effective supervision on the process between adjacent domains. Experiments show that both local and global cycle-consistency are important for the success of MCCAN, which outperforms CCADN in terms of denoising quality with slightly less computation resource consumption.

References

[1]
Asha Anoosheh, Eirikur Agustsson, Radu Timofte, and Luc Van Gool. 2018. ComboGAN: Unrestrained scalability for image domain translation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 783–790.
[2]
I. Arapakis, E. Efstathopoulos, V. Tsitsia, S. Kordolaimi, N. Economopoulos, S. Argentos, A. Ploussi, and E. Alexopoulou. 2014. Using “iDose4” iterative reconstruction algorithm in adults’ chest–abdomen–pelvis CT examinations: Effect on image quality in relation to patient radiation exposure. British Journal of Radiology 87, 1036 (2014).
[3]
Hu Chen, Yi Zhang, Weihua Zhang, Peixi Liao, Ke Li, Jiliu Zhou, and Ge Wang. 2017. Low-dose CT denoising with convolutional neural network. In Proceedings of the 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI’17). IEEE, Los Alamitos, CA, 143–146.
[4]
Yunjey Choi, Minje Choi, Munyoung Kim, Jung-Woo Ha, Sunghun Kim, and Jaegul Choo. 2017. StarGAN: Unified generative adversarial networks for multi-domain image-to-image translation. arxiv:1711.09020.
[5]
Leon A. Gatys, Alexander S. Ecker, and Matthias Bethge. 2016. Image style transfer using convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2414–2423.
[6]
Michael Green, Edith M. Marom, Eli Konen, Nahum Kiryati, and Arnaldo Mayer. 2018. Learning real noise for ultra-low dose lung CT denoising. In Proceedings of the International Workshop on Patch-Based Techniques in Medical Imaging. 3–11.
[7]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 770–778.
[8]
Jason B. Hobbs, Noah Goldstein, Kimberly E. Lind, Deirdre Elder, Gerald D. Dodd III, and James P. Borgstede. 2018. Physician knowledge of radiation exposure and risk in medical imaging. Journal of the American College of Radiology 15, 1 (2018), 34–43.
[9]
Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A. Efros. 2017. Image-to-image translation with conditional adversarial networks. arxiv:1611.07004.
[10]
Justin Johnson, Alexandre Alahi, and Li Fei-Fei. 2016. Perceptual losses for real-time style transfer and super-resolution. In Proceedings of the European Conference on Computer Vision. 694–711.
[11]
Eunhee Kang, Hyun Jung Koo, Dong Hyun Yang, Joon Bum Seo, and Jong Chul Ye. 2018. Cycle consistent adversarial denoising network for multiphase coronary CT angiography. arxiv:1806.09748.
[12]
Boaz Karmazyn, Donald P. Frush, Kimberly E. Applegate, Charles Maxfield, Mervyn D. Cohen, and Robert P. Jones. 2009. CT with a computer-simulated dose reduction technique for detection of pediatric nephroureterolithiasis: Comparison of standard and reduced radiation doses. American Journal of Roentgenology 192, 1 (2009), 143–149.
[13]
Taeksoo Kim, Moonsu Cha, Hyunsoo Kim, Jung Kwon Lee, and Jiwon Kim. 2017. Learning to discover cross-domain relations with generative adversarial networks. In Proceedings of the International Conference on Machine Learning. 1857–1865.
[14]
Zhoubo Li, Lifeng Yu, Joshua D. Trzasko, David S. Lake, Daniel J. Blezek, Joel G. Fletcher, Cynthia H. McCollough, and Armando Manduca. 2014. Adaptive nonlocal means filtering based on local noise level for CT denoising. Medical Physics 41, 1 (2014), 011908.
[15]
Jin Liu, Jianhua Ma, Yi Zhang, Yang Chen, Jian Yang, Huazhong Shu, Limin Luo, et al. 2017. Discriminative feature representation to improve projection data inconsistency for low dose CT imaging. IEEE Transactions on Medical Imaging 36, 12 (2017), 2499–2509.
[16]
Jiaying Liu, Wenhan Yang, Shuai Yang, and Zongming Guo. 2018. Erase or fill? Deep joint recurrent rain removal and reconstruction in videos. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3233–3242.
[17]
Ming-Yu Liu and Oncel Tuzel. 2016. Coupled generative adversarial networks. In Advances in Neural Information Processing Systems. 469–477.
[18]
Zihao Liu, Xiaowei Xu, Tao Liu, Qi Liu, Yanzhi Wang, Yiyu Shi, Wujie Wen, Meiping Huang, Haiyun Yuan, and Jian Zhuang. 2019. Machine vision guided 3D medical image compression for efficient transmission and accurate segmentation in the clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 12687–12696.
[19]
Armando Manduca, Lifeng Yu, Joshua D. Trzasko, Natalia Khaylova, James M. Kofler, Cynthia M. McCollough, and Joel G. Fletcher. 2009. Projection space denoising with bilateral filtering and CT noise modeling for dose reduction in CT. Medical Physics 36, 11 (2009), 4911–4919.
[20]
William W. Mayo-Smith, Amy K. Hara, Mahadevappa Mahesh, Dushyant V. Sahani, and William Pavlicek. 2014. How I do it: Managing radiation dose in CT. Radiology 273, 3 (2014), 657–672.
[21]
Mehdi Mirza and Simon Osindero. 2014. Conditional generative adversarial nets. arxiv:1411.1784.
[22]
John Nuyts, Bruno De Man, Patrick Dupont, Michel Defrise, Paul Suetens, and Luc Mortelmans. 1998. Iterative reconstruction for helical CT: A simulation study. Physics in Medicine & Biology 43, 4 (1998), 729.
[23]
Rui Qian, Robby T. Tan, Wenhan Yang, Jiajun Su, and Jiaying Liu. 2018. Attentive generative adversarial network for raindrop removal from a single image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2482–2491.
[24]
Hongming Shan, Yi Zhang, Qingsong Yang, Uwe Kruger, Mannudeep K. Kalra, Ling Sun, Wenxiang Cong, and Ge Wang. 2018. 3-D convolutional encoder-decoder network for low-dose CT via transfer learning from a 2-D trained network. IEEE Transactions on Medical Imaging 37, 6 (2018), 1522–1534.
[25]
Aaron Sodickson, Pieter F. Baeyens, Katherine P. Andriole, Luciano M. Prevedello, Richard D. Nawfel, Richard Hanson, and Ramin Khorasani. 2009. Recurrent CT, cumulative radiation exposure, and associated radiation-induced cancer risks from CT of adults. Radiology 251, 1 (2009), 175–184.
[26]
Jifeng Wang, Xiang Li, and Jian Yang. 2018. Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1788–1797.
[27]
Tianchen Wang, Xiaowei Xu, Jinjun Xiong, Qianjun Jia, Haiyun Yuan, Meiping Huang, Jian Zhuang, and Yiyu Shi. 2020. ICA-UNet: ICA inspired statistical UNet for real-time 3D cardiac cine MRI segmentation. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention. 447–457.
[28]
Jelmer M. Wolterink, Tim Leiner, Max A. Viergever, and Ivana Išgum. 2017. Generative adversarial networks for noise reduction in low-dose CT. IEEE Transactions on Medical Imaging 36, 12 (2017), 2536–2545.
[29]
Fang Xu and Klaus Mueller. 2007. Real-time 3D computed tomographic reconstruction using commodity graphics hardware. Physics in Medicine & Biology 52, 12 (2007), 3405.
[30]
Xiaowei Xu, Yukun Ding, Sharon Xiaobo Hu, Michael Niemier, Jason Cong, Yu Hu, and Yiyu Shi. 2018. Scaling for edge inference of deep neural networks. Nature Electronics 1, 4 (2018), 216–222.
[31]
Xiaowei Xu, Qing Lu, Lin Yang, Sharon Hu, Danny Chen, Yu Hu, and Yiyu Shi. 2018. Quantization of fully convolutional networks for accurate biomedical image segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 8300–8308.
[32]
Xiaowei Xu, Tianchen Wang, Yiyu Shi, Haiyun Yuan, Qianjun Jia, Meiping Huang, and Jian Zhuang. 2019. Whole heart and great vessel segmentation in congenital heart disease using deep neural networks and graph matching. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention. 477–485.
[33]
Xiaowei Xu, Tianchen Wang, Jian Zhuang, Haiyun Yuan, Meiping Huang, Jianzheng Cen, Qianjun Jia, Yuhao Dong, and Yiyu Shi. 2020. ImageCHD: A 3D computed tomography image dataset for classification of congenital heart disease. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention. 77–87.
[34]
Qingsong Yang, Pingkun Yan, Yanbo Zhang, Hengyong Yu, Yongyi Shi, Xuanqin Mou, Mannudeep K. Kalra, Yi Zhang, Ling Sun, and Ge Wang. 2018. Low-dose CT image denoising using a generative adversarial network with Wasserstein distance and perceptual loss. IEEE Transactions on Medical Imaging 37, 6 (2018), 1348–1357.
[35]
Zili Yi, Hao (Richard) Zhang, Ping Tan, and Minglun Gong. 2017. DualGAN: Unsupervised dual learning for image-to-image translation. In Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV’17). 2868–2876.
[36]
Chenyu You, Qingsong Yang, Lars Gjesteby, Guang Li, Shenghong Ju, Zhuiyang Zhang, Zhen Zhao, et al. 2018. Structurally-sensitive multi-scale deep neural network for low-dose CT denoising. IEEE Access 6 (2018), 41839–41855.
[37]
Jiawei Zhang, Yuzhen Jin, Jilan Xu, Xiaowei Xu, and Yanchun Zhang. 2018. MDU-Net: Multi-scale densely connected U-Net for biomedical image segmentation. arxiv:1812.00352.
[38]
Jiawei Zhang, Yanchun Zhang, and Xiaowei Xu. 2021. Pyramid U-Net for retinal vessel segmentation. arxiv:2104.02333.
[39]
Jiawei Zhang, Yanchun Zhang, Shanfeng Zhu, and Xiaowei Xu. 2020. Constrained multi-scale dense connections for accurate biomedical image segmentation. In Proceedings of the 2020 IEEE International Conference on Bioinformatics and Biomedicine (BIBM’20). IEEE, Los Alamitos, CA, 877–884.
[40]
Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. arxiv:1703.10593.

Cited By

View all
  • (2024)Improving efficiency of DNN-based relocalization module for autonomous driving with server-side computingJournal of Cloud Computing: Advances, Systems and Applications10.1186/s13677-024-00592-113:1Online publication date: 25-Jan-2024
  • (2024)Enhancing trust transfer in supply chain finance: a blockchain-based transitive trust modelJournal of Cloud Computing: Advances, Systems and Applications10.1186/s13677-023-00557-w13:1Online publication date: 2-Jan-2024
  • (2024)InteractNet: Social Interaction Recognition for Semantic-rich VideosACM Transactions on Multimedia Computing, Communications, and Applications10.1145/366366820:8(1-21)Online publication date: 12-Jun-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Journal on Emerging Technologies in Computing Systems
ACM Journal on Emerging Technologies in Computing Systems  Volume 17, Issue 4
October 2021
446 pages
ISSN:1550-4832
EISSN:1550-4840
DOI:10.1145/3472280
  • Editor:
  • Ramesh Karri
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Journal Family

Publication History

Published: 21 July 2021
Accepted: 01 April 2021
Revised: 01 December 2020
Received: 01 May 2020
Published in JETC Volume 17, Issue 4

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Adversarial network
  2. computed tomography
  3. deep learning
  4. image denoising
  5. image translation

Qualifiers

  • Research-article
  • Refereed

Funding Sources

  • National Key Research and Development Program of China
  • Science and Technology Planning Project of Guangdong Province, China
  • Guangdong Peak Project
  • National Natural Science Foundatio of China

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)14
  • Downloads (Last 6 weeks)0
Reflects downloads up to 14 Sep 2024

Other Metrics

Citations

Cited By

View all
  • (2024)Improving efficiency of DNN-based relocalization module for autonomous driving with server-side computingJournal of Cloud Computing: Advances, Systems and Applications10.1186/s13677-024-00592-113:1Online publication date: 25-Jan-2024
  • (2024)Enhancing trust transfer in supply chain finance: a blockchain-based transitive trust modelJournal of Cloud Computing: Advances, Systems and Applications10.1186/s13677-023-00557-w13:1Online publication date: 2-Jan-2024
  • (2024)InteractNet: Social Interaction Recognition for Semantic-rich VideosACM Transactions on Multimedia Computing, Communications, and Applications10.1145/366366820:8(1-21)Online publication date: 12-Jun-2024
  • (2024)Low-Light Salient Object Detection by Learning to Highlight the Foreground ObjectsIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.337710834:8(7712-7724)Online publication date: 1-Aug-2024
  • (2023)Shot Boundary Detection Using Color Clustering and Attention MechanismACM Transactions on Multimedia Computing, Communications, and Applications10.1145/359592319:6(1-23)Online publication date: 12-Jul-2023
  • (2023)E-detector: Asynchronous Spatio-temporal for Event-based Object Detection in Intelligent Transportation SystemACM Transactions on Multimedia Computing, Communications, and Applications10.1145/358436120:2(1-20)Online publication date: 27-Sep-2023
  • (2023)Mirror Segmentation via Semantic-aware Contextual Contrasted Feature LearningACM Transactions on Multimedia Computing, Communications, and Applications10.1145/356612719:2s(1-22)Online publication date: 17-Feb-2023
  • (2023)Beyond the Parts: Learning Coarse-to-Fine Adaptive Alignment Representation for Person SearchACM Transactions on Multimedia Computing, Communications, and Applications10.1145/356588619:3(1-19)Online publication date: 25-Feb-2023
  • (2023)Roadside Unit-based Unknown Object Detection in Adverse Weather Conditions for Smart Internet of VehiclesACM Transactions on Management Information Systems10.1145/355492313:4(1-21)Online publication date: 3-Jan-2023
  • (2023)SMNet: Synchronous Multi-Scale Low Light Enhancement Network With Local and Global ConcernIEEE Transactions on Multimedia10.1109/TMM.2023.325414125(9506-9517)Online publication date: 8-Mar-2023
  • Show More Cited By

View Options

Get Access

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media