Cross-contrast Fusion and Aggregation Network for Multi-contrast MRI Super-resolution
Authors: 
Yue Ding, Tao Zhou, Ye WuAuthors Info & Claims
Yue Ding, Tao Zhou, Ye WuAuthors Info & ClaimsPages 102 - 107
Abstract
Magnetic resonance imaging (MRI) super-resolution (SR) can restore high-quality high-resolution (HR) images from low-resolution (LR) ones, which is helpful for clinical diagnosis and treatment. Different from SR restoration using a single-contrast, multi-contrast SR restoration has received increasing attention because different modalities can provide complementary information to improve the SR quality. However, fusing multi-contrast features as well as capturing cross-level correlations among different layers are still challenging. In this paper, we propose a Cross-contrast Fusion and Aggregation Network (CFA-Net) for multi-contrast MRI super-resolution, which can effectively explore complementary information from multi-contrast images to improve target-image SR quality. Specifically, the multi-contrast images are passed through multiple residual Swin Transformer blocks (RSTB) to learn hierarchical feature representations at different layers. Then, we present a Cross-contrast Fusion Module (CFM) to fuse the cross-contrast features in a layer-wise strategy, which can capture the complementary information from multi-contrast MR images. Moreover, a cross-level Feature Aggregation Module (FAM) is proposed to integrate cross-level features from CFMs for exploring the interaction between different layers. Experimental results on three multi-contrast MR datasets demonstrate that our method performs better than other state-of-the-art SR methods.
References
[1]
Charlotta Andersson, Johan Kihlberg, Tino Ebbers, Lena Lindström, Carl-Johan Carlhäll, and Jan E Engvall. 2016. Phase-contrast MRI volume flow–a comparison of breath held and navigator based acquisitions. BMC Medical Imaging 16, 1 (2016), 1–8.
[2]
Kanwal K Bhatia, Anthony N Price, Wenzhe Shi, Jo V Hajnal, and Daniel Rueckert. 2014. Super-resolution reconstruction of cardiac MRI using coupled dictionary learning. In IEEE ISBI. 947–950.
[3]
Anthony G Christodoulou, Haosen Zhang, Bo Zhao, T Kevin Hitchens, Chien Ho, and Zhi-Pei Liang. 2013. High-resolution cardiovascular MRI by integrating parallel imaging with low-rank and sparse modeling. IEEE Transactions on Biomedical Engineering 60, 11 (2013), 3083–3092.
[4]
Chun-Mei Feng, Huazhu Fu, Shuhao Yuan, and Yong Xu. 2021. Multi-contrast MRI super-resolution via a multi-stage integration network. In International Conference on Medical Image Computing and Computer Assisted Intervention. Springer, 140–149.
[5]
Chun-Mei Feng, Kai Wang, Shijian Lu, Yong Xu, and Xuelong Li. 2021. Brain MRI super-resolution using coupled-projection residual network. Neurocomputing 456 (2021), 190–199.
[6]
Christian Ledig, Lucas Theis, Ferenc Huszár, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, 2017. Photo-realistic single image super-resolution using a generative adversarial network. In IEEE Computer Vision and Pattern Recognition Conference. 4681–4690.
[7]
Jingyun Liang, Jiezhang Cao, Guolei Sun, Kai Zhang, Luc Van Gool, and Radu Timofte. 2021. Swinir: Image restoration using swin transformer. In IEEE International Conference on Computer Vision. 1833–1844.
[8]
Bee Lim, Sanghyun Son, Heewon Kim, Seungjun Nah, and Kyoung Mu Lee. 2017. Enhanced deep residual networks for single image super-resolution. In IEEE Computer Vision and Pattern Recognition Conference Workshops. 136–144.
[9]
I. C. London. [n. d.]. IXI dataset (2015). http://brain-development.org/ixi-dataset/.
[10]
Qing Lyu, Hongming Shan, Cole Steber, Corbin Helis, Chris Whitlow, Michael Chan, and Ge Wang. 2020. Multi-contrast super-resolution MRI through a progressive network. IEEE Transactions on Medical Imaging 39, 9 (2020), 2738–2749.
[11]
Qing Lyu, Hongming Shan, and Ge Wang. 2020. MRI super-resolution with ensemble learning and complementary priors. IEEE Transactions on Computational Imaging 6 (2020), 615–624.
[12]
Oskar Maier, Bjoern H Menze, Janina von der Gablentz, Levin Häni, Mattias P Heinrich, Matthias Liebrand, 2017. ISLES 2015-A public evaluation benchmark for ischemic stroke lesion segmentation from multispectral MRI. Medical Image Analysis 35 (2017), 250–269.
[13]
James PB O’connor, Alan Jackson, Geoff JM Parker, Caleb Roberts, and Gordon C Jayson. 2012. Dynamic contrast-enhanced MRI in clinical trials of antivascular therapies. Nature reviews Clinical oncology 9, 3 (2012), 167–177.
[14]
David Owen, Andrew Melbourne, Zach Eaton-Rosen, David L Thomas, 2018. Deep convolutional filtering for spatio-temporal denoising and artifact removal in arterial spin labelling MRI. In International Conference on Medical Image Computing and Computer Assisted Intervention. Springer, 21–29.
[15]
Youwei Pang, Lihe Zhang, Xiaoqi Zhao, and Huchuan Lu. 2020. Hierarchical dynamic filtering network for rgb-d salient object detection. In European Conference on Computer Vision. Springer, 235–252.
[16]
Sérgio Pereira, Adriano Pinto, Victor Alves, and Carlos A Silva. 2016. Brain tumor segmentation using convolutional neural networks in MRI images. IEEE Transactions on Medical Imaging 35, 5 (2016), 1240–1251.
[17]
Andrea Rueda, Norberto Malpica, and Eduardo Romero. 2013. Single-image super-resolution of brain MR images using overcomplete dictionaries. Medical Image Analysis 17, 1 (2013), 113–132.
[18]
Feng Shi, Jian Cheng, Li Wang, Pew-Thian Yap, and Dinggang Shen. 2015. LRTV: MR image super-resolution with low-rank and total variation regularizations. IEEE Transactions on Medical Imaging 34, 12 (2015), 2459–2466.
[19]
Jun Shi, Qingping Liu, Chaofeng Wang, Qi Zhang, Shihui Ying, and Haoyu Xu. 2018. Super-resolution reconstruction of MR image with a novel residual learning network algorithm. Physics in Medicine & Biology 63, 8 (2018), 085011.
[20]
Tzu-An Song, Fan Yang, Samadrita Roy Chowdhury, Kyungsang Kim, Keith A Johnson, Georges El Fakhri, Quanzheng Li, and Joyita Dutta. 2019. PET image deblurring and super-resolution with an MR-based joint entropy prior. IEEE Transactions on Computational Imaging 5, 4 (2019), 530–539.
[21]
Radu Timofte, Vincent De Smet, and Luc Van Gool. 2014. A+: Adjusted anchored neighborhood regression for fast super-resolution. In ACCV. Springer, 111–126.
[22]
Sébastien Tourbier, Xavier Bresson, Patric Hagmann, Jean-Philippe Thiran, Reto Meuli, and Meritxell Bach Cuadra. 2015. An efficient total variation algorithm for super-resolution in fetal brain MRI with adaptive regularization. NeuroImage 118 (2015), 584–597.
[23]
Eric Van Reeth, Ivan WK Tham, Cher Heng Tan, and Chueh Loo Poh. 2012. Super-resolution in magnetic resonance imaging: a review. Concepts in Magnetic Resonance Part A 40, 6 (2012), 306–325.
[24]
Lei Xiang, Yong Chen, Weitang Chang, Yiqiang Zhan, Weili Lin, Qian Wang, and Dinggang Shen. 2018. Deep-learning-based multi-modal fusion for fast MR reconstruction. IEEE Transactions on Biomedical Engineering 66, 7 (2018), 2105–2114.
[25]
Kai Xuan, Shanhui Sun, Zhong Xue, Qian Wang, and Shu Liao. 2020. Learning mri k-space subsampling pattern using progressive weight pruning. In International Conference on Medical Image Computing and Computer Assisted Intervention. Springer, 178–187.
[26]
Jure Zbontar, Florian Knoll, Anuroop Sriram, Tullie Murrell, Zhengnan Huang, Matthew J Muckley, Aaron Defazio, Ruben Stern, Patricia Johnson, Mary Bruno, 2018. fastMRI: An open dataset and benchmarks for accelerated MRI. arXiv preprint arXiv:1811.08839 (2018).
[27]
Kun Zeng, Hong Zheng, Congbo Cai, Yu Yang, Kaihua Zhang, and Zhong Chen. 2018. Simultaneous single-and multi-contrast super-resolution for brain MRI images based on a convolutional neural network. Computers in Biology and Medicine 99 (2018), 133–141.
[28]
Hong Zheng, Xiaobo Qu, Zhengjian Bai, Yunsong Liu, Di Guo, Jiyang Dong, Xi Peng, and Zhong Chen. 2017. Multi-contrast brain magnetic resonance image super-resolution using the local weight similarity. BMC Medical Imaging 17 (2017), 1–13.
[29]
Tao Zhou, Huazhu Fu, Geng Chen, Jianbing Shen, and Ling Shao. 2020. Hi-net: hybrid-fusion network for multi-modal MR image synthesis. IEEE Transactions on Medical Imaging 39, 9 (2020), 2772–2781.
[30]
Tao Zhou, Mingxia Liu, Kim-Han Thung, and Dinggang Shen. 2019. Latent representation learning for Alzheimer’s disease diagnosis with incomplete multi-modality neuroimaging and genetic data. IEEE Transactions on Medical Imaging 38, 10 (2019), 2411–2422.
Index Terms
- Cross-contrast Fusion and Aggregation Network for Multi-contrast MRI Super-resolution
Recommendations
Model-Guided Multi-Contrast Deep Unfolding Network for MRI Super-resolution Reconstruction
MM '22: Proceedings of the 30th ACM International Conference on MultimediaMagnetic resonance imaging (MRI) with high resolution (HR) provides more detailed information for accurate diagnosis and quantitative image analysis. Despite the significant advances, most existing super-resolution (SR) reconstruction network for ...
Accurate Multi-contrast MRI Super-Resolution via a Dual Cross-Attention Transformer Network
Medical Image Computing and Computer Assisted Intervention – MICCAI 2023AbstractMagnetic Resonance Imaging (MRI) is a critical imaging tool in clinical diagnosis, but obtaining high-resolution MRI images can be challenging due to hardware and scan time limitations. Recent studies have shown that using reference images from ...
Cross-contrast mutual fusion network for joint MRI reconstruction and super-resolution
AbstractMagnetic Resonance Imaging (MRI) is a widely used medical imaging technique that has become an essential tool for diagnosing various diseases and visualizing internal structures and tissues in the human body. MRI reconstruction and super-...
Highlights- We propose a novel Cross-contrast Mutual Fusion Network for Joint MRI Reconstruction and Super-Resolution.
- A Multi-contrast Feature Aggregation (MFA) module is presented to learn the multi-contrast feature representations.
- A Multi-...
Comments
Information & Contributors
Information
Published In
March 2023
824 pages
ISBN:9781450399029
DOI:10.1145/3594315
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 02 August 2023
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ICCAI 2023
ICCAI 2023: 2023 9th International Conference on Computing and Artificial Intelligence
March 17 - 20, 2023
Tianjin, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 83Total Downloads
- Downloads (Last 12 months)67
- Downloads (Last 6 weeks)2
Reflects downloads up to 14 Sep 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format