Abstract
A new algorithm is developed to jointly recover a temporal sequence of images from noisy and under-sampled Fourier data. Specifically we consider the case where each data set is missing vital information that prevents its (individual) accurate recovery. Our new method is designed to restore the missing information in each individual image by “borrowing” it from the other images in the sequence. As a result, all of the individual reconstructions yield improved accuracy. The use of high resolution Fourier edge detection methods is essential to our algorithm. In particular, edge information is obtained directly from the Fourier data which leads to an accurate coupling term between data sets. Moreover, data loss is largely avoided as coarse reconstructions are not required to process inter- and intra-image information. Numerical examples are provided to demonstrate the accuracy, efficiency and robustness of our new method.
Similar content being viewed by others
Data Availability
All data sets are publicly available. MATLAB codes used for all numerical results are available from the authors upon request.
Notes
We will use jump function and edge function interchangeably throughout our exposition.
To be clear, the methods cited here are primarily re-weighted \(\ell _1\) regularization methods (and often referred to as such), since the weights are iteratively adapted. By contrast the VBJS method does not iteratively adapt the weights, so it is a weighted \(\ell _1\) regularization scheme.
To be clear, the typical MMV reconstruction process uses multiple observations at a single snapshot in time, that is, when there is no change in the underlying scene. This is in contrast to the problem of interest in this investigation, which considers a sequence of observations over time during which the underlying scene changes.
Standard weighted \(\ell _1\) regularization schemes typically scale the weights to be inversely proportional to the magnitudes of the components in the sparse domain of the solution at each iteration. Numerical experiments in [2] demonstrate that the VBJS approach is more robust, especially in low SNR environments.
These are reasonable assumptions in many applications, such as the cars and tanks considered in our synthetic aperture radar (SAR) data example.
Using the smallest or average radius would cause the reconstructed edge curve to sometimes lie inside the actual edge curve, resulting in an inaccurate change mask, \(\tilde{C}_j\) in (28).
To the best of our knowledge, however, these procedures all use pixelated reconstructed images to detect the change, where as in our approach we use edge maps generated from the Fourier data to avoid data loss.
Comparing two matrices by simply checking the relation for all elements (\(A = B \iff A(k,l) = B(k,l)\) for all k, l) is not appropriate since any single misidentified edge point, for example, due to noise, may cause the objects in two single object edge masks of the same object, say at corresponding times j and \(j+1\), to be determined as different.
Although we incorporate information from \(J = 6\) data sets, for better visualization we display only four.
References
Gelb, A., Tadmor, E.: Detection of edges in spectral data. Appl. Comput. Harmon. Anal. 7(1), 101–135 (1999)
Adcock, B., Gelb, A., Song, G., Sui, Y.: Joint sparse recovery based on variances. SIAM J. Sci. Comput. 41(1), 246–268 (2019). https://doi.org/10.1137/17M1155983
Gelb, A., Scarnati, T.: Reducing effects of bad data using variance based joint sparsity recovery. J. Sci. Comput. 78(1), 94–120 (2019)
Scarnati, T., Gelb, A.: Accurate and efficient image reconstruction from multiple measurements of fourier samples (2020)
Candès, E.J., Romberg, J., Tao, T.: Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52(2), 489–509 (2006)
Candès, E.J., Romberg, J.K., Tao, T.: Stable signal recovery from incomplete and inaccurate measurements. Commun. Pure Appl. Math. J. Issued Courant Inst. Math. Sci. 59(8), 1207–1223 (2006)
Candès, E.J., Tao, T.: Near-optimal signal recovery from random projections: universal encoding strategies? IEEE Trans. Inf. Theory 52(12), 5406–5425 (2006)
Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006)
Candès, E.J., Wakin, M.B., Boyd, S.P.: Enhancing sparsity by reweighted \(\ell _1\) minimization. J. Fourier Anal. Appl. 14(5–6), 877–905 (2008)
Chartrand, R., Yin, W.: Iteratively reweighted algorithms for compressive sensing. In: IEEE international conference on acoustics, speech and signal processing, pp. 3869–3872 (2008)
Daubechies, I., DeVore, R., Fornasier, M., Gunturk, C.S.: Iteratively re-weighted least squares minimization for sparse recovery. Commun. Pure Appl. Math. J. Issued Courant Inst. Math. Sci. 63(1), 1–38 (2010)
Liu, Y., Ma, J., Fan, Y., Liang, Z.: Adaptive-weighted total variation minimization for sparse data toward low-dose X-ray computed tomography image reconstruction. Phys. Med. Biol. 57(23), 7923–7956 (2012). https://doi.org/10.1088/0031-9155/57/23/7923
Langer, A.: Automated parameter selection for total variation minimization in image restoration. J. Math. Imaging Vis. 57(2), 239–268 (2017)
Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 6, 679–698 (1986)
Sobel, I., Feldman, G.: A \(3\times 3\) isotropic gradient operator for image processing. A Talk at the Stanford Artificial Project, 271–272 (1968)
Jin-Yu, Z., Yan, C., Xian-Xiang, H.: Edge detection of images based on improved Sobel operator and genetic algorithms. In: IEEE international conference on image analysis and signal processing, pp. 31–35 (2009)
Gao, W., Zhang, X., Yang, L., Liu, H.: An improved Sobel edge detection. In: IEEE 3rd international conference on computer science and information technology, vol. 5, pp. 67–71 (2010)
Gelb, A., Tadmor, E.: Detection of edges in spectral data II. Nonlinear enhancement. SIAM J. Num. Anal. 38(4), 1389–1408 (2000)
Archibald, R., Gelb, A., Platte, R.: Image reconstruction from undersampled Fourier data using the polynomial annihilation transform. J. Sci. Comput. (2015). https://doi.org/10.1007/s10915-015-0088-2
Martinez, A., Gelb, A., Gutierrez, A.: Edge detection from non-uniform Fourier data using the convolutional gridding algorithm. J. Sci. Comput. 61(3), 490–512 (2014)
Gelb, A., Song, G.: Detecting edges from non-uniform Fourier data using Fourier frames. J. Sci. Comput. 71(2), 737–758 (2017)
Viswanathan, A., Gelb, A., Cochran, D.: Iterative design of concentration factors for jump detection. J. Sci. Comput. 51(3), 631–649 (2012)
Xie, W., Deng, Y., Wang, K., Yang, X., Luo, Q.: Reweighted \(\ell _1\) regularization for restraining artifacts in FMT reconstruction images with limited measurements. Opt. Lett. 39(14), 4148–4151 (2014)
Landi, G.: The Lagrange method for the regularization of discrete ill-posed problems. Comput. Optim. Appl. 39(3), 347–368 (2008)
Wen, Y.-W., Chan, R.H.: Parameter selection for total-variation-based image restoration using discrepancy principle. IEEE Trans. Image Process. 21(4), 1770–1781 (2011)
Yang, X., Hofmann, R., Dapp, R., Van de Kamp, T., dos Santos Rolo, T., Xiao, X., Moosmann, J., Kashef, J., Stotzka, R.: TV-based conjugate gradient method and discrete L-curve for few-view CT reconstruction of X-ray in vivo data. Opt. Express 23(5), 5368–5387 (2015)
Gong, C., Zeng, L.: Adaptive iterative reconstruction based on relative total variation for low-intensity computed tomography. Signal Process. 165, 149–162 (2019)
Vogel, C.R.: Regularization parameter selection methods. Soc. Ind. Appl. Math. (2002). https://doi.org/10.1137/1.9780898717570.ch7
Sanders, T., Platte, R.B., Skeel, R.D.: Effective new methods for automated parameter selection in regularized inverse problems. Appl. Numer. Math. 152, 29–48 (2020)
Shchukina, A., Kasprzak, P., Dass, R., Nowakowski, M., Kazimierczuk, K.: Pitfalls in compressed sensing reconstruction and how to avoid them. J. Biomol. NMR 68(2), 79–98 (2017). https://doi.org/10.1007/s10858-016-0068-3
Kang, M.-S., Kim, K.-T.: Compressive sensing based SAR imaging and autofocus using improved Tikhonov regularization. IEEE Sens. J. 19(14), 5529–5540 (2019). https://doi.org/10.1109/JSEN.2019.2904611
Churchill, V., Archibald, R., Gelb, A.: Edge-adaptive \(\ell _2\) regularization image reconstruction from non-uniform Fourier data. Inverse Probl. Imaging. 13 (2019). https://doi.org/10.3934/ipi.2019042
Archibald, R., Gelb, A., Yoon, J.: Polynomial fitting for edge detection in irregularly sampled signals and images. SIAM J. Numer. Anal. 43(1), 259–279 (2005)
Song, P., Mota, J.F.C., Deligiannis, N., Rodrigues, M.R.D.: Coupled dictionary learning for multimodal image super-resolution. In: IEEE global conference on signal and information processing (GlobalSIP), pp. 162–166 (2016). https://doi.org/10.1109/GlobalSIP.2016.7905824
Song, P., Deng, X., Mota, J.F.C., Deligiannis, N., Dragotti, P.L., Rodrigues, M.R.D.: Multimodal image super-resolution via joint sparse representations induced by coupled dictionaries. IEEE Trans. Comput. Imaging 6, 57–72 (2020). https://doi.org/10.1109/TCI.2019.2916502
Rigie, D.S., Rivière, P.J.L.: Joint reconstruction of multi-channel, spectral CT data via constrained total nuclear variation minimization. Phys. Med. Biol. 60(5), 1741–1762 (2015). https://doi.org/10.1088/0031-9155/60/5/1741
Kazantsev, D., Jørgensen, J.S., Andersen, M.S., Lionheart, W.R.B., Lee, P.D., Withers, P.J.: Joint image reconstruction method with correlative multi-channel prior for X-ray spectral computed tomography. Inverse Probl. 34(6), 064001 (2018). https://doi.org/10.1088/1361-6420/aaba86
Eslahi, N., Foi, A.: Joint sparse recovery of misaligned multimodal images via adaptive local and nonlocal cross-modal regularization. In: IEEE 8th international workshop on computational advances in multi-sensor adaptive processing (CAMSAP), pp. 111–115 (2019). https://doi.org/10.1109/CAMSAP45676.2019.9022478
Kazantsev, D., Lionheart, W.R.B., Withers, P.J., Lee, P.D.: Multimodal image reconstruction using supplementary structural information in total variation regularization. Sens. Imaging 15(1), 97 (2014). https://doi.org/10.1007/s11220-014-0097-5
Chen, Y., Fang, R., Ye, X.: Joint image edge reconstruction and its application in multi-contrast MRI (2017) arXiv:1712.02000 [math.NA]
Chen, G., Hay, G.J., Carvalho, L.M.T., Wulder, M.A.: Object-based change detection. Int. J. Remote Sens. 33(14), 4434–4457 (2012). https://doi.org/10.1080/01431161.2011.648285
Hussain, M., Chen, D., Cheng, A., Wei, H., Stanley, D.: Change detection from remotely sensed images: From pixel-based to object-based approaches. ISPRS J. Photogramm. Remote. Sens. 80, 91–106 (2013). https://doi.org/10.1016/j.isprsjprs.2013.03.006
Tewkesbury, A.P., Comber, A.J., Tate, N.J., Lamb, A., Fisher, P.F.: A critical synthesis of remotely sensed optical image change detection techniques. Remote Sens. Environ. 160, 1–14 (2015). https://doi.org/10.1016/j.rse.2015.01.006
Inglada, J., Mercier, G.: A new statistical similarity measure for change detection in multitemporal SAR images and its extension to multiscale change analysis. IEEE Trans. Geosci. Remote Sens. 45(5), 1432–1445 (2007)
Thonfeld, F., Feilhauer, H., Braun, M., Menz, G.: Robust change vector analysis (RCVA) for multi-sensor very high resolution optical satellite data. Int. J. Appl. Earth Obs. Geoinf. 50, 131–140 (2016)
Ye, S., Chen, D., Yu, J.: A targeted change-detection procedure by combining change vector analysis and post-classification approach. ISPRS 114, 115–124 (2016). https://doi.org/10.1016/j.isprsjprs.2016.01.018
McDermid, G.J., Linke, J., Pape, A.D., Laskin, D.N., McLane, A.J., Franklin, S.E.: Object-based approaches to change analysis and thematic map update: challenges and limitations. Can. J. Remote. Sens. 34(5), 462–466 (2008)
Chen, G., Zhao, K., Powers, R.: Assessment of the image misregistration effects on object-based change detection. ISPRS J. Photogramm. Remote. Sens. 87, 19–27 (2014). https://doi.org/10.1016/j.isprsjprs.2013.10.007
Hsiao, Y.-T., Chuang, C.-L., Jiang, J.-A., Chien, C.-C.: A contour based image segmentation algorithm using morphological edge detection. In: IEEE international conference on systems, man and cybernetics, vol. 3, pp. 2962–2967 (2005). https://doi.org/10.1109/ICSMC.2005.1571600
Papari, G., Petkov, N.: Adaptive pseudo dilation for gestalt edge grouping and contour detection. IEEE Trans. Image Process. 17(10), 1950–1962 (2008)
Papari, G., Petkov, N.: Edge and line oriented contour detection: state of the art. Image Vis. Comput. 29(2–3), 79–103 (2011)
Gao, F., Wang, M., Cai, Y., Lu, S.: Extracting closed object contour in the image: remove, connect and fit. Pattern Anal. Appl. 22(3), 1123–1136 (2019)
Archibald, R., Gelb, A.: A method to reduce the Gibbs ringing artifact in MRI scans while keeping tissue boundary integrity. IEEE Trans. Med. Imaging 21(4), 305–319 (2002)
Archibald, R., Chen, K., Gelb, A., Renaut, R.: Improving tissue segmentation of human brain MRI through preprocessing by the Gegenbauer reconstruction method. Neuroimage 20(1), 489–502 (2003)
Afaq, Y., Manocha, A.: Analysis on change detection techniques for remote sensing applications: a review. Ecol. Inf. 63, 101310 (2021)
Ash, J.N.: A unifying perspective of coherent and non-coherent change detection. In: Zelnio, E., Garber, F.D. (eds.) Algorithms for Synthetic Aperture Radar Imagery XXI, vol. 9093, pp. 90–98. SPIE, (2014). https://doi.org/10.1117/12.2054338. International Society for Optics and Photonics
Li, H., Gong, M., Wang, Q., Liu, J., Su, L.: A multiobjective fuzzy clustering method for change detection in SAR images. Appl. Soft Comput. 46, 767–777 (2016). https://doi.org/10.1016/j.asoc.2015.10.044
Ji, S., Xue, Y., Carin, L.: Bayesian compressive sensing. IEEE Trans. Signal Process. 56(6), 2346–2356 (2008)
Tipping, M.E.: Sparse Bayesian learning and the relevance vector machine. J. Mach. Learn. Res. 1, 211–244 (2001)
Wipf, D.P., Rao, B.D.: Sparse Bayesian learning for basis selection. IEEE Trans. Signal Process. 52(8), 2153–2164 (2004)
Boyd, S., Parikh, N., Chu, E.: Distributed optimization and statistical learning via the alternating direction method of multipliers (2011). https://doi.org/10.1561/2200000016
Ellsworth, M., Thomas, C.: A fast algorithm for image deblurring with total variation regularization. Unmanned Tech Solutions 4 (2014)
Lalwani, G., Livingston Sundararaj, J., Schaefer, K., Button, T., Sitharaman, B.: Synthesis, characterization, in vitro phantom imaging, and cytotoxicity of a novel graphene-based multimodal magnetic resonance imaging - X-ray computed tomography contrast agent. J. Mater. Chem. B 2(22), 3519–3530 (2015). https://doi.org/10.1039/C4TB00326H
Glaubitz, J., Gelb, A.: High order edge sensors with \(\ell ^1\) regularization for enhanced discontinuous Galerkin methods. SIAM J. Sci. Comput. 41(2), 1304–1330 (2019)
Acknowledgements
This work is partially supported by the NSF grants DMS #1521661 (GS), DMS #1912685 (AG), DMS #1939203 (GS), AFOSR grant #FA9550-18-1-0316 (AG), and ONR MURI grant #N00014-20-1-2595 (AG).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors have not disclosed any competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Xiao, Y., Glaubitz, J., Gelb, A. et al. Sequential Image Recovery from Noisy and Under-Sampled Fourier Data. J Sci Comput 91, 79 (2022). https://doi.org/10.1007/s10915-022-01850-7
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10915-022-01850-7