article

An Expectation-Maximization Algorithm for Blind Separation of Noisy Mixtures Using Gaussian Mixture Model

Authors:

Shan WangAuthors Info & Claims

Circuits, Systems, and Signal Processing, Volume 36, Issue 7

Pages 2697 - 2726

https://doi.org/10.1007/s00034-016-0424-2

Published: 01 July 2017 Publication History

Abstract

In this paper, we propose a new expectation-maximization (EM) algorithm, named GMM-EM, to blind separation of noisy instantaneous mixtures, in which the non-Gaussianity of independent sources is exploited by modeling their distribution using the Gaussian mixture model (GMM). The compatibility between the incomplete-data structure of the GMM and the hidden variable nature of the source separation problem leads to an efficient hierarchical learning and alternative method for estimating the sources and the mixing matrix. In comparison with conventional blind source separation algorithms, the proposed GMM-EM algorithm has superior performance for the separation of noisy mixtures due to the fact that the covariance matrix of the additive Gaussian noise is treated as a parameter. Furthermore, the GMM-EM algorithm works well in underdetermined cases by incorporating any prior information one may have and jointly estimating the mixing matrix and source signals in a Bayesian framework. Systematic simulations with both synthetic and real speech signals are used to show the advantage of the proposed algorithm over conventional independent component analysis techniques, such as FastICA, especially for noisy and/or underdetermined mixtures. Moreover, it can even achieve similar performance to a recent technique called null space component analysis with less computational complexity.

References

[1]

S. Amari, A. Cichocki, Adaptive blind signal processing-neural network approaches. Proc. IEEE 86(10), 2026---2048 (1998)

[2]

A.J. Bell, T.J. Sejnowski, An information-maximization approach to blind separation and blind deconvolution. Neural Comput. 7, 1129---1159 (1995)

Digital Library

[3]

A. Belouchrani, J.F. Cardoso, Maximum likelihood source separation for discrete sources, in Elsevier EUSIPCO'94 (Edinburgh, 1994)

[4]

J. Bilmes, A Gentle Tutorial on the EM Algorithm and Its Application to Parameter Estimation for Gaussian Mixture and Hidden Markov Models, Technical Report ICSI-TR-97-021 (University of Berkelym 1997). http://www.citeseer.nj.nec.com/blimes98gentle.html

[5]

J.F. Cardoso, B.H. Laheld, Equivariant adaptive source separation. IEEE Trans. Signal Process. 44(12), 3017---3030 (1996)

Digital Library

[6]

J.F. Cardoso, A. Souloumiac, Blind beamforming for non-Gaussian signals, in IEE Proceedings F on Radar and Signal Processing, vol. 140(6) (1993), pp. 362---370

[7]

P. Comon, M. Rajih, Blind identification of underdetermined mixtures based on the characteristic function. Signal Process. 86(9), 2671---2681 (2006)

Digital Library

[8]

A.P. Dempster, N.M. Laird, D.B. Rubin, Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. Ser. B 39(1), 1---38 (1977)

[9]

J.L. Gauvain, C.H. Lee, Maximum a posteriori estimation for multivariate Gaussian mixture observations of chains. IEEE Trans. Speech Audio Process. 2(2), 291---298 (1994)

[10]

F. Gu, H. Zhang, D. Zhu, Blind separation of complex sources using generalised generating function. IEEE Signal Process. Lett. 20(1), 71---74 (2013)

[11]

F. Gu, H. Zhang, W. Wang, D. Zhu, Generalized generating function with tucker decomposition and alternating least squares for underdetermined blind identification. EURASIP J. Adv. Signal Process. (2013).

[12]

F. Gu, H. Zhang, D. Zhu, Blind separation of non-stationary sources using continuous density Markov models. Digit. Signal Process. 23(5), 1549---1564 (2013)

Digital Library

[13]

F. Gu, H. Zhang, Y. Xiao, A Bayesian approach to blind separation of mixed discrete sources by Gibbs sampleing, in Lecture Notes on Computer Science, vol. 6905 (2011), pp. 463---475

Digital Library

[14]

http://ai.korea.ac.kr/classes/2004/cse827/doc/map.1994.ieee.291, Accessed in 2013

[15]

Q. Huang, J. Yang, Y. Xue, Y. Zhou, Temporally correlated source separation based on variational Kalman smoother. Digit. Signal Process. 18(3), 422---433 (2008)

Digital Library

[16]

Q. Huo, C. Chan, Bayesian Adaptive Learning of the Parameters of the Hidden Markov Model for Speech Recognition, HKU CSIS Technical Report TR-92-08 (1992). http://www.csis.hku.hk/research/techreps/document/TR-92-08

[17]

W. Hwang, J. Ho, Y. Lin, Null Space Component Analysis for Noisy Blind Source Separation, Technical Report TR-IIS-13-001 (2014). http://www.iis.sinica.edu.tw/page/library/TechReport/tr2013/tr13.html

[18]

A. Hyvärinen, Fast and robust fixed-point algorithms for independent component analysis. IEEE Trans. Neural Netw. 10(3), 626---634 (1999)

Digital Library

[19]

A. Karfoul, L. Albera, G. Birot, Blind underdetermined mixture identification by joint canonical decomposition of HO cumulants. IEEE Trans. Signal Process. 58(2), 638---649 (2010)

Digital Library

[20]

S. Kim, C.D. Yoo, Underdetermined blind source separation based on generalized Gaussian distribution, in Proceedings of the 16th IEEE Signal Processing Society Workshop on Machine Learning for Signal Processing (Arlington, VA, 2006), pp. 103---108

[21]

K.H. Knuth, Informed source separation: a Bayesian tutorial, in Proceedings of the 13th European Signal Processing Conference (EUSIPCO 2005) (Antalya, 2005)

[22]

S. Kullback, R.A. Leibler, On information and sufficiency. Ann. Math. Stat. 22(1), 79---86 (1951)

[23]

L.D. Lathauwer, J. Castaing, J.F. Cardoso, Fourth-order cumulant-based blind identification of underdetermined mixtures. IEEE Trans. Signal Process. 55(6), 2965---2973 (2007)

Digital Library

[24]

L.D. Lathauwer, J. Castaing, Blind identification of underdetermined mixtures by simultaneous matrix diagonalization. IEEE Trans. Signal Process. 56(3), 1096---1105 (2008)

Digital Library

[25]

C.-H. Lee, J.-L. Gauvain, Speaker adaptation based on MAP estimation of HMM parameters, in Proceedings of the IEEE International Conference on Acoustics Speech and Signal Processing (1993), pp. 558---561

Digital Library

[26]

J.Q. Li, A.R. Barron, Mixture density estimation, in Advances in Neural Information Processing Systems, vol. 12 (MIT Press, Cambridge, 2000), pp. 279---285

Digital Library

[27]

X. Luciani, A.L.F. de Almeida, P. Comon, Blind identification of underdetermined mixtures based on the characteristic function: the complex case. IEEE Trans. Signal Process. 59(2), 540---553 (2011)

Digital Library

[28]

J. Ma, L. Xu, M.I. Jordan, Asymptotic convergence rate of the EM algorithm for Gaussian mixtures. Neural Comput. 12, 2881---2907 (2000)

Digital Library

[29]

E. Moulines, J.F. Cardoso, E. Gassiat, Maximum likelihood for blind separation and deconvolution of noisy signals using mixture models, in International Conference on Acoustics, Speech, and Signal Processing (Munich, 1997), pp. 3617---3620

[30]

S. Peng, W. Hwang, Null space pursuit: an operator-based approach to adaptive signal processing. IEEE Trans. Signal Process. 58(5), 2475---2483 (2010)

Digital Library

[31]

M. Razaviyayn, M. Hong, Z. Luo, A Unified Convergence Analysis of Block Successive Minimization Methods for Nonsmooth Optimization, arXiv:1209.2385 {math.OC} (2012)

[32]

T. Routtenberg, J. Tabrikian, MIMO-AR system identification and blind source separation for GMM-distributed sources. IEEE Trans. Signal Process. 57(5), 1717---1730 (2009)

Digital Library

[33]

T. Rydn, EM versus chain Monte Carlo for estimation of hidden Markov models: a computational perspective. Bayesian Anal. 3, 659---688 (2008)

[34]

G. Schwarz, Estimation the dimension of a model. Annu. Stat. 6(2), 461---464 (1978)

[35]

H. Snoussi, A.M. Djafari, Unsupervised learning for source separation with mixture of Gaussians prior for sources and Gaussian prior for mixture coefficients, in Proceedings of the 2001 IEEE Signal Processing Society Workshop on Neural Networks for Signal Processing XI (2001), pp. 293-302

[36]

H. Snoussi, J. Idier, Bayesian blind separation of generalized hyperbolic processes in noisy and underdetermined mixtures. IEEE Trans. Signal Process. 54(9), 3257---3269 (2006)

Digital Library

[37]

S. Sun, C. Peng, W. Hou, J. Zheng, Y. Jiang, X. Zheng, Blind source separation with time series variational Bayes expectation maximization algorithm. Digit. Signal Process. 12(1), 17---33 (2012)

Digital Library

[38]

K. Todros, J. Tabrikian, Blind separation of independent sources using Gaussian mixture model. IEEE Trans. Signal Process. 55(7), 3645---3658 (2007)

[39]

P. Tseng, Convergence of a block coordinate descent method for nondifferentiable minimization. J. Optim. Theory Appl. 109, 475---494 (2001)

Digital Library

[40]

L. Xu, M.I. Jordan, On convergence properties of the EM algorithm for Gaussian mixtures. Neural Comput. 8, 129---151 (1996)

Digital Library

[41]

Y. Zhang, X. Shi, C.H. Chen, A Gaussian mixture model for underdetermined independent component analysis. Signal Process. 86(6), 1538---1549 (2006)

Digital Library

[42]

Y. Zhao, Image segmentation using temporal-spatial information in dynamic scenes, in Proceedings of the IEEE International Conference on Machine Learning and Cybernetics (2003)

Cited By

Liu H(2022)Clothing Nanometer Antimite and Antibacterial Based on Deep Learning TechnologyJournal of Nanomaterials10.1155/2022/49161972022Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1155/2022/4916197
Azam MBouguila N(2019)Bounded Generalized Gaussian Mixture Model with ICANeural Processing Letters10.1007/s11063-018-9868-749:3(1299-1320)Online publication date: 1-Jun-2019
https://dl.acm.org/doi/10.1007/s11063-018-9868-7
Schwartz BGannot SHabets ESchwartz BGannot SHabets E(2017)Two Model-Based EM Algorithms for Blind Source Separation in Noisy EnvironmentsIEEE/ACM Transactions on Audio, Speech and Language Processing10.1109/TASLP.2017.273843825:11(2209-2222)Online publication date: 1-Nov-2017
https://dl.acm.org/doi/10.1109/TASLP.2017.2738438

An Expectation-Maximization Algorithm for Blind Separation of Noisy Mixtures Using Gaussian Mixture Model

Recommendations

PARAFAC-Based Blind Identification of Underdetermined Mixtures Using Gaussian Mixture Model

This paper presents a novel algorithm, named GMM-PARAFAC, for blind identification of underdetermined instantaneous linear mixtures. The GMM-PARAFAC algorithm uses Gaussian mixture model (GMM) to model non-Gaussianity of the independent sources. We show ...
Blind separation of non-stationary sources using continuous density hidden Markov models

Blind source separation (BSS) has attained much attention in signal processing society due to its 'blind' property and wide applications. However, there are still some open problems, such as underdetermined BSS, noise BSS. In this paper, we propose a ...
Blind source separation of more sources than mixtures using generalized exponential mixture models

Blind source separation is discussed with more sources than mixtures in this paper. The blind separation technique includes two steps. The first step is to estimate a mixing matrix, and the second is to estimate sources. If the sources are sparse, the ...

Comments

Information & Contributors

Information

Published In

cover image Circuits, Systems, and Signal Processing

Circuits, Systems, and Signal Processing Volume 36, Issue 7

July 2017

412 pages

ISSN:0278-081X

Issue’s Table of Contents

Copyright © Copyright © 2017 Springer Science+Business Media New York.

Publisher

Birkhauser Boston Inc.

United States

Publication History

Published: 01 July 2017

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 22 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Liu H(2022)Clothing Nanometer Antimite and Antibacterial Based on Deep Learning TechnologyJournal of Nanomaterials10.1155/2022/49161972022Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1155/2022/4916197
Azam MBouguila N(2019)Bounded Generalized Gaussian Mixture Model with ICANeural Processing Letters10.1007/s11063-018-9868-749:3(1299-1320)Online publication date: 1-Jun-2019
https://dl.acm.org/doi/10.1007/s11063-018-9868-7
Schwartz BGannot SHabets ESchwartz BGannot SHabets E(2017)Two Model-Based EM Algorithms for Blind Source Separation in Noisy EnvironmentsIEEE/ACM Transactions on Audio, Speech and Language Processing10.1109/TASLP.2017.273843825:11(2209-2222)Online publication date: 1-Nov-2017
https://dl.acm.org/doi/10.1109/TASLP.2017.2738438

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents