Abstract
We analyze the dynamical behaviors of semi-supervised learning in the framework of on-line learning by using the statistical-mechanical method. A student uses several correlated input vectors in each update. The student is given a desired output for only one input vector out of these correlated input vectors. In this model, we derive simultaneous differential equations with deterministic forms that describe the dynamical behaviors of order parameters using the self-averaging property in the thermodynamic limit. We treat the Hebbian and Perceptron learning rules. As a result, it is shown that using unlabeled data is effective in the early stages for both of the two learning rules. In addition, we show that the two learning rules have qualitatively different dynamical behaviors. Furthermore, we propose a new algorithm that improves the generalization performance by switching the number of input vectors used in an update as the time step proceeds.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Saad, D. (ed.): On-Line Learning in Neural Networks. Cambridge University Press, Cambridge (1998)
Urakami, M., Miyoshi, S., Okada, O.: Statistical mechanics of on-line learning when a moving teacher goes around an unlearnable true teacher. J. Phys. Soc. Jpn. 76, 044003 (2007)
Hirama, T., Hukushima, K.: On-line learning of an unlearnable true teacher through mobile ensemble teachers. J. Phys. Soc. Jpn. 77, 094801 (2008)
Chapelle, O., Scholkopf, B., Zien, A.: Semi-Supervised Learning. MIT Press, Cambridge (2006)
Zhu, X., Goldberg, A.B.: Introduction to Semi-Supervised Learning: Synthesis Lectures on Artificial Intelligence and Machine Learning. Morgan & Claypool Publishers, city of publication (2009)
Engel, A., Broeck, C.: Statistical Mechanics of Learning. Cambridge University Press, Cambridge (2001)
Nakao, K., Narukawa, Y., Miyoshi, S.: Statistical mechanics of on-line learning using correlated examples. IEICE Trans. Inf. Syst. E94–D(10), 1941–1944 (2011)
Nishimori, H.: Statistical Physics of Spin Glasses and Information Processing: An Introduction. Oxford University Press, Oxford (2001)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Fujii, T., Ito, H., Miyoshi, S. (2016). A Theoretical Analysis of Semi-supervised Learning. In: Hirose, A., Ozawa, S., Doya, K., Ikeda, K., Lee, M., Liu, D. (eds) Neural Information Processing. ICONIP 2016. Lecture Notes in Computer Science(), vol 9948. Springer, Cham. https://doi.org/10.1007/978-3-319-46672-9_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-46672-9_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-46671-2
Online ISBN: 978-3-319-46672-9
eBook Packages: Computer ScienceComputer Science (R0)