Abstract—Deep learning has shown its great potential and function in algorithm research and practical application (such as speech recognition, natural language processing, computer vision). Deep learning is a kind of new multilayer neural network learning algorithm, which alleviates the optimization difficulty of traditional deep models and arouses wide attention in the field of machine learning. Firstly, the origin of deep learning is discussed and the concept of deep learning is also introduced. Secondly, according to the architectural characteristics, deep learning algorithms are classified into three classes, this paper emphatically introduces deep networks for unsupervised and supervised learning model and elaborates typical deep learning models and the corresponding extension models. This paper also analyzes both advantage and disadvantage of each model and points out each extension method's inheritance relationship with the corresponding typical model. Finally, applications of deep learning algorithms is illustrated, the remaining issues and the future orientation are concluded as well.

Index Terms—deep learning, auto-encoder, restricted boltzmann machine, convolutional neural network, deep neural network

Cite: Shuo Hu, Yaqing Zuo, Lizhe Wang, and Peng Liu, "A Review about Building Hidden Layer Methods of Deep Learning," Vol. 7, No. 1, pp. 13-22, February, 2016. doi: 10.12720/jait.7.1.13-22