Constructive Neural Network Learning

Shaobo Lin Wenzhou University Jinshan Zeng Jiangxi Normal University Xiaoqin Zhang Wenzhou University

Machine Learning mathscidoc:1903.41001

IEEE Transactions on Cybernetics, 49, (1), 2019.1
In this paper, we aim at developing scalable neural network-type learning systems. Motivated by the idea of constructive neural networks in approximation theory, we focus on constructing rather than training feed-forward neural networks (FNNs) for learning, and propose a novel FNNs learning system called the constructive FNN (CFN). Theoretically, we prove that the proposed method not only overcomes the classical saturation problem for constructive FNN approximation, but also reaches the optimal learning rate when the regression function is smooth, while the state-of-the-art learning rates established for traditional FNNs are only near optimal (up to a logarithmic factor). A series of numerical simulations are provided to show the efficiency and feasibility of CFN.
Constructive neural network learning, generalization error, neural networks, saturation
[ Download ] [ 2019-03-19 20:25:11 uploaded by JinshanZeng ] [ 868 downloads ] [ 0 comments ]
  title={Constructive Neural Network Learning},
  author={Shaobo Lin, Jinshan Zeng, and Xiaoqin Zhang},
  booktitle={IEEE Transactions on Cybernetics},
Shaobo Lin, Jinshan Zeng, and Xiaoqin Zhang. Constructive Neural Network Learning. 2019. Vol. 49. In IEEE Transactions on Cybernetics.
Please log in for comment!
Contact us: | Copyright Reserved