Geometric Interpretation of Nonlinear Approximation Capability for Feedforward Neural Networks

This paper presents a preliminary study on the nonlinear approximation capability of feedforward neural networks (FNNs) via a geometric approach. Three simplest FNNs with at most four free parameters are defined and investigated. By approximations on one-dimensional functions, we observe that the Chebyshev-polynomials, Gaussian, and sigmoidal FNNs are ranked in order of providing more varieties of nonlinearities. If neglecting the compactness feature inherited by Gaussian neural networks, we consider that the Chebyshev-polynomial-based neural networks will be the best among three types of FNNs in an efficient use of free parameters.

• Type: Conference
• Year: 2004
• Where: ISNN
• Authors: Bao-Gang Hu, Hong-Jie Xing, Yujiu Yang
• Keywords: Feedforward Neural Networks, Free Parameters, ISNN 2004, Neural Networks