Relative Loss Bounds for Multidimensional Regression Problems

10 years 5 months ago
Relative Loss Bounds for Multidimensional Regression Problems
We study on-line generalized linear regression with multidimensional outputs, i.e., neural networks with multiple output nodes but no hidden nodes. We allow at the final layer transfer functions such as the softmax function that need to consider the linear activations to all the output neurons. The weight vectors used to produce the linear activations are represented indirectly by maintaining separate parameter vectors. We get the weight vector by applying a particular parameterization function to the parameter vector. Updating the parameter vectors upon seeing new examples is done additively, as in the usual gradient descent update. However, by using a nonlinear parameterization function between the parameter vectors and the weight vectors, we can make the resulting update of the weight vector quite different from a true gradient descent update. To analyse such updates, we define a notion of a matching loss function and apply it both to the transfer function and to the parameterizat...
Jyrki Kivinen, Manfred K. Warmuth
Added 01 Nov 2010
Updated 01 Nov 2010
Type Conference
Year 1997
Where NIPS
Authors Jyrki Kivinen, Manfred K. Warmuth
Comments (0)