Kernel Conjugate Gradient for Fast Kernel Machines

11 years 1 months ago
Kernel Conjugate Gradient for Fast Kernel Machines
We propose a novel variant of the conjugate gradient algorithm, Kernel Conjugate Gradient (KCG), designed to speed up learning for kernel machines with differentiable loss functions. This approach leads to a better conditioned optimization problem during learning. We establish an upper bound on the number of iterations for KCG that indicates it should require less than the square root of the number of iterations that standard conjugate gradient requires. In practice, for various differentiable kernel learning problems, we find KCG consistently, and significantly, outperforms existing techniques. The algorithm is simple to implement, requires no more computation per iteration than standard approaches, and is well motivated by Reproducing Kernel Hilbert Space (RKHS) theory. We further show that data-structure techniques recently used to speed up kernel machine approaches are well matched to the algorithm by reducing the dominant costs of training: function evaluation and RKHS inner pr...
Nathan D. Ratliff, J. Andrew Bagnell
Added 29 Oct 2010
Updated 29 Oct 2010
Type Conference
Year 2007
Authors Nathan D. Ratliff, J. Andrew Bagnell
Comments (0)