The Kernel Least-Mean-Square Algorithm

The combination of the famed kernel trick and the least-mean-square (LMS) algorithm provides an interesting sample by sample update for an adaptive filter in reproducing Kernel Hilbert Spaces (RKHS), which is named here the KLMS. Unlike the accepted view in kernel methods, this paper shows that in the finite training data case, the KLMS algorithm is well-posed in RKHS without the addition of an extra regularization term to penalize solution norms as was suggested by Kivinen and Smale in [1], [2]. This result is the main contribution of the paper and enhances the present understanding of the LMS algorithm with a machine learning perspective. The effect of the KLMS stepsize is also studied from the viewpoint of regularization. Two experiments are presented to support our conclusion that with finite data the KLMS algorithm can be readily used in high dimensional spaces and particularly in RKHS to derive nonlinear, stable algorithms with comparable performance to batch, regularized solutio...

• Type: Journal
• Year: 2008
• Where: TSP
• Authors: Weifeng Liu, Puskal P. Pokharel, Jose C. Principe
• Keywords: Algorithms, Famed Kernel Trick, KLMS, TSP 2008