Sciweavers

Share
JMLR
2006

Stability Properties of Empirical Risk Minimization over Donsker Classes

11 years 1 months ago
Stability Properties of Empirical Risk Minimization over Donsker Classes
We study some stability properties of algorithms which minimize (or almost-minimize) empirical error over Donsker classes of functions. We show that, as the number n of samples grows, the L2diameter of the set of almost-minimizers of empirical error with tolerance (n) = o(n- 1 2 ) converges to zero in probability. Hence, even in the case of multiple minimizers of expected error, as n increases it becomes less and less likely that adding a sample (or a number of samples) to the training set will result in a large jump to a new hypothesis. Moreover, under some assumptions on the entropy of the class, along with an assumption of Komlos-Major-Tusnady type, we derive a power rate of decay for the diameter of almost-minimizers. This rate, through an application of a uniform ratio limit inequality, is shown to govern the closeness of the expected errors of the almost-minimizers. In fact, under the above assumptions, the expected errors of almost-minimizers become closer with a rate strictly ...
Andrea Caponnetto, Alexander Rakhlin
Added 13 Dec 2010
Updated 13 Dec 2010
Type Journal
Year 2006
Where JMLR
Authors Andrea Caponnetto, Alexander Rakhlin
Comments (0)
books