Sciweavers

Share
PAMI
2012

Sparse Algorithms Are Not Stable: A No-Free-Lunch Theorem

6 years 7 months ago
Sparse Algorithms Are Not Stable: A No-Free-Lunch Theorem
Abstract—We consider two desired properties of learning algorithms: sparsity and algorithmic stability. Both properties are believed to lead to good generalization ability. We show that these two properties contradict each other. That is, a sparse algorithm can not be stable and vice versa. Thus, one has to trade off sparsity and stability in designing a learning algorithm. In particular, our general result implies that ℓ1-regularized regression (Lasso) cannot be stable, while ℓ2-regularized regression is known to have strong stability properties and is therefore not sparse.
Huan Xu, Constantine Caramanis, Shie Mannor
Added 28 Sep 2012
Updated 28 Sep 2012
Type Journal
Year 2012
Where PAMI
Authors Huan Xu, Constantine Caramanis, Shie Mannor
Comments (0)
books