Hardness of Learning Halfspaces with Noise

9 years 8 days ago
Hardness of Learning Halfspaces with Noise
Learning an unknown halfspace (also called a perceptron) from labeled examples is one of the classic problems in machine learning. In the noise-free case, when a halfspace consistent with all the training examples exists, the problem can be solved in polynomial time using linear programming. However, under the promise that a halfspace consistent with a fraction (1 − ε) of the examples exists (for some small constant ε > 0), it was not known how to efficiently find a halfspace that is correct on even 51% of the examples. Nor was a hardness result that ruled out getting agreement on more than 99.9% of the examples known. In this work, we close this gap in our understanding, and prove that even a tiny amount of worst-case noise makes the problem of learning halfspaces intractable in a strong sense. Specifically, for arbitrary ε, δ > 0, we prove that given a set of examples-label pairs from the hypercube a fraction (1 − ε) of which can be explained by a halfspace, it is N...
Venkatesan Guruswami, Prasad Raghavendra
Added 11 Jun 2010
Updated 11 Jun 2010
Type Conference
Year 2006
Where FOCS
Authors Venkatesan Guruswami, Prasad Raghavendra
Comments (0)