On Information Regularization

12 years 4 months ago
On Information Regularization
We formulate a principle for classification with the knowledge of the marginal distribution over the data points (unlabeled data). The principle is cast in terms of Tikhonov style regularization where the regularization penalty articulates the way in which the marginal density should constrain otherwise unrestricted conditional distributions. Specifically, the regularization penalty penalizes any information introduced between the examples and labels beyond what is provided by the available labeled examples. The work extends (Szummer and Jaakkola, 2003) to multiple dimensions, providing a regularizer independent of the covering of the space used in the derivation. In addition we lay the learning theoretical framework for classification with information regularization and provide a sample complexity bound. We illustrate the regularization principle in practice by restricting the class of conditional distributions to be logistic regression models and constructing the regularization p...
Adrian Corduneanu, Tommi Jaakkola
Added 01 Nov 2010
Updated 01 Nov 2010
Type Conference
Year 2003
Where UAI
Authors Adrian Corduneanu, Tommi Jaakkola
Comments (0)