Sciweavers

Share
JMLR
2012

Online Incremental Feature Learning with Denoising Autoencoders

7 years 1 months ago
Online Incremental Feature Learning with Denoising Autoencoders
While determining model complexity is an important problem in machine learning, many feature learning algorithms rely on cross-validation to choose an optimal number of features, which is usually challenging for online learning from a massive stream of data. In this paper, we propose an incremental feature learning algorithm to determine the optimal model complexity for large-scale, online datasets based on the denoising autoencoder. This algorithm is composed of two processes: adding features and merging features. Specifically, it adds new features to minimize the objective function’s residual and merges similar features to obtain a compact feature representation and prevent over-fitting. Our experiments show that the proposed model quickly converges to the optimal number of features in a large-scale online setting. In classification tasks, our model outperforms the (non-incremental) denoising autoencoder, and deep networks constructed from our algorithm perform favorably compar...
Guanyu Zhou, Kihyuk Sohn, Honglak Lee
Added 27 Sep 2012
Updated 27 Sep 2012
Type Journal
Year 2012
Where JMLR
Authors Guanyu Zhou, Kihyuk Sohn, Honglak Lee
Comments (0)
books