Proximal Methods for Sparse Hierarchical Dictionary Learning

We propose to combine two approaches for modeling data admitting sparse representations: on the one hand, dictionary learning has proven effective for various signal processing tasks. On the other hand, recent work on structured sparsity provides a natural framework for modeling dependencies between dictionary elements. We thus consider a tree-structured sparse regularization to learn dictionaries embedded in a hierarchy. The involved proximal operator is computable exactly via a primal-dual method, allowing the use of accelerated gradient techniques. Experiments show that for natural image patches, learned dictionary elements organize themselves in such a hierarchical structure, leading to an improved performance for restoration tasks. When applied to text documents, our method learns hierarchies of topics, thus providing a competitive alternative to probabilistic topic models.

• Type: Conference
• Year: 2010
• Where: ICML
• Authors: Rodolphe Jenatton, Julien Mairal, Guillaume Obozinski, Francis Bach
• Keywords: Dictionary Elements, ICML 2010, Machine Learning, Signal Processing Tasks, Tree-structured Sparse Regularization