Sparse Coding with Invariance Constraints

We suggest a new approach to optimize the learning of sparse features under the constraints of explicit transformation symmetries imposed on the set of feature vectors. Given a set of basis feature vectors and invariance transformations, from each basis feature a family of transformed features is generated. We then optimize the basis features for optimal sparse reconstruction of the input pattern ensemble using the whole transformed feature family. If the prede£ned transformation invariance coincides with an invariance in the input data, we obtain a less redundant basis feature set, compared to sparse coding approaches without invariances. We demonstrate the application to a test scenario of overlapping bars and the learning of receptive £elds in hierarchical visual cortex models.

• Type: Conference
• Year: 2003
• Where: ICANN
• Authors: Heiko Wersing, Julian Eggert, Edgar Körner
• Keywords: Basis Feature, Basis Feature Vectors, Feature Vectors, ICANN 2003