Visual recognition faces the difficult problem of recognizing objects despite the multitude of their appearances. Ample neuroscientific evidence shows that the cortex uses a topographic code to represent visual stimuli. We therefore develop a bilinear probabilistic model that learns transformations to build an invariant topographic code in an unsupervised way. Simulations for the simple over-complete linear case yield V1 like Gabor receptive fields that are arranged in orientation, frequency and position maps. For the computationally more powerful case of two control units, we show that an application to natural inputs placed at different positions and with a consistent relative transformation (e.g. rotation), leads to an invariant topographic output representation and hence a relative implementation of the transformation in the control units, i.e. the Gabor receptive fields are transformed accordingly.