Abstract. In this paper we address the task of image categorization using a new similarity measure on the space of Sparse Multiscale Patches (SMP). SMPs are based on a multiscale transform of the image and provide a global representation of its content. At each scale, the probability density function (pdf ) of the SMPs is used as a description of the relevant information. The closeness between two images is defined as a combination of Kullback-Leibler divergences between the pdfs of their SMPs. In the context of image categorization, we represent semantic categories by prototype images, which are defined as the centroids of the training clusters. Therefore any unlabeled image is classified by giving it the same label as the nearest prototype. Results obtained on ten categories from the Corel collection show the categorization accuracy of the SMP method.