Exploiting Distribution Skew for Scalable P2P Text Clustering

K-Means clustering is widely used in information retrieval and data mining. Distributed K-Means variants have already been proposed, but none of the past algorithms scales to large numbers of nodes. In this work we describe a new P2P algorithm which significantly reduces the communication costs involved by exploiting distribution skew, naturally found in text and other datasets. The algorithm achieves high clustering quality and requires no synchronization between peers. An extensive evaluation with up to 100.000 peers shows the algorithm's effectiveness and scalability as well as its ability to cope with churn.

• Type: Conference
• Year: 2008
• Where: DBISP2P
• Authors: Odysseas Papapetrou, Wolf Siberski, Fabian Leitritz, Wolfgang Nejdl
• Keywords: Database, DBISP2P 2008, K-means Clustering, K-Means Variants, Past Algorithms Scales