Sciweavers

Share
PODS
2009
ACM

An efficient rigorous approach for identifying statistically significant frequent itemsets

11 years 11 days ago
An efficient rigorous approach for identifying statistically significant frequent itemsets
As advances in technology allow for the collection, storage, and analysis of vast amounts of data, the task of screening and assessing the significance of discovered patterns is becoming a major challenge in data mining applications. In this work, we address significance in the context of frequent itemset mining. Specifically, we develop a novel methodology to identify a meaningful support threshold s for a dataset, for which the number of itemsets with support at least s yields a substantial deviation from what would be expected in a random dataset with the same number of transactions and the same individual item frequencies. These itemsets can then be flagged as statistically significant with a small false discovery rate. Our methodology hinges on a Poisson approximation; we show that the distribution of the number of itemsets with Harvard School of Engineering and Applied Sciences, Cambridge, MA, USA. Email: kirsch@eecs.harvard.edu. Supported in part by NSF Grant CNS-0721491 and a...
Adam Kirsch, Michael Mitzenmacher, Andrea Pietraca
Added 25 Nov 2009
Updated 25 Nov 2009
Type Conference
Year 2009
Where PODS
Authors Adam Kirsch, Michael Mitzenmacher, Andrea Pietracaprina, Geppino Pucci, Eli Upfal, Fabio Vandin
Comments (0)
books