Sciweavers

PODS
2010
ACM

An optimal algorithm for the distinct elements problem

13 years 8 months ago
An optimal algorithm for the distinct elements problem
We give the first optimal algorithm for estimating the number of distinct elements in a data stream, closing a long line of theoretical research on this problem begun by Flajolet and Martin in their seminal paper in FOCS 1983. This problem has applications to query optimization, Internet routing, network topology, and data mining. For a stream of indices in {1, . . . , n}, our algorithm computes a (1 ± ε)approximation using an optimal O(ε−2 +log(n)) bits of space with 2/3 success probability, where 0 < ε < 1 is given. This probability can be amplified by independent repetition. Furthermore, our algorithm processes each stream update in O(1) worst-case time, and can report an estimate at any point midstream in O(1) worst-case time, thus settling both the space and time complexities simultaneously. We also give an algorithm to estimate the Hamming norm of a stream, a generalization of the number of distinct elements, which is useful in data cleaning, packet tracing, and da...
Daniel M. Kane, Jelani Nelson, David P. Woodruff
Added 10 Jul 2010
Updated 10 Jul 2010
Type Conference
Year 2010
Where PODS
Authors Daniel M. Kane, Jelani Nelson, David P. Woodruff
Comments (0)