Mining high-speed data streams

9 years 7 months ago
Mining high-speed data streams
Many organizations today have more than very large databases; they have databases that grow without limit at a rate of several million records per day. Mining these continuous data streams brings unique opportunities, but also new challenges. This paper describes and evaluates VFDT, an anytime system that builds decision trees using constant memory and constant time per example. VFDT can incorporate tens of thousands of examples per second using off-the-shelf hardware. It uses Hoeffding bounds to guarantee that its output is asymptotically nearly identical to that of a conventional learner. We study VFDT's properties and demonstrate its utility through an extensive set of experiments on synthetic data. We apply VFDT to mining the continuous stream of Web access data from the whole University of Washington main campus. Categories and Subject Descriptors H.2.8 [Database Management]: Database Applications-data mining; I.2.6 [Artificial Intelligence]: Learning-concept learning; I.5.2...
Pedro Domingos, Geoff Hulten
Added 25 Aug 2010
Updated 25 Aug 2010
Type Conference
Year 2000
Where KDD
Authors Pedro Domingos, Geoff Hulten
Comments (0)