Evaluation Methods for Focused Crawling

12 years 6 months ago
Evaluation Methods for Focused Crawling
The exponential growth of documents available in the World Wide Web makes it increasingly difficult to discover relevant information on a specific topic. In this context, growing interest is emerging in focused crawling, a technique that dynamically browses the Internet by choosing directions that maximize the probability of discovering relevant pages, given a specific topic. Predicting the relevance of a document before seeing its contents (i.e., relying on the parent pages only) is one of the central problem in focused crawling because it can save significant bandwidth resources. In this paper, we study three different evaluation functions for predicting the relevance of a hyperlink with respect to the target topic. We show that classification based on the anchor text is more accurate than classification based on the whole page. Moreover, we introduce a method that combines both the anchor and the whole parent document, using a Bayesian representation of the Web graph structure...
Andrea Passerini, Paolo Frasconi, Giovanni Soda
Added 28 Jul 2010
Updated 28 Jul 2010
Type Conference
Year 2001
Where AIIA
Authors Andrea Passerini, Paolo Frasconi, Giovanni Soda
Comments (0)