Sciweavers

Share
ECAI
2008
Springer

Reinforcement Learning with Classifier Selection for Focused Crawling

8 years 4 months ago
Reinforcement Learning with Classifier Selection for Focused Crawling
Focused crawlers are programs that wander in the Web, using its graph structure, and gather pages that belong to a specific topic. The most critical task in Focused Crawling is the scoring of the URLs as it designates the path that the crawler will follow, and thus its effectiveness. In this paper we propose a novel scheme for assigning scores to the URLs, based on the Reinforcement Learning (RL) framework. The proposed approach learns to select the best classifier for ordering the URLs. This formulation reduces the size of the search space for the RL method and makes the problem tractable. We evaluate the proposed approach on-line on a number of topics, which offers a realistic view of its performance, comparing it also with a RL method and a simple but effective classifier-based crawler. The results demonstrate the strength of the proposed approach.
Ioannis Partalas, Georgios Paliouras, Ioannis P. V
Added 19 Oct 2010
Updated 19 Oct 2010
Type Conference
Year 2008
Where ECAI
Authors Ioannis Partalas, Georgios Paliouras, Ioannis P. Vlahavas
Comments (0)
books