IJCAI
2007
13 years 5 months ago
2007
Estimating the rate of Web page updates helps in improving the Web crawler’s scheduling policy. But, most of the Web sources are autonomous and updated independently. Clients li...
WEBDB
2001
Springer
13 years 8 months ago
2001
Springer
Web crawlers generate significant loads on Web servers, and are difficult to operate. Instead of running crawlers at many “client” sites, we propose a central crawler and We...
CEEMAS
2005
Springer
13 years 10 months ago
2005
Springer
Abstract. In this paper we compare our selection based learning algorithm with the reinforcement learning algorithm in Web crawlers. The task of the crawlers is to find new inform...
ICDE
2003
IEEE
14 years 5 months ago
2003
IEEE
Web crawlers generate significant loads on Web servers, and are difficult to operate. Instead of repeatedly running crawlers at many "client" sites, we propose a central...