Sciweavers

Share
WWW
2003
ACM

Distributed Indexing of the Web Using Migrating Crawlers

9 years 11 months ago
Distributed Indexing of the Web Using Migrating Crawlers
Due to the tremendous increase rate and the high change frequency of Web documents, maintaining an up-to-date index for searching purposes (search engines) is becoming a challenge. The traditional crawling methods are no longer able to catch up with the constantly updating and growing Web. Realizing the problem, in this paper we suggest an alternative distributed crawling method with the use of mobile agents. Our goal is a scalable crawling scheme that minimizes network utilization, keeps up with document changes, employs time realization, and is easily upgradeable. Keywords Migrating Crawler, Search Engine, Mobile Agents, Web Crawling
Odysseas Papapetrou, Stavros Papastavrou, George S
Added 22 Nov 2009
Updated 22 Nov 2009
Type Conference
Year 2003
Where WWW
Authors Odysseas Papapetrou, Stavros Papastavrou, George Samaras
Comments (0)
books