178 search results - page 2 / 36 » Scheduling Algorithms for Web Crawling |
STOC
2002
ACM
14 years 5 months ago
2002
ACM
WWW
2006
ACM
14 years 6 months ago
2006
ACM
The web crawler space is often delimited into two general areas: full-web crawling and focused crawling. We present netSifter, a crawler system which integrates features from thes...
IADIS
2004
13 years 6 months ago
2004
There is a great amount of information on the web that can not be accessed by conventional crawler engines. This portion of the web is usually called hidden web data. To be able t...
WWW
2004
ACM
14 years 6 months ago
2004
ACM
The massive distribution of the crawling task can lead to inefficient exploration of the same portion of the Web. We propose a technique to guide crawlers exploration based on the...
WWW
2007
ACM
14 years 6 months ago
2007
ACM
Near-duplicate web documents are abundant. Two such documents differ from each other in a very small portion that displays advertisements, for example. Such differences are irrele...