4 search results - page 1 / 1 » Efficient Crawling Through URL Ordering |
CN
1998
13 years 4 months ago
1998
In this paper we study in what order a crawler should visit the URLs it has seen, in order to obtain more "important" pages first. Obtaining important pages rapidly can ...
KDD
2008
ACM
14 years 5 months ago
2008
ACM
A large fraction of the URLs on the web contain duplicate (or near-duplicate) content. De-duping URLs is an extremely important problem for search engines, since all the principal...
IC
2009
13 years 2 months ago
2009
- Crawling web pages written in Arabic or any other language with limited content in the web may, at first, seem to parallel the process of crawling the English content. However, t...
WWW
2005
ACM
14 years 5 months ago
2005
ACM
In this paper we present the Infocious Web search engine [23]. Our goal in creating Infocious is to improve the way people find information on the Web by resolving ambiguities pre...