Sciweavers

299 search results - page 3 / 60
» User-centric Web crawling
Sort
View
FTIR
2010
69views more  FTIR 2010»
13 years 3 months ago
Web Crawling
Christopher Olston, Marc Najork
CN
1998
54views more  CN 1998»
13 years 4 months ago
Efficient Crawling Through URL Ordering
In this paper we study in what order a crawler should visit the URLs it has seen, in order to obtain more "important" pages first. Obtaining important pages rapidly can ...
Junghoo Cho, Hector Garcia-Molina, Lawrence Page
CORR
2012
Springer
292views Education» more  CORR 2012»
12 years 17 days ago
Optimal Threshold Control by the Robots of Web Search Engines with Obsolescence of Documents
A typical web search engine consists of three principal parts: crawling engine, indexing engine, and searching engine. The present work aims to optimize the performance of the cra...
Konstantin Avrachenkov, Alexander N. Dudin, Valent...
ADBIS
2003
Springer
173views Database» more  ADBIS 2003»
13 years 10 months ago
UCYMICRA: Distributed Indexing of the Web Using Migrating Crawlers
Due to the tremendous increase rate and the high change frequency of Web documents, maintaining an up-to-date index for searching purposes (search engines) is becoming a challenge....
Odysseas Papapetrou, Stavros Papastavrou, George S...
SIGIR
2002
ACM
13 years 4 months ago
Do TREC web collections look like the web?
We measure the WT10g test collection, used in the TREC-9 and TREC 2001 Web Tracks, and the .GOV test collection used in the TREC 2002 Web and Interactive Tracks, with common measu...
Ian Soboroff