Selection in Scale-Free Small World

Abstract. In this paper we compare our selection based learning algorithm with the reinforcement learning algorithm in Web crawlers. The task of the crawlers is to find new information on the Web. We performed simulations based on data collected from the Web. The collected portion of the Web is typical and exhibits scale-free small world (SFSW) structure. We have found that on this SFSW, the weblog update algorithm performs better than the reinforcement learning algorithm. It finds the new information faster than the reinforcement learning algorithm and has better new information/all submitted documents ratio.

• Type: Conference
• Year: 2005
• Where: CEEMAS
• Authors: Zsolt Palotai, Csilla Farkas, András Lörincz
• Keywords: CEEMAS 2005, Reinforcement Learning Algorithm, Update Algorithm Performs, Web Crawlers