SBotMiner: Large Scale Search Bot Detection

14 years 4 months ago
SBotMiner: Large Scale Search Bot Detection
In this paper, we study search bot traffic from search engine query logs at a large scale. Although bots that generate search traffic aggressively can be easily detected, a large number of distributed, low rate search bots are difficult to identify and are often associated with malicious attacks. We present SBotMiner, a system for automatically identifying stealthy, low-rate search bot traffic from query logs. Instead of detecting individual bots, our approach captures groups of distributed, coordinated search bots. Using sampled data from two different months, SBotMiner identifies over 123 million bot-related pageviews, accounting for 3.8% of total traffic. Our in-depth analysis shows that a large fraction of the identified bot traffic may be associated with various malicious activities such as phishing attacks or vulnerability exploits. This finding suggests that detecting search bot traffic holds great promise to detect and stop attacks early on. Categories and Subject Descriptors ...
Fang Yu, Yinglian Xie, Qifa Ke
Added 01 Mar 2010
Updated 02 Mar 2010
Type Conference
Year 2010
Where WSDM
Authors Fang Yu, Yinglian Xie, Qifa Ke
Comments (0)