Sciweavers

WEBDB
2004
Springer

Spam, Damn Spam, and Statistics: Using Statistical Analysis to Locate Spam Web Pages

14 years 5 months ago
Spam, Damn Spam, and Statistics: Using Statistical Analysis to Locate Spam Web Pages
The increasing importance of search engines to commercial web sites has given rise to a phenomenon we call “web spam”, that is, web pages that exist only to mislead search engines into (mis)leading users to certain web sites. Web spam is a nuisance to users as well as search engines: users have a harder time finding the information they need, and search engines have to cope with an inflated corpus, which in turn causes their cost per query to increase. Therefore, search engines have a strong incentive to weed out spam web pages from their index. We propose that some spam web pages can be identified through statistical analysis: Certain classes of spam pages, in particular those that are machine-generated, diverge in some of their properties from the properties of web pages at large. We have examined a variety of such properties, including linkage structure, page content, and page evolution, and have found that outliers in the statistical distribution of these properties are hig...
Dennis Fetterly, Mark Manasse, Marc Najork
Added 02 Jul 2010
Updated 02 Jul 2010
Type Conference
Year 2004
Where WEBDB
Authors Dennis Fetterly, Mark Manasse, Marc Najork
Comments (0)