Sciweavers

Share
KDD
2007
ACM

Practical guide to controlled experiments on the web: listen to your customers not to the hippo

10 years 9 days ago
Practical guide to controlled experiments on the web: listen to your customers not to the hippo
The web provides an unprecedented opportunity to evaluate ideas quickly using controlled experiments, also called randomized experiments (single-factor or factorial designs), A/B tests (and their generalizations), split tests, Control/Treatment tests, and parallel flights. Controlled experiments embody the best scientific design for establishing a causal relationship between changes and their influence on user-observable behavior. We provide a practical guide to conducting online experiments, where end-users can help guide the development of features. Our experience indicates that significant learning and return-oninvestment (ROI) are seen when development teams listen to their customers, not to the Highest Paid Person's Opinion (HiPPO). We provide several examples of controlled experiments with surprising results. We review the important ingredients of running controlled experiments, and discuss their limitations (both technical and organizational). We focus on several areas tha...
Ron Kohavi, Randal M. Henne, Dan Sommerfield
Added 30 Nov 2009
Updated 30 Nov 2009
Type Conference
Year 2007
Where KDD
Authors Ron Kohavi, Randal M. Henne, Dan Sommerfield
Comments (0)
books