Sciweavers

Share
NIPS
2004

A Large Deviation Bound for the Area Under the ROC Curve

9 years 13 days ago
A Large Deviation Bound for the Area Under the ROC Curve
The area under the ROC curve (AUC) has been advocated as an evaluation criterion for the bipartite ranking problem. We study large deviation properties of the AUC; in particular, we derive a distribution-free large deviation bound for the AUC which serves to bound the expected accuracy of a ranking function in terms of its empirical AUC on an independent test sequence. A comparison of our result with a corresponding large deviation result for the classification error rate suggests that the test sample size required to obtain an -accurate estimate of the expected accuracy of a ranking function with -confidence is larger than that required to obtain an -accurate estimate of the expected error rate of a classification function with the same confidence. A simple application of the union bound allows the large deviation bound to be extended to learned ranking functions chosen from finite function classes.
Shivani Agarwal, Thore Graepel, Ralf Herbrich, Dan
Added 31 Oct 2010
Updated 31 Oct 2010
Type Conference
Year 2004
Where NIPS
Authors Shivani Agarwal, Thore Graepel, Ralf Herbrich, Dan Roth
Comments (0)
books