Sciweavers

Share
ICDM
2010
IEEE

Active Learning from Multiple Noisy Labelers with Varied Costs

8 years 18 days ago
Active Learning from Multiple Noisy Labelers with Varied Costs
In active learning, where a learning algorithm has to purchase the labels of its training examples, it is often assumed that there is only one labeler available to label examples, and that this labeler is noise-free. In reality, it is possible that there are multiple labelers available (such as human labelers in the online annotation tool Amazon Mechanical Turk) and that each such labeler has a different cost and accuracy. We address the active learning problem with multiple labelers where each labeler has a different (known) cost and a different (unknown) accuracy. Our approach uses the idea of adjusted cost, which allows labelers with different costs and accuracies to be directly compared. This allows our algorithm to find low-cost combinations of labelers that result in high-accuracy labelings of instances. Our algorithm further reduces costs by pruning under-performing labelers from the set under consideration, and by halting the process of estimating the accuracy of the labelers a...
Yaling Zheng, Stephen D. Scott, Kun Deng
Added 03 Mar 2011
Updated 03 Mar 2011
Type Journal
Year 2010
Where ICDM
Authors Yaling Zheng, Stephen D. Scott, Kun Deng
Comments (0)
books