A reputation system for selling human computation

We describe a reputation-driven market that motivates human computation sellers (workers) to produce optimal levels of quality when quality is not immediately measurable and contracts specifying the level of quality cannot be enforced. We consider the implications of making quality instantaneously verifiable up to a known threshold (“partial verification”), as might be the case when the market maker possesses a sufficiently advanced artificial intelligence apparatus. We find that such a verification system allows the market to attain a Pareto optimal equilibrium with less patient workers, and, in addition, makes the market more resistant to pseudonym attacks. Categories and Subject Descriptors K.4.4 [Computers and Society]: Electronic Commerce— payment schemes, security General Terms Economics Keywords human computation, reputation systems, trust, adverse selection, moral hazard, feedback, contract theory

• Type: Conference
• Year: 2009
• Where: KDD
• Authors: Trevor Burnham, Rahul Sami
• Keywords: Human Computation, Human Computation Sellers, KDD 2009, Pareto Optimal Equilibrium