TurKit
Top Cited Papers
- 3 October 2010
- proceedings article
- Published by Association for Computing Machinery (ACM)
Abstract
Mechanical Turk (MTurk) provides an on-demand source of human computation. This provides a tremendous opportunity to explore algorithms which incorporate human computation as a function call. However, various systems challenges make this difficult in practice, and most uses of MTurk post large numbers of independent tasks. TurKit is a toolkit for prototyping and exploring algorithmic human computation, while maintaining a straight-forward imperative programming style. We present the crash-and-rerun programming model that makes TurKit possible, along with a variety of applications for human computation algorithms. We also present case studies of TurKit used for real experiments across different fields.Keywords
This publication has 13 references indexed in Scilit:
- Exploring iterative and parallel human computation processesPublished by Association for Computing Machinery (ACM) ,2010
- Financial incentives and the "performance of crowds"Published by Association for Computing Machinery (ACM) ,2009
- Finding causes of program output with the Java WhylinePublished by Association for Computing Machinery (ACM) ,2009
- Harnessing the wisdom of crowds in wikipediaPublished by Association for Computing Machinery (ACM) ,2008
- Crowdsourcing user studies with Mechanical TurkPublished by Association for Computing Machinery (ACM) ,2008
- LabelMe: A Database and Web-Based Tool for Image AnnotationInternational Journal of Computer Vision, 2007
- Games with a PurposeComputer, 2006
- Becoming WikipedianPublished by Association for Computing Machinery (ACM) ,2005
- Labeling images with a computer gamePublished by Association for Computing Machinery (ACM) ,2004
- The Staircase-Method in PsychophysicsThe American Journal of Psychology, 1962