Guilds and Quality

From crowdresearch
Jump to: navigation, search

Abstract

A unique byproduct of the asymmetrical relationship between workers and requesters in MTurk is a inability of incremential price increases to effect task quality. Observing this phenonemon, how will the relationship between price and quality play out in a crowdsourcing platform that allows for market forces to playout?

References

Quality
  • 2. Quality Management in Crowdsourcing using Gold Judges Behavior

https://slack-files.com/files-pri-safe/T03R8AQ66-F0N6UQN07/p267-kaza.pdf?c=1455991941-48214c77849db01f97d888d56953c0e921804ff1

  • 3. Shepherding the Crowd Yields Better Work

http://www.eecs.berkeley.edu/~bjoern/papers/dow-shepherd-cscw2012.pdf

  • 4. Financial Incentives and the “Performance of Crowds”

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.173.8632&rep=rep1&type=pdf

  • 5. Quality Control for Real-time Ubiquitous Crowdsourcing

http://www0.cs.ucl.ac.uk/staff/l.capra/publications/ubicrowd11.pdf

  • 6. Programmatic Gold: Targeted and Scalable Quality Assurance in Crowdsourcing

http://publicassets.s3.amazonaws.com/papers/HCOMP2011_philosopher_stone.pdf

  • 7. Managing the Quality of Large-Scale Crowdsourcing

http://trec.nist.gov/pubs/trec20/papers/TUD_DMIR.crowdsourcing.4-20update.pdf

Guilds
  • 1. S.R. Epstein. Craft Guilds, Apprenticeship, and Technological Change in Preindustrial Europe. The Journal of Economics History, Vol. 58, N°3, September 1998.

File:Craft guilds apprenticeship and technological change.pdf

  • 3. Nicolas Ducheneaut, Nicholas Yee, Eric Nickell, and Robert J. Moore. 2006. "Alone together?": exploring the social dynamics of massively multiplayer online games. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '06), Rebecca Grinter, Thomas Rodden, Paul Aoki, Ed Cutrell, Robin Jeffries, and Gary Olson (Eds.). ACM, New York, NY, USA, 407-416

http://pubs.aeaweb.org/doi/pdfplus/10.1257/jep.28.4.169

Crowdsourcing
  • 1. Beyond Mechanical Turk: An Analysis of Paid Crowd Work Platforms.

https://www.ideals.illinois.edu/bitstream/handle/2142/73639/138_ready.pdf?sequence=2

  • 2. Collaboratively Crowdsourcing Workflows with Turkomatic.

https://bid.berkeley.edu/files/papers/kulkarni-turkomatic-cscw2012.pdf

Simulations and Crowd Research
  • 1. Crowdsourcing User Studies With Mechanical Turk

http://www-users.cs.umn.edu/~echi/papers/2008-CHI2008/2008-02-mech-turk-online-experiments-chi1049-kittur.pdf

  • 2. Behavioral Mechanism Design: Optimal Crowdsourcing Contracts and Prospect Theory. David Easley, Arpita Ghosh. Proc. 16th ACM Conference on Economics and Computation (EC), 2015.

http://www.arpitaghosh.com/papers/EC15-full.pdf

  • 4. Game Theory and Incentives in Human Computation Systems, Arpita Ghosh. Book chapter. Handbook of Human Computation, Springer 2013.

http://www.arpitaghosh.com/papers/games_hcomp.pdf

  • 5. Galen Pickard, Wei Pan, Iyad Rahwan, Manuel Cebrian, Riley Crane, Anmol Madan, and Alex Pentland. Time-critical social mobilization. Science, 334:509–512, 2011.

DOI: http://dx.doi.org/10.1126/science.1205869

  • 6. Eleni Koutrouli and Aphrodite Tsalgatidou. Reputation Systems Evaluation Survey. ACM Comput. Surv. 48, 3, Article 35, December 2015.

DOI: http://dx.doi.org/10.1145/2835373 - File:Reputation evaluation survey A35-koutrouli.pdf

Simulation

http://crowdresearch.stanford.edu/w/index.php?title=SocioDynamicsModel