WinterMilestone 1 Dubs

From crowdresearch
Jump to: navigation, search

Click on the EDIT Button on the top-right of this page to edit.

Experience the life of a Worker on Mechanical Turk

Reflect on your experience as a worker on Mechanical Turk. What did you like? What did you dislike? If you're from outside the USA and unable to access MTurk, try try the worker sandbox, or other sites like CrowdFlower or Microworkers or Clickworker.

Experience the life of a Requester on Mechanical Turk


First of all, this is definitely a hard problem to solve. There are infinite possibilities of works that can be requested and it’s complicated to create a system that can not only serve all the possibilities, but also maintain a good traction with all of them. Usually, there is a trade-off with specificity and traction. How to make a good service for a diverse group of requesters. Here, we will try to consider some opinions we had about the requesting systems, with some comments that can lead to future implementations/suggestions.

Good qualities/Positive aspects

  • Straight-forward publication. Except from the very beginning (login, etc), there are not a lot of doubts about how to use the system. It’s a very direct process, but a limited one.
  • There is a certificate system. Although there are a lot of ways to improve it, it’s interesting to avoid scammers/bad workers/bad intentions in the community.
  • Pre-organized models. There are some models to help you get ready to create something, so you don’t necessarily need to start from the scratch.
  • HTML edition. There are possibilities to edit the HTML and there is some bootstrap/css on it already.
  • It generates a .csv file that can later be used for other programs/analysis.
  • Cheaper than hiring a group of people to do the same. It’s pretty cheap to the requester to get work done and it’s a pretty random group of people, which is good for statistics.

Bad qualities/Negative aspects

  • Limited edition options. It’s possible to edit the HTML, but the way the structure is constructed makes it really hard to truly make a task different from the sample ones provided. An API or a system that could connect different systems/apis/etc with them in an easy/intuitive way would be a huge step forward to guarantee that all kind of tasks could be done with the application.
  • Old design. We believe that especially when we’re dealing with money, the design takes an important place: it inspires reliability. The way MTurk is currently designed doesn’t really bring any kind of confidence for someone that wants to invest money.
  • It’s restricted for people outside the US. We are Brazilians and, although we could use it since one of our members is in the US right now, it’s a really bad problem. Currency can be a big deal for a lot of workers. For instance, the Brazilian currency is around 1 to 4 right now. It means that a Brazilian worker would really benefit for a payment made in dollars from the same system and would be great for requesters (not too good for american workers though).
  • The data visualization is poor. Even though the results page is interesting, it could have a way better system to visualize data. D3.js could be really useful to truly bring meaning to the data acquired. Certainly, it would depend on the acquired data, but it would be a good possibility for a lot of parameters or sample batches.
  • Payment systems could be easier to navigate. For instance, the MTurk fees are not explained, so it’s not possible to know the exact total amount of money needed before finishing the batch. It would be really useful to know it before. Also, sometimes you don’t have enough money to make a batch public on the MTurk account and it takes too long to put money again. Maybe, we could have an easier way to refill the account.
  • tags to match workers/requesters

Explore alternative crowd-labor markets

Compare and contrast the crowd-labor market you just explored (TaskRabbit/oDesk/GalaxyZoo) to Mechanical Turk.



  • What do you like about the system / what are its strengths?
  • What do you think can be improved about the system?


  • What do you like about the system / what are its strengths?
  • What do you think can be improved about the system?

Flash Teams

  • What do you like about the system / what are its strengths?
  • What do you think can be improved about the system?

Milestone Contributors

Our contributors:

  • Flavio Scorpione - @scorpione
  • Gabriel Bayomi - @gbayomi
  • Henrique Orefice - @horefice
  • Lucas Bamidele - @lucasbamidele
  • Teogenes Moura - @teomoura