Milestone 1 testset

From crowdresearch
Jump to: navigation, search

Experience the life of a Worker on Mechanical Turk

As a worker on mTurk some of the things I liked were:

  • Tasks require low intelligence and sometimes trivial.
  • Tasks require very little time, can be done in breaks.

Things I disliked about mTurk:

  • Monotonous tasks, most often annoying because it requires low intelligence.
  • The pay is very little.

Experience the life of a Requester on Mechanical Turk

  • Some individuals are used to the questions in social sciences thus the population answering certain questions is quite familiar with answers. Skewed results.
  • I liked that the cost of conducting such experiments is really cheap. Any other form of conducting a survey would be more expensive.
  • Large access to a literate population of various demography. Good for results from various demographic studies.
  • Based on the past performance of the worker, it makes it easier to evaluate what set of workers are good for a study and just selecting such workers for a study is also a possibility.

Alternate crowd-labor markets - TaskRabbit

This form of crowdsourcing only gets task specific and proficient people for each task. For instance for carpentry or task that require a very specific skill is easier to get workers for since these people have already been verified by TaskRabbit. Since most of these tasks are in-person tasks: criminal record check, insurance, trained workers is key. Compared to AMT, TaskRabbit takes into account all these and can increase the efficiency of these tasks since specialized and skilled individuals are hired only. City specific work provided employment to a large section of the society and prevents monopoly by individual workers since it is very easy to enter the competition. Unlike AMT, task performance rating matters a significant amount in the determination of wage for the worker.




  • The format of chopping documents and validating with at least 2 individuals makes evaluation of the crowdsourcing task easy.
  • Another thing that makes the system very useful is that the task could be done at any time making it very versatile.

Possible Improvements

  • Increase in monetary incentive, even if it is minimal for workers with higher quality ratings i.e people who did better in the past.



  • mClerk uses SMS, and this makes the system very easily scalable even at places without internet. It also only rewards people who have answered correctly. Strong motivation to do the work correctly. Gamifying the approach through bonuses and jackpots was the key. These can clearly increase the incentive to do really well in these tasks.

Possible Improvements:

  • Through internet, larger images and not through MMS. Automated rewards at the end of the day or rewards based on number of people also invited by the person to join will increase crowdsourcing reach.
  • Partner with Telecom Providers to automatically recharge phones with the amount or to reach a larger mass.

Flash Teams

What do you like about the system / what are its strengths?

  • The thing that I like about the system most is that this form of crowdsourcing can accomplish complex tasks and tries its best to micromanage tasks and divide tasks so as to achieve work objectives in a timely manner.

What do you think can be improved about the system?

  • The disadvantage of systems like these is that they can only accomplish tasks that can be divided into sub tasks easily and highly interdependent tasks that require frequent communication and high team coordination are difficult to achieve through system. Thus systems like Foundry can be improved to address this goal.
  • Another improvement can be having systems to reassign tasks to people if task objectives are completed faster and rewards for the same.