WinterMilestone 1 DineshDhakal

From crowdresearch
Revision as of 05:07, 17 January 2016 by Dineshdhakal (Talk | contribs)

Jump to: navigation, search

Submission for Winter Milestone 1 by Dinesh Dhakal

Experience the life of a Worker on Mechanical Turk

My request to sign-up on Amazon Mechanical was rejected. The list of tasks was visible but I was unable to proceed to see the HITs. Meanwhile, I tried the Amazon Mechanical Turk sandbox and it was a different experience than what I had expected.

What did you like?

  • The sheer number of HITs available. The popularity of the platform ensures the huge number of HITs available. At any point there were over 2 Million HITs available.
  • Clear List Items with Reward and Time Allotted mentioned for each HIT. This should help the worker judge the estimated effort required
  • A directory of qualifications setup by the requesters.
  • The sorting options are quite relevant and helpful
  • The check-box to quickly view only the jobs you are eligible for
  • A good dashboard that reflects all your earnings and also the ability to view detailed statement of HITs solved on a per day basis

Dashboard dineshd.png

Status detail.png

  • The ability to quickly contact the requester and can be used to convey with issues, suggestions or requests.

Contact requester.png

What did you dislike?

  • Country restrictions - This restriction is simply discouraging for me as a worker. The internet is a very powerful tool that spans across boundaries, use it well.
  • The rewards for a lot of assignments are extremely low.
  • Some of the requests with utterly low payments make it comparable to a cheap labor camp exploiting workers.
  • Opaque decision making in terms of why worker account application is accepted or rejected.
  • Outdated legacy-looking UI elements.
  • No engine to match requesters to probable workers
  • No transparency to the worker as to why their submissions were rejected.
  • Some approvals take too long.

Experience the life of a Requester on Mechanical Turk

Reflect on your experience as a requester on Mechanical Turk.

What did you like?

  • Templates to help create most widely submitted tasks quickly and with minimal effort.
  • Ability to set the level of qualification required to maintain the quality of results.
  • Freedom to create batches of HITs with fairly well manageable parameters.
  • Ability to provide bonus to workers for each assignment.
  • Well defined APIs to interface with AWS based systems.

What did you dislike?

  • Outdated legacy-looking UI elements.
  • No way to target workers based to their skill sets
  • No universally defined skill sets for workers. This would help in targeting workers that have a good track record in a certain domain of work, eg. image labelling.
  • Interoperability with external systems - should provide well defined APIs to interract with any external system that the requester might have. (non AWS based).
  • Though well defined, limited number of templates for creation of new HITs
  • No Quick-start guide or tutorial to get started speedily.
  • Small but annoying bugs that were rather easy to hit. Eg:- Going to workers tab under Manage and selecting workers directly somehow did not bing the worker to an Assignment ID, while clicking on worker ID through Results page did.

File:Bonus no assign id

Explore alternative crowd-labor markets

Microworkers is another platform that aims to connect the workers and requesters in the micro-tasks domain.

  • First and foremost, I am surprised how easy the sign-up process was. It was a breeze. A simple sign-up form and an email confirmation. That's it. Workers are not required to wait for approval unlike Amazon Turk.
  • Better categorized than Amazon turk
  • The number of tasks available is very low as compared to Amazon Turk.
  • Payments options like Paypal and Skrill available for the workers

File:Microworkers signup

This is how the home page looks like after signing in.

File:Microworkers home

Comparing this with MTurk, both the platforms are similar in a lot of aspects.

  • The sorting options are same as that on Amazon Mechanical Turk.
  • Rewards are more or less in the same ranges. Still underpaid according to me.
  • The power resides with the requester and rejection is still not explained to the worker.



  • What do you like about the system / what are its strengths?
  • What do you think can be improved about the system?


  • What do you like about the system / what are its strengths?
  • What do you think can be improved about the system?

Flash Teams

  • What do you like about the system / what are its strengths?
  • What do you think can be improved about the system?

Milestone Contributors

Slack usernames of all who helped create this wiki page submission: @kessyena, @reneelin, @mahsa, @marim_g