WinterMilestone 1 DineshDhakal

From crowdresearch
Revision as of 05:35, 17 January 2016 by Dineshdhakal (Talk | contribs)

Jump to: navigation, search

Submission for Winter Milestone 1 by Dinesh Dhakal

Experience the life of a Worker on Mechanical Turk

My request to sign-up on Amazon Mechanical was rejected. The list of tasks was visible but I was unable to proceed to see the HITs. Meanwhile, I tried the Amazon Mechanical Turk sandbox and it was a different experience than what I had expected.

What did you like?

  • The sheer number of HITs available. The popularity of the platform ensures the huge number of HITs available. At any point there were over 2 Million HITs available.
  • Clear List Items with Reward and Time Allotted mentioned for each HIT. This should help the worker judge the estimated effort required
  • A directory of qualifications setup by the requesters.
  • The sorting options are quite relevant and helpful
  • The check-box to quickly view only the jobs you are eligible for
  • A good dashboard that reflects all your earnings and also the ability to view detailed statement of HITs solved on a per day basis

Dashboard dineshd.png

Status detail.png

  • The ability to quickly contact the requester and can be used to convey with issues, suggestions or requests.

Contact requester.png

What did you dislike?

  • Country restrictions - This restriction is simply discouraging for me as a worker. The internet is a very powerful tool that spans across boundaries, use it well.
  • The rewards for a lot of assignments are extremely low.
  • Some of the requests with utterly low payments make it comparable to a cheap labor camp exploiting workers.
  • Opaque decision making in terms of why worker account application is accepted or rejected.
  • Outdated legacy-looking UI elements.
  • No engine to match requesters to probable workers
  • No transparency to the worker as to why their submissions were rejected.
  • Some approvals take too long.

Experience the life of a Requester on Mechanical Turk

Reflect on your experience as a requester on Mechanical Turk.

What did you like?

  • Templates to help create most widely submitted tasks quickly and with minimal effort.
  • Ability to set the level of qualification required to maintain the quality of results.
  • Freedom to create batches of HITs with fairly well manageable parameters.
  • Ability to provide bonus to workers for each assignment.
  • Well defined APIs to interface with AWS based systems.

What did you dislike?

  • Outdated legacy-looking UI elements.
  • No way to target workers based to their skill sets
  • No universally defined skill sets for workers. This would help in targeting workers that have a good track record in a certain domain of work, e.g. image labelling.
  • Interoperability with external systems - should provide well defined APIs to interact with any external system that the requester might have. (non AWS based).
  • Though well defined, limited number of templates for creation of new HITs
  • No Quick-start guide or tutorial to get started speedily.
  • Small but annoying bugs that were rather easy to hit. Eg:- Going to workers tab under Manage and selecting workers directly somehow did not bind the worker to an Assignment ID, while clicking on worker ID through Results page did.

Dineshd Bonus no assign id.png

Explore alternative crowd-labor markets

Microworkers is another platform that aims to connect the workers and requesters in the micro-tasks domain.

A few differences

  • First and foremost, I am surprised how easy the sign-up process was. It was a breeze. A simple sign-up form and an email confirmation. That's it! Workers are not required to wait for approval unlike Amazon Turk.

Dineshd microworkers signup.png

  • Better categorized than Amazon turk
  • Very few tasks available at any point as compared to Amazon Turk
  • Payments options like Paypal and Skrill available for the workers
  • Better set of RESTful APIs that can be used with external systems as desired by the requester

This is how the home page looks like after signing in.

Dineshd microworkers home.png


Comparing this with MTurk, both the platforms are similar in a lot of aspects.

  • The sorting options are same as that on Amazon Mechanical Turk.
  • Rewards are more or less in the same ranges. Still underpaid according to me.
  • The power resides with the requester and rejection is still not explained to the worker.
  • Very limited targeting of Worker groups based on a rating system for each category.



  • What do you like about the system / what are its strengths?
  • What do you think can be improved about the system?


  • What do you like about the system / what are its strengths?
  • What do you think can be improved about the system?

Flash Teams

  • What do you like about the system / what are its strengths?
  • What do you think can be improved about the system?

Milestone Contributors

Slack usernames of all who helped create this wiki page submission: @dineshd