Difference between revisions of "WinterMilestone 1 DineshDhakal"

From crowdresearch
Jump to: navigation, search
Line 11: Line 11:
 
* The check-box to quickly view only the jobs you are eligible for  
 
* The check-box to quickly view only the jobs you are eligible for  
 
* A good dashboard that reflects all your earnings and also the ability to view detailed statement of HITs solved on a per day basis
 
* A good dashboard that reflects all your earnings and also the ability to view detailed statement of HITs solved on a per day basis
[[File:dashboard_dineshd.png | 400px]]
+
[[File:dashboard_dineshd.png | 800px]]
  
[[File:status_detail.png | 400px]]
+
[[File:status_detail.png | 800px]]
 
* The ability to quickly contact the requester and can be used to convey with issues, suggestions or requests.
 
* The ability to quickly contact the requester and can be used to convey with issues, suggestions or requests.
[[File:contact_requester.png | 400px]]
+
[[File:contact_requester.png | 800px]]
  
 
===What did you dislike?===
 
===What did you dislike?===
Line 47: Line 47:
 
* No Quick-start guide or tutorial to get started speedily.
 
* No Quick-start guide or tutorial to get started speedily.
 
* Small but annoying bugs that were rather easy to hit. Eg:- Going to workers tab under Manage and selecting workers directly somehow did not bing the worker to an Assignment ID, while clicking on worker ID through Results page did.
 
* Small but annoying bugs that were rather easy to hit. Eg:- Going to workers tab under Manage and selecting workers directly somehow did not bing the worker to an Assignment ID, while clicking on worker ID through Results page did.
[[File:Dineshd_Bonus_no_assign_id.png | 400px]]
+
[[File:Dineshd_Bonus_no_assign_id.png | 800px]]
  
 
== Explore alternative crowd-labor markets ==
 
== Explore alternative crowd-labor markets ==
Line 56: Line 56:
 
* First and foremost, I am surprised how easy the sign-up process was. It was a breeze. A simple sign-up form and an email confirmation. That's it. Workers are not required to wait for approval unlike Amazon Turk.
 
* First and foremost, I am surprised how easy the sign-up process was. It was a breeze. A simple sign-up form and an email confirmation. That's it. Workers are not required to wait for approval unlike Amazon Turk.
  
[[File:dineshd_microworkers_signup.png | 400px]]
+
[[File:dineshd_microworkers_signup.png | 800px]]
  
  
Line 70: Line 70:
 
This is how the home page looks like after signing in.
 
This is how the home page looks like after signing in.
  
[[File:dineshd_microworkers_home.png | 400px]]  
+
[[File:dineshd_microworkers_home.png | 800px]]  
  
  

Revision as of 05:29, 17 January 2016

Submission for Winter Milestone 1 by Dinesh Dhakal

Experience the life of a Worker on Mechanical Turk

My request to sign-up on Amazon Mechanical was rejected. The list of tasks was visible but I was unable to proceed to see the HITs. Meanwhile, I tried the Amazon Mechanical Turk sandbox and it was a different experience than what I had expected.

What did you like?

  • The sheer number of HITs available. The popularity of the platform ensures the huge number of HITs available. At any point there were over 2 Million HITs available.
  • Clear List Items with Reward and Time Allotted mentioned for each HIT. This should help the worker judge the estimated effort required
  • A directory of qualifications setup by the requesters.
  • The sorting options are quite relevant and helpful
  • The check-box to quickly view only the jobs you are eligible for
  • A good dashboard that reflects all your earnings and also the ability to view detailed statement of HITs solved on a per day basis

Dashboard dineshd.png

Status detail.png

  • The ability to quickly contact the requester and can be used to convey with issues, suggestions or requests.

Contact requester.png

What did you dislike?

  • Country restrictions - This restriction is simply discouraging for me as a worker. The internet is a very powerful tool that spans across boundaries, use it well.
  • The rewards for a lot of assignments are extremely low.
  • Some of the requests with utterly low payments make it comparable to a cheap labor camp exploiting workers.
  • Opaque decision making in terms of why worker account application is accepted or rejected.
  • Outdated legacy-looking UI elements.
  • No engine to match requesters to probable workers
  • No transparency to the worker as to why their submissions were rejected.
  • Some approvals take too long.

Experience the life of a Requester on Mechanical Turk

Reflect on your experience as a requester on Mechanical Turk.

What did you like?

  • Templates to help create most widely submitted tasks quickly and with minimal effort.
  • Ability to set the level of qualification required to maintain the quality of results.
  • Freedom to create batches of HITs with fairly well manageable parameters.
  • Ability to provide bonus to workers for each assignment.
  • Well defined APIs to interface with AWS based systems.

What did you dislike?

  • Outdated legacy-looking UI elements.
  • No way to target workers based to their skill sets
  • No universally defined skill sets for workers. This would help in targeting workers that have a good track record in a certain domain of work, eg. image labelling.
  • Interoperability with external systems - should provide well defined APIs to interract with any external system that the requester might have. (non AWS based).
  • Though well defined, limited number of templates for creation of new HITs
  • No Quick-start guide or tutorial to get started speedily.
  • Small but annoying bugs that were rather easy to hit. Eg:- Going to workers tab under Manage and selecting workers directly somehow did not bing the worker to an Assignment ID, while clicking on worker ID through Results page did.

Dineshd Bonus no assign id.png

Explore alternative crowd-labor markets

Microworkers is another platform that aims to connect the workers and requesters in the micro-tasks domain.

A few differences

  • First and foremost, I am surprised how easy the sign-up process was. It was a breeze. A simple sign-up form and an email confirmation. That's it. Workers are not required to wait for approval unlike Amazon Turk.

Dineshd microworkers signup.png


  • Better categorized than Amazon turk
  • The number of tasks available is very low as compared to Amazon Turk
  • Payments options like Paypal and Skrill available for the workers
  • Better set of APIs that can be used with various systems as desired by the requester


This is how the home page looks like after signing in.

Dineshd microworkers home.png


Similarities

Comparing this with MTurk, both the platforms are similar in a lot of aspects.

  • The sorting options are same as that on Amazon Mechanical Turk.
  • Rewards are more or less in the same ranges. Still underpaid according to me.
  • The power resides with the requester and rejection is still not explained to the worker.
  • Very limited targeting of Worker groups based on a rating system for each category.

Readings

MobileWorks

  • What do you like about the system / what are its strengths?
  • What do you think can be improved about the system?

Daemo

  • What do you like about the system / what are its strengths?
  • What do you think can be improved about the system?

Flash Teams

  • What do you like about the system / what are its strengths?
  • What do you think can be improved about the system?

Milestone Contributors

Slack usernames of all who helped create this wiki page submission: @dineshd