Difference between revisions of "WinterMilestone 5 stormsurfer"

From crowdresearch
Jump to: navigation, search
m (Brief introduction and problem being solved)
Line 12: Line 12:
 
*'''Finding good workers:''' e.g. workers that do a task proficiently.
 
*'''Finding good workers:''' e.g. workers that do a task proficiently.
  
''Task ranking preferences, however, largely vary from user to user, and at different points in time, the same user may have different preferences.'' How do we maximize utility for both parties at any given moment? Currently, other systems face a similar problem: Facebook allows the user to choose rank between "top stories" and "most recent," and our very own voting platforms allows users to rank between various sorting methods, including "top," "new," "best," and "needs comments/views."
+
''Task ranking preferences, however, largely vary from user to user, and at different points in time, the same user may have different preferences.'' How do we maximize utility for both parties at any given moment? Currently, other systems face a similar problem: Facebook allows the user to choose rank between "top stories" and "most recent," and our very own voting platform allows users to rank between various sorting methods, including "top," "new," "best," and "needs comments/views."
  
 
''Why not allow the user to choose in Daemo?''
 
''Why not allow the user to choose in Daemo?''

Revision as of 17:30, 14 February 2016

System

Please use the following template to write up your introduction section this week.

Brief introduction and problem being solved

For workers, task ranking boils down to two things:

  • Finding good requesters: e.g. requesters that pay well, requesters that accept/reject HITs fairly/on-time, requesters that maintain strong/positive communication.
  • Finding good tasks: e.g. interesting tasks, tasks that match the worker's abilities, tasks that allow the work to learn something new, tasks that pay a high wage (money/time).

For requesters, task ranking boils down to simply one thing:

  • Finding good workers: e.g. workers that do a task proficiently.

Task ranking preferences, however, largely vary from user to user, and at different points in time, the same user may have different preferences. How do we maximize utility for both parties at any given moment? Currently, other systems face a similar problem: Facebook allows the user to choose rank between "top stories" and "most recent," and our very own voting platform allows users to rank between various sorting methods, including "top," "new," "best," and "needs comments/views."

Why not allow the user to choose in Daemo?

Modules of the system

I thus propose allowing the user to choose between several different rankings, including:

  • Dynamic ranking, based on a combination of requester and task quality.
  • Requester ranking, based on requester quality (as with Boomerang).
  • Wage ranking, based on estimated wage (from workers who have performed similar tasks) for a task.
  • Discovery ranking, which includes new tasks that the worker can explore.

Module 1: dynamic ranking

Problem and limitations

Module preview

Crowdsourcing platforms such as Amazon Mechanical Turk, Clickworker, and Microworkers fail to address all aspects of reputation and ranking between workers and requesters. Workers cannot easily search for and find requesters and tasks they are interested in, and requesters cannot easily be matched with high-quality workers who are proficient in their specific tasks.

Currently, Boomerang solves one aspect of the problem by introducing "a reputation system that differs from traditional rate-and-leave systems by 'boomeranging' the accuracy of the rating decision back to directly impact the user. This feedback loop means that giving someone a high rating increases the likelihood of working with that individual again, while giving a low rating reduces that likelihood." Boomerang thus effectively matches high-quality workers with high-quality requesters (and vice versa), yet no match is made in regard to task category (classification) and worker proficiency with respect to that category.

System details

Module 2: requester ranking

Problem and limitations

Module preview

System details

Module 3: wage ranking

Problem and limitations

Module preview

System details

Module 4: discovery ranking

Problem and limitations

Module preview

System details

Milestone contributors

Slack usernames of all who helped create this wiki page submission: @shreygupta98