View source for WinterMilestone 5 stormsurfer

== System ==

'''Please use the following template to write up your introduction section this week.'''

=== Brief introduction and problem being solved ===

For workers, task ranking boils down to two things:
*'''Finding good requesters:''' e.g. requesters that pay well, requesters that accept/reject HITs fairly/on-time, requesters that maintain strong/positive communication.
*'''Finding good tasks:''' e.g. interesting tasks, tasks that match the worker's abilities, tasks that allow the work to learn something new, tasks that pay a high wage (money/time).

For requesters, task ranking boils down to simply one thing:
*'''Finding good workers:''' e.g. workers that do a task proficiently.

''Task ranking preferences, however, largely vary from user to user, and at different points in time, the same user may have different preferences.'' How do we maximize utility for both parties at any given moment? Currently, other systems face a similar problem: Facebook allows the user to choose rank between "top stories" and "most recent," and our very own voting platform allows users to rank between various sorting methods, including "top," "new," "best," and "needs comments/views."

''Why not allow the user to choose in Daemo?''

=== Modules of the system ===

I thus propose allowing the user to choose between several different rankings, including:
*'''Dynamic ranking''', based on a combination of requester and task quality.
*'''Requester ranking''', based on requester quality (as with Boomerang).
*'''Wage ranking''', based on estimated wage (from workers who have performed similar tasks) for a task.
*'''Discovery ranking''', which includes new tasks that the worker can explore.

=== Module 1: dynamic ranking ===

==== Problem and limitations ====

Crowdsourcing platforms such as Amazon Mechanical Turk, Clickworker, and Microworkers fail to address '''all aspects''' of reputation and ranking between workers and requesters. Workers cannot easily search for and find requesters and tasks they are interested in, and requesters cannot easily be matched with high-quality workers who are proficient in their specific tasks.

Currently, ''Boomerang'' solves '''one aspect''' of the problem by introducing "a reputation system that differs from traditional rate-and-leave systems by 'boomeranging' the accuracy of the rating decision back to directly impact the user. This feedback loop means that giving someone a high rating increases the likelihood of working with that individual again, while giving a low rating reduces that likelihood." Boomerang thus effectively matches high-quality workers with high-quality requesters (and vice versa), ''yet no match is made in regard to task category (classification) and worker proficiency with respect to that category.''

==== Module preview and system details ====

In this module, we introduce the dynamic ranking feature, an algorithmic approach to categorizing and ranking requester tasks. Requesters can tag tasks into categories such as "transcribing," "summarizing," and "image tagging," and workers can search for tasks based on tags using the search utility. Machine learning will be used to train the system to automatically categorize tasks.

The dynamic ranking feature is unique in that it builds on Boomerang to take into account not only worker and requester ratings but also the ''task'' completed. For example, a worker who may have completed tasks in two different categories for the same requester may be proficient in one area (e.g. "transcribing") but not another (e.g. "image tagging"). The requester, therefore, is likely to rate the worker high with respect to the tasks in the first area ("transcribing") but low with respect to the second ("image tagging"). Given that the worker completed a roughly equal amount of each type of task, with the current state of Boomerang, ''all'' of the requester’s tasks would most likely appear in the ''middle'' of the worker’s task feed, and the requester’s tasks would be available to the worker at approximately an ''average'' decay.

However, the dynamic ranking feature will have an updated ranking algorithm that takes into account task category, resulting in a '''divergence''' in both the ''position'' where a requester’s different tasks will appear in the worker’s task feed and the ''decay'' at which they will be available. In the above example, the specific requester’s "transcribing" tasks will appear at the ''top'' of the worker’s task feed, and the worker will have ''early'' access to these tasks, because the worker was rated ''high'' by the requester for this specific category of tasks. On the other hand, the specific requester’s "image tagging" tasks will appear at the ''bottom'' of the worker’s task feed, and the worker will have ''delayed'' access to these tasks, because the worker was rated ''low'' by the requester for this specific category of tasks.

As more requesters rate the worker with respect to the various task categories, the feature will learn which types of tasks the user is interested and proficient in. In the previous example, tasks categorized as "transcribing" by new requesters will begin to populate the worker’s task feed, below the "transcribing" tasks create by (high) rated requesters but above "image tagging" tasks created by all requesters.

As a result, workers and requesters will not only be matched on the basis of ratings between workers and requesters but also the type of task that was completed. This will allow workers to easily search for and find requesters and tasks they are ''interested'' in and requesters to easily be matched with ''high-quality'' workers who are ''proficient'' in their specific tasks.

=== Module 2: requester ranking ===

==== Problem and limitations ====

==== Module preview and system details ====

=== Module 3: wage ranking ===

==== Problem and limitations ====

==== Module preview and system details ====

=== Module 4: discovery ranking ===

==== Problem and limitations ====

==== Module preview and system details ====

== Milestone contributors ==

Slack usernames of all who helped create this wiki page submission: @shreygupta98