Difference between revisions of "Crowdresearch:WinterMilestone 3 Carpe Noctem - Reputation"

From crowdresearch
Jump to: navigation, search
Line 1: Line 1:
 
To improve reputation system beyond Boomerang, we can look into some more creative ways of calculating and utilizing the reputation. A successful reputation system should more or less accurately reflect the quality of work and areas of expertise of each worker. It should reflect the trustworthiness and quality of requesters. Such reputation system should also be resistant to hackers who may try find shortcuts to rack up reputation.
 
To improve reputation system beyond Boomerang, we can look into some more creative ways of calculating and utilizing the reputation. A successful reputation system should more or less accurately reflect the quality of work and areas of expertise of each worker. It should reflect the trustworthiness and quality of requesters. Such reputation system should also be resistant to hackers who may try find shortcuts to rack up reputation.
  
== How to improve reputation system beyond Boomerang? ==
+
== Worker Reputation ==
 +
First, let’s focus on reputation for workers. Due to a wide range of types of work, using a single reputation score may not accurately represent a worker’s work ethics, quality of work, areas of expertise, level of experiences etc.
 +
 
 +
Instead, we will break down the reputation into subcategories. The subcategories include whether work finishes tasks on time (timeliness), work quality, communication skill/style, work ethics, areas of expertise.
 +
 
 +
Each category has its own metrics. For example,
 +
> work quality would be based on the satisfaction ratings from requesters. Areas of expertise would be computed based on worker task history, task size, level of difficulty and so on.
 +
> if workers have been receiving good scores on web development consistently, the task, he would be rated high in the area of web development.
 +
> Finally, a comprehensive reputation score will be calculated based on [timeliness, work quality, communication skills, work ethics] as well as areas of expertise [web dev, machine learning, UX design, …].
 +
 
 +
This score is used as an overall measurement of worker’s reputation for humans but when matching requesters with workers, the computer will use the specific subcategories to recommend workers. The specifics of how each subcategory reputation is computed is too long and detailed to put it here but can use StackOverflow reputation as a reference.
 +
 
 +
== Requester Reputation ==
 +
Similar to worker reputation, requester reputation can be calculated based on a variety of criterias. They are [task quality, response time, payment timeliness, easy-to-work]. Like worker reputation, the comprehensive reputation is for humans, while the actually recommendation is based on subcategories.
 +
 
 +
== For Newcomers ==
 +
All users start with one reputation point, and reputation can never drop below 1. To help new users jump start their journey, tutorials and mentors will be provided on demand.  Reputation can go to negative if workers receive many low score (<4 on scale of 10). This would make sure newcomers are not at a natural disadvantage comparatively.
 +
 
 +
== How Workers, Requesters Find Each Other ==
 +
Subcategory scores will be used by computer algorithm to recommend workers to requesters and vice versa. Request
 +
 
  
== How would reputation work for new comers - workers or requesters? ==
 
  
== How do workers find relevant work, and requesters find ideal worker? ==
 
  
 
== Milestone Contributors ==
 
== Milestone Contributors ==

Revision as of 20:42, 31 January 2016

To improve reputation system beyond Boomerang, we can look into some more creative ways of calculating and utilizing the reputation. A successful reputation system should more or less accurately reflect the quality of work and areas of expertise of each worker. It should reflect the trustworthiness and quality of requesters. Such reputation system should also be resistant to hackers who may try find shortcuts to rack up reputation.

Worker Reputation

First, let’s focus on reputation for workers. Due to a wide range of types of work, using a single reputation score may not accurately represent a worker’s work ethics, quality of work, areas of expertise, level of experiences etc.

Instead, we will break down the reputation into subcategories. The subcategories include whether work finishes tasks on time (timeliness), work quality, communication skill/style, work ethics, areas of expertise.

Each category has its own metrics. For example, > work quality would be based on the satisfaction ratings from requesters. Areas of expertise would be computed based on worker task history, task size, level of difficulty and so on. > if workers have been receiving good scores on web development consistently, the task, he would be rated high in the area of web development. > Finally, a comprehensive reputation score will be calculated based on [timeliness, work quality, communication skills, work ethics] as well as areas of expertise [web dev, machine learning, UX design, …].

This score is used as an overall measurement of worker’s reputation for humans but when matching requesters with workers, the computer will use the specific subcategories to recommend workers. The specifics of how each subcategory reputation is computed is too long and detailed to put it here but can use StackOverflow reputation as a reference.

Requester Reputation

Similar to worker reputation, requester reputation can be calculated based on a variety of criterias. They are [task quality, response time, payment timeliness, easy-to-work]. Like worker reputation, the comprehensive reputation is for humans, while the actually recommendation is based on subcategories.

For Newcomers

All users start with one reputation point, and reputation can never drop below 1. To help new users jump start their journey, tutorials and mentors will be provided on demand. Reputation can go to negative if workers receive many low score (<4 on scale of 10). This would make sure newcomers are not at a natural disadvantage comparatively.

How Workers, Requesters Find Each Other

Subcategory scores will be used by computer algorithm to recommend workers to requesters and vice versa. Request



Milestone Contributors

  • Michelle Chan : @michellechan
  • Manoj Pandey : @manojpandey
  • Lucas Qiu  : @lucasq
  • Mengnan Wang : @mengnan