Reputation System Ideas

From crowdresearch
Jump to: navigation, search

Some Reputation System Ideas

William Dai @williamdai

1. Implement a trial period for new workers in which tasks of all categories are open to them. If they do well, the requestor can note the type of task performed (task type = skill). Over time the worker will be allowed to do just the type of task he/she consistently gets good ratings for (maintain % good to bad?)

Requestors would simply highlight the skills exhibited by the workers, and similarly can choose to allow workers with N number of "badges" with #category of skill. The requestor in this case would be determining directly the quality of worker they wish to find for their next project, and so wouldn't willingly inflate reviews. If the workers' main issue is finding suitable tasks, the #category labels may help them identify what types of work suit them best.

2. To prevent unfair drops in rating, an incremental rating system that would allow only small increases and drops at any given time can help prevent sudden drops in reputation, especially if said reputation ranking can affect future pay and tasks.

The only options for rating would be N-small_increment, N, N+small_increment. this will, over time, provide adequate ratings, and prevent sudden irrational inflation/deflation. there can always be an option to contact the platform with outrageous/outstandingly poor performance. For example, if I were a requestor wanting to rate a worker with ranking 4/5, I could only choose among 3.9, 4.0, and 4.1. This would deter people from making extra accounts to boost ratings, can be used in conjunction with weighted rating systems similar to the page rank algorithm (who gives matters), and will prevent excessive negative results from retaliation ratings.

3. Instead of having ratings decide how much work is available to a given worker, or the amount of pay received, which are very important and will lead to increased/continued pleading and rate inflation, consider a system in which all work is available to everyone - but only a limited number can work on a given task. The higher ranked workers get to do the task first if there are too many workers wishing to perform a task. This incentives higher ratings without much potential for demotivating newer workers.

New workers can start with a 4/5 rating. Requestors can be told that 4 signifies acceptable work quality etc. But having itemized ratings wouldn't be bad either.