Reputation System Working Idea

From crowdresearch
Jump to: navigation, search

Idea

@rijul

Requesters/workers are organized into seven "bands"(A, B, C, D, E, F, G), in order of importance to their counterpart. Here is the algorithm I suggest should be used to rank requesters/workers on their partner's feeds as well as to be a part of the mechanism that determines order of task release. Note that for the sake of simplification, I'm taking the perspective of a given requester, although this should work from the point of view of the worker as well.

1) Check+ is represented in calculations as +1, check as 0 and check- as -1. New workers are given the rating -0.01 (just so that it is slightly below check).

2) Bands B,D and G would consist of workers the requester has not rated previously, each ranked in accordance to their global averages. Band B would comprise of check+ workers, band D of check(all workers with same rating, so ordered randomly, or by some other measure) and band G of check- workers. Band E would be made up of new workers.

2) For all other workers (in essence, workers the requester has rated previously), we would calculate the value of 'R'. Where,


R=[(Number of times rated check+ by requester)*(+1)]+[(Number of times rated check- by requester)*(-1)]


a) Workers with R=0 are placed in band C

b) Workers with R>0 are placed in band A. They are now ranked by their individual scores (high to low)*

c) Workers with R<0 are placed in band G. They are now ranked by their individual scores (high to low)*


*In case two or more scores are equal, we incorporate global ranking. Workers with higher global rankings are positioned higher.


I believe this addresses the issue of intra-band ranking, providing the requester with the workers he prefers most, each time.

Check rating system.jpg

Band A: ✓+ Rated by requester

Band B: ✓+ Rated by other requesters

Band C: ✓ Rated by requester

Band D: ✓ Rated by other requesters

Band E: New

Band F: ✓- Rated by requester

Band G: ✓- Rated by other requesters


Limitations

Band D is likely to be extremely narrow, since only one particular average value is used.