WinterMilestone 3 raymondsums

From crowdresearch
Jump to: navigation, search

Reputation-related Idea

Implement an alternative to present reputation system. For example, reputation now shows percentage based on the last 100 or 1000 tasks done in the specific category of work. This will be more forgiving for new users and 1 or 2 rejections early on in the beginning of their turking career will not be as devastating.

And having a set number in a specific category of work is also important so that the requesters can set a requirement that workers must have some experience in similar tasks. This will prevent certain users from gaming the system by doing a large quantity of easy tasks and then accepting highly specialized or high value tasks.

Both workers and requesters will rate each other upon completion of task and review of task respectively. An occasional turker will be assigned to review tasks at random to either verify the ratings given - by reviewing both the HIT instructions and quality of work done. This independent reviewer will be picked at random from the pool of turkers and reimbursed for their time. This should hopefully enforce the credibility of the ratings and acts as the quality control.