Milestone 4 researchinprogress Reputation: Mutual Rating System
Problem statement (problem being addressed):
How might we design better reputation systems?
An important aspect of crowdsourcing markets is to have a system which makes it easier for workers to choose the reputed requesters. It is also important for the requesters to choose the workers who have higher rating to ensure quality.
So, a reputation system is necessary to provide an efficient mechanism to choose the desired target .This can be achieved by having ratings based on experience and quality of work done for the workers. And, ratings based on promptness, task-clarity and fairness for the requesters. But, the question is that who has the power to provide these ratings? What if the ratings are biased and don't convey proper information? Since, these ratings are kept in mind while assigning the tasks,it might be unfair, as it will provide some capable workers lesser work than they deserve. This problem has to be dealt with, to ensure fairness and design a better reputation system.
Credit for the idea (inspiration):
The idea is inspired from the ideas stated in Different_Levels_of_workers,Bring_Top_Workers_Closer_to_Requesters, Expose_worker_skills and User_Rating_System which brings out the need of having a platform which would prove beneficial to the workers and requesters. A reputation system design, which would save time and let one choose the best out of a set of workers and requesters.
Explanation of the idea:
The idea proposes creating an individual profile on the platform which has the information of the worker and requester including the ratings which are given to them.
A,reputation system where the requesters have to grade the quality of work done by the workers which decides their ratings.
-> These ratings(on a scale of 0-10) can be provided on a categorical basis like overall efficiency, experience, time efficiency, behaviour, skills as per the requirement etc. This is how other requesters get to know which workers are good at what and they can choose their targeted workers effectively.
-> For all the HITs completed by the workers they will be rated by each requester and their overall(average) rating will be visible to other workers and requesters.
-> These ratings can categorize workers as experts, intermediate and novice ; which motivates the workers to improve their chances of getting HITs and work on their skillset.
-> A top-worker who deserves the task is also motivated to work in a recommended way as his/her skills are exposed and rated accordingly.
-> A novice will have to be given a head-start for acquiring reputation. To make this possible, some weekly/monthly challenges (including sample tasks) can be hosted by the platform which motivates the novice category to increase their ratings. This would provide them the experience of working on the platform . Or, the novice workers can be given small tasks by the experts who would inturn rate the work done by them and might even provide some bonus.
-> The workers can form groups sharing similar interests, skills and task preferences to collaborate and explore opportunities.
At the worker side:
-> The requester will be rated by the worker on the basis of overall experience, promptness, fairness, HIT description, behaviour etc .
-> Using these ratings the other workers will be able to select the task under a particular requester which would prove beneficial. In a similar way, These ratings can categorize requesters as experts, intermediate and novice ; this motivates the requesters to improve their chances of getting more workers.
-> The top requesters get a chance to gather more workers for better completion of their HITs.
-> The novice requesters can get popular on the platform by posting more and more HITs , level by level.
-> The requesters may even form communities and groups to discuss the opportunities and methods to gain access to the top-workers, which will be helpful for novice requesters.
The platform will have a search tool which would help requesters choose the category of workers on the basis of their ratings. And, the workers also get to choose the requesters they want to work with. The top workers/requesters would be visible at the top as per their ratings.