Crowdresearch:Open Governance Through Managed Services (Milestone 5)
The Challenge in building a reputation system
In a particular team all are best and all 10 employees are doing equally good, if we disclose that A is chosen for a particular task everyone will run after A, where as in reality B is equally good as A or others if not disclosed we will all watch the job done by others, and may choose one of the job. Provide Equal opportunity.
Developing reliable reputation system is always a open challenge that limit the effectiveness of reputations, a user enjoying a highly valued reputation would imply that the user has conducted good transactions in the past, however his present transactions are kept aside.
Reputation system are always based on past data, or the near past data, each day in human life is different where he encounters a series of events which can turn out to be the influencing agents to change behaviors, John did social work while he was in college but later on he performed a series of illegitimate tasks which has brought his reputation down.
There can be 3 types of reputation - Bad / Good / No reputation / mixed reputation (perform illegitimate jobs, but still help the helpless)
How do you earn reputation in the real world?
Earning reputation is a continuous process and can be earned through delivering good services over a period of time.
The way we earn reputation in real world
- Helping others solve a complex task and earn reputation as a skilled worker
- Establish as a Business man & delivering social responsibility
- Doctors earn reputation by successfully executing a critical surgical operation
- Doing Illegal activity – adding fear factor brings negative reputation
- Being a research scholar
- By Working for reputed organizations
- Get political portfolio
- Graduates from reputed Institutes earn reputation by virtue of the stamp provided by the institute
The reputation system may differ from demography to demography
How to improve reputation system beyond Boomerang?
IDEA 1 - Remove Reputation system
- Remove the concept of reputation system and have a neutral platform where workers come and work based on their choice. This reduces the human tendency to only look for task from requester with High rating, not everyone can have a high rating, what will happen to tasks which are from coming from requester having low rating, same applies to requester, a requester would always look for a reputed worker what will happen to other workers having low rating. Everybody wants a good boss, but if we don't get we don't quit jobs, we live with it get adjusted to it.
- However there should be way to establish Open channels between requester & worker through a facilitator so that they can give feedback to each other and set each others expectation.
Lets say a requester had posted a task of designing a logo 100+ workers came up with different concept but the requester pick ups one, but he does not disclosed to the other the one which has picked.
IDEA 2 - Establish a career path with yearly review cycles
- Rating system is not on a individual task base but a series of tasks performed over a period of time
- Conceptual model developed from real life - The platform should allow rewards to worker in the form of a promotion, Provide roles to workers, as happens in Industry, Jr. worker, Sr.worker, Lead worker based on past performance records, engagement with the system, task approval rate, minimal task rejections
- Peer to peer rating system throughout the year where each workers based on concrete reasoning & facts also get the chance to rate other workers on a quarterly basis.
- The first level reviewers can be workers/requester, basically a process of self-appraisal
- Second level - Requester
- Third level - Platform Owners
- Minimum engagement with Crowd sourcing platform : 1 year
- Task approval rate : 90% and above
- Task rejections: less than 10%
- Based on yearly review the system will Create groups – low performers, high performance workers/requester and over time the objective of the worker would be to reach the top of the ladder.
- For “short-lived” workers/requester who have conducted few transactions the ratings will not be valid
- Allow the workers/ requester to rate who has been in system for a certain period
- Enable Points based reputation