Crowdresearch:Open Governance Through Managed Services (Milestone 5)
The Challenge in building a reputation system
Developing reliable reputation system is always a open challenge there is a limit to measure the effectiveness of reputation, a user enjoying a highly valued reputation would imply that the user has conducted good transactions in the past, however his present transactions are kept aside while assessing the reputation
Reputation system are always based on past data, or the near past data, each day in human life is different where he encounters a series of events which can turn out to be the influencing agents to change behaviors, John used to help others in solving maths problems while he was in college and became a reputed problem solver but later on he performed a series of illegitimate tasks which has brought his reputation down in the society.
Reputation has 3 variants - Bad / Good / No reputation(always neutral) / mixed reputation (perform illegitimate jobs, but still help the helpless)
How do you earn reputation in the real world?
Earning reputation is a continuous process and can be earned through delivering good services over a period of time.
The way we earn reputation in real world
- Helping others solve a complex task and earn reputation as a skilled worker
- Establish as a Business man & delivering social responsibility
- Doctors earn reputation by successfully executing a critical surgical operation
- Doing Illegal activity – adding fear factor brings negative reputation
- Become a scientist & discover
- By working for reputed organizations
- Get a political portfolio and solve critical public issues
- Do good Social work
- Graduates from reputed Institutes earn reputation by virtue of the stamp provided by the institute
- High Social class with high income group
How to improve reputation system beyond Boomerang?
IDEA 1 - Establish a career path with quarterly/yearly review cycles
Built on a predefined framework
- Let the rating system be not on a individual task but a series of tasks performed by a worker over a period of time, the rating achieved by workers & requester will remain valid for a year.
- Conceptual model developed from real life - The platform should offer rewards to worker in the form of a promotion, raising rates based on skill & seniority, Provide roles to workers, as happens in Industry, Jr. worker, Sr.worker, Lead worker based on past performance records, engagement with the system, task approval rate, minimal task rejections
- Peer to peer rating system throughout the year where each workers based on concrete reasoning & facts also get the chance to rate other workers on a quarterly basis.
- The first level reviewers can be workers/requester, basically a process of self-appraisal
- Second level - Requester
- Third level - Platform Owners
- For “short-lived” workers/requester who have conducted few transactions the ratings will not be valid, Minimum engagement with Crowd sourcing platform should be a year
- Task approval rate : 90% and above
- Task rejections: less than 10%
- Based on yearly review the system will Create groups – low performers, high performance workers/requester and over time the objective of the worker would be to reach the top of the ladder.
IDEA 2 - Transitivity
The foundation of social network systems friends of a friend', In situations where A trusts B and B trusts C, transitivity concerns the extent to which A trusts C. If one can be trusted as a worker/requester, one can be also trusted to recommend or endorse another worker/requester thus building a chain of reputation system.