Crowdresearch:Open Governance Through Managed Services (Milestone 5)
The Challenge in building a reputation system
Developing reliable reputation system is always a open challenge that limit the effectiveness of reputations, a user enjoying a highly valued reputation would imply that the user has conducted good transactions in the past, however his present transactions are kept aside while assessing the reputation and generally comes in
Reputation system are always based on past data, or the near past data, each day in human life is different where he encounters a series of events which can turn out to be the influencing agents to change behaviors, John used to help others in solving maths problems while he was in college and got reputed as a problem solver but later on he performed a series of illegitimate tasks which has brought his reputation down in the society.
Reputation has 3 variants - Bad / Good / No reputation(always neutral) / mixed reputation (perform illegitimate jobs, but still help the helpless)
How do you earn reputation in the real world?
Earning reputation is a continuous process and can be earned through delivering good services over a period of time.
The way we earn reputation in real world
- Helping others solve a complex task and earn reputation as a skilled worker
- Establish as a Business man & delivering social responsibility
- Doctors earn reputation by successfully executing a critical surgical operation
- Doing Illegal activity – adding fear factor brings negative reputation
- Become a scientist & discover
- By working for reputed organizations
- Get a political portfolio and solve critical public issues
- Do good Social work
- Graduates from reputed Institutes earn reputation by virtue of the stamp provided by the institute
- High Social class with high income group
How to improve reputation system beyond Boomerang?
IDEA 1 - Establish a career path with quarterly/yearly review cycles
Built on a predefined framework
- Let the rating system be not on a individual task but a series of tasks performed by a worker over a period of time, the rating achieved by workers & requester will remain valid for a year.
- Conceptual model developed from real life - The platform should offer rewards to worker in the form of a promotion, raising rates based on skill & seniority, Provide roles to workers, as happens in Industry, Jr. worker, Sr.worker, Lead worker based on past performance records, engagement with the system, task approval rate, minimal task rejections
- Peer to peer rating system throughout the year where each workers based on concrete reasoning & facts also get the chance to rate other workers on a quarterly basis.
- The first level reviewers can be workers/requester, basically a process of self-appraisal
- Second level - Requester
- Third level - Platform Owners
- For “short-lived” workers/requester who have conducted few transactions the ratings will not be valid, Minimum engagement with Crowd sourcing platform should be a year
- Task approval rate : 90% and above
- Task rejections: less than 10%
- Based on yearly review the system will Create groups – low performers, high performance workers/requester and over time the objective of the worker would be to reach the top of the ladder.
IDEA 2 - Body shopping revisited
Body shopping is the practice of consultancy firms recruiting information technology workers in order to contract their services out on a short-term basis.
Let the requester post the requirement to the consultancy firm, and then distribute the task to individual based on workers interest & skill
Forming independent body who would be dealing with workers issues relating to payment, unfair ratings, task rejections, fair wages and can settle issue by directly talking to requester, similarly they can also solve problems for requester in finding a reliable & skilled worker
IDEA 3 - Transitivity
Transitivity is a highly desired property of a trust metric.In situations where A trusts B and B trusts C, transitivity concerns the extent to which A trusts C. Without transitivity, trust metrics are unlikely to be used to reason about trust in more complex relationships.
The intuition behind transitivity follows everyday experience of 'friends of a friend' (FOAF), the foundation of social networks. However, the attempt to attribute exact formal semantics to transitivity reveals problems, related to the notion of a context. For example, defines conditions for the limited transitivity of trust, distinguishing between direct trust and referral trust. Similarly, shows that simple trust transitivity does not always hold, based on information on the Advogato model and, consequently, have proposed new trust metrics.
The simple, holistic approach to transitivity is characteristic to social networks (FOAF, Advogato). It follows everyday intuition and assumes that trust and trustworthiness apply to the whole person, regardless of the particular context. If one can be trusted as a friend, one can be also trusted to recommend or endorse another friend. Therefore, transitivity is semantically valid without any constraints. and is a natural consequence of this approach.
The more thorough approach distinguishes between different contexts of trust, and does not allow for transitivity between contexts that are semantically incompatible or inappropriate. Contextual approach may, for instance, distinguish between trust in a particular competence, trust in honesty, trust in the ability to formulate a valid opinion, or trust in the ability to consolidate another's opinions. Contextual approach is often used in trust-based service composition. The understanding that trust is contextual is a foundation of a collaborative filtering.