Difference between revisions of "Crowdresearch:Open Governance Through Managed Services (Milestone 5)"

From crowdresearch
Jump to: navigation, search
Line 8: Line 8:
 
Reputation system are always based on past data, or the near past data, each day in human life is different where he encounters a series of events which can turn out to be the influencing agents to change behaviors, John did social work while he was in college but later on he performed a series of illegitimate tasks which has brought his reputation down.
 
Reputation system are always based on past data, or the near past data, each day in human life is different where he encounters a series of events which can turn out to be the influencing agents to change behaviors, John did social work while he was in college but later on he performed a series of illegitimate tasks which has brought his reputation down.
  
There can be 3 types of reputation - Bad / Good / No reputation / mixed reputation (perform illegitimate jobs, but still help the helpless)
+
Reputation has 3 variants - Bad / Good / No reputation(always neutral) / mixed reputation (perform illegitimate jobs, but still help the helpless)
  
  
Line 34: Line 34:
 
'''IDEA 1 - Remove Reputation system - Provide equal Opportunity'''
 
'''IDEA 1 - Remove Reputation system - Provide equal Opportunity'''
  
*Remove the concept of reputation system and have a neutral platform where workers come and work based on their choice. This reduces the human tendency to only look for task from requester with High rating, not everyone can have a high rating, what will happen to tasks which are from coming from requester having low rating, same applies to requester, a requester would always look for a reputed worker what will happen to other workers having low rating. Everybody wants a good boss, but if we don't get we don't quit jobs, we live with it get adjusted to it.
+
Boomerang is trying to establish a relationship between a worker & requester using the following equation
 +
Low rated W = Low rated R
 +
Medium rated W = Medium rated R
 +
High rated W = High rated R
  
*However there should be way to establish Open channels between requester & worker through a facilitator so that they can give feedback to each other and set each others expectation.
+
To set the context let's take an example, there are 10 workers and all the 10 workers are equally performing good, for a particular task, say task 1 if we disclose that the output of worker A is selected then for the next set of task the requester will run after A, where as in reality worker B is equally good as A and others. Similarly, let's say a requester had posted a task of designing a logo 100+ workers came up with different concept but the requester pick up one, but he is not making a public announcement that which logo he has picked up.  
  
To set the context let's take an example, there are 10 workers and all the 10 workers are equally doing good, for a particular task if we disclose that the output of A is chosen for the next task everyone(requester) will run after A, where as in reality B is equally good as A and others.
+
This in a way will provide equal opportunity to all workers & requester.
This in a way will provide Equal opportunity to all workers & requester.
+
  
+
*Remove the concept of reputation system and have a neutral platform where workers come and work based on their choice. This reduces the human tendency to only look for task from requester with High rating, not everyone can have a high rating, what will happen to tasks which are from coming from requester having low rating, same applies to requester, a requester would always look for a reputed worker what will happen to other workers having low rating. Everybody wants a good boss, but if we don't get we don't quit jobs, we live with it get adjusted to it.
Similarly, let's say a requester had posted a task of designing a logo
+
 
100+ workers came up with different concept but the requester pick ups one, but he is not making a public announcement that which logo he has picked up. In a way he is does
+
*However, there should be way to establish Open channels between requester & worker through a facilitator so that they can give feedback to each other and set each others expectation prior working on a task
  
  
Line 49: Line 51:
 
'''IDEA 2 - Establish a career path with yearly review cycles'''
 
'''IDEA 2 - Establish a career path with yearly review cycles'''
  
*Rating system is not on a individual task base but a series of tasks performed over a period of time
+
*Let the rating system be not on a individual task but a series of tasks performed by a worker over a period of time
  
 
*Conceptual model developed from real life - The platform should allow rewards to worker in the form of a promotion, Provide roles to workers, as happens in Industry, Jr. worker, Sr.worker, Lead worker based on past performance records, engagement with the system, task approval rate, minimal task rejections
 
*Conceptual model developed from real life - The platform should allow rewards to worker in the form of a promotion, Provide roles to workers, as happens in Industry, Jr. worker, Sr.worker, Lead worker based on past performance records, engagement with the system, task approval rate, minimal task rejections
Line 65: Line 67:
  
 
*Based on yearly review the system will Create groups – low performers, high performance workers/requester and over time the objective of the worker would be to reach the top of the ladder.
 
*Based on yearly review the system will Create groups – low performers, high performance workers/requester and over time the objective of the worker would be to reach the top of the ladder.
 
  
 
*For “short-lived” workers/requester who have conducted few transactions the ratings will not be valid
 
*For “short-lived” workers/requester who have conducted few transactions the ratings will not be valid
Line 73: Line 74:
  
  
'''IDEA 3 - Create workers guild'''
+
'''IDEA 3 - Body shopping revisited '''
 +
 
 +
Body shopping is the practice of consultancy firms recruiting information technology workers in order to contract their services out on a short-term basis.[https://en.wikipedia.org/wiki/Body_shopping]
 +
 
 +
Let the requester post the requirement to the consultancy firm, and then distribute the task to individual based on workers interest & skill
 +
 
 +
Forming independent body who would be dealing with workers issues relating to payment, unfair ratings, task rejections, fair wages and can settle issue by directly talking to requester, similarly they can also solve problems for requester in finding a reliable & skilled worker
 +
 
 +
 
 +
 
 +
'''IDEA 4 - Transitivity '''
 +
 
 +
Transitivity is a highly desired property of a trust metric.In situations where A trusts B and B trusts C, transitivity concerns the extent to which A trusts C. Without transitivity, trust metrics are unlikely to be used to reason about trust in more complex relationships.
 +
 
 +
The intuition behind transitivity follows everyday experience of 'friends of a friend' (FOAF), the foundation of social networks. However, the attempt to attribute exact formal semantics to transitivity reveals problems, related to the notion of a context. For example,[42] defines conditions for the limited transitivity of trust, distinguishing between direct trust and referral trust. Similarly,[43] shows that simple trust transitivity does not always hold, based on information on the Advogato model and, consequently, have proposed new trust metrics.
 +
 
 +
The simple, holistic approach to transitivity is characteristic to social networks (FOAF, Advogato). It follows everyday intuition and assumes that trust and trustworthiness apply to the whole person, regardless of the particular context. If one can be trusted as a friend, one can be also trusted to recommend or endorse another friend. Therefore, transitivity is semantically valid without any constraints. and is a natural consequence of this approach.
  
A body who would be dealing with workers issues relating to payment, unfair ratings, task rejections, fair wages and can settle issue by directly talking to requester
+
The more thorough approach distinguishes between different contexts of trust, and does not allow for transitivity between contexts that are semantically incompatible or inappropriate. Contextual approach may, for instance, distinguish between trust in a particular competence, trust in honesty, trust in the ability to formulate a valid opinion, or trust in the ability to consolidate another's opinions. Contextual approach is often used in trust-based service composition.[44] The understanding that trust is contextual is a foundation of a collaborative filtering.
Let the requester post the work in the workers guild, and then distributed to individual based on workers interest & skill
+

Revision as of 07:50, 29 January 2016

The Challenge in building a reputation system

Developing reliable reputation system is always a open challenge that limit the effectiveness of reputations, a user enjoying a highly valued reputation would imply that the user has conducted good transactions in the past, however his present transactions are kept aside.

Reputation system are always based on past data, or the near past data, each day in human life is different where he encounters a series of events which can turn out to be the influencing agents to change behaviors, John did social work while he was in college but later on he performed a series of illegitimate tasks which has brought his reputation down.

Reputation has 3 variants - Bad / Good / No reputation(always neutral) / mixed reputation (perform illegitimate jobs, but still help the helpless)


How do you earn reputation in the real world?

Earning reputation is a continuous process and can be earned through delivering good services over a period of time.

The way we earn reputation in real world

  • Helping others solve a complex task and earn reputation as a skilled worker
  • Establish as a Business man & delivering social responsibility
  • Doctors earn reputation by successfully executing a critical surgical operation
  • Doing Illegal activity – adding fear factor brings negative reputation
  • Being a research scholar
  • By Working for reputed organizations
  • Get political portfolio
  • Graduates from reputed Institutes earn reputation by virtue of the stamp provided by the institute
  • High Social class with income group

The reputation system may differ from demography to demography


How to improve reputation system beyond Boomerang?

IDEA 1 - Remove Reputation system - Provide equal Opportunity

Boomerang is trying to establish a relationship between a worker & requester using the following equation Low rated W = Low rated R Medium rated W = Medium rated R High rated W = High rated R

To set the context let's take an example, there are 10 workers and all the 10 workers are equally performing good, for a particular task, say task 1 if we disclose that the output of worker A is selected then for the next set of task the requester will run after A, where as in reality worker B is equally good as A and others. Similarly, let's say a requester had posted a task of designing a logo 100+ workers came up with different concept but the requester pick up one, but he is not making a public announcement that which logo he has picked up.

This in a way will provide equal opportunity to all workers & requester.

  • Remove the concept of reputation system and have a neutral platform where workers come and work based on their choice. This reduces the human tendency to only look for task from requester with High rating, not everyone can have a high rating, what will happen to tasks which are from coming from requester having low rating, same applies to requester, a requester would always look for a reputed worker what will happen to other workers having low rating. Everybody wants a good boss, but if we don't get we don't quit jobs, we live with it get adjusted to it.
  • However, there should be way to establish Open channels between requester & worker through a facilitator so that they can give feedback to each other and set each others expectation prior working on a task


IDEA 2 - Establish a career path with yearly review cycles

  • Let the rating system be not on a individual task but a series of tasks performed by a worker over a period of time
  • Conceptual model developed from real life - The platform should allow rewards to worker in the form of a promotion, Provide roles to workers, as happens in Industry, Jr. worker, Sr.worker, Lead worker based on past performance records, engagement with the system, task approval rate, minimal task rejections

Appraisal process

  • Peer to peer rating system throughout the year where each workers based on concrete reasoning & facts also get the chance to rate other workers on a quarterly basis.
  • The first level reviewers can be workers/requester, basically a process of self-appraisal
  • Second level - Requester
  • Third level - Platform Owners

Criteria

  • Minimum engagement with Crowd sourcing platform : 1 year
  • Task approval rate : 90% and above
  • Task rejections: less than 10%
  • Based on yearly review the system will Create groups – low performers, high performance workers/requester and over time the objective of the worker would be to reach the top of the ladder.
  • For “short-lived” workers/requester who have conducted few transactions the ratings will not be valid
  • Allow the workers/ requester to rate who has been in system for a certain period
  • Enable Points based reputation


IDEA 3 - Body shopping revisited

Body shopping is the practice of consultancy firms recruiting information technology workers in order to contract their services out on a short-term basis.[1]

Let the requester post the requirement to the consultancy firm, and then distribute the task to individual based on workers interest & skill

Forming independent body who would be dealing with workers issues relating to payment, unfair ratings, task rejections, fair wages and can settle issue by directly talking to requester, similarly they can also solve problems for requester in finding a reliable & skilled worker


IDEA 4 - Transitivity

Transitivity is a highly desired property of a trust metric.In situations where A trusts B and B trusts C, transitivity concerns the extent to which A trusts C. Without transitivity, trust metrics are unlikely to be used to reason about trust in more complex relationships.

The intuition behind transitivity follows everyday experience of 'friends of a friend' (FOAF), the foundation of social networks. However, the attempt to attribute exact formal semantics to transitivity reveals problems, related to the notion of a context. For example,[42] defines conditions for the limited transitivity of trust, distinguishing between direct trust and referral trust. Similarly,[43] shows that simple trust transitivity does not always hold, based on information on the Advogato model and, consequently, have proposed new trust metrics.

The simple, holistic approach to transitivity is characteristic to social networks (FOAF, Advogato). It follows everyday intuition and assumes that trust and trustworthiness apply to the whole person, regardless of the particular context. If one can be trusted as a friend, one can be also trusted to recommend or endorse another friend. Therefore, transitivity is semantically valid without any constraints. and is a natural consequence of this approach.

The more thorough approach distinguishes between different contexts of trust, and does not allow for transitivity between contexts that are semantically incompatible or inappropriate. Contextual approach may, for instance, distinguish between trust in a particular competence, trust in honesty, trust in the ability to formulate a valid opinion, or trust in the ability to consolidate another's opinions. Contextual approach is often used in trust-based service composition.[44] The understanding that trust is contextual is a foundation of a collaborative filtering.