Jsilver Reputation ideas

From crowdresearch
Jump to: navigation, search

Reputation and Other Ideas by Jsilver

I will post a collection of my "wish-list" reputation ideas (among other ideas), some of which have been scattered in my mind and notes since January 2015 and even much prior, mostly based on my experience (as an oDesk worker, team manager, client since 2012). I've shared several/many of them countless times here on Slack (publicly and privately), Meteor, and Wikis. This repository is a work in progress.


Acronyms used:

RS= Reputation system

End-of-task Itemized Rating (Applicable to worker and client)

A client would rate a worker based on skills used in a particular task and based on other criteria (domains: quality of work; communication/responsiveness; cooperation and work ethics/professionalism, deadline/turnaround; efficiency (like cost and time); and accuracy/consistency).

This RS is not perfect but would reflect a rating for each skill used in a task or project, rather than the typical "limited-view" 5-star rating. This is essential so that future clients would not bet blind on such a worker; If done in conjunction with interviewing the worker and checking the worker's job history, the client would have a very good idea of the worker's skills and experiences.

On the other hand, a worker could rate a client based on that 6-domain criteria above plus another criteria: knowledge of task. (criteria mentioned here may not be accurate nor final) As the platform evolves, it would have a substantial collection of rating data from workers and clients. By then, each worker would have worked with many clients and vice versa. If necessary, the platform would be able to put more weight on ratings given by skilled clients/workers (those who are able to rate workers/clients more accurately/honestly than others).


Implicit signals (Applicable to worker and client)

Rating is done during job application evaluation phase. This is a great way of providing reputation coverage throughout the worker (and client) population. I believe this could be implemented much easier than other reputation systems. See [1]


Rating/Ranking Weighting

  • Wherever necessary (forum, upvoting, etc.), rating/ranking a worker who has recently worked higher than a worker who has last worked many months or years ago.
  • Wherever necessary, rating/ranking a worker who has done more work higher than a worker who has done much less work in a given period.
  • Wherever necessary, rating/ranking a senior worker (platform member for a longer time) higher than a newer worker.


Tiered service fees (applicable to workers and clients)

Meteor link:

Worker performance (based on rating) would be tied to varying service fee.

Proposed worker tiered service fee matrix: (proposal may not be final) 5% to 15%. New workers would start at 10% and change based on performance.

Proposed client tiered service fee matrix: (proposal may not be final) 5% to 15%. New clients would start at 10% and change based on performance.

Top-performing workers would be eligible for decreasing service fees to as low as 5% or 6%. On the other hand, poor-performing workers would be subject to increasing fees up to 12% to 15% -- before getting banned from the platform. Almost similar policy for clients. Service fee change would be implemented after x amount of work completed, or after x period (quarterly, semi-annually, etc.).

For clients: Offer them lower fees for regularly giving feedback to workers they hire, and providing regular/continuous workload, quick and fair/high payment, quick communication and good ethics, detailed task descriptions, and interaction in the platform forum.

Join date could also be one of the criteria to consider.


Access to more tasks and better-paying tasks

Worker's access to more tasks and better-paying tasks is subject to his/her performance (rating). We could flip this and apply to clients simultaneously: client's access to the top workers could be contingent upon workers' historical rating for that client.


Security, and management of trolls/bad actors

Several ideas to improve security and to prevent the spread of bad apples among the reputable and innocent ones:

  • Ability to report and flag scam and spam clients and job posts
  • Make it expensive hard, and/or less enticing for trolls to do their thing
  • Prevent people from making new/duplicate accounts
  • The platform forum should have forum moderators, and allow the community to help flag and report trolls and spammer and scammers
  • Track data like IP address, etc.; use cookies
  • Slow down trolls' access to the platform
  • Limit certain/most users' access to certain areas so malicious behaviors can be confined and easier to detect
  • Try to hide the troll/bad actor from the community


PEAK and Z index

PEAK: http://crowdresearch.stanford.edu/w/index.php?title=Milestone_8_sanjoseSpartans_Foundation2

Z index: (private √+√√- rating + Skill/Task category A rating + Task category B rating + ....)/ 100 ? (calculation unsure)

"Z" for "zen" or "zenith", or we could call it D index instead (D for Daemo).


Summer Milestone 11 Submissions

Complete (End-to-End) Reputation Cycle

This is a combination of Implicit Signals and End-of-task Itemized Rating. Please read the details above. This Rep Cycle would yield reputation from both sides of the task (at the start and end), and help increase reputation coverage regardless of quantity of workers hired (non-hired workers still get reputation). To incentivize clients to create implicit signals, the platform would offer them lower fees based on implicit signal quantity (this idea is subject to refinement).

Peer Feedback/Rating

Workers will assess their peers' work and give them reputation. To incentivize workers, the platform would offer them lower fees based on feedback quantity (this idea is subject to refinement). The disadvantage of this is not all work output can be assessed (e.g., those output subject to Non-Disclosure Agreements).

Varied Ideas

  • Use a combination of monetary and non-monetary incentives (workers and clients). I believe this has a great synergy. The main idea is to attenuate the focus on money and foster and strengthen cooperation and sharing among the community.
  • Use the prototype task to clarify and refine task details, find the right workers, and determine fair pricing.
  • Create a Custodian Group (CG) to lead, manage, and protect the platform and its community.
  • Democratize learning and provide access to lifelong learning opportunities which would have a major impact on workers' lives: a. Daemo would partner with MOOCs so we can help workers improve their skills and earn more. As a partner, MOOCs would get more clients, while workers would get discounts on these MOOCs. MOOC certificates would be easily integrated to a worker's profile.; b. Daemo would partner with subscribe-only research repositories so workers could get access to them for free or a much smaller fee.
  • Create a Daemo marketplace where workers can sell "evergreen work output" (e.g. an app or tshirt design)
  • Daemo can generate more jobs by hiring workers within the platform. Daemo can also crowdsource platform necessities, like Daemo's logo design as an example.
  • Crowdsource the skills test questions: To create unique, tough skills test questions, we can ask platform's workers to anonymously and privately submit test questions they think are tough and unique but would be answerable by skilled people in their skill categories. Daemo will then sift through their suggestions.
  • For further customization, worker task feed should show more full-time related work, if the worker specifies in the profile that he's looking for full-time work.
  • If Daemo will allow ads, ads in the platform should be work-related, like insurance, an accounting app etc.
  • In the near future, Daemo would offer paid healthcare and pension, and/or performance- or Daemo index-based profit-sharing.
  • Top wish: An initial global minimum wage (floor) of between $7 to $10 hour would really be awesome for workers worldwide; This price range would serve as a mandatory wage floor, rather than a requirement... I believe each government's wage data and recommended wage are unreliable.
  • Possible error page message (with Daemo logo beside or above this message): "Error: Our servers are currently experiencing a technical problem. This is probably temporary and should be fixed soon. Sorry, please try again in a few minutes."
  • Daemo in-built message history should be a default feature. We could archive it or delete it after 6 months or so to control bandwidth usage. Message history should only be accessible to a limited set of people (customer service people/Dispute team) -- for example, in case of a dispute between worker and client. File access should be conferred to relevant people only.
  • We need to delegate and not put all the weight on the LB.

References

  • These seem beneficial to our project: [6], [7]