Open Honest Crowdsourcing - One idea on Daemo Reputation System by AngelaRichmondFuller

From crowdresearch
Jump to: navigation, search

Open honest crowdsourcing

That's it. That's the big idea!

This is underpinned by Daemo's foundations of building trust and distributing power

What does this mean in practice?

Real Authentic Honest Workers and Requesters

  1. real profiles that workers can link together in collectives (should they so choose) that could be re-formed for each task.
  2. links to GitHub, LinkedIn, Twitter
  3. Skills assessed. Workers can self-certify and then complete tasks set by Daemo to demonstrate skills. This could happen in either:

- an incubator space (perhaps working on tasks for charities and NGOs for no or reduced pay - an idea that we've been throwing around which Claudia has outlined as a way of onboarding newbies at [1]). I would also add another element in that the worker's true ability of the listed qualifications/certifications will also be assessed through the output. There could be 360 reviews by co-workers and mentors which will determine if the worker will be invited to join the site or not. This could be outlined as being the "interview stage". People in the real world aren't paid for preparing for and attending interviews so why should workers online? Daemo doesn't have to accept just any old worker. Another option if we want to pay people for this it to make this a separate arm of Daemo or separate not-for-profit company that the profitable arm of Daemo funds. If the worker doesn't get accepted after the incubation stage, they will be told why, what skills they need to improve and what they need to do in order to develop those skills through learning pathways, for example, take a course, read an article, do an assignment etc. Also, the worker would then be told when they can re-apply.


- Daemo's internal exams/tasks for each skill where workers who pass can gain particular badges such as those I outlined in my evaluation of the Freelancer platform [2]

Other elements to Daemo's reputation system:

  1. referral system. Provide workers and requesters to refer people to different tasks. Providing these connections should work to built trust. If x worker is recommended by z worker and then he does a bad job, perhaps it could reflect in both worker x and worker y's reputation
  2. provide a mechanism whereby the tasks that requesters post are of a good quality so that both the requesters and workers expectations are aligned to begin with.
  3. Reputation based on algorithms - something like that Hidden Markov Model which estimates the latent and dynamic levels of expertise based on observed characteristics (educational background and certification, work history, feedback score (at each milestone and at the end), rehire rates, wage, how whether or not workers list the skills at hand on their profiles, at the time of application as outlined in The Utility of Skills in Online Labor Markets that @trygve reviewed.

This leads to my other "ONE idea" I listed on Meteor related to a focus on skills: Workers' and requesters' reputation ratings linked directly to each skill area rather than an aggregate rating for all the tasks incorporating various skills all bundled into one. [3]