Summer Milestone 11 Freelancer reputation system evaluation by AngelaRichmondFuller

From crowdresearch
Jump to: navigation, search


Reviewed by @arichmondfuller (WORK in PROGRESS)

Summary of the reputation system

  • On Freelancer both requesters and workers rate each other and leave comments regarding performance by giving 1 to 5 star ratings and written comments as soon as a project or contest is completed and payment has been turned over.
  • Each project page will show a list of bidders that requesters can choose from, with freelancer ratings and bid amounts available at a glance.
  • They give each other 1-5 star ratings related to quality and timeliness.
  • View profile page containing worker reputation elements at
  • Workers profile pages show a 1 to 5 star average rating for all completed projects along with the average price they charge per hour.
  • Workers can outline their education, certificates and qualifications on their profile page.
  • Workers can collect badges to highlight their commitment and skills such as completed 100 projects, made over $1000, provided 500 bits of feedback in the forum, passed English Exam Level 3, etc.
  • Workers can also take exams provided by Freelancer to demonstrate various skills. These exams such as Freelancer Orientation, English Level 3, Numeracy Level 1, Employer Orientation, Worker Orientation, etc. are viewable on the profile page.
  • View this posted task to see elements of requester reputation at The posted tasks show the requesters' 1 to 5 star average ratings along with how many tasks they have posted. You can click through for further details. It also shows how many projects were completed and how many reviews given.
  • Requesters are rated on: Clarity in Specification; Communication; Payment Promptness; Professionalism; Would Work for Again
  • There is also an indication of whether or not the requester's payment method is verified.

Identify what is working and what is failing on the crowdsourcing platform

  • Star ratings are inflated
  • Workers often resort to begging requesters for 5 star ratings at the end of projects.
  • Requesters often withhold payment for tasks.
  • Several scammers operate on this site if we believe the reviews identified on a quick google search.
  • More work needs to be done to verify requester accounts
  • Often worker accounts are a company that has a group of workers who do the work. The star rating may not tell the full story if it's actually several different people doing the work.
  • Because copies of the chat aren't saved by the system, workers often say things that bully the requester and then are immediately erased so there is no evidence of the demands. I imagine this happens the other way as well.

Find inequality between how requesters and workers are treated by the system

  • Because payment at milestones is optional, workers are often uncertain as to whether or not they will be paid in full at the end of a task.
  • Requesters are subject to begging for 5 star ratings upon completion of a task.
  • Workers bully requesters at the beginning of tasks asking for more than 50% paid up front before any work is done.
  • Because the milestone payment process is not formalised on the platform novice requesters at times feel that workers bully them into more payment than deserved being paid up front.