Milestone 4 pentagram Results: Peer Review System

From crowdresearch
Jump to: navigation, search

Problem Statement


The quality of the submitted HIT is questionable or does not meet expectations of the requester

Solution


This problem is addressed by involving both the requesters and workers in making decision on the quality of the submitted HIT i.e. a peer-reviewing system in addition to the existing mechanism(where the requester reviews HITs)

Description of solution


Flow Diagram

Trust2 peer review system.png

Notes

  • Based on the reputation of the worker, the submitted HITs are reviewed for quality
  • The review policy : High reputation workers will review HITs submitted by low reputation workers and vice-versa
  • In addition the reviews can be personally monitored by the requester
  • The reviewers may also be paid a small percentage of the original wage assigned for the task