Milestone 4 Results Testset

From crowdresearch
Jump to: navigation, search

Peer Review

Reading through the two ideas for the theme of Results, we came up with the following problems and solutions:


  • How can the reviewers be picked?
  • What are they paid?
  • How can their status stay anonymous?
  • Are they the same as other workers?


  • Maybe they have a reputation/given a reputation amount by the Requester
  • Both the requester and reviewer get rated
  • High reputation reviewers review within their own groups and low reputations workers in theirs (there are multiple groups of workers who are organised based on the reputation)
  • Payment can be managed as a separate amount instead of as a percentage of the HIT amount, because the reviewers probably have to apply as well to review a piece of work (and all workers have to be reviewers at some point)

What we realized was that both ideas were collectively emphasizing the notion of a peer reviewer, a moderator of sorts who manages the work of workers and tasks of requesters.