Milestone 4 ZSpace

From crowdresearch
Jump to: navigation, search

Design Axes for 3 Themes

TASK CLARIFICATION

  • Similarities :
  1. Designing better HITs is a bottleneck for both worker and requester.
  2. Most solutions look for worker feedback on HIT designs. They can indicate how good a HIT design was in form of upvotes.
  • Differences :
  1. Some solutions focus on gradual improvement through feedback and upvotes, others involve intermediary systems like artificial turker for task filtering before it is posted.
  2. There is need for good searching tools that help turkers search for tasks that match their skill-set. For this the solutions propose recommender systems that utilize that match their interests and skills to the available HITs.

RESULT

  • Similarities :
  1. To get good results submissions need to be reviewed before they reach the requester.
  • Differences :
  1. One solution proposes peer evaluation where same task is reviewed by turkers on same level. This is not very useful in complex tasks.
  2. Other solution is to develop a responsibility hierarchy similar to corporate world. This can be useful to handle complex tasks and the requester can contact and hold responsible only a few managers who are on the top of the pyramid.

REPUTATION

  • Similarities :
  1. Transparency is vital in building reputation for both worker and requester. A worker works with assurance if he/she whom they are working for. Requester would also like their work to be done by workers with a good record and required skills.
  2. There is a need to differentiate workers based on their skills and performance.
  3. Requester should be able to exercise control on who accepts his HITs and thus control quality of submissions.
  • Differences :
  1. Workers can be differentiated on the basis of their skill-set or performance in tasks.
  2. Higher reputation does not mean higher pay in both the solutions but it ensures higher work.

List of Ideas and the Theme they belong to

TASK CLARIFICATION

Metahit

RESULT

Parallel and iterative

REPUTATION

Milestone 4 ZSpace Reputation: SKYNET A HIT-Turker Matching System