Milestone 5 ZSpace Storyboard: xPERT :Make your HIT count

From crowdresearch
Jump to: navigation, search

File:XPERT.pdf

xPERT

Merge1 1.jpg Merge1 2.jpg Merge1 3.jpg Merge1 4.jpg Merge1 5.jpg Merge1 6.jpg Merge1 7.jpg Merge1 8.jpg

Current Situation

  • There is no mechanism for a worker to showcase his skills and get better opportunity as well as remuneration appropriate for his/her skill set.
  • At the same time a requester cannot differentiate between worker when the HIT requires some special skills.

Idea

  • A worker grading system can grade a worker under different categories like paragraph editing, image description or language translation, transcribing.
  • The grading shall be done based on the results of the previous HITs of the worker and the reviews provided by the requester.
  • The requester can choose a certain percentage of his HITs to be available to only highly rated workers and then subsequent HITs to be open for everyone to attempt. This might not be useful in surveys that require random representation but for other tasks it can ensure quality.

Possible Benefits

  • Every worker has a profile which mirrors his experience.
  • It gives experience workers better prospectus of good work. This might inspire the newbies to build a good profile and the experienced ones to maintain theirs.
  • Requester can decide the distribution of HITs and consequently remuneration among workers of different grade.

Possible Challenges

  • Newbies might find it difficult to get high paying work at first.

Solution: Certain percentage of HITs must be open to all workers.

  • If a Turker is good at attempting HITs but he/she is not regular at Mturk, this will be reflected in the profile. This might have adverse affect on both requester and Turker. As requester might miss on a good worker and a good worker might not have access to appropriate work.

Solution: The grading system should be sensitive to address the stated problem and keep record of active period.