Milestone 3 ams TrustIdea 2: Checkpoint System

From crowdresearch
Jump to: navigation, search


  1. Setting checkpoints on the basis of HITs completed by a worker for a single requester, so that worker can be paid off regularly in batches
  2. Allowing requester to frequently keep a tab of the quality of work being submitted and keeping the good workers motivated by paying him/her instantly

Ams 3 Checkpoint system.jpg


Let us explain the system with the help of an example:

  1. A requesters uploads a work consisting of 1000 HITs
  2. The requester choose a number of HITS, say 50, as a base checkpoint, based on the HIT pay and time to complete one HIT
  3. A new worker must complete the number of HITs of the base checkpoint before he/she may continue with further HITs.
  4. Requester is notified of the completion of the checkpoint from the worker
  5. Based on the quality of the work, the requester can accept or decline the HITs
  6. The requester might also accept the HITs completed by the worker, so that he gets paid, but refuse to offer any further HITs to him/her
  7. If the requester accepts the work, he may assign 200 HITs (a random number assumed) for the next checkpoint of the accepted worker
  8. Once the worker completes these 200 HITs, the requester has to again verify and pay the worker for completing his checkpoint
  9. The requester can then assign more HITs (say 300) or less HITs (say 100) for the next checkpoint, based on the confidence of the requester in the worker. The requester may also attach a note explaining the change in the number of HITs for the next checkpoint.

Need for Checkpoints

Worker is Regularly Paid

  • Checkpoints will force a requester to pay outstanding wages before he can accept more results from a quality worker
  • Worker will be assured that he doesn't get 'mass rejected'
  • Worker will be paid immediately for his work

Requester can Manage Work Easily

  • Initially there might be an overhead of constantly acknowledging and paying off when a checkpoint is complete
  • Requester can easily identify a quality worker from a below average worker. This allows requesters to discontinue with workers whom they feel are not at par with the quality required, at an early stage. The requester will have a choice to accept the work from the checkpoint of a below average worker, so that the worker does not feel cheated, and then simply notify that worker that he is not allowed to continue. If the requester chooses to reject the work on the basis of quality or requirements, then the situation of mass rejection can be avoided as checkpoints will ensure that an average worker doesn't recklessly complete HITs only to get rejected later
  • Requesters can communicate inbetween checkpoints to clarify any doubts, criticize or appreciate the work to keep the worker motivated.

Additional Aspects

Multiple Workers

There is a high possibility that multiple workers are working on the same job. This might lead to depletion of HITs for each worker, which results in a situation where all the HITs of the job get finished but many or all workers working on that job might not complete enough HITs to reach their next checkpoint. In such a case, the requester will be simply notified of the work of all the workers, and he/she will review each worker and pay their outstanding wages accordingly