Milestone 3 ZSpace PowerIdea 1: Result Submission
At present the result to a HIT is submitted to the requester directly who then has to sieve out the bad ones. This requires a lot of time and patience. It would be really helpful if the submission were directed through some system that partially verifies the result before it reaches the requester, saving his time and energy.
When the results are submitted, automated system can check for the correct results and rate them. This shall be really effective in case of HITs that require a definitive response. However this can be expanded to incorporate the HITs that require open-ended responses, by looking for certain keyword in the results.
This cannot be used for the HITs that focus on creativity or designing.
Same task can be assigned to more than one worker and a task is approved if same result is produced by majority of the workers assigned with the task. This system again works well with the HITs requiring well-defined answers like captcha recognition, mathematical expressions.
This approach assumes the democratic principle that the majority tends to be right. However this may not be the situation always. For example: In HITs requiring the Turker to translate a paragraph, the majority tends to use the less reliable but faster way of using Google Translate thus giving an answer which is wrong even though it is the majority one.
In iterative solution, the results of one Turker are assigned as task to another Turker who further improvises the task or rates the previously done task. This way ensures that the final result obtained by the requester has better quality. This process can work well where the above mentioned processes fail.
The problem occurs when a less experienced Turker is entrusted with task of improving more experienced Turker's task. There is a chances of the task being deteriorated rather than being improvised.
- The end result has a higher probability of have been attempted correctly.Thus, the requester has to reassign the HIT less often.
- The time in accessing the results is reduced for the requester.