Milestone 4 Team Innovation 2 Results: Automated quality checks of submitted work

From crowdresearch
Jump to: navigation, search

Making Quality Checks Simpler

We propose an automated quality check system that clearly shows workers how to get their work approved.

First, the requester sets up the system for their task:

1. The requester clearly states the task requirements for the worker to read and understand. 2. The requester tells the system certain requirements must be met for work to be accepted.

Then, when a worker works on a task:

1. The worker submits their completed work. 2. The system checks to see if it meets the requester's parameters. 3. If it doesn't, the system highlights errors and asks the worker to resubmit, until the work meets the desired parameters.

Problems addressed by this idea

  • work quality control
  • requester time investment in checking work
  • worker time investment in correcting work
  • unfair rejections or ratings

How this idea solves problems

Automating the work approval system even in part saves time for everyone, and helps ensure that work submitted is of reasonable quality. It also hopefully heads off some unfair rejections and ratings before they can happen, since certain mistakes simply won't be accepted by the system.


Not all important qualities can be checked automatically, and work will still have to be reviewed by the requester by some qualities. The automated system may be clunky and refuse to accept good work if not designed intelligently, or if unreasonable parameters are used by the requester.

Idea inspired by

The ideas involving tasks being approved (by either workers or an artificial Turker) before they can go live; also, some current requesters use a system that auto-checks for certain qualities.