Milestone 0 Design

From crowdresearch
Revision as of 21:43, 24 June 2015 by Angelarichmondfuller (Talk | contribs) (Feedback)

Jump to: navigation, search

This document is a collaboratively-edited "spec" for Milestone 0. Make it better!

The goal of Milestone 0 is to give requesters an idea of how to improve their task, and who to choose to complete it, in a way that holistically combines microtasks (short tasks) and macro tasks (long tasks).

How much?

After a requester describes the title of the task, we ask them to upload a dataset if they have one. If they’ve uploaded a dataset, we ask them to estimate how long each item will take. We then set the Milestone 0 limit to be the equivalent of ~10 minutes of work. Example: 2-minute task means 5 tasks is Milestone 0. If there is no dataset, we ask them to author a Milestone that is the smallest unit of work they can use to evaluate quality (e.g., a sketch if you’re asking for a logo, or your best portfolio item).

They then set the maximum number of workers who do Milestone 0 before we show them.We recommend 3. They select the worker level they need (reputation), which guides who is able to submit to Milestone 0.

Launch!

Any sufficiently qualified individual via the reputation requirements can submit Milestone 0, first come first serve, up to the maximum the user chose. The task has an open feedback textbox to send suggestions to the requester about how to improve the task design.

Possible issue with 1st come 1st serve I'm wondering if workers will feel pressure to be online all the time to be able to "snag" the tasks they want to do. The tasks will always go to the workers sitting there waiting to pounce on a task. Is there some sort of way we could allocate workers to tasks to keep the workers from feeling like they always have to be online to get the good work? (comments by ARF)

Feedback

When Milestone 0 completes, we notify the requester.They look at the results. If this will be a large-scale (many person) task, they will likely want to make changes to the task design in response. If it's a small-scale task, it makes more sense to just message the workers and talk with them. Following this, the requester can either relaunch Milestone 0 (to iterate) or author the next milestone. Authoring the next milestone means qualifying some or all of the workers who did Milestone 0 to go ahead --- in other words, picking who you'll go ahead with which will also trigger payment to those who completed Milestone 0 (ARF).

Future work

Milestone 0 results can be used to create training data for onboarding new workers once the task launches to everyone.