Milestone 0 Design

From crowdresearch
Jump to: navigation, search

This document is a collaboratively-edited "spec" for Milestone 0. Make it better!

The goal of Milestone 0 is to give requesters an idea of how to improve their task, and who to choose to complete it, in a way that holistically combines microtasks (short tasks) and macro tasks (long tasks).

How much?

After a requester describes the title of the task, we ask them to upload a dataset if they have one. If they’ve uploaded a dataset, we ask them to estimate how long each item will take and to indicate their availability??? (ARF). We then set the Milestone 0 limit to be the equivalent of ~10 minutes of work. Example: 2-minute task means 5 tasks is Milestone 0. If there is no dataset, we ask them to author a Milestone that is the smallest unit of work they can use to evaluate quality (e.g., a sketch if you’re asking for a logo, or your best portfolio item). If the requester cannot estimate the smallest unit of work or/ and the amount of time, they can refer to templates to estimate both of them. (Karolina)

They then set the maximum number of workers who do Milestone 0 before we show them.We recommend 3. They select the worker level they need (reputation), which guides who is able to submit to Milestone 0.


Any sufficiently qualified individual via the reputation requirements can submit Milestone 0, first come first serve, up to the maximum the user chose. The task has an open feedback textbox to send suggestions to the requester about how to improve the task design.

Possible issue with 1st come 1st serve I'm wondering if workers will feel pressure to be online all the time to be able to "snag" the tasks they want to do. The tasks will always go to the workers sitting there waiting to pounce on a task. If we have a calendar of availability for people on the platform, could there be a way of allocate workers to tasks to keep the workers from feeling like they always have to be online to get the good work? I'm just thinking of how I felt when the comments in our Slack channel kept getting archived; I felt an ever present sense of panic when not online checking the channels. Also, what are the mechanisms for a worker to grab the task - submitting the completed work for Milestone 0 or just by indicating "this one's mine" somehow and then, in the case of Macro-tasks, taking a bit more time to polish up and submit Milestone 0. (comments by ARF)


When Milestone 0 completes, we notify the requester.They look at the results. If this will be a large-scale (many person) task, they will likely want to make changes to the task design in response. If it's a small-scale task, it makes more sense to just message the workers and talk with them. Following this, the requester can either relaunch Milestone 0 (to iterate) or author the next milestone. Authoring the next milestone means qualifying some or all of the workers who did Milestone 0 to go ahead --- in other words, picking who you'll go ahead with which will also trigger a payment reminder to those who completed Milestone 0 (ARF).

Future work

Milestone 0 results can be used to create training data for onboarding new workers once the task launches to everyone.