Milestone 5 Improving Task Authoring with a Project Manager by Team1

From crowdresearch
Revision as of 23:06, 14 February 2016 by Kamilamananova (Talk | contribs)

Jump to: navigation, search

Task Authorship

Study introduction

Requesters have an influence on the outcomes of workers, so our hypothesis is that bad authorship leads to a lower outcome quality, because of the ambiguity of tasks and its following misunderstanding. In STUDY 1 we propose a one-part experiment, using a crowdsourcing platform for the experimental environment, where we will test Project Manager (PM) and compare the results in the form of Analysis of variance (ANOVA) results before and after.

By using the PM in our experiment, we show how the quality of delivered tasks is changing analyzing the rejection rate and input data.

Study method

Study 1 and all subsequent experiments reported in this paper were conducted using a proprietary microtasking platform that outsources crowd work to workers on the Amazon MechanicalTurk microtask market. We will restrict workers to those who do not have a "Master Qualification” to reach the more general skilled workers. A follow up survey will give us more insights on the demographics of the workers.

We introduce a new concept for crowdsourcing called "PM" (Project Manager), where PM is the person who is hired by requesters to be a supervisor of a group of workers to help create better tasks, finish a required task in a high quality and rating workers. Firstly, requester provides the task that he want to be done and hire workers to perform it, then the requester may choose the best worker from the hired workers to be PM or may hire him individually. After that, the PM's must provide the workers with details and describe them the needed job if the task description provided by requester is not clear enough. During the task performing, PM is monitoring the quality of work and should deliver the task to the requester on time and rate the workers. Here is a list of the responsibilities of the PM:

  • rating of workers
  • task setup (task description, configuration of patterns)
  • managing task fixing (checking delivered work)
  • Providing a template of a requested work task or similar task
  • writing a task feed list for a worker
  • implementation checking with a task feed list
  • provide information to the requester about implementation process, if needed
  • checking quality of the final work (implementation )

Method specifics and details

Experimental Design for the study

Measures from the study

What do we want to analyze?


@seko - Sekandar Matin, @purynova - Victoria Purynova, @ahmednasser - Ahmed Nasser, @kamila - Kamila Mananova