Milestone 5 TuringMachine Mockup: CrowdSourced HITs creation using Task Design Dashboard & Recommendation System

Author

Neil Gaikwad

Influence and Related work

MTurk task design templates are limited in functionality and flexibility. It has often seen that the ambitious instructions and confusing task flow affects workers productivity. The Future of Work, Kittur et.al. 2013, authors highlight one of the complains: “Too often the job itself is badly designed or is messed up and there is a degree of misunderstanding between the worker and the job engineer.” Furthermore, the paper elaborates 12 different key aspects of crowd-computing design. One of them Crowd Work Process focuses on How might we design the workflow and control the quality of work?

Past research has shown that the lack of skills, misunderstanding, and irresponsible behaviors are some of the main causes of low quality submissions. Various techniques can be used to increase the quality of work. Templates can be one of the possible solutions to design clear instructions. The PixelPerfect team has proposed an excellent idea of new template framework. Figure below highlights the some of these techniques. The paper Shepherding the crowd yields better work, Dow et.al. 2012 proposed an idea of Rubric that can be helpful for quality measurement.

Link to the Storyboard

CrowdSourced HITs creation using Task Design Dashboard

Problem Design, Setting, & Environment

Sequence Thumbnails

Task Design Dashboard: Classic Theme
Task Design Dashboard: Task Categorization & Selection
Task Design Dashboard: Drag & Drop Task Design
Task Design Dashboard: Automated Code generation & customization
Task Design Dashboard: Expert Workforce Management
Task Design Dashboard: Task & Trend Analytics
Task Design Dashboard: Dark Theme
Scenario Storyboard

Task Design Dashboard: Classic Theme

Design the task using a web browser or iPad: Task specific Drag & Drop Controls with the Task Design Board

Task Design Dashboard: Task Categorization & Selection

Lego Blocks for designing the complex crowdsourcing tasks Crowd sourcing systems can be designed to solve large number of problems. The eco-system can involve tasks and work that require different kind of expertise and skills, please see Genomes of Collective Intelligence Framework, Malone et.al 2010. These tasks can be broken down into various categories as shown in the figure below