Milestone 4 pixelpals

From crowdresearch
Jump to: navigation, search

Submission of milestone 4


For each of the three themes you've chosen, talk with your team about:

  • What do these ideas tend to have in common?
  • How do these ideas tend to differ?

Use your insights from this conversation to generate design axes like we discussed in last week's meeting (see slide notes if you'd like). Write up a short paragraph (2-4 sentences) for each theme summarizing the similarities, differences, and axes inspired.


Similarity between Resolution Ideas: 1. There is a need of a moderating system/redressing system to solve worker-requesters disputes. 2. Since money is the major concern of the disputes. The output of the system would affect the workers or requester or the system in terms of money.

Differences: 1. How the moderating system is formed: (i). could comprise of committee of pre-determined moderators nominated in some way. (ii). Committee of moderators which comprises of unbiased requesters and workers not associated with the task and is formed every time for a dispute. This ensures that the moderator committee is not fixed and their judgements remains unbiased. (iii). Automated system: doesn’t consists of any moderator. Instead the rating given by the workers to the requesters and ratings by requesters to the workers would decide whether the worker/requester could be trusted or not. (iv). Talking directly to the requester to solve the dispute. 2. Resolution of Money: (i).If worker is wrong in the dispute, his amount of money from the task goes to the requester. If requester is found wrong, he pays the requester in addition to paying to the worker. (ii). Moderators to be paid fixed amount of money from the one which loses the dispute. (iii). Fixed Fine to be paid by the one who loses the dispute. If worker loses the dispute, he has to pay fine also in addition to paying the moderator. Same thing happens for requester also. Ax.jpg


Theme Results: How might workers+requesters work together to produce higher-quality results?


The solutions involve extra levels of review to ensure good quality. The extra review causes costs to platform or reviewers. The cost increment is justified by the trade-off between cost and higher quality results produces thereof.

Differences: The solutions vary in terms of who is responsible for the extra review task – peer workers, requester or third part experts/managers? The level of interactions required between workers and requester vary with the type of model. Some models work on collaborative systems, others separate worker/requester portions definitely. Some models involve improving review system over time with experience and feedback. Others present a fixed, ready-to-implement system. Problem: Workers with specialised skills might want to solve only higher level problems to get higher pay but not get the chance since they are new to the platform. At the same time, workers with novice skills might be able to only efficiently do part of the problem.

Idea: To facilitate faster shift to high pay, high skill tasks for new workers: Division of larger tasks into two types of subtasks: one with moderate skill requirement and one with expert skill requirement. Once users solve the easy subtask with high quality results, only then will they allowed to move to task with higher skill requirement. This ensures that workers are not rated only according to their time spent on the platform rather more on their suitability to the task at hand. To simplify the review portion for requesters, some tasks can have a prerequisite of solving ‘x’ number of similar tasks to be eligible for doing the task. ‘x’ will be a nominal number that can be crossed in a span of 1 week.

Task Clarity


  • Net result coming out of the user should be desired result expected by requester.
  • Workers need to upvote whether tasks clarity is there or not.


  • Creating tasks which are easily adaptable to the workers.
  • Tricks to maximize the workers output could be different.

Problem : Difficulty for workers to understand the task and judge whether they have sufficient skills to do the task. Solution: Designing of task on the basis of usability.

Idea : Use Human Computer Interaction tools to create proper tasks

  • Do Task Analysis of your tasks that is, get the tasks solved by large group of peope.
  • Analyse to what set of people are able to do task nicely.
  • Create multiple versions of task and do Between-group experiment and In-group experiment both to understand more about usability aspects
  • Ask changes in tasks from user.
  • Make those changes and again do task analysis.
  • Finalize it and describe clearly the crux of task and required skill set.

List of Ideas and the Theme they belong to

Provide them in whatever format you want - diagrams, sketches, table, or a combination.

Milestone 4 pixelpals Resolution: Revaluation System

Milestone 4 pixelpals Results:faster shift to high pay

Milestone 4 pixelpals Task Clarity:Use Human Computer Interaction tools to create proper tasks