WinterMilestone 1 Philanthrope

From crowdresearch
Revision as of 12:43, 17 January 2016 by Adityakumarakash (Talk | contribs) (Daemo)

Jump to: navigation, search

This is Milestone-1 submission of team Philanthrope. We are a team of two (@shyam.jvs @adityakumarakash), studying Computer Science at the Indian Institute of Technology Bombay. This page contains an essence of our work done in week 1.

Experience as a Worker on Mechanical Turk

We have done 1.5$ worth tasks as a worker on the Sandbox version of Amazon Mechanical Turk. Being out of the US, we faced issues getting ourselves registered on the actual Mturk and hence the sandboxed version. It took us around 3 hours to earn this money. The overall experience while using Mturk was pleasant except for a few shortcomings that we could notice. We talk about the +ve and -ve points of Mturk briefly below.

Positives

  • A smoothly running platform and a scalable solution for crowd-sourcing at large.
  • Simple and easy to understand user interface for someone who is new to the platform.
  • Allows searching and sorting of HITs based on reward amount, time alloted, expiration date and so on.
  • A system for requesting and obtaining qualifications which help add credibility to the worker.
  • Statistics to gauge performance like HIT submission rate, HIT acceptance rate, HIT rejection rate, etc.
  • Being a web service, it is pretty accessible from almost any smart device with an internet connection. Thus, power to earn lies at your finger tips.
  • On philosophical grounds, these tasks give a feeling of fulfillment to the workers and make them think that their time is being used constructively.

Negatives

  • No direct channel of communication between worker and employer. So no negotiation possible.
  • Tasks are typically paid less due to their morose nature which in turn lead to worker dissatisfaction. A feeling of not being compensated enough for the time and effort put in often arises. The worker might eventually give up considering that alternative jobs could be monetarily more fruitful.
  • Because responses are simply rejected in case they are not satisfactory, without any further feedback, workers do not have a chance to know what skills they lack on.
  • Employers might turn out to be fraudsters who take work but reject to pay, saying that the work was not satisfactory. Workers have no way to punish such fraudulent employers.
  • Communication among workers is also not present. This is a serious drawback because, important information like an employer being fraud, etc cannot be spread across workers. This has led to third party platforms like Turkopticon and Dynamo that aid communication.

Experience the life of a Requester on Mechanical Turk

Reflect on your experience as a requester on Mechanical Turk. What did you like? What did you dislike? Also attach the CSV file generated when you download the HIT results. If you're from outside the USA and unable to access MTurk, you can try the MTurk requester sandbox, or CrowdFlower or Microworkers or Clickworker

Explore alternative crowd-labor markets

Compare and contrast the crowd-labor market you just explored (TaskRabbit/oDesk/GalaxyZoo) to Mechanical Turk.

Readings

MobileWorks

Strengths of MobileWorks

  • Cheap way for human OCR
  • Simple interface which allows to get large user base without any training

Improvements / Suggestions

  • The platform could be simultaneously developed for smart phones eg. based on apps
  • Payments for tasks could be adjusted based on their difficulty
  • Choice between language and hand written characters and printed articles could be added
  • Data on how often subjects felt urge to use the system with time could help in increasing the excitement of the system

Daemo

Strengths of Daemo

  • Maintaining a balance in power by using representatives from both workers and requesters in a democratic fashion is really good
  • Prototyping the task and further discussion on task between requester and worker helps improve task specification and consistency between deliverable of worker and expectation of requester
  • Establishing trust between involved parties improves work quality

Suggestions

Flash Teams

Strengths of Flash Teams

  • Allows macro tasks based on engineering to be done effectively using crowd sourcing
  • Exploit the potential of crowd by allowing expert collaboration
  • Faster completion of the tasks as compared to traditional ways

Improvements

  • Management of users based on geographical regions would help better collaboration and facilitate meetings
  • Ownership of the work done would boost confidence in the system and maintain trust on other users
  • More sophisticated technique to assemble the experts, based on factors as working speed, geography, interests etc. would increase productivity

Milestone Contributors

Slack usernames of all who helped create this wiki page submission: @kessyena, @reneelin, @mahsa, @marim_g