Milestone 2 C2C

From crowdresearch
Jump to: navigation, search

Attend a Panel to Hear from Workers and Requesters

Deliverable

Report on some of the observations you gathered during the panel.

Worker Perspectives:

  • It is possible to do the minimum wage with your right skills and patient.
  • It seems to be a lot more numbers of low paying requesters.
  • Workers seem to avoid single-penny tasks.
  • It is very hard to find good work due to a wide range of jobs and unclear descriptions.
  • There is a learning curve to understand what HITs work for you.
  • There are a lot of tips and tricks on how to use MTurk.
  • It’s painless since the approved payments are guaranteed. (but the signup process is painful.)
  • A masters qualification requirement affects the amount of work available.
  • Scams consistently pop up. This perhaps means that the masters qualification doesn’t help screening the scams well.
  • Workers would want to have a skill test like O-Desk to prove their qualifications.

Requester Perspectives:

  • Since anyone can be a requester, there are lots of different types of jobs posted (e.g. image tagging vs. web research).
  • Requesters include detailed descriptions to help guide workers (e.g. Wikipedia featured article criteria)
  • Requesters set an arbitrary price when submitting the task.
  • Requesters look for ways the workers game their submissions and require additional work to try to screen fraudulent workers.
  • Some requesters break complex (difficult + time consuming) tasks into a few micro tasks.
  • Amazon will only take a cut of money from the requester’s end (not the worker’s).
  • New HITs are always posted at the peak time of the day (morning, lunch break, evening).
  • Time of the week (weekend, weekday) or time of the year (holidays, break, etc.) also have an impact on the amount of responses.
  • The payment also varies depended on the time of the year (lower in Summer).
  • Due to the large amount of workers, requesters tend to reply to workers slowly (or don’t even reply).
  • Requesters are not able to select a certain group of workers that matches the tasks.

Reading Others' Insights

Worker perspective: Being a Turker

1) What observations about workers can you draw from the readings? Include any that may be are strongly implied but not explicit.

  • Job sorting system is complicated and ineffective
  • The best incentive is money.
  • Many workers have full-time job and work on this as a part-timer.

2) What observations about requesters can you draw from the readings? Include any that may be are strongly implied but not explicit.

  • Unfair paid. (Task level VS money)

Worker perspective: Turkopticon

1) What observations about workers can you draw from the readings? Include any that may be are strongly implied but not explicit.

  • Rejection from requesters are only give negative feedbacks. It affects reputation and might cost money.
  • Hard to know if the person does his/her best in the job.

2) What observations about requesters can you draw from the readings? Include any that may be are strongly implied but not explicit.

  • There is a big gap in communication between requesters and workers.
  • Large amount of workers make things hard for requesters to interact and prioritize each individual worker.

Requester perspective: The Need for Standardization in Crowdsourcing

1) What observations about workers can you draw from the readings? Include any that may be are strongly implied but not explicit.

  • Workers have different backgrounds and experiences.
  • There is no effective way to rate good/bad workers.
  • The experience and approval rate cannot be used to guarantee the worker.

2) What observations about requesters can you draw from the readings? Include any that may be are strongly implied but not explicit.

  • Requesters seems to be at advantage on standardization of task.
  • Gaming system is difficult but inevitable.

Both perspectives: A Plea to Amazon: Fix Mechanical Turk

1) What observations about workers can you draw from the readings? Include any that may be are strongly implied but not explicit.

  • Workers cannot contact the requesters directly unless they get a permission.

2) What observations about requesters can you draw from the readings? Include any that may be are strongly implied but not explicit.

  • All workers get paid equally no matter they are good or bad workers.
  • Hard to access the APIs.

Synthesize the Needs You Found

Worker Needs

Workers needs “HIGH PAYMENT” = Right tasks. Right rewards.

Evidence: All workers are looking for the same thing, low effort & high return tasks. (Low effort tasks depends on the worker’s preferences and experiences.)

Interpretation: It is obvious that money is the best incentive for workers. However, the biggest obstacle is an ineffective communication between workers and requesters (e.g. uninformative descriptions or task), and between system and workers (e.g. complicated task sorting.)

Requester Needs

Requesters need “QUALITY RESULTS”. = Right time. Right targets. Right results.

Evidence: All requesters are looking for reliable results from qualified targets by paying the lowest amount of money.

Interpretation: “WHEN” and “HOW” to post the right HITs have a significantly impact on the results. The better understanding of worker in terms of qualifications (experience, academic, etc.) and behaviors (working time, job preferences, etc.) could lead to better results.

Platform Needs

Both workers and requesters are looking for an interactive platform that helps them learn more about each other better. The platform supposed to be a two-way communication that provides different kind of information needed for both sides. This should help eliminate the existing issues and provide a fair trade both sides, ideally, it should be as simple as “the better quality tasks means higher payment”.