WinterMilestone 2 stormsurfer

From crowdresearch
Revision as of 14:56, 24 January 2016 by Shreygupta (Talk | contribs)

Jump to: navigation, search

Attend a panel to hear from workers and requesters

Report on some of the observations you gathered during the panel.

Reading others' insights

Worker perspective: Being a Turker

  • Crowdsourcing (definition): the act of a company or institution taking a function once performed by employees and outsourcing it to an undefined (and generally large) network of people in the form of an open call.
  • What observations about workers can you draw from the readings? Include any that may be strongly implied but not explicit.
    • 80% of the tasks are carried out by the 20% most active Turkers.
    • The majority of Turkers (56%) are U.S. based, but there is a growing number of Indian Turkers (36%) and other nationalities, and nearly one-third of respondents had a median annual income of <$10,000.
    • For posters on Turker Nation, the primary reason for Turking is to earn money (money comes up in the discussions the most). All of the other benefits of Turking are simply side-benefits (Turkers also enjoy doing tasks that are fun, interesting, and/or educational). Some Turkers accept a slightly lower pay if a task is more enjoyable, but most dismiss the idea.
    • Turkers seem to be responsible for promoting fair pay in the system (MTurk).
    • Highest earnings for the most experienced Turkers are approximately $15,000 per year (full-time job), and the minimum is $50, although the researchers do not know how long the Turkers worked. Some workers are on MTurk as a full-time job, whereas others only spend a few hours. Those who only spend a few hours are often not doing well financially either; they use their earnings from MTurk as a supplement to their regular salary ("hand-to-mouth"). Many Turkers are living under difficult circumstances. Not all individuals prefer MTurk and are only Turking as a consequence of the economy; they would much rather prefer to have their salary back. (However, it is important to note that a $15k salary would be considered good in several countries outside of the U.S., including India.)
    • Turkers are interested in their earning potential and set targets for themselves (e.g. $10/day).
    • Primary use of Turker Nation (as well as other forums) is to rate requesters as bad/good with respect to fair pay, approval/rejection of tasks, and comments (which can be demeaning). Consequences of low approval ratings (<90% prevents workers from accessing certain tasks) and blocking (large amounts can cause a ban from MTurk) simply prevent them from earning money, and they feel insulted. Workers value communication; they dislike when a requester rejects their work without a valid reason. They value communication, and positive worker-requester relationships can develop. However, workers generally accept a rejection on a HIT when there is a valid fault in their work. On forums, they are asked to be objective in their approvals/disapprovals of requesters and refrain from posting too quickly (flaming) when their HITs are rejected.
    • A pay rate of $4-6/hour is seen by workers as an acceptable rate and is, in fact, higher than the average pay rate for many tasks on MTurk (yet lower than the minimum wage in the U.S.). However, workers understand the tradeoffs and benefits of working on MTurk. Benefits for Turkers: set hours are not required, money does not have to be spent on transportation costs, and judgments are restricted to the work submitted rather than personal appearance.
    • Invisible work: workers must spend time searching for tasks and often accept poorer paying jobs as an interim means to the bigger goal of better paying (and often more interesting) work.
    • Interestingly, while workers want to adhere to Amazon’s Terms of Service, they do not want the U.S. government to legislate and regulate MTurk. They believe in collective individual actions and believe that if the government intervened, it would lead to a closing of MTurk, which acts as a safety net for many Turkers. Some distrust the government; furthermore, they do not want other individuals making decisions for them.
    • Problems for Turkers: employers who don’t pay, identifying scams, and the cost (to workers) of poorly designed tasks.
    • Problems with MTurk: information asymmetry and the imbalance of power (workers cannot rate/block requesters).
  • What observations about requesters can you draw from the readings? Include any that may be strongly implied but not explicit.
    • Several requesters do improve their relationship with workers, and positive worker-requester relationships can develop over time. Requesters can improve over time (e.g. paying faster, being more specific in their tasks) as they gain feedback.
    • Some requesters outright mass reject tasks and give demeaning comments, but this is not always the case; many times, there is a reason to reject a task.
    • Problems with MTurk: information asymmetry and the imbalance of power (requesters can rate/block workers).

Worker perspective: Turkopticon

  • What observations about workers can you draw from the readings? Include any that may be strongly implied but not explicit.
  • What observations about requesters can you draw from the readings? Include any that may be strongly implied but not explicit.

Requester perspective: Crowdsourcing User Studies with Mechanical Turk

  • What observations about workers can you draw from the readings? Include any that may be strongly implied but not explicit.
  • What observations about requesters can you draw from the readings? Include any that may be strongly implied but not explicit.

Requester perspective: The Need for Standardization in Crowdsourcing

  • What observations about workers can you draw from the readings? Include any that may be strongly implied but not explicit.
  • What observations about requesters can you draw from the readings? Include any that may be strongly implied but not explicit.

Both perspectives: A Plea to Amazon: Fix Mechanical Turk

  • What observations about workers can you draw from the readings? Include any that may be strongly implied but not explicit.
  • What observations about requesters can you draw from the readings? Include any that may be strongly implied but not explicit.

Needfinding by browsing MTurk-related forums, blogs, Reddit, etc.

List out the observations you made while doing your fieldwork. Links to examples (posts/threads) would be extremely helpful.

Synthesis

List out your most salient and interesting needs for workers and requesters. Please back up each one with evidence: at least one observation, and ideally an interpretation as well.

Worker needs

Requester needs

Milestone contributors

Slack usernames of all who helped create this wiki page submission: @shreygupta98