WinterMilestone 2 zrehmani

From crowdresearch
Jump to: navigation, search

Research Engineering (Test Flight)

Congrats on getting setup locally!

Tutorials

By the end of this week, we expect everyone in the test flight group to be well acquainted with our codebase, and have familiarity with Angular, Django, and Material Design. Check out Introduction to Research Engineering for documentation and tutorials. If you choose to brush up on some of your skills with a tutorial (this is completely optional), please share a screenshot indicating the completion of your tutorial in #engineering-deliver on Slack by Sunday at 8pm PST.

Completed an AngularJS tutorial from CodeAcademy

Attend a Panel to Hear from Workers and Requesters

Deliverable

Report on some of the observations you gathered during the panel.

Essentially, a worker wants to be understood and recognized as an employee, with fair wages that are paid on time, and some sense of input, rather than being simply seen as machines that generate an output, machines that they can reject without realizing that these workersdepend on the work.

On the other hand, a requester wants high quality results that are delivered on time and fit the requester's expectations. At times, this may result in a dichotomy in which neither the requesters nor the workers reach their individual goals.

Reading Others' Insights

Worker perspective: Being a Turker

1) What observations about workers can you draw from the readings? Include any that may be are strongly implied but not explicit.

  • Workers are disadvantaged when it comes to a requester-worker relationship.
  • Workers are not able to communicate or rate requesters in the same way that requesters can.
  • Workers mostly use AMT for supplemental income.

2) What observations about requesters can you draw from the readings? Include any that may be are strongly implied but not explicit.

  • Requesters face a lot of mistrust from workers, especially if they do not have a good repertoire.
  • Requesters must be open to communication and very quick at accepting HITs if workers are to return to their HITs.

Worker perspective: Turkopticon

1) What observations about workers can you draw from the readings? Include any that may be are strongly implied but not explicit.

  • There is a variety of reasons people work as workers. For some, it may be to "kill time," and for others, it may be to put food on the table for them and their families
  • The idea of Humans-as-a-Service (HaaS) sounds much like Software-as-a-Service (SaaS), which I perceive as dehumanizing.
  • When workers complete a task, they do so without any guarantee that they will be paid, even if they put forth their full effort. Psychologically, this lack of guarantee may discourage workers from trying 110%.

2) What observations about requesters can you draw from the readings? Include any that may be are strongly implied but not explicit.

  • Requesters can be thieves, choosing not to pay the worker even if the worker completes the task to the requester's expectations.
  • Requesters have more autonomy than the workers; a 50/50 relationship may serve as a better relationship between the two parties to hold more accountability for both.

Requester perspective: Crowdsourcing User Studies with Mechanical Turk

1) What observations about workers can you draw from the readings? Include any that may be are strongly implied but not explicit.

  • Older HITs tend to be forgotten
  • Tasks with more detail tend to have higher quality results done by the workers.

2) What observations about requesters can you draw from the readings? Include any that may be are strongly implied but not explicit.

  • The more time that requesters spends detailing their tasks, the higher the quality they expect.
  • Requesters see Mechanical Turk as a means to complete large-scale tasks with little financial risk and time commitment.

Requester perspective: The Need for Standardization in Crowdsourcing

1) What observations about workers can you draw from the readings? Include any that may be are strongly implied but not explicit.

  • Workers need to learn the interface before starting, and often the interfaces are hard to use.
  • Workers might have trouble understanding the instructions given by a requester.
  • Workers cannot communicate directly with the requesters.

2) What observations about requesters can you draw from the readings? Include any that may be are strongly implied but not explicit.

  • Requesters are less concerned about reputation on AMT than workers are.
  • Requesters have concerns about the quality of work submitted.
  • Requesters have more control over HITs than workers.

Both perspectives: A Plea to Amazon: Fix Mechanical Turk

1) What observations about workers can you draw from the readings? Include any that may be are strongly implied but not explicit.

  • Workers cannot do anything about work that has been rejected
  • Workers are not able to communicate with requesters
  • Workers cannot rate requesters

2) What observations about requesters can you draw from the readings? Include any that may be are strongly implied but not explicit.

  • It's hard for requesters to distinguish between good and bad workers.
  • Requesters have to build their own interfaces if they want a good HIT
  • Requesters are not able to handle bad quality of work

Synthesize the Needs You Found

List out your most salient and interesting needs for workers, and for requesters. Please back up each one with evidence: at least one observation, and ideally an interpretation as well.

Worker Needs

A set of bullet points summarizing the needs of workers.

  • Workers need to be respected by their employers. Evidence: Sanjay said in the worker panel that he wrote an angry email to a requester who mass-rejected his work. Interpretation: this wasn't actually about the money; it was about the disregard for Sanjay's work ethic.
  • Workers want a higher guarantee that the hard work they put in will be rewarded with the wages that they deserve. Evidence: Turkopticon talks about wage theft, the process by which requesters can choose not to pay workers even if the workers deserve their wages. Interpretation: The requesters have much more autonomy/power in the worker/requester relationship.

Requester Needs

A set of bullet points summarizing the needs of requesters.

  • Requesters need to trust the results they get from workers. Evidence: In this thread on Reddit (linked), a requester is struggling to know which results to use and which ones to reject or re-post for more data. Interpretation: it's actually quite difficult for requesters to know whether 1) a worker tried hard but the question was unclear or very difficult or an edge case, or 2) a worker wasn't really putting in a best effort.
  • Requesters need to be held more accountable for rejections (maybe by limiting the number of rejections they can give?) Evidence: The wage theft suggested by the Turkoptikon. Interpretation: Workers and Requesters have to have a more equal relationship.
  • Requesters need to have more detailed task descriptions. Evidence: The Crowdsourcing User Studies Article mentions that tasks that are more detailed have better quality results. Interpretation: If requesters put in work, the workers will put in work as well.