Milestone 2 crowdshrimp

From crowdresearch
Revision as of 22:47, 11 March 2015 by Kristinehoang (Talk | contribs)

Jump to: navigation, search

Needs Observed from Panels

By Kristine Hoang

Workers need: to more easily find HITs on AMT; to reach out to other Turkers and learn from them; to find a community to belong to; to make money at a certain rate; to make money while setting one's own schedule; to easily perform tasks using clear instructions; to avoid unpaid time; to find reliable, reoccurring work; to easily organize their workday tasks (which calls for a more intuitive interface); to acquire tools that can get them more easily started with AMT; to more easily contact requestors

Requestors need: to save money when assigning tasks; to find workers who won't bail on them or perform poorly; to check that workers are following instructions and rules

Readings

Being a Turker By Kristine Hoang

This reading contains analyses showing how Turker Nation members act as economic actors. It also analyzes Turkers as a form of "invisible work" using an "undefined network of people," as Wired Magazine called them.

So, what do Turkers turk for? It is for money or enjoyment and pastime? How do they view AMT -- is a reliable, more permanent source of income or something to support them temporarily? Do they like that they can control their own hours and don't have to worry about extraneous costs that come with a 9-5 office job, like paying for gas and lunch with co-workers?

This reading shows that the design of AMT supports the needs of Requestors over Turkers. There is a clear imbalance of power in which Turkers can be rated but Requestors cannot. This imbalance may be why Turkers need to have a support system within forums: they need to see where they stand with other Turkers.

1) What observations about workers can you draw from the readings? Workers need: to make money, pay for rent; to set their own schedule; to set income goals; to find others to empathize with and who can support them.

2) What observations bout requesters can you draw from the readings?

Requestors need: to get the work done in a timely manner; to save as much money as possible.

Turkopticon: Interrupting Worker Invisibility in Amazon Mechanical Turk

By Kristine Hoang

This paper argues that human computation currently relies on worker invisibility. This is similar to the role Yelp plays in restaurants, offering a checks and balances system where reviews and ratings keep restaurants in check with quality and customer service. This is what Turkopticon tries to solve as it allows workers to publicize and evaluate their relationships with employers. It allows workers to create and use reviews of employers when choosing employers on AMT.

1) What observations about workers can you draw from the readings? Workers need: to have their voices heard; to rate requestors; to make sure they get paid fairly

2) What observations bout requesters can you draw from the readings?

Requestors need: (similarly as above reading) to get the work done in a timely manner; to save as much money as possible.

Crowdsourcing User Studies with Mechanical Turk

My main question from this reading: micro-task = micro-payment, is this okay?

In this reading, the authors ask how requestors can identify "malicious users" -- that is, users who would "game" the system and provide nonsense answers in order to decrease time spent on the task and increase their rate of pay. For requestors, the authors argue that they need to have verifiable questions as part of the task and multiple ways to suspect responses.

1) What observations about workers can you draw from the readings? Workers need: to work less but make more money

2) What observations bout requesters can you draw from the readings?

Requestors need: to identify malicious users; to provide verifiable tasks before tasks begin

The Need for Standardization in Crowdsourcing

By Kristine Hoang

This paper argues that standardization can provide more equal opportunities for AMT workers. Workers should be trained like they've signed a contract with a regular 9-5 track even if it is remote. Job screening and training give an incentive for performance, and bad performance results in large consequences (losing one's job).

So how can we have more structure in AMT marketplace? AMT still deals with pricing work, predicting completion time, gaining quality, and the lack of ease in searching for jobs; this paper argues that more structure can fix these things.

The paper notes that one interesting solution offered is the thriving "curated garden," where firms recruit and train workers for standardized tasks, resulting in more scalability and cost savings.


1) What observations about workers can you draw from the readings? Workers need: to be treated like real employees; to have a checks and balances system to protect them from bad requestors; to more easily find jobs; to get paid a fair wage for an equivalent task

2) What observations bout requesters can you draw from the readings?

Requestors need: to find a structured way to price work, to have more standardized completion time expectations, and a standardized way to measure quality

A Place to Amazon: Fix Mechanical Turk! By Kristine Hoang

How can we establish a true reputation for AMT workers? This is the question of this paper. An interesting point I read here was: "When workers cannot differentiate easily good from bad workers, they tend to assume that every worker is bad." It seems like requestors are concerned about getting quality work with the amount of money they offer, so why not create a way they could know what they're getting? Low wages will make good workers leave the market, filling it up with bad ones.

1) What observations about workers can you draw from the readings? Workers need: to be able to trust requestors; to keep track of the work they're doing; to rate requestors

2) What observations bout requesters can you draw from the readings?

Requestors need: to post task with more ease