Milestone 2 crowdshrimp

From crowdresearch
Jump to: navigation, search

Needs Observed from Panels

By Kristine Hoang

Workers need: to more easily find HITs on AMT; to reach out to other Turkers and learn from them; to find a community to belong to; to make money at a certain rate; to make money while setting one's own schedule; to easily perform tasks using clear instructions; to avoid unpaid time; to find reliable, reoccurring work; to easily organize their workday tasks (which calls for a more intuitive interface); to acquire tools that can get them more easily started with AMT; to more easily contact requestors

Requestors need: to save money when assigning tasks; to find workers who won't bail on them or perform poorly; to check that workers are following instructions and rules

Readings

Being a Turker By Kristine Hoang

This reading contains analyses showing how Turker Nation members act as economic actors. It also analyzes Turkers as a form of "invisible work" using an "undefined network of people," as Wired Magazine called them.

So, what do Turkers turk for? It is for money or enjoyment and pastime? How do they view AMT -- is it a reliable, more permanent source of income or something to support them temporarily? Do they like that they can control their own hours and don't have to worry about extraneous costs that come with a 9-5 office job, like paying for gas and lunch with co-workers?

This reading shows that the design of AMT supports the needs of Requestors over Turkers. There is a clear imbalance of power in which Turkers can be rated but Requestors cannot. This imbalance may be why Turkers need to have a support system within forums: they need to see where they stand with other Turkers.

1) What observations about workers can you draw from the readings? Workers need: to make money, pay for rent; to set their own schedule; to set income goals; to find others to empathize with and who can support them.

2) What observations bout requesters can you draw from the readings?

Requestors need: to get the work done in a timely manner; to save as much money as possible.

Turkopticon: Interrupting Worker Invisibility in Amazon Mechanical Turk

By Kristine Hoang

This paper argues that human computation currently relies on worker invisibility. The idea of Turkopticon is similar to the role Yelp plays in restaurants, offering a checks and balances system where reviews and ratings keep restaurants in check with quality and customer service. This is what Turkopticon tries to solve as it allows workers to publicize and evaluate their relationships with employers. It allows workers to create and use reviews of employers when choosing employers on AMT.

1) What observations about workers can you draw from the readings? Workers need: to have their voices heard; to rate requestors; to make sure they get paid fairly

2) What observations bout requesters can you draw from the readings?

Requestors need: (similarly as above reading) to get the work done in a timely manner; to save as much money as possible.

Crowdsourcing User Studies with Mechanical Turk

My main question from this reading: micro-task = micro-payment, is this okay?

In this reading, the authors ask how requestors can identify "malicious users" -- that is, users who would "game" the system and provide nonsense answers in order to decrease time spent on the task and increase their rate of pay. For requestors, the authors argue that they need to have verifiable questions as part of the task and multiple ways to suspect responses.

1) What observations about workers can you draw from the readings? Workers need: to work less but make more money

2) What observations bout requesters can you draw from the readings?

Requestors need: to identify malicious users; to provide verifiable tasks before tasks begin

The Need for Standardization in Crowdsourcing

By Kristine Hoang

This paper argues that standardization can provide more equal opportunities for AMT workers. Workers should be trained like they've signed a contract with a regular 9-5 track even if it is remote. Job screening and training give an incentive for performance, and bad performance results in large consequences (losing one's job).

So how can we have more structure in AMT marketplace? AMT still deals with pricing work, predicting completion time, gaining quality, and the lack of ease in searching for jobs; this paper argues that more structure can fix these things.

The paper notes that one interesting solution offered is the thriving "curated garden," where firms recruit and train workers for standardized tasks, resulting in more scalability and cost savings.


1) What observations about workers can you draw from the readings? Workers need: to be treated like real employees; to have a checks and balances system to protect them from bad requestors; to more easily find jobs; to get paid a fair wage for an equivalent task

2) What observations bout requesters can you draw from the readings?

Requestors need: to find a structured way to price work, to have more standardized completion time expectations, and a standardized way to measure quality

A Place to Amazon: Fix Mechanical Turk! By Kristine Hoang

How can we establish a true reputation for AMT workers? This is the question of this paper. An interesting point I read here was: "When workers cannot differentiate easily good from bad workers, they tend to assume that every worker is bad." It seems like requestors are concerned about getting quality work with the amount of money they offer, so why not create a way they could know what they're getting? Low wages will make good workers leave the market, filling it up with bad ones.

1) What observations about workers can you draw from the readings? Workers need: to be able to trust requestors; to keep track of the work they're doing; to rate requestors

2) What observations bout requesters can you draw from the readings?

Requestors need: to post task with more ease

Conclusion

By Kristine Hoang

Workers' Needs A checks and balances system that ensures they can trust requestors. Requestors can reject good work and not pay for it if they wish. Workers need to know when they're going to get paid; how others have rated requestors; to be able to filter out spam; to more easily find work that fits their skills and desired payment; to receive job training; to reach out to other Turkers and members of the community; to better communicate with requestors

David from the panel says, "it’s a lot of commiseration – the workplace and the requestors – seem very faceless. A lot of people get immediately discouraged when they have their first rejection or aren’t able to find money for multiple days in a row." This shows me that human relationships behind the computer screen are also a need.


Requestors' Needs Requestors need a standardized way to charge for tasks; to differentiate between good workers and bad workers; to hire high quality performers (though not all are willing to pay the price).

Serge from the panel says, "the main is figuring out which questions we can to detect cheating." This shows me that requestors are afraid that workers will cheat them as much as workers are afraid requestors will cheat them.

What I can conclude from this is that creating a crowdsourcing platform that puts a face to the community can help in creating a checks and balance system for both parties. Perhaps we can create profiles with real work experience (like LinkedIn) and both workers and requestors can receive ratings. Ratings can be based on several categories. For example, for workers they can be rated on timeliness and accuracy. Requestors can be rated on communication and clarity.