Milestone 2 cder

From crowdresearch
Revision as of 12:54, 11 March 2015 by Aditimithal (Talk | contribs)

Jump to: navigation, search

Template for your submission for Milestone 2. Do not edit this directly - instead, make a new page at Milestone 2 YourTeamName or whatever your team name is, and copy this template over. You can view the source of this page by clicking the Edit button at the top-right of this page, or by clicking here.

Attend a Panel to Hear from Workers and Requesters

Deliverable

Report on some of the observations you gathered during the panel.

Reading Others' Insights

Worker perspective: Being a Turker

1) What observations about workers can you draw from the readings? Include any that may be are strongly implied but not explicit.

2) What observations about requesters can you draw from the readings? Include any that may be are strongly implied but not explicit.

Worker perspective: Turkopticon

1) What observations about workers can you draw from the readings? Include any that may be are strongly implied but not explicit.

2) What observations about requesters can you draw from the readings? Include any that may be are strongly implied but not explicit.

Requester perspective: Crowdsourcing User Studies with Mechanical Turk

1) What observations about workers can you draw from the readings? Include any that may be are strongly implied but not explicit.

2) What observations about requesters can you draw from the readings? Include any that may be are strongly implied but not explicit.

Requester perspective: The Need for Standardization in Crowdsourcing

1) What observations about workers can you draw from the readings? Include any that may be are strongly implied but not explicit.

2) What observations about requesters can you draw from the readings? Include any that may be are strongly implied but not explicit.

Both perspectives: A Plea to Amazon: Fix Mechanical Turk

1) What observations about workers can you draw from the readings? Include any that may be are strongly implied but not explicit.

2) What observations about requesters can you draw from the readings? Include any that may be are strongly implied but not explicit.

Do Needfinding by Browsing MTurk-related forums, blogs, Reddit, etc

List out the observations you made while doing your fieldwork. Links to examples (posts / threads) would be extremely helpful.

Many interesting observations were made regarding the needs of the workers and requesters :

MTurk forums are really great for observation, they provide good insights into the workers and requesters need. The workers and requesters get a platform to interact on the forums which proves beneficial.

1)Worker Needs:

a) Workers need an MTurk functionality which would help them subscribe to preferred requesters, categorize HITs ,get notifications for the same and be able to search the site to find a specific requester.

b) Workers need to be patient and develop trust in terms of payment transactions.

c) Workers need to learn the intricacies of the interface for each requester.

d)Workers need to trust the requesters and the HITs posted by them.


2) Requester Needs:


a) Requester needs to set the 'Time Allotted' limit for HITs to an amount of time much longer than the expected amount of time.

b) Requesters need to be clear about bonuses and be interactive with the Workers.

c) A requester needs to use a consent/intro page or paragraphs for his/her HIT request.

d) A requester needs to implement a custom qualification to grade workers and assign complexity to the HIT.

e)Requesters need to implement the “best practices” for each type of work. For example, the designing part)

f) Requesters need to understand the “Lifetime Approval Rate”

g) Requesters need to understand rejections and 'hardblocks.'

h) Every Requester needs to price its work unit without knowing the conditions of the market .


Synthesize the Needs You Found

List out your most salient and interesting needs for workers, and for requesters. Please back up each one with evidence: at least one observation, and ideally an interpretation as well.

Worker Needs

1. Example: Workers need an MTurk functionality which would help them subscribe to preferred requesters and get notifications for the same. Interpretation: this will increase the efficient on the worker’s side (efforts) as they get to choose whom they wish to work for and It will also save time . Evidence:Many people upvoted for this suggestion on the link provided in the previous section.

2. Workers need an MTurk feature to enable searching the website to find out a specific requester or the hit count. Interpretation:This feature would save time and effort, it would help a worker choose a specific kind of task and help him keep a record of his/her HITS. Evidence:Many people upvoted for this suggestion on the link provided in the previous section.

3. Workers need to be able to separate hits based on different categories . Interpretation: Category based HITs help a worker work efficiently and maximize his output. Evidence:In many forums it was a suggestion offered by many workers.

4. Workers need to be patient and develop trust in terms of payment transactions. Interpretation: The lack of trust arises due to the payment suspension of many Mturk workers. Since MTurk is quite a big and reputed crowd-sourcing platform it would definitely deal with this payment error and rectify it ASAP. Evidence: In this thread(http://www.reddit.com/r/mturk/comments/2yj04z/the_account_has_been_unsuspended_so_that_you_may/ ) the requester is having payment issues since his account has been suspended.


5. Workers need to adapt to the requirements of each requester and handle the intricacies. Interpretation: For ease of gaining maximum HITs the worker needs to comply with the working criteria and standards or the requester. Evidence: Since we know that experienced MTurks are likely to earn better wages on Mturk than the inexperienced , it highlights the fact that those people are aware of how to interact with their each requester and are comfortable working in their environment.

Requester Needs

A set of bullet points summarizing the needs of requesters.

1. Requester needs to set the 'Time Allotted' limit for HITs to an amount of time much longer than the expected amount of time. Interpretation: Time duration provided is an important factor in determining the number of workers one wants to get within a specific interval of time. It attracts more workers and the requester earns points.Evidence:http://www.mturkgrind.com/threads/lifetime-approval-rate-misunderstood-by-requesters.27458/ With MTurk support the requester s were able to resolve the problems caused to the workers due to rejection.

2. Requesters need to understand the “Lifetime Approval Rate” Interpretation:When the workers made a HIT for this requester, and it wasn’t approved for some duration , the requester claimed it to be "0% (0/0)" which was much less than the targeted HIT rate margin (90%) set by the requester. Evidence: http://www.mturkgrind.com/threads/lifetime-approval-rate-misunderstood-by-requesters.27458/ With MTurk support the requester s were able to resolve the problems caused to the workers due to rejection.

3. Requesters need to be clear about bonuses. Interpretation: If a bonus is offered,the potential amount (or range of possible amounts) and how to earn it, and how soon workers should expect it to be paid must be mentioned .

4. A requester needs to implement a custom qualification to grade workers and assign complexity to the HIT. Interpretation: This avoids time wastage as unnecessary extra HITS don’t need to be done.Plus, one can avoid mass rejections which can kill a requester’s reputation.

5. Requesters need to implement the “best practices” for each type of work. For example, the designing part) Interpretation: To attract more workers the designing part chosen by the requester should be well enough and legible.

6. Every Requester needs to price its work unit without knowing the conditions of the market . Interpretation: In order to get the work done the price limit to fetch workers needs to be set.

7. Requesters need to understand rejections and 'hardblocks.' Interpretation: Requesters reject a worker as they never intended to pay. Or even when the requester set up the hit incorrectly. The workers are even hardblocked without a reason and are sent an Amazon letter stating that another attempt may lead to jeopardy, nothing much can be done about it. Evidence:http://www.mturkgrind.com/threads/guideline-for-academic-requesters-on-mturk.26327/ (master’s reply)