Milestone 2 sanjosespartans

From crowdresearch
Jump to: navigation, search

Attend a Monday Panel to Talk with Expert Workers and Requesters

Deliverable

When talking about needfinding, it is best practice to organize your thoughts into three stages:

  • Observations: What you see and hear
  • Interpretations: Why you think you are hearing and seeing those things. What is driving those behaviors? This is the "recursive why" we talked about in team meeting.
  • Needs: These are the deeper, more fundamental driving motivators for people. As we talked about in team meeting, needs must be verbs, not nouns.

For more detail about these, see this week's meeting slides.

The deliverable for the panel subsection: report on some of the observations you gathered during the panel. You can hold back on interpretations and needs until you finish the rest of the observation-gathering in the next steps.

We have mentioned some of the observations that we gathered during the discussion:

Worker:

1. Many of the HIT's are academic or need much expert knowledge.

2. The HIT that have high pay are actually tough or require prior knowledge about them,even though people try to complete them ,but still the acceptance rate for the same are very less.

3.The tasks that are present on the AMT are essentially : Market Surveys,Research Tasks,Completion of questionnaires and some other tasks.

4. As per the discussions it was a common point mentioned that there is low sense of reliability among the Requesters and Workers.

5. The motive to earn money is the main source from the Workers point of view ,while they attempt to complete a HIT on the mTurk platform.

6. It helps them in getting an extra source of income without hindering their present jobs that they have and proves to be a booster for them.

7. They want to make use of their skills as in the case of oDesk so that they can get some freelance jobs as well.

8. Many platforms like AMT don't have proper channel of communication between the Worker and Requester ,so in case of any clarification or doubts there is no medium of interaction.

9. There are some workers who are working for longer period of time still they are not able to get that much money as some others are able to make while being new joinees and are able to get HIT that offer them more money.

10. Money,ethics and task at hand should be interesting are some of the things that a Worker looks into while attempting a HIT.

11. Many times it is also the case that a Pro Worker helps a newbie in getting to know the platform and helps out in his process of exploration and thereby help in earning good money.Any valuable guidance given by an experienced person helps the newbie to get a good understanding and he/she comes to know the trade practices and makes a rough list of do's and dont's that should be followed while attempting HIT's and proves to be a good learning experience.


Requester:

1. They don't have the authority to select workers according to their own choice for a particular task.

2. They have the problem that the tasks that were given out ,were they done seriously? And they don't have any assurance as to the task that they have submitted would be completed or not,so uncertainty prevails at their end also.

3. They generally try to avoid those workers whose demographic data is not correct.

4. They rarely reject the work as this proves to create a negative impression on the Worker's profile.This makes the approval rate not so useful also.

5. They have some problems related to the geography that in countries outside USA,some Workers can't attempt the HIT ,so pool of Worker's available to them is narrowed down in this case,and they have to make use of available pool only without any guarantee or assurance that their task would be completed and the level of perfection needed in the task would be achieved or not.

6. They prefer breaking down a big task into smallest possible task and then giving HIT for the same.

7. Some personal level tasks that they have are difficult to change them into turkable tasks.

8. If a HIT that they have submitted for completion is not responded well ,so they can then get a feedback from Worker's through forums discussion as to if it's correct or not and if the pricing and level of importance of the task are correct or not.

9. They sometimes feel that giving rating to the Worker's is not always a preferred option as at the end of the day a Worker is only concerned about the rating and not with the relationship developed with Requester or the knowledge that is gained by completing the task.Sometimes a Requester might think that some much better output can be achieved from a person ,so can give him a low rating but this hampers the relationship as its the rating that always plays an important role int he system.

10. Requesters would love to have a system where they can get some trivial tasks done by Worker's and on the basis of outcome of the results they can handpick their own set of worker's and then assign them some priority tasks as they feel it helps them to know a bit about the background of the Worker,the needs,aspiration and the capability of the person and then assign them a task suitable according to his skill set and caliber.

Reading Others' Insights

An hour in the library can save a year of fieldwork. Please read the following materials. These papers will also grow out our foundation of related work. The papers are copyrighted, so please don't redistribute them. You need to be signed in to view them.

Deliverables

1) What observations about workers can you draw from the readings? Include any that may be are strongly implied but not explicit.

1. Weighs interestingness of a task. (Can sometimes settle for a comparatively low paying task given its enjoyable).

2. Encounters difficulties finding relevant tasks. Spends a lot of time in search of a suitable task, pay rate, requester reputation to make an informed decision.

3. Suffers from information asymmetry, have limited resources to rate the requesters.

4. Assesses a task’s riskiness by evaluating requester’s responsiveness, payment history, task evaluation (Majority Rules).

5. Have a hope/desire/motivation to earn more.

6. Some still prefer a stable job over turking.

7. Stands vulnerable to fluctuating ratings, denigrate behavior/ slander, late payments.

8. Turkers share a supportive attitude amongst themselves. Turkers facing difficulties seek advice on turking forums. Share opinions about requesters.

9. Can sometimes lead to misleading haux unintentionally giving the impression of a scammer to the Requester.

10. Genuine feedbacks (good/bad) are appreciated.


2) What observations bout requesters can you draw from the readings? Include any that may be are strongly implied but not explicit.

1. No standard templates are available to set up a task. Requester has to design the task from scratch. This leads to non standard practices. Similar level task can be priced differently, have different quality requirements, etc. A Requester need not essentially be aware of what is the best approach to design a task.

2. Novices have to look at elsewhere on web to familiarize them with the Turking platform and to design HITS as no onboarding training is provided on AMT.

3. The reputation is subject to revision by workers based on his/her responsiveness, task quality assessment, payment frequency, etc.

4. Malicious user behaviour can lead to a lot of hassle for the requesters. The requester will have to waste his/her time and energy to find, review and remove sterile responses. The greater the number of responses, the higher is the risk of encountering such cases.

5. Faces difficulties in hand picking good workers.

6. Can be blacklabelled unduly.

7. Lack of good task prototypes can yield poor results. For e.g. A same question when asked in multiple choice form can create a difference in the output. Subjective tasks can be formulated in fill in the blank format.

Do Needfinding by Browsing MTurk-related forums, blogs, Reddit, etc

Deliverable

List out the observations you made while doing your fieldwork. Links to examples (posts / threads) would be extremely helpful.


Synthesize the Needs You Found

Deliverables

List out your most salient and interesting needs for workers, and for requesters. Please back up each one with evidence: at least one observation, and ideally an interpretation as well.

A set of bullet points summarizing the needs of workers.

  • Example: Workers need to be respected by their employers. Evidence: Sanjay said in the worker panel that he wrote an angry email to a requester who mass-rejected his work. Interpretation: this wasn't actually about the money; it was about the disregard for Sanjay's work ethic.

A set of bullet points summarizing the needs of requesters.

  • Example: requesters need to trust the results they get from workers. Evidence: In this thread on Reddit (linked), a requester is struggling to know which results to use and which ones to reject or re-post for more data. Interpretation: it's actually quite difficult for requesters to know whether 1) a worker tried hard but the question was unclear or very difficult or an edge case, or 2) a worker wasn't really putting in a best effort.