Difference between revisions of "Worker-Requester Panel: Meeting Notes by Dcthompson & Willtchiu"

From crowdresearch
Jump to: navigation, search
(Created page with " ---- '''Willtchiu's notes''' AMT Panel Notes: TurkerNation - platform that encourages ethical behaviour in workers, good at teaching new turkers the proper way of...")
 
Line 12: Line 12:
  
  
AMT Panel Notes:
+
TurkerNation:
 +
* platform that encourages ethical behaviour in workers, good at teaching new turkers the proper way of doing HITs
 +
* advises requesters on appropriate qualifications needed
  
TurkerNation - platform that encourages ethical behaviour in workers, good at teaching new turkers the proper way of doing HITs
+
'''Laura''' (worker)
advises requesters on appropriate qualifications needed
+
* Problem: HITs posted in TurkerNation chat room, sometimes not visible in threads.
 +
* On tackling HITs: Chooses HITs that do not risk approval rating (i.e. 99%)
 +
* Kept track of HITs, payment, and time it took.
 +
*#Problem: Gauging time spent on HITs are hard, because different people have different skill sets (i.e. some are slower, faster, more conscientious)
 +
*# Suggestion: Some sort of hourly wage prediction?
 +
* Problem: New worker’s reputation is very volatile, takes only 1 HIT to hurt < 100 HIT workers
 +
*# Idea: Streaks of accepted HITs give an extra boost to a worker’s reputation; makes it more forgiving if failing a HIT, yet more rewarding for consistently doing well
  
Laura       
 
Problem: HITs posted in TurkerNation chat room, sometimes not visible in threads.
 
On tackling HITs: Chooses HITs that do not risk approval rating (i.e. 99%)
 
Kept track of HITs, payment, and time it took.
 
Problem: Gauging time spent on HITs are hard, because different people have different skill sets (i.e. some are slower, faster, more conscientious)
 
Suggestion: Some sort of hourly wage prediction?
 
Problem: New worker’s reputation is very volatile, takes only 1 HIT to hurt < 100 HIT workers
 
Idea: Streaks of accepted HITs give an extra boost to a worker’s reputation; makes it more forgiving if failing a HIT, yet more rewarding for consistently doing well
 
  
Rochelle   Problem: Unpredictable when HITs are posted  
+
'''Rochelle'''
Prioritize higher paying HITs
+
*Problem: Unpredictable when HITs are posted  
        HITs are never scheduled out, edge of the seat waiting for HITs
+
*Prioritize higher paying HITs
        Need to make decisions between life and doing HITs
+
*HITs are never scheduled out, edge of the seat waiting for HITs
        HITs are posted at all times of the day
+
*Need to make decisions between life and doing HITs
alert groups that send out messages when new HITs are posted
+
*HITs are posted at all times of the day
Prioritizes worker approval ratings first, and requesters who have a good  reputation
+
*# alert groups that send out messages when new HITs are posted
Suggestion: Wants better interaction with requesters to clarify HITs
+
*Prioritizes worker approval ratings first, and requesters who have a good  reputation
Suggestion: People on forums give feedback on HITs; especially time estimates are important
+
*Suggestion: Wants better interaction with requesters to clarify HITs
Problem: Fluctuation of wages are very unpredictable; hard to budget income or spending because of this.
+
*Suggestion: People on forums give feedback on HITs; especially time estimates are important
Need: Tools similar to http://crowd-workers.com  
+
*Problem: Fluctuation of wages are very unpredictable; hard to budget income or spending because of this.
 +
*#Need: Tools similar to http://crowd-workers.com  
  
Chris       (Requester)
+
'''Chris''' (Requester)
        Workers are awesome at emailing requesters if HITs are unclear
+
* Workers are awesome at emailing requesters if HITs are unclear
 
Suggestion: It’ll be awesome to have more interaction between workers and requesters.
 
Suggestion: It’ll be awesome to have more interaction between workers and requesters.
 
Problem: When scaling HITs to much larger volume, requester’s time to respond to HITs is limited and they outsource, in effect, the job of answering Turkers’ emails.
 
Problem: When scaling HITs to much larger volume, requester’s time to respond to HITs is limited and they outsource, in effect, the job of answering Turkers’ emails.

Revision as of 00:11, 20 January 2016






Willtchiu's notes


TurkerNation:

  • platform that encourages ethical behaviour in workers, good at teaching new turkers the proper way of doing HITs
  • advises requesters on appropriate qualifications needed

Laura (worker)

  • Problem: HITs posted in TurkerNation chat room, sometimes not visible in threads.
  • On tackling HITs: Chooses HITs that do not risk approval rating (i.e. 99%)
  • Kept track of HITs, payment, and time it took.
    1. Problem: Gauging time spent on HITs are hard, because different people have different skill sets (i.e. some are slower, faster, more conscientious)
    2. Suggestion: Some sort of hourly wage prediction?
  • Problem: New worker’s reputation is very volatile, takes only 1 HIT to hurt < 100 HIT workers
    1. Idea: Streaks of accepted HITs give an extra boost to a worker’s reputation; makes it more forgiving if failing a HIT, yet more rewarding for consistently doing well


Rochelle

  • Problem: Unpredictable when HITs are posted
  • Prioritize higher paying HITs
  • HITs are never scheduled out, edge of the seat waiting for HITs
  • Need to make decisions between life and doing HITs
  • HITs are posted at all times of the day
    1. alert groups that send out messages when new HITs are posted
  • Prioritizes worker approval ratings first, and requesters who have a good reputation
  • Suggestion: Wants better interaction with requesters to clarify HITs
  • Suggestion: People on forums give feedback on HITs; especially time estimates are important
  • Problem: Fluctuation of wages are very unpredictable; hard to budget income or spending because of this.
    1. Need: Tools similar to http://crowd-workers.com

Chris (Requester)

  • Workers are awesome at emailing requesters if HITs are unclear

Suggestion: It’ll be awesome to have more interaction between workers and requesters. Problem: When scaling HITs to much larger volume, requester’s time to respond to HITs is limited and they outsource, in effect, the job of answering Turkers’ emails. Needs: Automatic judiciary system that will handle disputed cases for approval or disapproval. Idea: Turkers who have a very high approval rating can opt to be judges of cases of dispute http://crowd-workers.com (Tracks hourly wage, time completion, and HIT specifications)


Peter (Requester)

       Posted HITs on labels on bookshelves

Concern: We care about quality, and if a worker falls below a certain percentage of approval then we reject them. Need: A HIT inside an iframe, customizable interface for HITs

Xiao Posts survey type of HITs Problem: Hard to gauge attention span of survey takers, ensure truthful answers

SpamGirl

       Need: Built-in workflows capabilities