Difference between revisions of "WinterMilestone 2@anotherhuman"
From crowdresearch
Aarongilbee (Talk | contribs) |
Aarongilbee (Talk | contribs) |
||
(13 intermediate revisions by the same user not shown) | |||
Line 4: | Line 4: | ||
# What observations about workers can you draw from the readings? Include any that may be are strongly implied but not explicit. | # What observations about workers can you draw from the readings? Include any that may be are strongly implied but not explicit. | ||
# What observations about requesters can you draw from the readings? Include any that may be are strongly implied but not explicit. | # What observations about requesters can you draw from the readings? Include any that may be are strongly implied but not explicit. | ||
+ | |||
+ | ==Stolee, K. & Elbaum, S. (2010) Observations about Requestors== | ||
+ | |||
+ | #Requestor monitored the updates of tasks completion. | ||
+ | #Requestors devise methods to cull user diversity, user experience, and worker gaming of the system. | ||
+ | #Requestors devised routes to gather information that is normally anonymized by the turk system. | ||
+ | #"open ended answers helped us to understand points of confusion and why participants differed" | ||
+ | #"was the result of a misinterpreted question" | ||
+ | #requestors used the UI to approve work completed and access the results. | ||
+ | # a credit card is required to front load the requestor account.********** | ||
+ | # hit creation can be tested in the developer sandbox | ||
+ | # requestors find that Turk "provides a framework ... for recruiting, ensuring privacy, distributing payment, and collecting results." | ||
+ | #"results can be easily downloaded in a CSV format" | ||
+ | #Requestors "create custom qualification tests .. using the command line tool or API" | ||
+ | #Requestor who is a surveyor understand "the importance of having enough subjects (i.e. workers) of the right kind." | ||
+ | #Requestor "doubled the initial ... reward" | ||
+ | #requestor "sent emails to two internal mailing lists." | ||
+ | #Requestor might "observe students... instead of observing software engineers practicing." | ||
+ | #Requestor might "perform studies without human subjects." [bad practice] | ||
+ | #Requestor might "evaluate visualization designs, conduct surveys about information seeking behaviors, and perform NL annotations to train machine learning algorithms." | ||
+ | #Requestor might "leverage a global community... to solve a problem, classify data, refine a product, gather feedback" | ||
+ | #Requestor required "to pass a pretest." | ||
+ | #Researchers "estimated aptitude by measuring education and qualification score." | ||
+ | #Researchers create qualifications for works by using domain specific knowledge and quality of work history. | ||
+ | #Requestors evaluate work after completion. | ||
+ | #Resquestors made task templates and combined tasks with a shared type ID. | ||
+ | #Resquestors "presented [workers] with treated or untreated pipe for each task." | ||
+ | #Resquestors "could not impose their constraint and control for learning effects." | ||
+ | #Turk "caused us to waste some data." | ||
+ | #"An alternate [research] design would be to create..." | ||
+ | # requestors define the work goals, collect relevant information from the workers | ||
+ | # requestors "had less control over the [workers] participating... and variations caused by how prominently the study is displayed in the infrastructure search results." | ||
+ | #"even our study uses tasks that are much more complex and time consuming than those recommended by" Turk | ||
+ | #researchers "must consider if randomized assignment... is appropriate for their study" | ||
+ | |||
+ | ==Martin, D. & et al. (2014) Observations about Requestors== | ||
+ | |||
+ | #to request work such as image tagging, duplicate recognition, transcription, translation, object classification, and generate content | ||
+ | #rely on turk to curate and manage the quality of content for their tasks | ||
+ | #requesters becomes confused about what actions constitute a bad worker [one man's trash is another's treasure] | ||
+ | #requestors block who they consider bad workers | ||
+ | #requestors fundamentally ask for help from the masses and a judgment from the asker is fundamentally a dynamic that is highly disrespectful | ||
+ | |||
+ | ==Irani, L. & Silberman, M. (2013) Observations about Requestors== | ||
+ | #the best requesters use turk to complete large batches of micro-tasks | ||
+ | #requestors are not ask who, what, or where the workers come from [false?] | ||
+ | #requesters utilize multiple avenues to assess "workers" | ||
+ | #requesters create form fields for data entry | ||
+ | #requesters upload audio for transcription | ||
+ | #requestors create requirements for data entry to address worker quality issues | ||
+ | #requestors define the structure of data entry | ||
+ | #requestors create instructions for data entry | ||
+ | #requestors specify the pool of information to be processed | ||
+ | #requestors define the criteria for work acceptance such as approval rate, country of origin, and skill specific mastery | ||
+ | #requestors recruit thousands of workers within hours | ||
+ | #requestors maintain intellectual property rights | ||
+ | #requestors vet worker outputs through algorithms (majority rule) | ||
+ | #requestors avoid responding to workers due to quantity | ||
+ | #requestors only respond to workers when things happen en masse | ||
+ | #requesters act as business people | ||
+ | #requester shape the interaction with the crowd | ||
+ | #requestors pay Amazon money | ||
+ | #requestors review workers mutually | ||
+ | #requestors have to address the work of people from multiple nations | ||
+ | |||
+ | ==Irani, L. & Silberman, M. (2013) Observations about Workers== | ||
+ | #workers utilize screen names across many platforms | ||
+ | #workers will report their experiences with a requestor | ||
+ | #workers self evaluate their own work | ||
+ | #workers check (status/alert function) for approval and payment status for submitted work | ||
+ | #workers tolerate what they see on amazon turk and express outrage that requestors pay for the service without appropriate management | ||
+ | #workers respond with dispute messages | ||
+ | #workers convene as a mass to report problems to requestors (HIVE) | ||
+ | #workers give up intellectual property rights | ||
+ | #workers test their mturk task related skill sets | ||
+ | #workers respond to the quality of the data entry structure | ||
+ | #workers interpret instructions for data entry | ||
+ | #workers translate the information to be processed into data entry inputs | ||
+ | #workers read requirements beyond the scope of the intent of turk | ||
+ | #workers transcribe audio into form fields | ||
+ | #workers complete fields into requestor forms | ||
+ | #workers utilize multiple windows on the same screen | ||
+ | #workers utilize multiple tabs on the same browser | ||
+ | #workers forget the importance of ergonomics, rest, repetitive stress injuries | ||
+ | #turkers may not have learned about minimum wage laws | ||
+ | #turkers express 3 kinds of responses to turk: some do it for fun,cure boredom, or earn income[!!!!!turker types] | ||
+ | #turkers usually expect money from tasks | ||
+ | #workers see tasks posted from outside MTurk (?) | ||
+ | |||
+ | ==Stolee, K. & Elbaum, S. (2010) Observations about Workers== | ||
+ | #Workers might "select and configure predefined modules and connecting them." | ||
+ | #Workers try to avoid the search page and complete tasks. | ||
+ | #Workers see the qualifications but might not see the specifications of the requestor. | ||
+ | #Workers identify tasks that are of similar types to match their preferences. | ||
+ | #Workers "discover Hits by searching based on some criteria, such as titles, descriptions, keywords, reward or expiration date." | ||
+ | |||
+ | ==Martin, D. & et al. (2014) Observations about Workers== | ||
+ | #"view AMT as a labor market" | ||
+ | #"unfair rejection of work" | ||
+ | #"to receive pay for work" | ||
+ | #communicate with others through Turk | ||
+ | #workers identify scams | ||
+ | #workers move through poorly designed tasks | ||
+ | #workers develop relationships with requestors | ||
+ | #workers seek some form of relational reciprocity with requestors | ||
+ | #workers gather to collect information about tasks, the platform, and requestors | ||
+ | #workers protect their hall of fame and shame post at http://turkernation.com/forumdisplay.php?13-Requesters-Hall-of-Fame-Shame | ||
+ | #workers find work that they're happy with despite pay rate | ||
+ | #workers discuss money and methods to earn it best | ||
+ | #workers talk about fun, learning and play as a major reason for joining MTurk | ||
+ | #workers earn cash on the Mturk system | ||
+ | #workers contrasts tasks upon a play/pay continuum | ||
+ | #workers criticize the pay attitude in the forums | ||
+ | #workers rely upon MTurk to accentuate cash flow when real world work stops (The purpose for turk changes) | ||
+ | #workers select only the "best" oportunities for pay | ||
+ | #turkers compete with one another and ask questions regard what others earn | ||
+ | #turkers set their own targets | ||
+ | #turkers respond to external events in their lives and adjust how they interact with turk based from those events | ||
+ | #turkers schedule and allot certain times of the day to be on MTurk | ||
+ | #workers rely on it as a source of income -- partially because mturk is available, accessible, and easy to find work due to its requestor diversity | ||
+ | #workers might use turk as a breadline | ||
+ | #workers find mturk ideal because one doesn't have to consider the professional environment and transportation concerns | ||
+ | #workers avoid those requestors who are demeaning and practice mass rejection | ||
+ | #workers compare experiences | ||
+ | #workers seek out requestors based from responsiveness | ||
+ | #workers give positive and negative badges | ||
+ | #workers spend time searching for jobs | ||
+ | #workers need access to decent work | ||
+ | #workers avoid being blocked by requestors | ||
+ | #workers design HITs with requestors | ||
+ | #workers self-monitor communication practices with requestors | ||
+ | #workers expect quick pay | ||
+ | #workers sample tasks to test the requestor | ||
+ | #workers bag several hits from one requestor | ||
+ | #workers will work on several hits in multiple tabs in a browser | ||
+ | #workers examine how quickly a requestor responds to questions | ||
+ | #workers avoid majority rules grading practices -- probably used in ML labeling tasks | ||
+ | #turkers use a consensus scheme to assess requestors in the forums | ||
+ | #turkers follow rules of requestor good practice | ||
+ | #turkers seek to know how often a requestor is online, quick to responsive he is to a task, how polite the person is | ||
+ | #turkers base trust upon several dimesions - competence of the requestor, concern of the requestor, and the intergrity (consistency) of interactions with the requestor | ||
+ | #some turkers scam, others try to solve this scamming through social governance | ||
+ | #in threads inidividuals are accused of cheating qualifications | ||
+ | #turkers suffer from fatigue ("i was not paying enough attention) | ||
+ | #turker practice reciprocity with requestors | ||
+ | #turkers label individuals as flamers | ||
+ | #turkers can be overly sensitive to a rejection | ||
+ | |||
+ | #workers might loss work due to a bad connection if work is saved on the cloud. | ||
+ | |||
+ | ==TurkNation Bonus Observations about Requestors== | ||
+ | #requesters set automatic acceptance of hits after a certain period of time | ||
+ | #surprise bonuses create questions for workers | ||
+ | |||
+ | ==TurkNation Bonus Observations about Workers== | ||
+ | [http://bit.ly/1OLgqmC] | ||
+ | #workers wait at most 30 days for hit approval as set by turk policy | ||
+ | #workers define one form of hope with approval and time | ||
+ | [http://bit.ly/1VdPfVx] | ||
+ | #workers identify tasks that earn bonuses, especially if the bonus occur frequently | ||
+ | #turkers provide scripts for others to use in requestor specific situations | ||
+ | [http://bit.ly/1RDnXtz] | ||
+ | #new turkers demonstrate misunderstandings of the Turk system and may be unfairly | ||
+ | [bit.ly/1OCaX3H] | ||
+ | #turkers inherit the problems of bureaucracy without ever knowing how the system changes | ||
+ | |||
+ | |||
+ | |||
Need-finding | Need-finding | ||
− | + | Martin, D. & et al. (2014) | |
− | + | #"how to motivate better, cheaper, and faster performance... without paying much" | |
− | + | Stolee, K. & Elbaum, S. (2010) | |
− | + | # to eliminate worker variability | |
− | + | # to control variability | |
− | + | # researchers need "some understanding of the system capabilities and constraints" | |
− | + | # the system needs to direct to other services in the business space that are more apt for the task at hand. | |
− | + | # requestors need to have the ability to control the presentation of their work, by priority, sequence, preference, iterative, random, importance... | |
− | + | ||
− | + | Needs identified during the worker-requestor interviews | |
− | + | # to identify patterns of requestors | |
− | + | # to be able to receive work immediately | |
− | + | # to be be able to pause work demands | |
− | + | # to be able to SLEEP | |
− | + | # to meet a daily goal (how dos this daily goal shape decision making) | |
− | + | # to quickly test a hypothesis | |
− | + | # to write clear instructions | |
− | + | # to post a batch of questions | |
− | + | # to receive information on how to improve task design | |
− | + | # to engaging in conversation with workers | |
− | + | # to how to monitor approval ratings | |
− | + | # to gauge the level of threat a requestor is towards approval ratings | |
− | + | # to scatter work across requestors | |
− | + | # to select those who provide HITs with good background | |
− | + | # to have a name and email to contact people before HITs | |
− | + | # to post hits without much interaction | |
− | + | # to know people are on the other side | |
− | + | # to label X of this item | |
− | + | # to optimize worker speed in job design an shape UI | |
− | + | # to avoid rejecting workers | |
− | + | # to maintain quality assurance of workers | |
− | + | # to send out sample hits to test task | |
− | + | # to gauge preferences of requestors | |
− | + | # to be able to manage interactions with workers at scale | |
− | + | # to manage worker correspondence | |
− | + | # to rate workers fairly | |
− | + | # to automatically clarify and challenge qualification based rejection | |
− | + | # to meet face to face with others | |
− | + | # to manage time better as quantity grows (the challenges and methods change) | |
− | + | # to connect with outside services that expand one's professional capacities | |
− | + | # to have strong best-worker relationships (10 batches) | |
− | + | # to pre-schedule work at intervals | |
− | + | # to assure quality, truthful and honest responses | |
− | + | # to connect with worker communities | |
− | + | # to have a hit within an iFrame | |
− | + | # to be able to build one's own efforts | |
− | + | # to look at the hit and estimate the value of effort... (paper: Estimating Charlie's Run-time Estimator) | |
− | + | # to manage the expectations of work involved and run times tied to HITs | |
− | + | # to know the other worker's personalities and how it affects work | |
− | + | # to communicate with others the reality of a job | |
− | + | # to match job type with work preferences and skill sets | |
− | + | # to earn as much profit (revenue) as possible | |
− | + | # to match typing tasks with those who like typing tasks | |
+ | # to match button clickers with button clicking tasks | ||
+ | # to estimate the value of a day | ||
+ | # to have a buffer of work demands over time | ||
+ | # to avoid random responses | ||
+ | # to accept as work that is better than chance | ||
+ | # to avoid rejections on the reputation system | ||
+ | # to have a constant contact ability with the requestor | ||
+ | # to have a buffer of points for reputation management | ||
+ | # to have buffer time that accommodate interruptions and disabilities | ||
incredible enablers of scientific research | incredible enablers of scientific research |
Latest revision as of 09:51, 22 January 2016
Observations
- What observations about workers can you draw from the interview? Include any that may be are strongly implied but not explicit.
- What observations about requesters can you draw from the interview? Include any that may be are strongly implied but not explicit.
- What observations about workers can you draw from the readings? Include any that may be are strongly implied but not explicit.
- What observations about requesters can you draw from the readings? Include any that may be are strongly implied but not explicit.
Contents
- 1 Stolee, K. & Elbaum, S. (2010) Observations about Requestors
- 2 Martin, D. & et al. (2014) Observations about Requestors
- 3 Irani, L. & Silberman, M. (2013) Observations about Requestors
- 4 Irani, L. & Silberman, M. (2013) Observations about Workers
- 5 Stolee, K. & Elbaum, S. (2010) Observations about Workers
- 6 Martin, D. & et al. (2014) Observations about Workers
- 7 TurkNation Bonus Observations about Requestors
- 8 TurkNation Bonus Observations about Workers
Stolee, K. & Elbaum, S. (2010) Observations about Requestors
- Requestor monitored the updates of tasks completion.
- Requestors devise methods to cull user diversity, user experience, and worker gaming of the system.
- Requestors devised routes to gather information that is normally anonymized by the turk system.
- "open ended answers helped us to understand points of confusion and why participants differed"
- "was the result of a misinterpreted question"
- requestors used the UI to approve work completed and access the results.
- a credit card is required to front load the requestor account.**********
- hit creation can be tested in the developer sandbox
- requestors find that Turk "provides a framework ... for recruiting, ensuring privacy, distributing payment, and collecting results."
- "results can be easily downloaded in a CSV format"
- Requestors "create custom qualification tests .. using the command line tool or API"
- Requestor who is a surveyor understand "the importance of having enough subjects (i.e. workers) of the right kind."
- Requestor "doubled the initial ... reward"
- requestor "sent emails to two internal mailing lists."
- Requestor might "observe students... instead of observing software engineers practicing."
- Requestor might "perform studies without human subjects." [bad practice]
- Requestor might "evaluate visualization designs, conduct surveys about information seeking behaviors, and perform NL annotations to train machine learning algorithms."
- Requestor might "leverage a global community... to solve a problem, classify data, refine a product, gather feedback"
- Requestor required "to pass a pretest."
- Researchers "estimated aptitude by measuring education and qualification score."
- Researchers create qualifications for works by using domain specific knowledge and quality of work history.
- Requestors evaluate work after completion.
- Resquestors made task templates and combined tasks with a shared type ID.
- Resquestors "presented [workers] with treated or untreated pipe for each task."
- Resquestors "could not impose their constraint and control for learning effects."
- Turk "caused us to waste some data."
- "An alternate [research] design would be to create..."
- requestors define the work goals, collect relevant information from the workers
- requestors "had less control over the [workers] participating... and variations caused by how prominently the study is displayed in the infrastructure search results."
- "even our study uses tasks that are much more complex and time consuming than those recommended by" Turk
- researchers "must consider if randomized assignment... is appropriate for their study"
Martin, D. & et al. (2014) Observations about Requestors
- to request work such as image tagging, duplicate recognition, transcription, translation, object classification, and generate content
- rely on turk to curate and manage the quality of content for their tasks
- requesters becomes confused about what actions constitute a bad worker [one man's trash is another's treasure]
- requestors block who they consider bad workers
- requestors fundamentally ask for help from the masses and a judgment from the asker is fundamentally a dynamic that is highly disrespectful
Irani, L. & Silberman, M. (2013) Observations about Requestors
- the best requesters use turk to complete large batches of micro-tasks
- requestors are not ask who, what, or where the workers come from [false?]
- requesters utilize multiple avenues to assess "workers"
- requesters create form fields for data entry
- requesters upload audio for transcription
- requestors create requirements for data entry to address worker quality issues
- requestors define the structure of data entry
- requestors create instructions for data entry
- requestors specify the pool of information to be processed
- requestors define the criteria for work acceptance such as approval rate, country of origin, and skill specific mastery
- requestors recruit thousands of workers within hours
- requestors maintain intellectual property rights
- requestors vet worker outputs through algorithms (majority rule)
- requestors avoid responding to workers due to quantity
- requestors only respond to workers when things happen en masse
- requesters act as business people
- requester shape the interaction with the crowd
- requestors pay Amazon money
- requestors review workers mutually
- requestors have to address the work of people from multiple nations
Irani, L. & Silberman, M. (2013) Observations about Workers
- workers utilize screen names across many platforms
- workers will report their experiences with a requestor
- workers self evaluate their own work
- workers check (status/alert function) for approval and payment status for submitted work
- workers tolerate what they see on amazon turk and express outrage that requestors pay for the service without appropriate management
- workers respond with dispute messages
- workers convene as a mass to report problems to requestors (HIVE)
- workers give up intellectual property rights
- workers test their mturk task related skill sets
- workers respond to the quality of the data entry structure
- workers interpret instructions for data entry
- workers translate the information to be processed into data entry inputs
- workers read requirements beyond the scope of the intent of turk
- workers transcribe audio into form fields
- workers complete fields into requestor forms
- workers utilize multiple windows on the same screen
- workers utilize multiple tabs on the same browser
- workers forget the importance of ergonomics, rest, repetitive stress injuries
- turkers may not have learned about minimum wage laws
- turkers express 3 kinds of responses to turk: some do it for fun,cure boredom, or earn income[!!!!!turker types]
- turkers usually expect money from tasks
- workers see tasks posted from outside MTurk (?)
Stolee, K. & Elbaum, S. (2010) Observations about Workers
- Workers might "select and configure predefined modules and connecting them."
- Workers try to avoid the search page and complete tasks.
- Workers see the qualifications but might not see the specifications of the requestor.
- Workers identify tasks that are of similar types to match their preferences.
- Workers "discover Hits by searching based on some criteria, such as titles, descriptions, keywords, reward or expiration date."
Martin, D. & et al. (2014) Observations about Workers
- "view AMT as a labor market"
- "unfair rejection of work"
- "to receive pay for work"
- communicate with others through Turk
- workers identify scams
- workers move through poorly designed tasks
- workers develop relationships with requestors
- workers seek some form of relational reciprocity with requestors
- workers gather to collect information about tasks, the platform, and requestors
- workers protect their hall of fame and shame post at http://turkernation.com/forumdisplay.php?13-Requesters-Hall-of-Fame-Shame
- workers find work that they're happy with despite pay rate
- workers discuss money and methods to earn it best
- workers talk about fun, learning and play as a major reason for joining MTurk
- workers earn cash on the Mturk system
- workers contrasts tasks upon a play/pay continuum
- workers criticize the pay attitude in the forums
- workers rely upon MTurk to accentuate cash flow when real world work stops (The purpose for turk changes)
- workers select only the "best" oportunities for pay
- turkers compete with one another and ask questions regard what others earn
- turkers set their own targets
- turkers respond to external events in their lives and adjust how they interact with turk based from those events
- turkers schedule and allot certain times of the day to be on MTurk
- workers rely on it as a source of income -- partially because mturk is available, accessible, and easy to find work due to its requestor diversity
- workers might use turk as a breadline
- workers find mturk ideal because one doesn't have to consider the professional environment and transportation concerns
- workers avoid those requestors who are demeaning and practice mass rejection
- workers compare experiences
- workers seek out requestors based from responsiveness
- workers give positive and negative badges
- workers spend time searching for jobs
- workers need access to decent work
- workers avoid being blocked by requestors
- workers design HITs with requestors
- workers self-monitor communication practices with requestors
- workers expect quick pay
- workers sample tasks to test the requestor
- workers bag several hits from one requestor
- workers will work on several hits in multiple tabs in a browser
- workers examine how quickly a requestor responds to questions
- workers avoid majority rules grading practices -- probably used in ML labeling tasks
- turkers use a consensus scheme to assess requestors in the forums
- turkers follow rules of requestor good practice
- turkers seek to know how often a requestor is online, quick to responsive he is to a task, how polite the person is
- turkers base trust upon several dimesions - competence of the requestor, concern of the requestor, and the intergrity (consistency) of interactions with the requestor
- some turkers scam, others try to solve this scamming through social governance
- in threads inidividuals are accused of cheating qualifications
- turkers suffer from fatigue ("i was not paying enough attention)
- turker practice reciprocity with requestors
- turkers label individuals as flamers
- turkers can be overly sensitive to a rejection
- workers might loss work due to a bad connection if work is saved on the cloud.
TurkNation Bonus Observations about Requestors
- requesters set automatic acceptance of hits after a certain period of time
- surprise bonuses create questions for workers
TurkNation Bonus Observations about Workers
- workers wait at most 30 days for hit approval as set by turk policy
- workers define one form of hope with approval and time
- workers identify tasks that earn bonuses, especially if the bonus occur frequently
- turkers provide scripts for others to use in requestor specific situations
- new turkers demonstrate misunderstandings of the Turk system and may be unfairly
[bit.ly/1OCaX3H]
- turkers inherit the problems of bureaucracy without ever knowing how the system changes
Need-finding
Martin, D. & et al. (2014)
- "how to motivate better, cheaper, and faster performance... without paying much"
Stolee, K. & Elbaum, S. (2010)
- to eliminate worker variability
- to control variability
- researchers need "some understanding of the system capabilities and constraints"
- the system needs to direct to other services in the business space that are more apt for the task at hand.
- requestors need to have the ability to control the presentation of their work, by priority, sequence, preference, iterative, random, importance...
Needs identified during the worker-requestor interviews
- to identify patterns of requestors
- to be able to receive work immediately
- to be be able to pause work demands
- to be able to SLEEP
- to meet a daily goal (how dos this daily goal shape decision making)
- to quickly test a hypothesis
- to write clear instructions
- to post a batch of questions
- to receive information on how to improve task design
- to engaging in conversation with workers
- to how to monitor approval ratings
- to gauge the level of threat a requestor is towards approval ratings
- to scatter work across requestors
- to select those who provide HITs with good background
- to have a name and email to contact people before HITs
- to post hits without much interaction
- to know people are on the other side
- to label X of this item
- to optimize worker speed in job design an shape UI
- to avoid rejecting workers
- to maintain quality assurance of workers
- to send out sample hits to test task
- to gauge preferences of requestors
- to be able to manage interactions with workers at scale
- to manage worker correspondence
- to rate workers fairly
- to automatically clarify and challenge qualification based rejection
- to meet face to face with others
- to manage time better as quantity grows (the challenges and methods change)
- to connect with outside services that expand one's professional capacities
- to have strong best-worker relationships (10 batches)
- to pre-schedule work at intervals
- to assure quality, truthful and honest responses
- to connect with worker communities
- to have a hit within an iFrame
- to be able to build one's own efforts
- to look at the hit and estimate the value of effort... (paper: Estimating Charlie's Run-time Estimator)
- to manage the expectations of work involved and run times tied to HITs
- to know the other worker's personalities and how it affects work
- to communicate with others the reality of a job
- to match job type with work preferences and skill sets
- to earn as much profit (revenue) as possible
- to match typing tasks with those who like typing tasks
- to match button clickers with button clicking tasks
- to estimate the value of a day
- to have a buffer of work demands over time
- to avoid random responses
- to accept as work that is better than chance
- to avoid rejections on the reputation system
- to have a constant contact ability with the requestor
- to have a buffer of points for reputation management
- to have buffer time that accommodate interruptions and disabilities
incredible enablers of scientific research avoids: anonymous, frustrating, complicated, tricky absolutely unpredictable, great, terrible power, frustrating,empowering, lonely, isolated, tiring diverse and amazing futuristic dynamic humans
what rules of thumb do people use? to try 2 minutes of work and then retry
types or requestors: volume, income