Difference between revisions of "Qualitative Analysis RDQA"
From crowdresearch
Aarongilbee (Talk | contribs) (→Hairball Map: What might happen outside of Turk? How might Requesters manipulate tasks as a response?) |
Aarongilbee (Talk | contribs) (→Hairball Map: What might happen outside of Turk? How might Requesters manipulate tasks as a response?) |
||
| Line 39: | Line 39: | ||
<blockquote> | <blockquote> | ||
| − | ---Begin Task Submission--- ---Evidence--- | + | {| |
| − | + | !---Begin Task Submission--- | |
| − | 1.Worker submits work GENERIC | + | !---Evidence--- |
| − | + | |- | |
| − | 2.1 Requester mass rejection parameter kicks in GENERIC | + | |1.Worker submits work |
| − | + | |GENERIC | |
| − | 3.1 Requester team screens rejected tasks [Account 46] | + | |- |
| − | + | |2.1 Requester mass rejection parameter kicks in | |
| − | 4.1 Requester team submits results report to Worker [Account 46] | + | |GENERIC |
| − | + | |- | |
| − | 5.1 Requester team posts to worker review page [Account 46] | + | |3.1 Requester team screens rejected tasks |
| − | + | |[Account 46] | |
| − | 2.2 Requester sends verification email (UNKNOWN) [Account 56] | + | |- |
| − | + | |4.1 Requester team submits results report to Worker | |
| − | 2.3 Requester sends automated email [Account 62] | + | |[Account 46] |
| − | + | |- | |
| − | 2.3.1 includes a task ticket confirmation [Account 17] | + | |5.1 Requester team posts to worker review page |
| − | + | |[Account 46] | |
| − | ----[INCOMPLETE INFORMATION]--- | + | |- |
| − | + | |2.2 Requester sends verification email (UNKNOWN) | |
| − | + | |[Account 56] | |
| − | ---Begin Generic Email Response--- | + | |- |
| − | + | |2.3 Requester sends automated email | |
| − | 1. Worker writes email to requester GENERIC | + | |[Account 62] |
| − | + | |- | |
| − | 2.1 Requester responds to email quickly GENERIC | + | |2.3.1 includes a task ticket confirmation |
| − | + | |[Account 17] | |
| − | ---something happens--- | + | |- |
| − | + | |----[INCOMPLETE INFORMATION]--- | |
| − | 2.2 Requester does not receive email GENERIC | + | | |
| − | + | |- | |
| − | 2.3 Requester marks worker's email as "spam" [Account 17] | + | |---Begin Generic Email Response--- |
| − | + | | | |
| − | NOTE: 17 is vengeful worker. "make sure I was paid my 20 cents". | + | |- |
| + | |1. Worker writes email to requester | ||
| + | |GENERIC | ||
| + | |- | ||
| + | |2.1 Requester responds to email quickly | ||
| + | |GENERIC | ||
| + | |- | ||
| + | |---something happens--- | ||
| + | |- | ||
| + | |2.2 Requester does not receive email | ||
| + | |GENERIC | ||
| + | |- | ||
| + | |2.3 Requester marks worker's email as "spam" | ||
| + | |[Account 17] | ||
| + | |- | ||
| + | |NOTE: 17 is vengeful worker. "make sure I was paid my 20 cents". | ||
Might have acted in a way to have pushed requester to mark | Might have acted in a way to have pushed requester to mark | ||
emails as "spam". | emails as "spam". | ||
| + | | | ||
| + | |} | ||
</blockquote> | </blockquote> | ||
Revision as of 23:59, 22 March 2016
Contents
Overview
Method
TurkOpticon Reviews
High Rating Requester Review provided by Turkopticon User
Low Rating Requester Review provided by Turkopticon User
Results
Turker thoughts about pay on Survey Tasks
| Data: TurkOpticon 5 Votes v. | All Others for Survey Tasks |
|---|---|
| Welch Two Sample T-Test | Students' T-Test |
| p = 0.005428 | p = 0.003642 |
Data CSV from March 19 [1]
data: dat$Tasks.w..5s and dat$All.Others
t = 3.3403, df = 12.791, p-value = 0.005428
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
1.976024 9.246208
sample estimates:
mean of x mean of y
10.863968 5.252852
Hairball Map: What might happen outside of Turk? How might Requesters manipulate tasks as a response?
---Begin Task Submission--- ---Evidence--- 1.Worker submits work GENERIC 2.1 Requester mass rejection parameter kicks in GENERIC 3.1 Requester team screens rejected tasks [Account 46] 4.1 Requester team submits results report to Worker [Account 46] 5.1 Requester team posts to worker review page [Account 46] 2.2 Requester sends verification email (UNKNOWN) [Account 56] 2.3 Requester sends automated email [Account 62] 2.3.1 includes a task ticket confirmation [Account 17] 1. Worker writes email to requester GENERIC 2.1 Requester responds to email quickly GENERIC 2.2 Requester does not receive email GENERIC 2.3 Requester marks worker's email as "spam" [Account 17] NOTE: 17 is vengeful worker. "make sure I was paid my 20 cents". Might have acted in a way to have pushed requester to mark emails as "spam".
Tasks w/ 5s All Others
Mean 10.86396774 5.252851782
SD 4.807581499 2.259549664
N 10 10
P 0.003642357
