Milestone 1 jicsorcerers

From crowdresearch
Revision as of 23:48, 4 March 2015 by Skawas (Talk | contribs) (mClerk)

Jump to: navigation, search

Hi, There!

This is JICsorcerers! We are students from Human Centered Design & Engineering at University of Washington.

So, why JICsorcers? J- Jordan, I- India, C- China. We are from different countries and we are working on building a better crowdsourcing platform together.

Experience the life of a Worker on Mechanical Turk

Question: Reflect on your experience as a worker on Mechanical Turk. What did you like? What did you dislike?

Yibo: I've worked on 3 different kinds of tasks. Quite a story...

Task 1: Find the author's name and email address

In this task, I'm assigned for some online articles, some of them are sports review, some are about education, and so forth. There are authors from CNN, NPC, etc. Here are some features for this task.

  • $0.05 for each
  • Did 4, all of them are pending
  • Approximately 10 minutes for each, if it takes over 10 minutes, I'll skip it.
  • All author's emails are really hard to find, but some of them are not impossible. For example, I found the email of one journalist in the description of Youtube video she uploaded.
  • Most importantly, I feel this task is kind of immoral. I mean searching for other people's email address. Should there exist any control for the task?

Task 2: Type in contents of supermarket receipt

I'm assigned for receipts of different markets. I need to type in each item's name, price, and choose quantity and type (item, coupon or void). No intelligence challenge in this work...

  • $0.05 for the first 10 items. $0.01 for 4 items after that.
  • Did 2, get 1 approved
  • Time depends on how many items and how many text are there on the receipt. I will choose to skip ones with too long item name...

Task 3: Survey of Strategic Feeling and Craving of Food

This survey claimed it took 60 minutes, but it actually took me 90 minutes. I need to first finished the survey, get the code, and then enter the code. However, after I finished the survey and got the code, the task got canceled! I cannot find it anymore!

The survey consists 2 parts. In part 1, I rate my subjective craving of food towards different food images. There are videos that try to influence my current emotion. I rate my subject feeling before and after watching those videos. Part 2 is a very long survey. There are even some private questions in it. I feel uncomfortable about it. Another thing is, I need to download and install a plug-in called Inquisit to start the survey. It makes me worry about my computer safety.

Things I like and dislike

I like

  • the opportunities of making money
  • different tasks available
  • ability to filter on those only available for me
  • sorting by different metrics, like revenue, date, etc.

I dislike

  • lack of monitoring of task content
  • lack of obligation of paying rewards (or cheating)
  • sometimes conflicts between reservations for the same task
  • Unfair reward (to be honest, it should be paid much higher than $1 for a 20 minute task)

Feedback from MTurk Support

After I report the failure to find the task I've worked on for one and a half hour last night, I got an email this morning:

Hello Yibo,

Thank you for your email, my name is Robin and I am more than happy to help you today.

Thank you for contacting Mechanical Turk! Unfortunately, as HITs are created by the Requester, we have very limited visibility into the design. If you are experiencing a problem with a HIT on Mechanical Turk, we recommend reaching out to them directly. You can reach out to a Requester by following these steps:

1. Go to the HITs tab on the Worker website and search for HITs containing the Requesters name. 2. Click on the title of the HIT in question. 3. On the lower right-hand side of the box, click Contact the Requester of this HIT. 4. Enter your question about the HIT and submit.

Your name and e-mail address will be made available to the requester when you click submit the "Contact the Requester of this HIT" form.

After you have sent your message, you may need to wait a few days for a reply. Some Requesters do not monitor their messages every day.

I hope this helps. Please let us know if you have any additional questions.

Thank you for your inquiry. Did I solve your problem?

If yes, please click here:

If no, please click here:

Best regards,

Robin R Amazon Mechanical Turk

Please note: this e-mail was sent from an address that cannot accept incoming e-mail. To contact us again, select the Contact Us link related to your inquiry below.



However, the key point is that I cannot find the requester's information anymore! I should have noted down them before I accepted and started working on that HIT.

Experience the life of a Requester on Mechanical Turk

Reflect on your experience as a requester on Mechanical Turk. What did you like? What did you dislike? Also attach the CSV file generated when you download the HIT results.

Explore alternative crowd-labor markets

Compare and contrast the crowd-labor market you just explored (TaskRabbit/oDesk/GalaxyZoo) to Mechanical Turk.

Task Rabbit

Compared to MTurk, in Task Rabbit...

  • finding a task is more based on "search", you need to enter keywords and find the write category.
  • client knows who the tasker will be before s/he starts the task
  • client chooses from suggested taskers, and there's detailed profile, such as desired salary
  • there's more information for a tasker, including ratings, reviews, tasks accomplished before
  • more in-person tasks, physical labors are more popular, such as cleaning and moving.
  • you don't need SSN to register as a Client (the same as requester in MTurk)

Galaxy Zoo

Compared to MTurk, in Galaxy Zoo...

  • it is a crowdsourced astronomy project, the task of it is very specific
  • it is a volunteered job without paid
  • each galaxy was seen by many different participants, which means the same task is assigned to multiple volunteers; having multiple independent classifications of the same object allows Galaxy Zoo to assess how reliable the results are


The website is down for maintenance...



What do you like about the system / what are its strengths?

Mobile web-based platform which makes it accessible for a larger population who otherwise might not have access to the web. MobileWorks system leveraged the following opportunity for a crowdsource platform for the mobile phone devices:

- Mobile phone users are 50% higher than desktop devices, and most of these phones are capable of surfing the web.

- Internet access on the phone is cheaper and have a variety of day/ three days internet plans. because of the ubiquity of inexpensive cell phones, even people with low income have access to the Internet.

-The mobile Internet is an cost-effective way to send micro-tasks to lower-income people

-It broaden the "Turkers" participation to marginalized workers.

What do you think can be improved about the system?

- Improve the Usability metrics- observe where errors might have occurred based per task vs. per works.

- The accuracy of the tasks might have been affected by the mobile device's screens.


What do you like about the system / what are its strengths?

- mClerks is also a Mobile-Based platform that made the crowdsourcing tasks accessible to a larger population of lower income works and it took advantage of all the features mentioned above.

- This is platform is not only limited to text. It leveraged sending small images via ordinary SMS, enabling novel distribution of graphical tasks.

- Payment mechanism in the form of mobile airtime (mobile credit) makes signing up and participating more visible and accessible.

What do you think can be improved about the system?

The testing of the system feedback and motivational mechanism used to encourage the users could be improved.

Randomization of the tasks assigned to the different people in the same circles of communication

Flash Teams

  • What do you like about the system / what are its strengths?
  • What do you think can be improved about the system?