Milestone 1 csushornets

From crowdresearch
Jump to: navigation, search

Experience the life of a Worker on Mechanical Turk

Reflect on your experience as a worker on Mechanical Turk. What did you like? What did you dislike?

I liked that it provides a lot of information regarding HITs at the main screen. Things like a descriptive title, reward amount per HIT, expiration date, and HITs available are all very simple and helpful things for the worker. The ability to preview a HIT before accepting one was a good design choice also. It allows the worker to gauge whether or not they are acceptable for the task at hand and to judge the exactly the work that the requester is asking for. The ability to return a task once it has been accepted was also useful if the task that was accepted was not what the worker was expecting or if the amount of work for the reward seemed insufficient.

I felt that since the work can be so broad, the site lacked some menus or filters to navigate the work better. It has a search bar, but I felt it would benefit from the addition of a simple categorical drop down list that would filter based on tags that a requester can put on their requests. It also doesn't filter out those that are "invite only", this seemed to be a very basic necessity for the many requests that will only accept certain workers. Another small irk with the site was that the default setting for the "for which you are qualified" check box is unchecked. This means that the moment a worker opens the page they may be looking at a ton of HITs they are unable to work on because they aren't qualified. There is also little information on how a worker obtains certifications from requesters.

Experience the life of a Requester on Mechanical Turk

File:Csushornets mturk batch results.csv

Reflect on your experience as a requester on Mechanical Turk. What did you like? What did you dislike? Also attach the CSV file generated when you download the HIT results.

Using the Mechanical Turk interface was very intuitive and took little time to set up a batch of tasks. Results came quickly and all the responses were legitimate. I liked the ability to choose from a group of possible task types and the default templates for each type give the requester the ability to quickly customize a task while keeping the look and feel workers are used too. I disliked the fees charged by amazon, 10% of the task reward amount seemed steep and the minimum of $0.005 fee meant there was a 50% fee on tasks with a reward of $0.01.

Explore alternative crowd-labor markets

Compare and contrast the crowd-labor market you just explored (TaskRabbit/oDesk/GalaxyZoo) to Mechanical Turk.

Amazon Turk is that Turk is set up in a way that nearly anyone can sign up and start completing tasks in their free time with no training needed. oDesk on the other hand is set up for professionals that already have an education and work experience can find freelance jobs to work on from anywhere they have an internet connection. Amazon Turk is about having large numbers of people do bite sized tasks, whereas oDesk is a place where someone can hire a professional to either work indefinitely on an hourly basis or pay a fair amount to complete a complicated task. The pay for workers on oDesk is far better but there is also an expectation that the person being hired is skilled.

Readings

MobileWorks

  • What do you like about the system / what are its strengths?
  1. A focus on simplicity is nice for that system. Since the tasks on there were simplistic in nature there was no need to add a ton of features that would otherwise confuse workers.
  2. The addition of assigning tasks to multiple workers to increase the quality of the answers was a very good idea that lead to more accurate results.
  • What do you think can be improved about the system?
  1. The amount of different tasks it can assign are limited by the phones' hardware and internet speeds.
  2. The highest accuracies were only obtainable through the use of multiple entries from different workers. This means that those posting tasks may have to pay for more people to do their tasks, which could in turn mean lower pay for the individual worker.

mClerk

  • What do you like about the system / what are its strengths?
  1. The biggest plus about the system is that it uses regular SMS messages, which is low cost and thus accessible to low income workers.
  2. It doesn't require an internet connection through their phone web browser.
  3. It also takes advantage of multiple user input to increase accuracy.
  4. Payments don't require a bank account and are done manually.
  5. SMS messages are incredibly inexpensive, even to the point of being included with normal phone plans.
  6. The system was able to spread quickly from giving users incentives for recruiting other productive members.
  • What do you think can be improved about the system?
  1. SMS messages can be lost or delayed in comparison to an online system.
  2. Low end phones typically can only hold a small amount of SMS messages, and can take up the place of messages that a user would use for other reasons.
  3. Most of the issues stem from the use of the SMS message system itself since they have their own faults.
  4. There should be a way for users to get money for their work rather than just phone credits.

Flash Teams

  • What do you like about the system / what are its strengths?
  1. It splits tasks into modular pieces and has separate teams work on the individual pieces. This allows for focused development within teams.
  2. At the same time it makes sure that the individual pieces are basically inputs and outputs of other pieces, meaning that the pieces can basically be put together to create a larger finished product.
  3. It allows for dynamic changes in the teams when the need arises.
  4. It all contributes to its biggest strength of allowing teams of people that otherwise never worked together to make something of decent quality, even when they aren't co-located.
  • What do you think can be improved about the system?
  1. The system needs a lot of information about the users before setting up the basis of the teams.
  2. A lot of the initial setting up sounds to be manual from the requester.
  3. A formal way of getting users experience might help the process of getting the correct people in the right teams.