Difference between revisions of "WinterMilestone 1 Dubs"

From crowdresearch
Jump to: navigation, search
Line 12: Line 12:
 
On the other hand, people from other countries might benefit from the dollar currency of the system. Although it would still be a low payment, it would certainly be a way better deal to them than to americans/strong currency countries. Unfortunately, the platform is not available around the world.
 
On the other hand, people from other countries might benefit from the dollar currency of the system. Although it would still be a low payment, it would certainly be a way better deal to them than to americans/strong currency countries. Unfortunately, the platform is not available around the world.
  
===Good qualities/Positive aspects===
+
===Positive aspects===
  
 
*UI is very simple and the most important tools are found easily.
 
*UI is very simple and the most important tools are found easily.
Line 18: Line 18:
 
*(If the platform was available around the world, it would be an interesting way to make money).
 
*(If the platform was available around the world, it would be an interesting way to make money).
  
 
+
===Negative aspects===
===Bad qualities/Negative aspects===
+
  
 
*The payment is really low. There is no way to count on that as the primary source of money on the US at least. There are a lot of better options to earn money out there.
 
*The payment is really low. There is no way to count on that as the primary source of money on the US at least. There are a lot of better options to earn money out there.
 
*Tasks need to be done precisely (concentration and speed needed). If you can't do it correctly, you just won't get the money.
 
*Tasks need to be done precisely (concentration and speed needed). If you can't do it correctly, you just won't get the money.
 
*It's not easy to find work and the certificate system sometimes just make it harder instead of really helping people to get qualified work.
 
*It's not easy to find work and the certificate system sometimes just make it harder instead of really helping people to get qualified work.
*Workers are very be vulnerable to untrustworthy requesters. There is no feedback for the worker and a lot of scam activities can happen,
+
*Workers are very be vulnerable to untrustworthy requesters. There is no feedback for the worker and a lot of scam activities can happen.
*UI is simple, but it is not welcoming and it's also very outdated. It gives the impression that the platform was abandoned, or is harder to use than it actually is.
+
*UI is simple, but it is not welcoming and it's also very outdated. It gives the impression that the platform was abandoned, or is harder to use than it actually is.
  
 
== Experience the life of a Requester on Mechanical Turk ==
 
== Experience the life of a Requester on Mechanical Turk ==

Revision as of 19:11, 17 January 2016

Click on the EDIT Button on the top-right of this page to edit.

Experience the life of a Worker on Mechanical Turk

Considerations

The most important thing here is to understand that the worker is a fundamental problem in the system. It is even hard to make positive aspects about it because the payment are very low and it's not easy to find work.

As a requester, we decided to make a task to ask workers the main problem with the platform and payment was the most frequent issue followed by finding work.

On the other hand, people from other countries might benefit from the dollar currency of the system. Although it would still be a low payment, it would certainly be a way better deal to them than to americans/strong currency countries. Unfortunately, the platform is not available around the world.

Positive aspects

  • UI is very simple and the most important tools are found easily.
  • You can find some decent tasks (extra money).
  • (If the platform was available around the world, it would be an interesting way to make money).

Negative aspects

  • The payment is really low. There is no way to count on that as the primary source of money on the US at least. There are a lot of better options to earn money out there.
  • Tasks need to be done precisely (concentration and speed needed). If you can't do it correctly, you just won't get the money.
  • It's not easy to find work and the certificate system sometimes just make it harder instead of really helping people to get qualified work.
  • Workers are very be vulnerable to untrustworthy requesters. There is no feedback for the worker and a lot of scam activities can happen.
  • UI is simple, but it is not welcoming and it's also very outdated. It gives the impression that the platform was abandoned, or is harder to use than it actually is.

Experience the life of a Requester on Mechanical Turk

Considerations

First of all, this is definitely a hard problem to solve. There are infinite possibilities of works that can be requested and it’s complicated to create a system that can not only serve all the possibilities, but also maintain a good traction with all of them. Usually, there is a trade-off with specificity and traction. How to make a good service for a diverse group of requesters. Here, we will try to consider some opinions we had about the requesting systems, with some comments that can lead to future implementations/suggestions.

For that part, we made a simple survey asking for the workers their opinion about what are the main problems about MTurk and

Positive aspects

  • Straight-forward publication. Except from the very beginning (login, etc), there are not a lot of doubts about how to use the system. It’s a very direct process, but a limited one.
  • There is a certificate system. Although there are a lot of ways to improve it, it’s interesting to avoid scammers/bad workers/bad intentions in the community.
  • Pre-organized models. There are some models to help you get ready to create something, so you don’t necessarily need to start from the scratch.
  • HTML edition. There are possibilities to edit the HTML and there is some bootstrap/css on it already.
  • It generates a .csv file that can later be used for other programs/analysis.
  • Cheaper than hiring a group of people to do the same. It’s pretty cheap to the requester to get work done and it’s a pretty random group of people, which is good for statistics.

Negative aspects

  • Limited edition options. It’s possible to edit the HTML, but the way the structure is constructed makes it really hard to truly make a task different from the sample ones provided. An API or a system that could connect different systems/apis/etc with them in an easy/intuitive way would be a huge step forward to guarantee that all kind of tasks could be done with the application.
  • Old design. We believe that especially when we’re dealing with money, the design takes an important place: it inspires reliability. The way MTurk is currently designed doesn’t really bring any kind of confidence for someone that wants to invest money.
  • It’s restricted for people outside the US. We are Brazilians and, although we could use it since one of our members is in the US right now, it’s a really bad problem. Currency can be a big deal for a lot of workers. For instance, the Brazilian currency is around 1 to 4 right now. It means that a Brazilian worker would really benefit for a payment made in dollars from the same system and would be great for requesters (not too good for american workers though).
  • The data visualization is poor. Even though the results page is interesting, it could have a way better system to visualize data. D3.js could be really useful to truly bring meaning to the data acquired. Certainly, it would depend on the acquired data, but it would be a good possibility for a lot of parameters or sample batches.
  • Payment systems could be easier to navigate. For instance, the MTurk fees are not explained, so it’s not possible to know the exact total amount of money needed before finishing the batch. It would be really useful to know it before. Also, sometimes you don’t have enough money to make a batch public on the MTurk account and it takes too long to put money again. Maybe, we could have an easier way to refill the account.
  • There are no tag system or any way to provide a specific information/ranking about your performance for specific tasks. Sometimes, someone is really good at filling surveys, but not that good at processing signals. It would be interesting to see that function.

Here is our csv file.

Explore alternative crowd-labor markets

TaskRabbit

The idea of behind TaskRabbit, according to themselves is "outsourcing household errands and skilled tasks to trusted people in your community.". Basically, it is an online and mobile marketplace that allows each neighborhood to outsource small tasks.

The biggest factor differentiating MTurk from TaskRabbit is the key role played by computer automation on the former. While Mechanical Turk heavily relies on artificial intelligence to deliver digital tasks such as image recognition and OCR, TaskRabbit has been facing trouble getting traction from human-only based tasks. While the background idea of outsourcing mechanical flows of your day-to-day routine is basically the same in both of them, MTurk has the intricate advantage of not only being a provider of digital services but also the infrastructure provided by its main company, Amazon. All in all, for what we're able to sense from both products as of today, it seems that getting people to outsource their mundane activities rather than their digital ones seems to be a huge yet not impossible challenge faced by any initiative aiming at getting people to work together using collective intelligence.

Readings

MobileWorks

  • What do you like about the system / what are its strengths?
  • What do you think can be improved about the system?

Daemo

  • What do you like about the system / what are its strengths?
  • What do you think can be improved about the system?

Flash Teams

  • What do you like about the system / what are its strengths?
  • What do you think can be improved about the system?

Milestone Contributors

Our contributors:

  • Flavio Scorpione - @scorpione
  • Gabriel Bayomi - @gbayomi
  • Henrique Orefice - @horefice
  • Lucas Bamidele - @lucasbamidele
  • Teogenes Moura - @teomoura