WinterMilestone 1 by Team - witty

From crowdresearch
Jump to: navigation, search

Experience the life of a Worker on Mechanical Turk

Amazon Mechanical Turk rejected our registration , so we tried MTurk developer sandbox. As a worker, we faced some difficulties due to the following reasons:

  1. Tasks were often incompletely or inadequately described.
  2. Some tasks had absurd expiration times (like 30 seconds), so we were unable to complete these in time.
  3. Few tasks took a large time for approval by the employer, often delaying payments.
  4. We didn't quite like the interface of the platform, there was a lot of scope for improvement in terms of UI/UX.
  5. It was a little difficult to communicate with requesters. In the age of instant messaging, we feel that there should be an option of either directly chatting with the requester or some other hassle-free process.

Some things we liked about the platform :

  1. Many of the tasks were simple, took less time and were paid accordingly.
  2. The dashboard containing the statistics of all the HITs was pretty useful to track the progress.

Experience the life of a Requester on Mechanical Turk

Things we didn't like :

  • Again, really poor UI.
  • Instructions to create a task were confusing. It took us a great deal of time to properly understand how to create an manage tasks.
  • The most difficult part was to create a 'Survey' project with their template.
  • Creating HITs individually was hard.
  • Lack of workers : Very few workers completed the tasks listed by us.

Some things we liked :

  • Tracking progress of tasks was convenient.
  • Creating projects using templates could have been a nice feature, if proper instructions were provided.
  • Feature of managing workers(bonus, block, unblock, assign a Qualification, or revoke a Qualification) was a plus.

CSV file : Media:Batch 119583 batch results.csv

Explore alternative crowd-labor markets

Microworkers

Things we liked :

  1. Better UI and UX.
  2. Available globally, accessible in mostly all countries.
  3. Quick response and review of profile after sign up.
  4. The tasks for workers were fairly well described and simple to perform.

Things we didn't like :

  1. Much more time was required to review the tasks by requesters. In some cases, more than 7 days were required, which we felt was absolutely unnecessary!
  2. As a requester, the tasks required that minimum available positions be 30, which was more than what one might be willing to provide in some cases.

Clickworkers

It was not available in India due to lack of projects.

Readings

MobileWorks

What do you like about the system / what are its strengths?

  • A great and noble initiative by MobileWorks to target the people at the bottom of pyramid, in developing countries like India.
  • It is designed for only low-end mobile phones which is a positive since smartphone penetration in India is still very low due to its relatively high cost.
  • The user interface is minimal and hence easy to understand. Sometimes too many features can be an overkill which results in user dissatisfaction.
  • Tasks are simple and take less time to be completed, which can be favorable as people can easily incorporate it into their routine.

What do you think can be improved about the system?

We found some limitations of the MobileWorks platform :

  • Due to the fact that it is limited only to low-end mobile phones, the efficiency and efficacy of the tasks would be less, since feature phones have limited capabilities when compared to smartphones.
  • Most of these phones would be using 2G plans as 3G and 4G are primarily an urban phenomenon. 2G does not give a very high access speed, which might lead to latency and slow completion of tasks.
  • This platform is incompetent to other crowdsourcing platforms as it is able to perform only simple and straightforward tasks, while others are capable of doing really complex HITs along with simple ones.

Some features which can be added :

  • Chat box or some easy of way of communicating with other workers/ requesters.
  • It can support more than 1 language, specially the local language of that area.

Daemo

What do you like about the system / what are its strengths?

  • User interface is great ! It gives a welcoming feel.
  • Hassle-free registration process.
  • It recognizes the limitations of other platforms and works on the two most important areas:reputation and task authorship, which is the essence of crowdsourcing platforms.
  • The Boomerang reputation system is a well-thought mechanism of evaluation , different from "traditional rate-and-leave systems".
  • The breakdown of a task into smaller 'milestones' with fixed time given to each smaller milestone is a great example of 'divide and conquer' !

What do you think can be improved about the system?

  • The process of rating can be a little daunting for new workers as they'll have a low rating at the start. They can be given some kind of 'headstart' which would encourage them to participate more. (The ratings could be decreased later on if the worker does not meet the expectations)

Flash Teams

What do you like about the system / what are its strengths?

  • Really liked how they have made this system like a finite automata machine, with blocks being treated as "nodes" having a fixed input whereas the output is dependent on what action is performed on the input but is of known type.
  • How they have implemented pipelining is impressive.
  • Team structures & DRIs to help manage things better also "Foundry abstracts away low-level management effort while allowing for high-level oversight and guidance. "
  • Easy to find and repair bugs/mistakes made by a particular team.

What do you think can be improved about the system?

  • It is being assumed that the output given by a particular block will be approved by the next block in sequence as an input which might not be true in some cases.
  • F2F interaction between teams and requester can do wonders for this system.

Engineering delivered

Screenshot from 2016-01-15 21-03-58.png

Milestone Contributors

Slack usernames of all who helped create this wiki page submission: @vrinda1994, @witty123