Milestone 1 iota

From crowdresearch
Jump to: navigation, search

Template for your submission for Milestone 1. Do not edit this directly - instead, make a new page at Milestone 1 YourTeamName or whatever your team name is, and copy this template over. You can view the source of this page by clicking the Edit button at the top-right of this page, or by clicking here.

Experience the life of a Worker on Mechanical Turk

Reflect on your experience as a worker on Mechanical Turk. What did you like? What did you dislike?


We tried out the role of a worker on the crowd-sourcing platform mTurk. Our observations and comments on the experience are as follows :

Actions

  1. Setup an account for the team and looked up for available jobs.
  2. Took up some simple jobs like signup tasks, search and verify tasks, commenting, etc.
  3. At the end of experiment, cumulatively we were able to make $4.51

Observations


General
  1. Simple, easy-to-use UI.
  2. Job listings are well categorised.
Good features
  1. The format for description of every task is very simple and easy to understand for any worker.
  2. Gives highlights of the task (like Time taken to finish, review period, employer profile)
  3. Easy to use mechanism to submit the proof of completion of task. Simple answering type to the questions asked by employer.
  4. Reporting misleading/non-working jobs.
  5. Well detailed user profile. This adds transparency to know more about the employer/requester.
  6. Incentives on the profile page to get reputations on particular categories. For example,one of our team member was personally motivated to get a five star reputation in the Twitter category as he was very active on that social media and hence the tasks seemed easy.
  7. Unique feature: Hire group jobs. Ability to join a team of experienced workers and get handpicked for a particular task by employer.
Disliked features
  1. There is no user personalised task suggestion system.For example, if a worker is constantly good at doing Twitter based tasks, there is no exclusive suggestions for him with only Twitter tasks.
  2. Work involved in the tasks and the payment corresponding to them are not justified in many cases.
  3. Reviews of submissions is sometimes very slow.
  4. The "not satisfactory review" for a submission are not very informative at times.

Experience the life of a Requester on Mechanical Turk

Reflect on your experience as a requester on Mechanical Turk. What did you like? What did you dislike? Also attach the CSV file generated when you download the HIT results.


We tried out the role of a employer on the crowd-sourcing platform mTurk. Our observations and comments on the experience are as follows :

Observations


General
  1. Employers can post their work to be done, as campaigns.
  2. Work submitted by workers is reviewed by the employer.
Good features
  1. Very detailed and exhaustive UI to describe the details of a campaign, and at the same time very easy to use and user-friendly.
  2. All jobs posted are reviewed and approved by the mTurk' admins before opening them to the workers. This avoids illegal,violent,spam and scam activities.
  3. Flexibility in choosing workers from specific countries.
  4. Unique feature: Hire groups. Assigning tasks only to group of select expert workers in a domain.
  5. Feature to award bonus to workers.
  6. Transactions page : Dedicated page to show all the monthly transactions in detail.
Disliked features
  1. Personally need to review and rate submissions to a task. It would be better if this is either automated or in-turn crowd-sourced.
  2. Hard to ensure that submissions are not duplicate or invalid.
  3. Approval process for a task can take a long time.
  4. The task creation process is itself cumbersome and time-consuming.


Explore alternative crowd-labor markets

Compare and contrast the crowd-labor market you just explored (TaskRabbit/oDesk/GalaxyZoo) to Mechanical Turk.

Readings

MobileWorks

Strengths


  1. Reachability to a very large community.
  2. Very specific and remote tasks can be completed. For example,translation of a script in a regional variant of a language
  3. Interface is as simple as possible.
  4. Worker satisfaction(for payment) is high,as individually the tasks are very small when divided.
  5. Targetted at the lower echelons of society, it has the promise to bring about their upliftment.
  6. Quality rating system is an efficient method to decide the remuneration for workers.

Scope for improvements and weaknesses


  1. With growth of the worker community at an unusually higher pace(due to extreme simplicity of work) than the tasks at hand, a point of stagnation may be reached when workers do not get enough work for them to be profitable while spending for their mobile internet data.
  2. Pre-processing(includes cost determination) and distribution for iterations will be resource intensive if it has to assure equal division of work.
  3. Instead of an application,that may involve platform issues,a browser based application for the same maybe more feasible as most of the modern day phones support a web browser.This can also span the community that uses any terminal with a browser.
  4. Smaller the division of the document to be converted,more the network traffic at the server.Considering the iterations for accuracy,the traffic at the server will only double.A feasible solution might be, on the go partitioning of the document rather than a pre-processing stage.That is,the user determines how much work he is capable of doing at the current point of time(with a limit). Based on his/her request the partitioning of the document happens,thereby reducing the network traffic of transmitting small packets again and again.
  5. The time taken to complete a task (Efficacy) must also be taken into consideration while payment. A quick response to a task must fetch more money.
  6. A system of allotting different groups for answering, validation and correction may be implemented and repeated till a converging result is achieved.


mClerk

  • What do you like about the system / what are its strengths?
  • What do you think can be improved about the system?


Flash Teams

  • What do you like about the system / what are its strengths?
  • What do you think can be improved about the system?


Strengths


  1. Complex tasks(like software engineering) can be crowd-sourced - places more trust on workers
  2. A team can be created in short notice, has a lot of flexibility and results are obtained quickly
  3. It incorporates Organisational Behaviour studies to effectively manage the process
  4. Reusable blocks, creates expertise/specialisation
  5. Pipe-lining, improves throughput and communication
  6. The user who may not have a clear idea of the steps to be followed can plan out the task with existing blocks
  7. Skilled people with little entrepreneurial inclination will be benefited by joining such a platform

Scope for improvements and weaknesses


  1. Workers can be associated with quality and commitment ratings, based on the tasks they have completed.
  2. Workers may be allowed to enroll in groups for a particular task. There will be a better understanding among the workers and the desired outputs may be obtained more quickly.
  3. A list of services that can be provided by the platform can be put up for users who have little idea about the technical details of a task and the platform can create the entire block chain for the user.