WinterMilestone 1 One Of Us

From crowdresearch
Jump to: navigation, search

Experience the life of a Worker on Mechanical Turk

Reflect on your experience as a worker on Mechanical Turk. What did you like? What did you dislike?

Positives

  • Signing up as a worker on Mechanical Turk was a straight-forward and easy process.
  • The tasks themselves were simple to complete and the directions were clear. Many times, it was a simple survey or email entry.
  • Searching got tasks that one could do was an easy process.

Negatives

  • Even though signing up was simple, the authentication of the account takes a couple days before the user can actually start doing tasks. For users outside the United States, the account can be rejected.
  • There is no tutorial about how a worker should actually go about. The worker is simply thrown into the system and expected to understand how to start completing tasks.
  • The design of the website seems very old. Something as simple as website design can negatively impact the morale of a worker.
  • There is no immediate feedback for the work done. Submissions can be pending for prolonged periods of time and the worker would not know whether or not he did a fair job. Especially for workers that recently signed up to the program, feedback would greatly help the worker get productive.
  • Many tasks involve simply taking personal info like email and phone numbers.

Experience the life of a Requester on Mechanical Turk

Positives

  • Tasks were easy to create and templates were provided to make the process even simpler.
  • Managing tasks were easy to do.
  • The layout of the website looks more modern which seems to show a bias toward the requesters.

Negatives

  • There seems to be a clear bias toward the requesters since they had the power to determine whether a worker could be paid. There should be an equal level of risk between the requesters and the workers. Perhaps it would be possible for the workers to be guaranteed a percentage of the total compensation of that task once completed. In return, the requesters could give the remainder of the compensation depending on the quality of the work.

Readings

MobileWorks

  • What do you like about the system / what are its strengths? MobileWorks takes advantage of the high penetration of mobile phones in India, an area which actively uses crowd-source markets. This concept of choosing a platform that maximizes the app exposure to workers should be a concern when creating Daemo. MobileWorks's approach to the problem of limited screen-space on mobile devices is very clever. By separating the original image into smaller pieces, the task can be simplified even further and result in a high accuracy.
  • What do you think can be improved about the system? MobileWork's innovative OCR technique unfortunately seems to also result in a large disadvantage. Since work is being broken down into smaller pieces, the worker does not feel like s/he is contributing to something substantial and quality may suffer. The system could be improved by incorporating other methods beyond just human OCR.

Daemo

  • What do you like about the system / what are its strengths? The system tries to give incentives for both the workers and the requesters to communicate through ratings. Traditionally, in mechanical turk, lack of clear directions or expectations would result in a compromise in quality of the task which would in term cause the requester to lower the compensation. This results in a vicious downward spiral which leaves both parties unsatisfied. Daemo allows requesters to provide clear tasks which will result in a higher quality of work and increased satisfaction of the requester.
  • What do you think can be improved about the system? The feedback system should be an ongoing process throughout the task. There should also be a "ban" system in place to filter out users which consistently do not fit the standards even after feedback.

Flash Teams

  • What do you like about the system / what are its strengths? The usage of a flash team seems to be an effective and efficient way to contract a project. This high level approach to solving tasks seems to also allow workers to understand the greater picture and potentially feel more attached to their work resulting in high quality work. Allowing for specialized teams to tackle tasks will also increase worker morale since they know they are on the team for their unique expertise.
  • What do you think can be improved about the system? There may be a problem of consistently forming a team of specialized individuals. These individuals would be experts in a particular area and generally consultants such as these would expect a higher compensation.

Milestone Contributors

Slack usernames of all who helped create this wiki page submission: @richard-shen, @smsn, @bryanjhsu