WinterMilestone 1 daxlab

From crowdresearch
Jump to: navigation, search

Experience the life of a Worker on Mechanical Turk

I couldn't get approval from Mechanical Turk so I did tasks on Microworkers and Amazon Mechanical Turk Sandbox.

Microworkers

I did HITs primarily on Microworkers and Mturk Workersandbox. I around 25 very simple tasks out of which 23 were accepted, 1 rejected and 1 is still pending. Most requesters were fast with their approval and managed to earn (0.70 + 1.41)$. The following is my feedback regarding MicroWorkers:

Pros

  • Easy to Join: Unlike MTurk, anyone can join and do tasks on Microworkers. MTurk rejected most of the applicants on this project and most of us non-US citizens had to go for alternatives.
  • Hire Group Tasks: A requester can invite specific workers to do the job. I was invited to do a sign up job but I didn't do it.
  • Time to finish: It is an estimate to how much time is required to finish the task. It is not a constraint but a rough estimate and a worker can take as much time as he wants.
  • Time to rate: The time which the requester takes to rate the task. The best part is we can sort the tasks based on TTR.
  • Categorization of requester: They have bronze, silver and gold medals for requesters based on the approval of tasks.
  • Fast approval by requesters
  • Submitting tasks is simple and fast
  • Has worker ratings

Cons

  • Poor Introduction: Their introduction page is very confusing and if someone new to HITs joins this website, he is not introduced to the site and how things work. A decent enough tutorial on "How to get started ?" should be there.
  • No multiple filtering: I wanted to look for jobs with multiple criteria selected and it is not possible. A worker should be able to see which jobs are highest paying of multiple types(Sign up/Vote/Click).
  • Poor relevancy: You can only select one method of sorting and it just naively sorted the jobs according to the type you select. It would be better if they use some ranking algorithm based on recency, payment and Time to finish.
  • No Search: Even though the filters help in finding jobs, a search would be preferred if I am trying to find a particular job.
  • No notifications: It doesn't notify me if some work I did was approved. However, they do have a notification for Hire Group tasks.
  • Urls not hyperlinked: Why aren't task urls in the tasks hyperlinked? Is it to prevent crawlers and bots?
  • No categorization of workers: Nothing to differentiate from a good worker and a bad worker apart.


Microworker.png

Amazon MTurk Sandbox

I used the Sandbox for doing a few HITS.

Cons

  • No Categorization/Filters: MTurk needs some categorization for the tasks. It has keywords which were very random and confusing.
  • Poor sorting structure: They have a dropdown regarding what criteria to sort from. From a UX point of view, it can be improved a lot by having sorting in tables.
  • No Introduction/Get Started
  • The time constraint is unnecessary.
  • No "Time to rate" category similar to Microworkers
  • No notifications

Pros

  • Search function
  • Easy and fast to submit tasks
  • Has a master category based on qualifications.

Mturk.png

Experience the life of a Requester on Mechanical Turk

I couldn't use the money I earned from Microworker to post a real job/campaign, so I used the Amazon MTurk Sandbox for posting a job.

Pros

  • Has predefined templates for a multiple category of projects to choose from.
  • Has a nice job management tab for seeing the progress of your jobs.
  • Has a Master qualification and we can also create custom qualifications.
  • Can publish tasks in batches.

Cons

  • Only two types of workers.
  • Some types of tasks require a Master qualification but they are not distinguishable in the list of categories. You have to click on create project and read the conditions to find out.
  • No access to requester's profile: I wanted to see how authentic and rewarding the requester was but I could not find a way to check out his details.
  • The process of creating a job needs to be simpler, cleaner and involve a lesser number of steps.
  • I could not find the url of my job which was posted and I had to login to the worker sandbox to find the url by searching.
  • Tasks are not Editable: I guess it is to prevent scamming, but I think requesters should have an initial allotted time before workers can see the HIT during which they can edit.
  • There was a bug in the HTML creation of Tagging the Image HIT.


Rmturk.png

Research Engineering (Test Flight)

Daemo.png

Explore alternative crowd-labor markets

Compare and contrast the crowd-labor market you just explored (TaskRabbit/oDesk/GalaxyZoo) to Mechanical Turk.

Readings

MobileWorks

  • What do you like about the system / what are its strengths?
  • What do you think can be improved about the system?

Daemo

  • What do you like about the system / what are its strengths?
  • What do you think can be improved about the system?

Flash Teams

  • What do you like about the system / what are its strengths?
  • What do you think can be improved about the system?

Milestone Contributors

Slack usernames of all who helped create this wiki page submission: @arjunpp