Milestone 1 Team Innovate

From crowdresearch
Jump to: navigation, search

Experience the life of a Worker on Mechanical Turk

Task:Reflect on your experience as a worker on Mechanical Turk. What did you like? What did you dislike?

Task 1: Labeling the photographs on mechanical turk for humans

In this task I was assigned to label the photographs using color markers.Some of the features for this task are as follows:

  • $0.05 for each task
  • labelled 3 photographs all of which are pending
  • Gave approximately 5 minutes foe each task

Task 2: Providing information about products

In this task I was assigned to provide information about various daily use products.Some of the features for this task are as follows:

  • $0.05 for each task
  • Gave information for approximately 3 products
  • Took around 10 minutes to provide information about product
  • All 3 pull requests are pending

Things I like & I dislike

The things that I like were:

  • Opportunity for making money
  • Filtering the task on basis of various factors
  • Variability of task types

The things that I dislike were:

  • No proper monitoring for task revenue
  • Cheating in paying rewards
  • Lack of monitoring of task content

Experience the life of a Requester on Mechanical Turk

Task:Reflect on your experience as a requester on Mechanical Turk. What did you like? What did you dislike? Also attach the CSV file generated when you download the HIT results.

Task 1 : Created a HIT for labeling photographs for human activities

The features of this task are as follows :

  • The cost of task was $0.02 for each individual HIT.
  • The task involved two categories,i.e., human detection & classification.
  • The task had around 10 images all of which are pending.

What I Like

  • Cheap labor available at mechanical turk.
  • Access to huge masses for doing work.
  • Availability to a variety of workforce.

What I dislike

  • Low accuracy of task.
  • Unavailability of specialized labor.
  • No proper monitoring over task.

The csv file of the results is here although it is still pending File:Enclose.csv.

Explore alternative crowd-labor markets

There were many other crowd sourcing platforms such as TaskRabbit, oDesk & GalaxyZoo. We explored the TaskRabbit platform.

Experience about TaskRabbit

About

The platform offers a limited specialized taskforce for a variety of tasks which includes cleaning, personal assistant, handy aid & moving help but is not limited to these only. The workers work form a group to accomplish this task.

TaskRabbit for Workers

The platform is open for workers willing to work in various fields. You offer service of any type which may interest you whether it canbe moving aid or cleaning or any other work. All you need to do is to create a profile using a Linkedin or Facebook account & then you start working.

TaskRabbit for Employers

The platform offers a diverse taskforce to employers for carrying out their task which can be done more efficiently. Also there is no cash payment. All the payments are made through secure gateway.

Pros

  • The taskforce is highly experienced & trustworthy.
  • The workers can outsource the errand that they do not want to do.
  • The entire payment process is handled securely online after a task is complete.

Cons

  • New workers find it difficult to get experience.
  • Workers may not get a fair wage.
  • The workers have no right to ask about wages.

Comparison with Mechanical Turk

Compared to the mechanical turk, TaskRabbit is a much secure & reliable platform but there is no direct interaction between employees & employers to discuss about the expenditure for the task. Since it is more secure , therefore, there is monitoring of the task content. TaskRabbit concentrates on individual talent, specialisations of workers whereas mechanical turk distributes microtasks to the masses which may be less effective in terms of output of HIT.

Readings

MobileWorks

A mobile crowdsourcing Platform for workers at the bottom of the pyramid

Strengths of the system

  • As Amazon’s Mechanical Turk has workers who are primarily the residants of USA and India. Since India have limited English Literacy and lack of access to a desktop computer so it makes crowdsourcing work inaccessible to a large population of Indian Population. It means for workers at the bottom of the pyramid. But in India mobile phone penetration is very high because they are simple and capable of surfing the internet.
  • The cost of mobile internet has got a decline so it is affordable by the workers at the bottom of the economic pyramid.
  • They also seems easy to handle . It also has increased the earning rate of the employees.
  • Biggest advantage of the system is that work can be done from anywhere , at anytime and according to our own comfort . It finally results to the flexibility rather bounding the workers .
  • It also results better accuracy as more and more workers are involved in it and thus provide closest correct result and the increasing employment also.

Improvement in the system

Despite having lots of positive points there are following improvements which i think has to considered:

  • There should be increased data rates which decreases the time to complete each tasks and thus results greater efficiency which means more tasks will be done by workers in less time.
  • As we know mobiles are very user-friendly but still there are lack of awareness is prevailing about crowdsourcing using mobiles . So there should be spreading of proper knowledge about how it works.

mClerk

Enabling Mobile Crowdsourcing in Developing Regions

Strengths of the system

Mobile Crowdsourcing is booming but still it has shortcomings which are tried to overcome by mClerk in the following way:

  • In this image-based tasks are distributed to a low-end mobile phones, using a little-known protocol that sends small bitmapped images via ordinary SMS messages. Due to this low-income workers are allowed to participate in the system using devices that they already own. Thus responses are also send via SMS which are affordable by the workers.
  • This system has greater accuracy because it involves continuous checking which means comparing again and again until it is completely matched with each other.

Improvement in the system

  • Digitalization speed should be improved which helps the workers to get high pay.
  • There should be improvement in the field of how the data entry speed can be improved on a mobile phone.

Flash Teams

Strength of the system

  • Flash teams consist of the experts from the crowd and manages the crowdsourced expert teams. In this input is given and then we get the desired ouput and then this output will be the input for the next task. The speciality of these flash teams are that teams can be combined as teams are modular their component tasks can be composed to form larger organization.
  • Flash Teams are also flexible in nature which means the.y can also shrink and also grow so that they can meet task demands by hiring additional members from the crowd.
  • They extended the paid crowdsourcing for expert work.
  • It enables complex work at crowd scale by automating the structures of traditional organizations.
  • Flash Teams also affords dynamic recruitment and coordination of on-demand expertise that is extremely difficult in offline scenarios.
  • As flash teams takes the advantage of timezone differences which potentially allow them to carry on uninterruptedly for days or weeks.

Improvement in the system

  • As controlled evaluation of flash teams was limited to napkin sketch design teams, so we have to plan to test other flash teams to better understand the types of tasks they are suited for.
  • Flash team experience found difficulties related to coordination and conflict. It also lack camaraderie-a spirit of familiarity. So in order to improve one solution can be given that to allow workers to find trusted colleagues and join into clusters that can be hired together.
  • There should be work in the field of exploring issues related to runtime course correction and dispute resolution.