Milestone 1 ZSpace

From crowdresearch
Jump to: navigation, search

Experience the life of a Worker on Mechanical Turk

  1. It is an open work scheme, mainly related to the open production side, where people get to work on fields they are interested.
  2. Anyone can participate, as long as there is work available.
  3. Well-structured tasks allow the on-the-task evaluation of the workers, and can automatically infer whether someone is a good fit for a task or not.
  4. In crowd-sourcing, you can treat each human as a weak classifier and then learn on top, where each judgement has a cost.
  5. In crowd-sourcing, there is very little friction in entering and leaving a job.
  6. In fact, this is the key crucial difference with traditional modes of employment; there is no interview and the employment is truly at will.
  7. It in fact gives you the access to various open challenges out in the science and technology in the world.
  8. It also helps in development of your breadths and depths of your knowledge set, thus helping not only in your personal development, but also contributing to the overall benefit to the society.
  9. It also acts as source of income for stay-at-home parents, unemployed and underemployed workers.

Experience the life of a Requester on Mechanical Turk

  1. Humans still significantly outperform the most powerful computers at completing such simple tasks as identifying objects in photographs – something children can do even before they learn to speak.
  2. Complex software applications based on the things computers do well, such as storing and retrieving large amounts of information or rapidly performing calculations.
  3. Amazon Mechanical Turk provides a wonderful service for service requesters (hereafter “Requesters”) to integrate Artificial Artificial Intelligence directly into their applications by making requests of humans.
  4. Requesters have the opportunity to approve completed Human Intelligence Tasks (HITs) before having to pay the workers and can also can specify that people who work on their tasks must first complete a qualification test.
  5. Businesses can create and publish HITs on Mechanical Turk using the web user interface, the web service API, or the command line tools
  6. It is a open platform beneficial to both workers and requesters, in aspects of work and economics.
  7. Collaboration through mturk gives the freedom to the requester to engage with humans from any part of the world solely based on the skill-set, without a need to take care of logistic and other concerns.
  8. Mturk is basically a hidden chess game, where the workers and requesters are pawns in the game, playing according to their advantages and disadvantages

Explore alternative crowd-labor markets


TaskRabbit is the smart way to get things done by connecting you with others in your neighborhood.

  • It connects you to qualified Taskers available to help, depending on what you need.
  • It has a team of smart, talented Taskers, who have all been thoroughly vetted.
  • It helps you live smarter, giving you more time to focus on what’s most important. Transparent hourly prices
  1. Once you post a task, you see hourly rates for the Taskers who are most qualified for your job.
  2. A minimum payment of one hour is required per task.
  3. The entire payment process (including reimbursements) is handled securely online after a task is complete.
  4. TaskRabbit takes an additional 20% service fee on each task so we can provide 24/7 Member Services support, full insurance on every task and our satisfaction guarantee.
  5. You can cancel your task online or by calling our Member Services team.
  6. If you cancel within six hours of the start time agreed upon with your Tasker, you will be required to pay for one hour of work.

Amazon mechanical turk

Amazon Mechanical turk contrasts this in the aspect of Requesters providing the Human Intelligence Tasks (HITs) and Workers select the one that interests them.

  • Requesters who want to get tasks done get access to a global, on-demand, 24 x 7 workforce
  • The system is responsive and robust that thousands of HITs get completed in minutes
  • They need to pay only when satisfied with the results
  1. For software developers, it solves the problem of building applications that need human intelligence.
  2. For businesses and entrepreneurs who want tasks completed, it solves the problem by providing accessing to vast network of human intelligence with necessary efficiencies
  3. Requesters can specify that people who work on their tasks must first complete a qualification test, so that only qualified people are allowed to work on HITs
  • On the other hand, workers can register and can work from home
  • They get to choose your own work hours, according to their convenience
  • As any other crowdsourcing platform, they get paid for doing good work
  1. Amazon Mechanical Turk limits the number of HITs a worker can do on a daily basis based on a number of factors including their past performance and account status
  2. Once a Requester approves a submitted HIT, Mechanical Turk will automatically transfer your earnings to your balance under Earnings
  3. You can transfer your earnings to your Amazon Payments account

ODesk oDesk, pays workers on an hourly basis, as opposed to paying piecemeal. This allowed requesters to get workers to focus on the hard case.

  • oDesk has an API as well, which can be used to further automate the process
  • When the need for human labor in long-term, it makes sense to ask the oDesk workers to first spend some time familiarizing themselves with the task.
  • All the workers use a common skype chatroom, where you can communicate with them in real time, informing them about system issues.
  1. oDesk runs a pretty strong identity verification scheme, which makes each worker a person tied to a real-world identity, as opposed to the disposable MTurk workerIDs.
  2. A disadvantage of oDesk is that most of the work ends up being more expensive than Mechanical Turk, since quick access isn't guaranteed



  • MobileWorks can be used for optical character recognition (OCR) tasks that can be completed by workers on low-end mobile phones through a web browser.

It is useful because of the following:

  1. Crowd workers do not require access to computers to complete the tasks, thus the tasks can be performed easily while travelling without the any hassle.
  2. According to the presented statistics mobile phone users in India are far more than people who have computers thus more workers can be involved in the task.
  3. The user interface of MobileWorks is very simple and easy to use as compared to Mturk which helps the workers to get familiarized with the system easily.
  4. Mobile Internet is available at cheaper rates making it economical for the workers.
  5. The tasks are not only text based but images can also be sent using the platform.
  6. People with very basic knowledge can also work and add to their revenue.

The system can address these issues for better results:

  1. It is mostly useful for text digitization
  2. There is no mechanism to differentiate average workers from good workers. If such a mechanism is devised then:
    • Task can be assigned based on efficiency.
    • Iterative mechanism can be used instead of recursive where good workers can be hierarchically assigned the task of verification.
    • For the same task output from efficient worker can be given more weight age then an average worker.
    • Compensation can be made according to the type of worker. Thus, giving workers better incentive for improvement.
  3. The speed of output is dependent on the mobile data service which is erratic in rural parts of India.
  4. Accuracy of tasks can be further improved by effectively selecting the tasks of appropriate difficulty.


  • mClerk enables image-based tasks to be distributed to low end mobile phones, using a little-known protocol that sends small bitmapped images via ordinary SMS messages. The work assigned is digitization of local-language text that is uniquely suited to the skills of low-income workers. To complete the task, users receive an image of a word via SMS, and send back the typed version. Workers transliterate the word in English, and the system later converts it to the local font.

The has four modules:

  1. image segmentation software
  2. a mobile crowdsourcing platform
  3. word aggregation code
  4. a payment mechanism.

It is useful because of the following:

  1. mClerk can be used on low end mobile phones, so no independent hardware is to be set up or purchased at the worker end.
  2. Internet services are not used thus availability of connection is not a bottleneck.
  3. The system is easy to use because workers are already familiar to sending bulk messages daily to family,friends.
  4. By eliminating the need for formal contracts and co-location between employer and employee, paid crowdsourcing could lower the barrier-to-entry in the global marketplace and provide a higher rate of pay than is available locally.
  5. Crowdsourced tasks can be completed on a flexible schedule, offering workers the opportunity to earn supplemental revenue during their commute to work or during other idle moments of the day.
  6. The procedure to refer someone or ask queries is rather simple. The worker gives a miss call and he is called back by mClerk for any clarification or troubleshooting. Similarly, if a mClerk worker wants to refer someone he can give a miss call from the registered number and mClerk calls back the new user to register.
  7. Cheap message packs are easily available.
  8. The payment mechanism is simple and does not require the user to have an account. The compensation is in the form of mobile credit.

The system can address these issues for better results:

  1. Tasks have a low resolution, this hinders the output.
  2. There is no mechanism to differentiate average workers from good workers as with MobileWorks.
  3. The transliterator also adds to inaccuracy while converting English to local language. This can be improved by:
    • Designing more efficient transliterator
    • Allowing workers to efficiently and directly key in local language. This is not being done presently due to the unavailability of local language font support in mobile phones.
  4. Words with lengths greater than 74 pixels have to be resized prior to transmission because of the capability of the used protocol. This resizing makes it difficult for users to decipher long words.
  5. The payment mechanism is in the form of mobile credit this limits its expansion. Most users perceive it as a easy way for mobile credit and it can not be translated into a full time work like MTurk.

Flash Teams

Flash teams framework is an innovative step designed to dynamically organize expert crowd source to carry out complex and innovative interdependent tasks. It has been said " A flash team is a linked set of modular tasks that draw upon paid experts from the crowd, on demand. The napkin sketch design team follows the user-centered design process to create a series of prototypes and iterate based on feedback to produce a user-tested software prototype within a day."Foundry is an application functioning as a flash team authoring environment with support for diverse expertise, and a runtime management platform.

It has following useful features:

  1. It allows outsourcing of broad-range of goals, like course designing, design prototyping and film animation.
  2. The efficient pipe-lining of sub-tasks allows the task to be completed in half the time taken by traditional teams.
  3. The transparency of task structure provides the organization an opportunity to leverage it for a better orchestration of crowd dynamics.
  4. The elasticity of the flash-teams allows the team to dynamically shrink or expand according to the needs of the project.
  5. Flash teams are the first to leverage the scale of paid crowdsourcing for expert work.
  6. Foundry’s composeable team structures are motivated by OB studies of the purpose and functioning of team structures, enabling coordination among team members.
  7. The possible diffusion of responsibility has been duly avoided by the appointment of Directly Responsible Individual.

The following mechanisms have been used by Foundry to optimize the work-organization:

  • Modular combination of teams
  • Elasticity
  • Pipe-lining

The system can address these issues for better results:

  1. As the task is broken into modular inter-dependent sub-tasks, it is hard to use quality control measures.
  2. More often than not will be the cases that teams do not always work according to plan, we need a mechanism to resolve conflicts and restructure the team with loss of expertise.
  3. Checking the authenticity of the worker's expertise as the project's success would be directly dependent on that.