WinterMilestone 1 Algorithmic HummingBirds

From crowdresearch
Jump to: navigation, search

INTRODUCTION

Crowdsourcing is the process of working, usually online, with a large group of people and is a combination of word 'crowd' and 'outsourcing'.


SUMMARY OF MEETING 1 '' The current set of crowdsourcing platforms do not ensure high quality results, produce inefficient tasks and suffer from poor worker-requester relationships. In order to overcome these issues, we propose a new standard (the next de-facto platform), Daemo which includes Boomerang, a reputation system that introduces alignment between ratings and likelihood of collaboration. We can accomplish this together with deadline induced milestones and the same for week 1 is: Summary on experiencing life with Mturk, exploration of alternative crowd labor markets and reading of some great research work like - Daemo, MobileWorks and FlashTeams.


EXPERIENCING LIFE ON MECHANICAL TURK '' Amazon Mechanical Turk (popularly MTurk) is a crowdsourcing Internet marketplace that enables individuals and requesters to coordinate the effective use of human intelligence to perform tasks with the help of Human Intelligence Tests (HITs) that computers are currently unable to do. Being located out of the United States geographically, I couldn't get the requisite permission to work on Mturk. I went ahead and tried out a few tasks anyway.


EXPLORING ALTERNATIVE CROWD LABOR MARKETS '' CLICKWORKER

Strengths The UI of the portal is great. It is self explanatory which is great for non technical users. Structure of the texts is well thought out. The flexibility provided because of the division into micro tasks is really convenient.

Weaknesses Domain knowledge maybe required to deal with crowdsourcing of scientific tasks. Job opportunities are great for specific groups. There is a penalty imposed for skipped tasks which may not always be fair to the worker.

MICROWORKERS

Strengths The rating is flexible. The UI is great with jobs and tasks which are easy to find that also match with your skills. There is also a great payment incentive scheme. There also exists unlimited positions to jobs.

Weaknesses There are tasks which impact your success rate and temporary success rate which might limit our performance. There were also complaints about the poor customer support on the site.

TASKRABBIT

Strengths they have an app for the purpose of crowdsourcing and the tasks also employs gamification techniques which should inculcate the fun component of the tasks. Full time positions are available.

Weaknesses They cater only to a limited geography and is available only in English. One may need specific domain experience to attempt some of the tasks.

AMAZON MECHANICAL TURK

Strengths Tasks (those which require qualifications and those that don't) are easy to find with simple and compact user interface and design which is self explanatory which would help a non technical user. The number of tasks is massive and of huge variety.

Weaknesses The user interface could have definitely been a lot better. Unfair schemes of wages have been adopted as it's surprisingly and extremely hard to earn on the platform.The initial login itself takes upto 48 hours.

UPWORK/oDesk

Strengths The UI is really state of the art and can cater to multimedia renderings.

Weaknesses It may require domain knowledge specially for some of the tasks. Getting started might be quite a challenge.

GALAXY ZOO

Strengths The UI is simple and easy to learn along with interactive design work and tables. It is available in many languages.

Weaknesses It may require domain knowledge specially for some of the tasks.

CROWDFLOWER

Strengths The UI is well designed with an instant payout scheme. It can support a variety of content and is geographically open to all continents. Awareness of behavioral tasks is available.

Weaknesses It may require technical audience specially for some of the tasks.


MANDATORY READINGS ''


MOBILE WORKS: A MOBILE CROWDSOURCING PLATFORM FOR WORKERS AT THE BOTTOM OF THE PYRAMID

Strengths Its a mobile phone based crowdsourcing platform intended to provide employment to marginalized workers in developing countries through the deployment of human optical character recognition (OCR) tasks that can be completed with great accuracy on low end phones using a simple web browser interface. The goal is to broaden participation in micro task market to enable livable wages to such workers. It also considers multiple entry error detection so it is competent at processing the tasks.

Ideas If we can minimize the use of the web by enabling offline task completion (at however small a scale), it would help the economy to some extent since we are considering marginalized workers in developing countries.


EXPERT CROWDSOURCING WITH FLASH TEAMS

Strengths Flash Teams is a framework for dynamically assembling and managing paid experts from the crowd which helps accomplish complex, interdependent goals or a sequence of linked tasks. It also introduces elasticity in the system and explores the feasibility of solving interdependent micro tasks via structured collaborations. In order to enable this, Foundry is introduced which is an end user authoring platform used to author modular tasks and run time manager which can then be extended to film animations and design prototyping. It helps effectively leverage diverse skills in a group and is scalable.

Ideas The idea is, no doubt, absolutely exciting. But I think the platforms like these and also our organisations like schools and colleges are more designed to cater to extroverts than to introverts (like myself). In this beautiful world of extroverts, I think we, introverts, too have something unique and beautiful to offer. I understand that the framework is scalable at different levels but I am thinking if there can be a new approach to it, say maybe, psychological or affective computing?


DAEMO: A CROWDSOURCED CROWDSOURCING PLATFORM

Strengths Collaborating efforts of a massive online community in order to build an upstream model of a new standard in crowdsourcing platforms with the help of boomerang, a reputation system that aligns the ratings the worker or requester receives and the likelihood of collaboration between them thus, building strong relationships between the two. We will also be hopefully bringing in fair principles of payment to crowd workers.

Ideas There is a certain redundancy of tasks across requesters and this results in many individuals repeatedly working on the same problem in isolation. So, if we could enable efficient image processing and pattern recognition systems to recognize patterns across tasks we might be able to minimize the redundancy. We can have efficient machine learning and artificial intelligence algorithms to apply similar principles to solve problems or tasks with similar patterns.