Milestone 6 PixelPerfect

From crowdresearch
Jump to: navigation, search

This is Team Pixel Perfect's submission towards the research-oriented component of Milestone 6.

Title

GenNextTurker : User categorization and task classification based platform.

Abstract

Existing crowd-worker platforms have a tall entry barrier making it difficult for new users to use the platform effectively. Moreover requesters would like to delegate tougher tasks to advanced workers. We therefore propose categorizing workers into tiers based on a rating metric - entry level workers receive aid to establish and familiarize themselves with the platform. Higher level workers earn privileges like offering mentorship, task completion tools etc.

A significant component of worker time goes into searching suitable tasks based on skillset, time, payment and other factors. We attempt to minimize this selection time through a smart task classification system.

Mentorship and task templates are additional features presented in view of our goals to help new users and increase task clarity.

Motivation

We are attempting to solve three major problems here :

  • Existing platforms work on a 'rich get richer' philosophy making it difficult for entry-level workers to contribute effectively. We focus on decentralizing power from the 'richer' users to the newer ones. Mentorship and task templates features have been added keeping new users in mind. (Issue of POWER)
  • Requesters expect good quality work from workers - they would want tougher, complex tasks to be done by more experienced workers. Our idea of a rating system helps introduce a trust component in evaluation of worker submissions. (Issue of TRUST)
  • Help reduce the task selection latency time (time required to select next task) and streamline workflow for workers. (Miscellaneous)

Related Work

Some work we came across which tried to address the issues similar to ours are laid below.

How is it related to our paper
Though the above paper uses a similar approach in selecting the most suited task, incorporating more or less the same parameters we use. However it provides the worker with a list of choices from which he may choose,they automatically assign tasks to the workers using their algorithm.

How is it related to our paper
This paper studied a thorough analysis of time spent in task-searching. The results from this paper can be used to construct our task classification system. For example, the results state that most tasks are chosen from the first or second page of search results. Thus, after applying our classification algorithm, we will aim to populate the first and second pages more.

Insight

The foundations and the features upon which model of proposal platform are laid down below :

Foundational ideas

  • Rating System : Quality output, that is what requesters expect from workers at the end of the day, but for that matter what goes under the hood plays a vital role. Obviously assigning highly complicated tasks to amateurs wouldn’t produce satisfactory output, so to overcome these issues workers are categorised into different tiers.
  • Efficient Search System : Requirement of third party tools to search for appropriate tasks made workers trade off their precious time for searching jobs. Incorporating this system will help workers search jobs easily based on certain parameters such as skillset, time requirement, money offered, etc.

Features

  • Task Templates : New requesters need to get over a lot of hurdles on crowdsourcing platforms and the most common among these is designing the task. The platform provides the requesters with a well structured task descriptor helping them rev up the task generation process and also improving upon task clarity.
  • Mentorship Program : Providing Mentors to novice workers to get them use to the turking environment.

System

Here is the work-flow of the system that our team proposes :


Rating System

Workers and requesters are rated keeping certain parameters into considerations like accuracy and promptness of submission for Workers, while task clarity and professionalism for requesters. Levelling up earns them more privileges and will unlock more complex and highly paid tasks. Workers and Requesters both are categorised into three tiers.

The basic working of the tier system is same for both the parties, workers and requesters, but they get different types of benefits out of that.
Tier.png
  • For workers : Initially starts off from the base level i.e. Novice. Workers are confined to certain set of tasks only. In case of in-availability of tasks which is highly likely for this tier, they are provided with computer generated tasks for computing the initial seeding of the worker. Based on the quality of output worker generate, experience, and lot of other factors decides whether the worker is qualified enough to move to the next tier. In tier 2, certain privileges are unlocked like HIT recommendation system. Moving on to tier 3, the apex of the rating category, highly complex tasks are opened to them.
  • For requesters : The requesters also starts from the base i.e. tier 3. The system goes same with requesters as with the workers. Moving to higher tier unlocks them privileges like tools for generating tasks increasing their productivity, reviewing the rejection proposals by lower levelled requesters, etc.


Efficient Search System

Incorporating a better search system would help workers search jobs much more efficiently by narrowing the search based on potential factors : skills required, worker rating, requester rating, time required, money offered, etc. This will help workers get rid of third party tools which requires a great investment of time.


Task Templates

Task clarity is of utmost importance as far as output quality is concerned. This help requesters by making task generation faster and much more structured. Requester will be asked whether to use task template or not. In case templates are used, requester will be presented a structured task descriptor consisting of the tags(type of work involved), skills required, description of the task, adding a video about the task(optional), etc.

Template.png

Mentorship Program

Experienced Workers can mentor novice workers for getting them over some issues that novice workers generally faces. Mentors(mainly experienced workers) can voluntarily opt for mentoring, Mentors have an option for which type of work they want to mentor. And talking about mentees, they have an option whether they need any mentor or not. Rationally, Mentors also seeks for their benefits, so in this method mentors would benefit by getting a boost in their rating, which might help them in getting into higher Tier unlocking privileges and tasks, or also by getting a share from the completed task. The boost in their rating will mainly depend on the feedback by the Mentees.

Evaluation

We aimed to solve fundamental issues of trust and power prevalent in current crowdsourcing platforms.

An easy to fill but detailed survey will be filled by the participant requesters and workers about their experience after completion of tasks will affect the ratings and compatibility of the parties involed , we expect the results from these feedback form will help determine the degree of success in rectifying the issues we aimed to resolve.

Rating System - We expect the rating system present on our platform to solve the trust problem faced by requesters regarding the work they get,our system insures that if the task is aptly priced, only a high level worker will be able to do the task ,which will reassure the requester about the quality. Our rating system also aims to solve the power problem seen in previous crowdsourcing platforms where requesters have the final say in matters of dispute and can reject work without justification , our system provides a separate rating for requesters and their rating and credibility will suffer if they indulge in unprofessional work,reducing the workforce(both quality and quantity wise) they have available for their jobs.

Efficient Search System - Workers often complain about a lot of time spent searching for work opportunities on the platform they've been using,our efficient search algorithm takes into account worker-requester compatibility,worker level and skillset for different type of tasks to match the worker with the jobs most suited to him,the worker can choose to sort the jobs either by time required for each job , or by payment depending upon need.This would assist the worker to concentrate on tasks that he takes on,as the burden of finding tasks is on the platform . Also,since the tasks available to the workers are those most suited for them,it guarantees high results and more trust between the worker and employer.

Task Templates - A problem faced by the requesters is that the task design system isn't flexible enough for tasks of increased complexity and also,it is often the case that requesters themselves don't have a fair grasp on the kind of results they expect,the introduction of task templates will help requesters to enhance the clarity of the tasks posted by them,which will increase the quality of results submitted by the workers.

Worker-Requester Mentorship Program - There is often a high-learning curve accompanying getting accustomed to the platform and achieve maximum productivity, the Worker-Requester mentorship program will connect novice participants to experienced workers/requesters who'll guide the new participants boosting their rate of growth on the platform

References