Introducing skill categories to bridge the gap between requesters and workers

From crowdresearch
Jump to: navigation, search

Abstract

Crowd sourcing is highly evolving business model where tasks are completed by a lot of people. Crowd sourcing is used in numerous backgrounds like information retrieval, marketing, software development and many more. Although there had been many platforms available for crowd sourcing, these platforms fails to meet the needs of requester and worker. This paper aims to find solutions to bridge the gap between the requester and the worker by introducing the concept of skill categories. the task feed ranking should leverage the knowledge of skills possessed by the workers. By doing so our platform will be able to provide a seamless connection between a requester and the worker. A requester will demand a specific category and the worker will search for the tasks that are in lines with his skill set. This whole process will improve the quality of task results as well as the time required to finish up the tasks.

Keywords

Crowd Sourcing; Skill Categories; Ranking with skill category

Introduction

The only goal a crowd sourcing platform has is to find relevant work for workers as well as finding equally satisfactory up to the mark results for a requester. In order to meet this goal a crowd sourcing platform should offer desired, hassle free tasks so that worker is able to gives his best efforts. If a worker is offered tasks according to his skills, he will be much more encouraged to finish up the tasks. This will improve the quality of work submitted. Another proposal is ranking the workers according to their skills. For example, the worker might have outstanding rating in web designing but average rating in translation of languages. hence a requester that demands high quality work will approach workers according to the high ranking in a specific skill he is looking for.

The problem at our hands

  • A requester will always have specific tasks. A given task might belong to a particular category. The top ranked workers of a given requester might not know how to do a task that the requester has requested. This problem will degrade the result quality as well as the requester have to devote more time in finding a suitable worker with required skill sets. For a given requester the top ranked workers by him will be higher up the ladder in terms of availability of tasks requested.
  • Let's say, the requester has been asking for a specific skill related task (Like translation), so he ranked the worker according to it. What if he now submitted another task that requires knowledge of website designing. His profile will still shows workers ranked according to the previous tasks. At the same time for a worker, the task feed will show the tasks that are submitted by the requester that has given him top rankings. The same worker does not have the necessary skills (web designing). Hence, the quality of his work will also suffer.

Creating Profiles

  • A worker must have a profile/account on the platform.
  • Every worker specifies various categories he is interested to work for.

4.jpg

Impact of making Categories

The categories can be made as drop down menu so that the worker have specific options to choose from. The categories can be modified anytime the worker wants. The task feed of a worker should be according to the his top ranked requester under his mentioned categories. If his top requester submits a task that is not enlisted in his area of expertise, that particular task should rank below in his task feed. Similarly, the worker list for a requester will be filtered out according the task category he has requested for as well as his top ranked workers.

Modifying the task feed and worker lists

  • A requester can see top ranked workers by other requester in a specific category.
  • A worker can see top ranked requester by other workers in a specific category.
  • Once a worker modifies his skill sets , the task feed will also change accordingly.

Benefits of categorization of tasks

The benefits of making categories as a mandatory field are manifold. Finding a suitable worker would be a lot easier for a requester since his feed will not only be according to his top ranked workers but also there skills. This will going to save him a lot of time. Similarly finding a work oh his interest a worker does not have to browse though all the tasks in his task feed. This is going to improve the quality of work as workers who does not a particular skill will not attempt such kind of tasks.

Rating with skill category

The overall rating of a worker will be a combination of his skill ratings. Such ranking will also encourage workers to not take up tasks that they're not sure they can finish up in time with the required quality.

2.jpg

Analysis

The proposed can be analysed by first defining few general categories like Software Development, Language Translation, Marketing etc. A subset of workers can be asked to pick tasks according to their mentioned skills. The time taken by the workers who have previously mentioned the skills is measured against the workers who does not filter out tasks according the skills. This metric can be measured to carefully analyzed to compare the results between the two cases. Along with this we can analyze how far does the worker has to 'scroll down the page' before finding something they are interested in working. Another metric according to which this proposed model can be measured is requester satisfaction. The ratio of total tasks accepted by the requester in case of specifying the skills and not specifying can be measured. Both these two metrics are enough proof of improving the overall efficiency of the platform

Follow up studies

  • Experimental studies to measure the time taken by workers to finish tasks in case of both specifying and not specifying the skill categories can be studied.
  • Analyze how far the workers scroll down the task feed to find relevant tasks.
  • Also, the ratio of task accepted and rejected in both the cases of specifying the skill categories can be analyzed.

Both of these studies can be done on a subset of requester and worker to obtain the sufficient iterations of the results.

Conclusion

In this paper, we have proposed a ranking system that leverages the knowledge of skill set of workers. Currently, we are searching for new features that can improve the accuracy of our prediction model for task feed ranking in crowd sourcing systems. Besides, we are conducting experiments to compare the performance among our proposed model of ranking using skill set mentioned by the worker and the skills required by requester. In the future, we will would want to show experimental results that will ensure the effectiveness of our proposed model.

References

  1. Martin D, Hanrahan B V, O’Neill J, Being a turker ACM, 2014:224-235
  2. Man-Ching Yuen, Irwin King, and Kwong-Sak Leung Task Recommendation in Crowdsourcing Systems
  3. Mahmood Hosseini, Keith Phalp, Jacqui Taylor, and Raian Ali The Four Pillars of Crowdsourcing: a Reference Model
  4. STEVEN P. DOW, ALANA GLASSCO, JONATHAN KASS, MELISSA SCHWARZ,DANIEL L. SCHWARTZ, and SCOTT R. KLEMMER Parallel Prototyping Leads to Better Design Results, More Divergence, and Increased Self-Efficacy
  5. Carrie J. Cai , Shamsi T. Iqbal , Jaime Teevan Chain Reactions: The Impact of Order on Microtask Chains

Contributors

  • Archana Kumari @dhankie
  • Dinesh Dhakal @dineshd