WinterMilestone 4 Adding Tags To Tasks For Better Categorization And Ranking

From crowdresearch
Jump to: navigation, search


The Crowdsource platforms like Mechanical Turk , suffer from the improper matching of tasks and workers , which results in bad work quality which is in truth, not due to worker inefficiency but because of wrong selection on the part of the worker , these uncertainties in the task requirements faced by the workers must be removed so as to accomplish better work quality and greater requester-worker trust , the improper matching of workers with the tasks result in poor quality of work leading to two major effects:

1) The requester labels the worker as a poor inefficient worker and provides less wages then expected by the worker.

2) The low wages dishearten the worker and result in anguish towards the requester.

These two effects ruin the requester-worker trust and results in a loss for the crowdsource platform . To achieve trust between requester and workers , the proper workers must be matched with their matching tasks so that they can perform their best , leading to better work quality .

We have seen that if the Tasks are provided with additional tags which describe certain properties of the task , this could aid in finding proper matching workers fulfilling the criteria .


The Worker-requester model is very essential for a successful crowdsource-platform , Workers must be matched with respected Requester , this is greatly accomplished by Boomerang reputation , boomerang will allow a better Worker-requester interaction , the negative effects of bad worker-requester matching results in a downward spiral for both requester and worker . A major factor in this downward spiral is non-descriptive tasks , unclear and misguiding tasks lead workers to produce different set of outputs which are efficient according to them , and unholy for the requesters , the requesters blame the workers and pay them poorly . This problem has been addressed by Daemo's prototype tasks which aims to enhance task quality by creating a feedback loop . Boomerang and Prototype tasks have reduced the gap between worker and requester but still one issue leads the system into downward spiral , Inefficient matching of task and worker . Each worker has a set of skills which are unique and distinguished and so are the requirements of a task , so it can be that a worker is really efficient in a particular skill but the task he selected requires a different skill which the worker lacks , and thus worker creates poor work , It should be noted that its not the worker who is inefficient but the wrong selection that is generating the poor work ! This can be eliminated by Providing ranking to the tasks , based on which those tasks would appear in a chronological order in the task feed in a different manner for each user according to their skill set, thus allowing the worker to intelligently choose the task which are more in-tune with his/her skills and thus can provide him/her with better wages and lead them to produce better quality work.

Our Idea for generating a ranking for the tasks would be by providing a list of tags which would be provided by the requester at the time of task creation , those tags would also be reviewed and modified by workers as in prototype tasks , the tags can also be reviewed by other requesters if the open governance idea comes into play , this would provide a strong solid tag list for each task , these set of tags would then be used to find most promising workers and then the task would be listed first in those workers' task feed , and similarly the process would create a chronologically ordered task list for each set of worker.

This idea helps in segregation of different types of tasks and also helps in ranking the tasks for each worker , leading to better selection of tasks and better work quality.

Related Work

Many different Organizations have been using tags for segregation and classification of objects for different users. We studied certain programming websites like codeforces ,codechef ,spoj.Codechef uses tagging system for tagging different programming problems so that different level of programmers can sort and work on the ones which they find matching with their set of skills.The organization has a discussion forum where segregation can be done based on different tags , thus users can find relevant information easily with the help of particular tags . Tags help novice programmers pick up easy problems first and help them save their time which could have been wasted if they solved very hard problems!

An example of the tagging system can be seen here ->

The codeforces platform provides programmers with a color rating after their performance in each contest which determines their level , red being the topmost level. The rating divides the programmers into two Divisions , Div 1 and Div 2 , when a contest is conducted on codeforces Div 2 (low rating programmers) cannot take part in Div 1 contest , this is a clear example of how a task can be tagged with certain properties and those workers which match can access the task quickly!

Another example of tagging is Quora which is a platform for Q&A , the problems on quora can be tagged which help in better match of users with those asking questions! Quora also gets the problems tagged manually , if somehow we could add tags automatically using machine learning , that would be a nice upgrade!


The tags must be selected from a list of already present tags which can determine each attribute on each scale , so as to easily classify each task , the tag field must be auto-complete field ,and must be modifiable to add certain peculiar tags for certain tasks .

Matching Algorithm

Once the list of tags is received and the list of workers with each having a set of skills,reputation score etc , we can use certain matching algorithms to find the best matching pairs like Stable Marriage Problem.


Achieving a better matching between worker and task is very crucial and must be solved quickly, this method aims at finding better matching tasks for each worker so as to produce good work quality and in turn help workers to each good wages!

Research Engineering

Made a pull request for the Advanced options on authoring Page issue #646

Made a pull request for autofocus for first fields on authentication pages #634


Slack handles:Team Bits-> @shivangi and @shivam

Github handles:-> insomniac12(Shivangi) and Curious72(Shivam)