WinterMilestone 4 beingcooper

From crowdresearch
Jump to: navigation, search

Introduction to Dynamic Task Pricing Model in Daemo

What's the problem you are solving

Pricing Model

Pricing of tasks is a very important factor in crowd-source platforms. A significant amount of workers' time is invested in searching tasks with right work-price balance[1]. Requesters depend a lot on prices as well. They are in search of a threshold amount that can help deliver quality work in time[2]. Although prototyping in Daemo does help requesters to find the right price[3],but number of workers involve in prototyping is small and price,once settled, remains constant throughout and affects rest of the workers. Since price assigned to a task plays such an important role, can we keep it dynamic? This submission introduces the concept of Dynamic Pricing.

Example

Imagine that a Daemo worker has proficiency in Python but no knowledge of Java. An average coding task at Daemo is priced at 1. He has two tasks on his task-feed, one is to write a python code for 0.8 and other is to write a Java code for the same price. He is more likely to choose the first one. Although priced less than average, but as he is quite good with Python, the task wont take much time and he need not learn any additional skill for that. He will certainly not choose the Java task as the efforts required there do not match with the stipend provided. However, an increment in the price may entice him to learn some basic Java. Workers may accept a task for a smaller price if they believe they can complete it quickly without the need of any extra effort and there are better chances of getting it accepted. Requesters also will not complain if there work is getting delivered at lesser cost and that too by domain specialized workers.

Related Work

  • In Pricing Tasks in Online Labor Markets, Yaron Singer and Manas Mittal have proposed a bidding mechanism which allocates assignments to an arriving worker only if her cost is below a certain threshold price that has been computed using previous bids, and pays her the threshold price.
  • In Truthful Incentives in Crowdsourcing Tasks using Regret Minimization Mechanisms, Adish Singla and Andreas Krause have designed a mechanism using multi-armed bandits for online auctions to procurement auctions under budget constraints using a regret minimization approach.

Concept

The idea is to keep the task pricing dynamic.

  • Step 1: Requester may finalize the price of the task with the help of prototyping. However, she wont release the task immediately.
  • Step 2: The task will first be released to the workers of the same domain (as the task) at a price slightly lower than the one calculated.
 For example, If the task is to develop a python code and the price decided is 1, it will first be released to the workers having experience in     
 Python and that too at a lower price, say 0.8.
  • Step 3: The requester will fix a time limit and if it turns out that the task has not received much attention in the allotted time period, the requester will have two options.
  • Step 4: Either she can increase the price of the task and still keep it under the same domain, or she can release the task to a larger domain with the same price(for example, from python coders to computer engineers).


Dynamic Pricing


Important Points

  1. The main idea behind this concept is that the time require to complete a task vary significantly from worker to worker.
  2. Workers having experience in the same domain as task will require appreciably less time for the task and hence, they may prefer to attempt it at a slightly lower price.
  3. This price will also ensure that they will face less competition as not many workers (specially from other domains) will attempt such a task at the initially offered price.
  4. This concept also ensures that requesters get quality work delivered, by domain specialized worker, at a price lower than the one calculated during the prototype phase.

Workers

This concept may sound a bit unfair to workers. There may be questions like why should a worker be paid less for a Python task just because he is good at it or this may encourage workers to delay submission in order to get better price. However, this concept actually provides more flexibility to the workers by opening a window of options for them.

  • Workers in crowd-source platforms come from various background with different needs.
  • Although monetary gain is the primary reason, for some it is more like a side income, for others it the case of survival[1].
  • This idea allows them to set priorities. They can accept the task at a slightly lower price with less competition (more chances of acceptance) or they may prefer to wait for the prices to go up and do it for a decent price but with more workers competing for it.
  • For example, a Daemo worker A whose aim is to earn some side income or to learn new skills may decide to work only on decently priced tasks, hence she may wait for prices to go up. On the other hand, another worker B, who is using Daemo as her primary source of income, may prefer a smaller price with less competition that can ensure more chances of earning.
  • So it becomes more suitable to the workers' needs as they are free to choose an option according to their current priorities.

Requesters

Requesters will have twofold advantage form this idea. Dynamic Pricing brings down cost and improves quality.

  • The financial burden will reduce as a portion of the task will be completed at a lower price than the one decided during the release. This will depend on the initial cost of the price and also on the timeline set by the requester.
  • Number of the workers belonging to that particular task-domain will also effect what portion of the task will get completed in the early phase.
  • We can safely eliminate the possibility that many of the non-domain workers (regarding that specific task) will be involved in the early submission phase as they may found no match between the price offered and the effort they will be putting in.
  • Hence, the quality will improve we early submissions will mostly consist of the submission by domain-specialized workers.

References

  1. Martin D, Hanrahan B V, O’Neill J, Being a turker ACM, 2014:224-235.
  2. Kittur A, Chi E H, Suh B.Crowdsourcing user studies with Mechanical TurkACM, 2008: 453-456.
  3. Stanford Crowd Research Collective.Dameo
  4. Y Singer, M Mittal, Pricing Tasks in Online Labor Markets
  5. Adish Singla, Andreas Krausemechanisms,Truthful Incentives in Crowdsourcing Tasks using Regret Minimization Mechanisms


Milestone Contributor

  • Sachin Sharma @beingcooper