Milestone 5 TuringMachine Mockup: Sustainable Reputation Mechanism
- 1 Author
- 2 Influence and Related work
- 3 Link to Storyboard
- 4 Reputation & Ranking: Leaderboard Based on Tranches
- 5 Risk Diversification with the Task Portfolio Manager
- 6 Behind the Interface: Ranking Mechanism
- 7 Behind the Interface: Algorithms - Automated Worker-Requestor Recommendations for the Portfolio
Influence and Related work
The Future of Work, Kittur etal 2013 and Genomes of Collective Intelligence Framework, Malone etal 2010 have shown that Reputation plays a big role in developing sustainable crowd sourcing communities. There is huge risk involve in the crowd sourcing work. Both requestors and workers are exposed to the risk related to payment or quality.
We build our idea of sustainable reputation using theories in structured finance and recommendation system. Currently there is no system or mechanism that can estimate and diversify the risk involved in the crowdsourcing work. We propose following reputation based portfolio diversification strategies to mitigate the risk:
- Diversified Task Portfolio for Workers: How should workers select the tasks that would diversify the risk of payment defaults and maximize the gain?
- Diversified Worker's Portfolio for Requestors:How should requestors select the workers so that they would diversify the risk of quality default and maximize the gain?
Link to Storyboard
Reputation & Ranking: Leaderboard Based on Tranches
Problem Design, Setting, & Environment
Risk Diversification with the Task Portfolio Manager
Problem Design, Setting, & Environment
Task Portfolio Manager
Behind the Interface: Ranking Mechanism
- Workers and requestors rate each other.
- An automated tranche creation algorithm clusters workers into A to E category.
Tranches Structure Penalties, Incentives, Risk of Default
- For the requestors in A tranche the minimum wage requirement is avg/below avg. However, for the requestors in D the minimum wage requirement is higher because he has a greater risk of default. Similar structure is designed for the worker. Please see figure below.
- Based on the ratings we can create the tranches of workers and requestors
- This structure exposes the risk involved in the crowdsourcing platforms.
- The workers can build the task portfolio using rating profiles and maximize the profit by diversifying the risk. For instance, a worker can select one task from tranche A, 3 from Avg, and rest from D.
The task flow
- Figure 3.1 shows the task claimed by 6 workers. This number can be larger.
Feedback & Visualization
- Figure below highlights the worker A's dashboard. Please read the diagram from #0 to #5 i.e. from the bottom to top
- The worker A receives real time feedback, motivational messages.
- The worker A can see the live task statistics and performance of his colleagues can motivate him to participate and do well in the task.
Over the period of time the system can build a network graph of workers and requestors who work well together. This can be further extended to build the teams that can work together on highly complex tasks.
Behind the Interface: Algorithms - Automated Worker-Requestor Recommendations for the Portfolio
- Diversified Task Portfolio for Workers: Recommend the combination of the tasks that would diversify the risk of payment defaults and maximize the gain. Workers can submit the diversification criteria.
- Diversified Worker's Portfolio for Requestors: Recommend the combination of the workers that would diversify the risk of quality default and maximize the gain. Requestor can submit the diversification criteria.
- We combine Recommendation engine with Tranches and automatically match up requestors and workers.
Combining Global and Local Effects
- Combine baseline method with latent factor model
Latent Factor Model for Workers to Requestors Recommendation
- Recommend workers to requestors