Milestone 3 PixelPerfect
Team Pixel Perfect's submission for Milestone 3.
- 1 Initial Brainstorm
- 1.1 Trust-related Ideas
- 1.1.1 How do requesters trust workers doing their tasks and the results they get?
- 1.1.2 How do requesters trust workers and vice versa?
- 1.1.3 How do workers trust they are not being exploited?
- 1.1.4 How do requesters trust they will get their results on time?
- 1.1.5 How do new workers get started quickly and earn the trust of their counterparts?
- 1.2 Power-related Ideas
- 1.1 Trust-related Ideas
- 2 Dive Deeper into Specific Ideas
- 3 Dark Horse idea
Crux of idea goes here.
How do requesters trust workers doing their tasks and the results they get?
- Mechanism on the platform where workers who have the appropriate skillset and demographics are eligible for the tasks.
- Unique identification requires each worker, requester to submit valid ID proof to verify their genuineness. This might be helpful in certain HITs which are demographics specific (eg. Surveys involving only native American women).
How do requesters trust workers and vice versa?
- We use the Facebook social graph model to conceptualize worker-requester relations and compatibility.
- We develop a crowdworker graph
(bipartite for now i.e no relations within workers or requesters) whose nodes are requester and workers, and compatibility is modeled by the edge weights.
- Compatibility develops through a two-way feedback system between workers and requesters after completion of a task. It is centralized
around a simple Yes/No question :
- Would you like to do a task from this requester again? [For workers]
- Would you delegate a task to this worker again? [For requesters]
- Compatibility plays a role in how visible a given requester’s HIT is, to various workers. Workers having higher compatibility with the
requester are more likely to see the HIT and therefore take it up.
- We develop a crowdworker graph
How do workers trust they are not being exploited?
- Implementation of Multi-level rejection system which also involves workers. Involvement of requesters and their counter parts makes rejection process much more fair and justifiable.
- Penalty system for delayed worker payment. Workers would be compensated for the delay in the payment as quite a large percentage of people rely on these platforms for their daily living.
- Changing the legal system. A common issue is work theft, where requesters do not acknowledge work submitted, however incorporate it in the results. This happens because requesters enjoy intellectual property rights over the work submitted and not the workers. A legal solution might be tough to implement considering the vast multitude of countries workers come from.
How do requesters trust they will get their results on time?
- Implementation of a Priority Queue system ,where tasks nearing their deadlines are given higher priority.
How do new workers get started quickly and earn the trust of their counterparts?
- New workers go through some initial computer generated tasks whose solutions are provided to them - we can gauge their skill set, accuracy and efficiency. This acts as their initial seed.
- Templates for design of HITs can be given to new requesters to enhance the clarity in their tasks.
What powers do workers have to feel respected and voice their opinion?
- Two-Stage Worker feedback system for a particular HIT and requester.
This is 2 line review of the idea.
- Issue column attached to every HIT where requesters can suggest improvement or give tips on completing the task.
This is a 2 line review of the idea.
What powers do workers have to find HITs quickly?
- The visibility of HITs on a workers dashboard will be determined by previous work, skill set etc.
- Workers can arrange HITs according to the time taken to complete the HIT, in order to facilitate their work plan in that particular sitting.
- Reducing idle worker time. A common issue on current crowdsourcing platforms is, certain weeks generate good payoff for workers while other weeks don’t. While there is no effective way to predict trends in task demand, we can look for compensation techniques. Auto-generated tasks whose solutions are known can be made available at all times. During idle scenarios (low task demand), these can be taken up for work. This enables workers to improve their experience and ratings during downtime, which in turn makes them eligible for tougher and more intricate tasks later on.
What powers do experienced users of the platform have?
- New rating system, separte for both workers and requesters. This system basically involves grading based on different factors like for workers it can be, efficiency, accuracy, skill level, credibility, compatibility, approval rate, experience, social graph, etc.
- Top-level worker/requesters have moderator like privileges.
- Moderators can classify tasks into various categories by tagging tasks with keywords. This would ease down the HITs filter process, making it much more accurate.
- High-skilled workers can access tasks having higher complexity and payscale.
Dive Deeper into Specific Ideas
For each of the 4 ideas (2 for trust, 2 for power), describe (using diagrams, sketches, storyboards, text, or some combination) the ideas in further detail.
Please create a separate wiki page for each of your ideas, so we can link to them individually. The title of the wiki page should be Milestone 3 followed by your team name and a description of the idea itself (ex: Milestone 3 YourTeamName TrustIdea 1: Automatic Pricing for Tasks based on Average Completion Time). Once done, post a link to each of your trust-related ideas to http://crowdresearch.meteor.com/category/milestone-3-trust-ideas and a link to each of your power-related ideas to http://crowdresearch.meteor.com/category/milestone-3-power-ideas when done, following the instructions at http://crowdresearch.meteor.com/posts/bXSNbqihjajASBQEL
- Milestone 3 YourTeamName TrustIdea 1: Description of some trust-related idea - http://crowdresearch.meteor.com/posts/TaNbEkmeoSSiv5vDB
- Milestone 3 YourTeamName TrustIdea 2: Description of another trust-related idea - http://crowdresearch.meteor.com/posts/ShEsGS3tx2Gdq9noJ
- Milestone 3 YourTeamName PowerIdea 1: Description of some power-related idea - http://crowdresearch.meteor.com/posts/HEiG2GyJWLFPzThYc
- Milestone 3 YourTeamName PowerIdea 2: Description of another power-related idea - http://crowdresearch.meteor.com/posts/qsv2bcRNjSL3PC8sC
Dark Horse idea
Describe your dark horse idea (using diagrams, sketches, storyboards, text, or some combination).
Please create a separate wiki page for your dark horse idea so we can link to it individually. Post the link on http://crowdresearch.meteor.com/category/milestone-3-dark-horse-ideas when done