Milestone 3 pixelpals

From crowdresearch
Revision as of 00:15, 19 March 2015 by Srishtysaha (Talk | contribs)

Jump to: navigation, search


Initial Brainstorm

The ideas you brainstormed, (at least 10 ideas for trust, and at least 10 ideas for power). Provide them in whatever format you want - diagrams, sketches, descriptions, or a combination (the wiki supports images, see here for instructions on uploading them).

Trust-related Ideas

1. Increasing trust by establishing a peer - rating system (5/7 Likert scale ) (by fellow workers ).Increasing trust by establishing a requester rating system ( 7 Likert scale ). Requester rate workers on a variety of criteria’s like Speed, Accuracy, Commitment, etc.

2. Automated pricing range establishment based on skills and average time to complete work.

3. Increasing trust by ensuring mandatory feedback from requesters upon each HIT rejection (maybe acception too). Increasing transparency increases trust b/w workers and requesters.

4. [Maybe] A forum/page where requesters can put up the HIT received as a response to their work to show an exceptionally poor or exceptionally great work. [Possible privacy concern.]The intended purpose is to rewards deserving works which will lead to possible rating increase.While at the same time, shame spammers/poor quality work so that these workers either improve or face ratings drop and possibly removal from the system.

5.How might we create a platform where workers can be confident about the task? Background: Workers need to be confident that they understand the goal of the task. Many times workers avoid tasks that have ambiguous goals or might result in errors that get them rejected. They become risk-averse when a task might be confusing and affect their reputation or payment. Solution: There should forum associated with the task where workers can ask questions and requesters can clarify them. There are certain common queries associated with a task which may be missed out and needs to be clarified. In this way, people can post queries in the public forum and everyone can benefit from it, reducing the ambiguity associated the task. In fact fellow workers can also clarify them. Solutions or clarifications can be upvoted or downvoted by the workers. Although there exists a mailing system, but generally workers mail the requesters for payment and other issues. Moreover requesters generally don’t respond so quickly and in many cases they don’t reply because there is no compulsion. In the embedded Question-Answer forum, if requesters don’t clarify the doubts it highlights that the requesters are not interested and are not helpful. On the basis of this workers can trust or distrust requesters. Alternatively we can also create a chat room where workers can clarify their doubts.

6.Requesters need workers who have appropriate demographics to do their tasks.Requesters have a concern regarding the demographics of the workers for surveys. The demographic information is self-reported by the workers. The information is not viewed as trustworthy by the requests. This is a problem for surveys and correct demographic information is required. Solution: IP based demography verification can easily be fooled by the use of proxies. A person sitting in India can easily claim to be from other part of the world say, USA through IP spoofing. We propose a verification system based on unique security code. For such HITS we intend to send OTP to the mobile device of the worker. Majority of the people in this world use mobile phones [1]. In US, connection per number of citizens is ~ 103%. Mobile phones are subscribed to Telephone service provider which identifies the network of the device. The worker will enter OTP before starting the task. Using this in addition to the IP address, requester can ascertain the demography of the worker. additionally, there are API’s in Android and Iphone OS which can tell the network subscriber. Workers with Smartphones can choose this option and can download an App which will receive the OTP and share the required network (area) information.

7.Requesters need workers to have appropriate skills which they require for certain tasks.Workers self-report their skills which are not viewed as trustworthy. Solution: To overcome this, the platform or the requester can have a short pre-requisite quiz. In the quiz specific questions could be asked to ensure the worker possess specific skills. Many workers might not like taking quiz every time and it could be difficult for requester to create such questions. Therefore, we propose the platform to take care of this. The platform could offer short-term courses for certain skills and at the end it could take exams and quizzes to certify the worker. Since, this certification would be provided by the platform the requester could trust it.For instance, one specific skill could be editing or reviewing a blog post fir which proficiency in English is a pre-requisite.In this way, workers won’t exaggerate their skill sets since the skill-set is verified by the platform. This, in addition to the short quiz before the exam could be more effective.

8.‘How might we make workers trust he requesters?’ The existing system is point based where there is rat-run for approval points on sites like TurkNation. Workers talk about requesters and rate them. We need to eliminate this point system but at the same time ensure workers are able to trust the requesters.We propose a certification system. Initially when a requester joins he/she is uncertified. According to certain amount of HITs completed and on the basis of other factors like payment given on time, responsiveness to the worker etc requester can get certification. Having a Certification means the Requester could be trusted.

9. Automated pricing range establishment based on skills and average time to complete work. Determine the "pricing range" for a work by calculating the avg completion time and existing pricing/wage standards. Ask certain skill criteria (questions) from each requester related to his task. Put these into pre defined categories associated the pre defined price ranges. Add the total price as value added bonus from such skill values to the price based on avg completion time. The requesters must price his works within this range determined for each of their tasks . Since the pricing will be determined and enforced by the crowd sourcing system, both workers and requesters will feel a sense of trust in the pricing system since (/if) they feel a sense of trust for the crowd sourcing system

10. Re-evaluation of a Rejected HIT by a centralized committee. Workers can request for re-evaluation of their rejected task by a pre-determined committee. This will help build trust amongst workers that the platform is fair and not biased towards requesters. In case of a re-evaluation request, the committee (possible made of some pre-determined requesters and workers with high rationgs and not connected to the task in question in any way) will study the request. They can even ask the requester to give justification of rejection. The decision of the committee will be the final. If the decision is in favour of the worker then the requester is made to pay the complete payment to the worker plus a 10% penalty which can go to the committee. In case of a requester favouring result the worker will be made to pay a small penalty fee. This will not only ensure a possible payment/funding source for the committee and improve its efficiency but also ensure both workers and requesters take this task(Rejection re-evaluation) seriously.

Power-related Ideas

Dive Deeper into Specific Ideas

For each of the 4 ideas (2 for trust, 2 for power), describe (using diagrams, sketches, storyboards, text, or some combination) the ideas in further detail.

Please create a separate wiki page for each of your ideas, so we can link to them individually. The title of the wiki page should be Milestone 3 followed by your team name and a description of the idea itself (ex: Milestone 3 YourTeamName TrustIdea 1: Automatic Pricing for Tasks based on Average Completion Time). Once done, post a link to each of your trust-related ideas to and a link to each of your power-related ideas to when done, following the instructions at

Trust-related Ideas

Power-related Ideas

Dark Horse idea

Describe your dark horse idea (using diagrams, sketches, storyboards, text, or some combination).