Milestone 3 ABC

From crowdresearch
Jump to: navigation, search

Template for your submission for Milestone 3. Do not edit this directly - instead, make a new page at Milestone 3 YourTeamName or whatever your team name is, and copy this template over. You can view the source of this page by clicking the Edit button at the top-right of this page, or by clicking here.

Initial Brainstorm

The ideas you brainstormed, (at least 10 ideas for trust, and at least 10 ideas for power). Provide them in whatever format you want - diagrams, sketches, descriptions, or a combination (the wiki supports images, see here for instructions on uploading them).

Trust-related Ideas

1) How can a new worker trust that a particular job is best suited for him/her: Develop a task recommender system

2) How can workers trust that the wage promised is appropriate for that job: Using statistics and crowdsourcing data, a supervised learning system can be devised to determine the optimal wage for a given task.

3) How can workers trust that the task information provided is valid: There could be a panel for screening the job posts, so that nobody can post jobs just like that. This will determine quality.

4) How can workers quickly recognize task tags according to their skills: The task recommender system will suggest the the appropriate tasks for a worker with a set of skill-set.

5) How will workers know exactly about the work duration of the task: We could add a feature to this system in form of Level of difficulty, for every HIT posted on the platform, which will be given by the Workers who have completed that HIT. So, when a new worker selects that HIT, he could get an estimate that how much time that task is going to take. These Workers, who have completed the tasks and whose reviews(level of difficulty) will be visible to the new worker, could be the ones who are trusted most by the given new worker or maybe who share the same skill set with him.

6) How can a worker trust that the work that he is doing is not meant for some illegal activity by the employer?: Profile completion of the requester must include relevant questions which can justify the intention of the requester posting the particular job. Social media screening to access background information. A panel in the Crowdsourcing platform must be made which will periodically monitor the requesters jobs and report anything suspicious.

7) How can a requester trust that the worker is not a spammer?: Background check, the worker's profile and reputation till now on MTurk.

8) How can a requester trust that the work done by the worker is genuine? Requester can take into account various parameters while evaluating the work of turkers. Some of the parameters are as follows:

-Time taken to complete the task: For instance, If a worker takes only a few seconds to complete a task that should take around a couple of minutes to complete, then obviously there is a serious doubt about the genuineness of the work.

-Various Anomalies: Just for the sake of completion of task, worker have the tendency to give random answers which is remotely different from the answers provided by the other workers. For example, in a survey task with lots of multiple choice questions, a worker can answer all his/her answers as choice “B”.


9) How can a requester trust about the validity of the worker's skillsets: A skill endorsement system (e.g. similar to the one in Linkedin) can be used where other requesters can endorse specific skills of the workers.

10) Good communication medium between turkers and requesters: Creating a chatting system between the requester and the Worker


Power-related Ideas

1) Who has the power to resolve disputes if there is an ongoing dispute between worker and requester?: Professional workers who have completed a broad range of jobs successfully and have a good reputation in their respective crowdsourcing platforms must be given power to mediate between workers and requesters.

2) Who has the power to provide help to the requesters while posting works: The system should provide assistance so that requesters can post the jobs without much hassle. Every time a requester creates a job, his job type will be noted. If the system finds out that the same job type has been used by the requester, quite a lot of times(very frequently), this information could be send to the system designers so that they could create a pre existing template for the same job type.

3) How can workers address their submissions to the requesters?: The requester can be given a power to decide whether he/she wants to review and devote enough time to validate each submission. If not, he/she can set a number of reviewers who will take care of the submissions who in turn will be paid by the requester only.

4) How might we threshold the wage for any task?: Using statistical analysis of the previous data, the most appropriate wage for a task can be determined. Consequently, a threshold can also be set on the wage for a particular task as well.

5) Who has the power to moderate spam reviews in the platform?: We believe that internal communication is the best way to settle any harsh reviews between the worker and the requester. But if that too isn't taking anywhere, the platform stakeholders need to intercept and take action about the situation.

6) How might we value the professional users of the platform: Professional workers who are working in the crowdsourcing platform for the past decade and so must be assigned some bonus and jackpots so that they will be motivated to work for another decade. They must be given freedom to edit other requester’s job and help other worker to do their task and at the same time criticize their work. Professional workers are important for the development of crowdsourcing platforms in the future.

7) How might we improve the performance of the workers and requesters participating in the crowdsourcing platforms?: In order to improve the efficiency of the novice workers and requesters proper documentation and technical support must be provided. Professional turkers can also aid workers with all their expertise and must provide deep insight into other participant’s task.

8) Who has the power to post the work: Requester can be divided into following two categories: Experienced Requester - The requester who has posted atleast 10 tasks should be given the power to post the task without any restrictions. New Requester- For the new requester, a proper channel has to be designed in order to validate the profile of requester and avoid the spammers.


Dive Deeper into Specific Ideas

Trust-related Ideas

Power-related Ideas

Dark Horse idea