Milestone 7 Hawkeye

From crowdresearch
Revision as of 00:11, 16 April 2015 by Kpuneetha (Talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search


A platform to perform background verification of worker and standardize payment.


Our platform takes into consideration the fundamental ideas of trust. We aim on increasing the level of trust between workers and requestors and propose a standardization for payment procedure. One important factor to increase the level of trust is communication and verification of the worker profiles. Communication between the requesters and workers is very important so that most of the issues or the doubts can be resolved at the lower level. Also we plan on having a peer referral system. We propose to standardize the payment system using bitcoins. The workers are updated regularly regarding the available jobs and the status of their on going job. Our platform consist of mediators who helps resolve conflicts between the worker and requestor and helps to guide the new workers to achieve best results.


It is important for the requestor to trust that the worker is genuine and not a bot. Our platform consists of a background check on every registered worker and is updated on their respective profile. Also peer workers can refer a worker for his skills and genuinety. Our suggested mode of payment is in form of bitcoins as it will eliminate the differences in the currency system. To keep the workers updated on the available work, notifications can be sent regularly.

Many a times though the formal communication is there between the requesters there might be some misunderstanding which can cause the conflicts between the two. Therefore, there is a high chance of conflicts between the worker and requestor which can be resolved by having a mediator between then. Also when a worker or requestor is new to the site, the mediator can help him/her get used to the site.

Related Work

Presently for verifying the genuinety of the worker captcha is used. But that is not very efficient in getting a fair idea about the authenticity of the worker. Also captcha is a very step in security, it can be improved many-folds.

The workers are presently paid according a value that is decided by the requestor. Also the currency with which the worker gets paid varies from country to country. This can lead to some workers being underpaid for the job or even overpaid.

When the issues between the requestor and worker crop up due to the misunderstanding between them for the work assigned or done, it is not resolved efficiently in the present systems.

Many-a-times the workers are not informed well about the work available because of their unavailability on the site all the time. So this will lead to a situation where the worker can lose some good opportunity.


The system we propose will have a background verification where the worker can link to any of the social networking sites to prove his genuinity or if the user has some personal issues then only email verification can be done according to the worker. Also when the peer workers are aware of the authenticity of the worker, he/she can recommend the worker so that the profile is updated accordingly for the public.

For every job, a standard number of bitcoins per hour is assigned. This depends on the level of difficulty of the job and the past experience of the worker. This can also be varied time to time according to the change of trends in the system.

One of the special features of our system is having a moderator who resolves issues between the worker and requestor. This can be resolved by checking the previous conversations of the two and the work done with respect to the assigned work by the requestor.


When the worker registers into the site, it is mandatory for him/her to provide the link to at least one of the social site which can serve as a source to check the background of the worker. If necessary, the worker can choose to stay anonymous, which can be displayed on the profile but without a link to the social profile in this case only email verification can be done. Though through the other personal profiles there can be an informal communication between the two so that can build up the trust before the workers take up the task, there can be online chat forums in which the moderators, workers or requesters all can communicate to each other any resolve their doubts this can be done outside the platform also using their social profiles, if needed incase of macro tasks the requesters can divide the work into several parts and each part can be evaluated by the requesters. Requesters and workers can communicate every interval of time when the work is in progress . There is also an option to the worker who can give a reference of a person who can attest to the authenticity of him/her, so this can be a part of authentication since someone is referring to the particular worker if needed the requester can ask him/her about the worker, this can go according the requester wish,if the requester wants the reference of the co-worker on the turk or some other professional who is not on Mturk, and these references can be contacted later if there is any. These are steps which can be taken for background verification.

Every worker can be classified into groups according to the level of experience and the area of expertise. The workers in the most experienced group can recommend the skills of any other worker. Also the worker can ask for a feedback on his/her job so as to prove his skill set so that it can help him/her provide a better profile.

Our system is portable and we propose an app which can send notifications to update the workers regarding the latest jobs that have been put up. It also helps improving the communication between the worker and the requestor.

The standard mode of payment online is bitcoins. It can ensure a uniformity in the payment procedure. Also the rate of payment can be standardised for every job to ensure that no worker is either underpaid or overpaid for a given work. We can have include machine learning techniques to understand the trends in the payment and suggest a decent payment for the requestor for a job or suggest the worker if the given pay is decent. Since lot of improvements needs to be done on bitcoins these can be thought of as an idea in future. Since we are including the communication between the requesters and the workers the moderators need not check out the progress of every workers task, a moderator moderating so many tasks is a big overhead and there are chances that requesters won’t like there work to be moderated all the tim , so we need moderators only for pre-tasks or post-tasks, thus as an addon feature our platform supports the role of a mediator who resolves the conflicts between the worker and requestor. The conflicts can be minimized between the worker and requester by having a moderator who is good enough to understand the problem and solve the issue. If the moderator is a person who doesn't have expertise in the particular field, it will only lead to more problems. So we are able to choose the moderators according to their expertise for a particular issue, that will improve the performance of the moderator and reduce the effect of the conflicts. This improves the level of the trust. The moderator is a worker who has high efficiency and good work commitment. Based on the number of HITs and the experience of the worker he/she is promoted to the level of a moderator. Also there is a constant rating for the moderator to keep him in check. The moderator is paid by both the worker and requestor depending on who approaches to resolve the conflict, the one who approaches will pay, in this way it will also not result in more number of disputes because the worker or the requester will approach only if there is a real problem because simply they won’t like to spend money and in this way the moderator need not check for all the rejects also since the requester or the worker will approach only if there is a genuine conflict. The disputes are sometimes the misunderstanding between the assigned work and the submitted work, or dissatisfaction in the work of the worker.

The moderator is just another worker with very good profile. So the platform promotes the worker to become a moderator if the work done is appreciated.

  1. Promote the top worker as a moderator based on the no of hits and quality of work and also the there can be a minimum criteria of how many years did he/she spend on Mturk.
  2. The moderator assigned to a particular conflict should be an expertise in that field, else it will be of not much of help having a moderator.
  3. The moderator checks all the initial conversations and exchanges between the worker and the requestor, to resolve the issue.
  4. Quality of work can be rated by the requester when satisfied with the work done.
  5. No of conflicts resolved by each worker or moderator will improve the position in ladder of being a moderator.
  6. Profile of each worker has to be assessed in order to increase the credibility.
  7. No of hits the moderator gets
  8. A moderator can also be demoted if (s)he doesn't keep up to the standards that are expected to be maintained. Reasons can be such that the worker and requester are not satisfied with the result. Or the number of HITs obtained by the moderator are less than a given specified value.

Another role of the moderator is to guide the new workers through the process of working , initially it is very difficult for the workers to get used to the platform , either there should be a proper documentation or these moderators can help.


The platform consists of a feedback system where the users can comment on the functionalities and the efficiency of the system with respect to various modules. The feedbacks serve as the criteria for improvement and evaluation of the platform.


Irani L C, Silberman M. Turkopticon: Interrupting worker invisibility in amazon mechanical turk. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 2013: 611-620.

Kittur A, Chi E H, Suh B. Crowdsourcing user studies with Mechanical Turk. Proceedings of the SIGCHI conference on human factors in computing systems. ACM, 2008: 453-456.