Issue tracker for requesters

From crowdresearch
Jump to: navigation, search

The idea for an issue tracker was inspired by the author's work on the social coding collaborative forum Github. Github is an open-source platform where developers can collaborate on open-source projects from all around the world. The Github issue tracker is an efficient, convenient and intuitive way of inspecting code, finding out issues with it and reporting it. Various other version control systems also use their own specific versions of error-reporting systems and bug-trackers. Our idea is to take that concept and apply it to crowdsourcing systems, in a manner that is described below.

Issue tracker

The issue tracker of our system is intended to be an integral part of they system. It is aimed at being built in a way that fosters and enhances the amount of trust the worker and the requester place in each other. This issue tracker is intended to be very similar to the Github issue tracker, with a few minor differences that I will highlight along the way.

Motivation

Most of the crowdsourcing systems that we have examined usually do not give a chance to the workers to actually improve upon the work that they have spent hours on. Once all that hard work gets rejected, it leads to heartburn and frustration. This is due to the inherent assumption that the only two states of work as far as crowdsourcing is concerned include either complete acceptance or complete rejection.

However, we have not taken into account one factor - improvement. We know that the requesters need to get their job done. Unless they are scammers, they are probably not getting their job done by rejecting the work of another worker. Also. the worker whose work has been rejected does not know the reason for his work being rejected. There is no feedback mechanism by which the worker can learn from his mistakes and make sure that he does not make those very same mistakes in a future project.

In light of the facts stated above, we feel that the problems that have been stated can be remedied easily by the introduction of a simple issue tracker.

How it works

The basic mechanism by which the issue tracker works is very simple. It works in a fashion very similar to the issue tracker that is built in to Github. However, as I have stated earlier, there are significant differences between the issue tracker of Github and the issue tracker that I am about to describe. For starters, this issue tracker is going to be built into a system that supports no -coding tasks as well. Therefore, without further ado, here is the mechanism by which our version of the issue tracker will work:

1. The requester posts a job on the system. A worker is assigned to that particular job. The worker does some work and submits it. If the work done by the worker is perfect, then all is good.

2. On the other hand, suppose the work done by the worker is not acceptable. In that case, the requester opens a new issue on the issue tracker. The requester has the option of assigning a deadline to the issue. If the issue is not fixed by that particular deadline, the work done by the worker will be rejected. This aspect is one of the ways in which this particular issue tracker differs from the one by Github.

3. Once a new issue has been opened, the worker is notified by means of email. The issue is assigned a number, and an issue webpage is created. The issue webpage facilitates discussion between the worker and the requester as to how the worker can improve his/her submission.

4. The worker can work on fixing the issue, and notify the requester by commenting on the issue page. By default, the requester gets notified every time a comment is posted on the issue web-page because it is assumed that since the requester created that issue, he is interested in following it.

5. The issues are private to the requester and the worker by default. However, they can be made available to the general community in case the requester of the task feels that it is necessary to bring a fresh perspective to the job that he originally posted.

6. The requester can allocate importance to an issue on a scale of 1 to 10, 10 being very important and one being not so important. There might be a subset of workers working on fixing issues for publicly available work. In such cases, the income of that particular worker will be directly proportional to the importance of the issue he/she is dealing with. This will depend on the importance rating assigned to the issue by the requester.

How it helps

The advantages of having an issue tracker build into the system are as follows:

1. The issue tracker allows for a feedback mechanism. It provides the requester with a way to communicate with the worker(s) in a systematic fashion about the issues that he/she feels exist in his/her work.

2. As the worker keeps fixing the issues prevalent in his work according to the needs of the requester, it provides the requester a more focussed way to get the most out of a given worker.

3. If the worker does not manage to fix the issue before the deadline, he/she knows that the work has been rejected due to his/her inability to complete the designated task on time. This is as opposed to a model where the worker has absolutely no idea why his work has been rejected.

4. Opening and discussing issues on the issue tracker allows the worker to incrementally improve upon the work that he/she is doing. This is as compared to a model where the worker will not learn from his mistakes because he/she has no idea what they are in the first place.

5. Since the issue tracker provides for an open medium of communication, this builds trust between the requester and the worker. The requester will not reject the worker's work without giving him a chance to fix it first (if he does that, then the worker has the right to not work for him in the future).