Milestone 3 pnp

From crowdresearch
Revision as of 23:38, 18 March 2015 by Kunalkhandelwal (Talk | contribs)

Jump to: navigation, search

Design notes for a future crowd work market

Vision of Future crowd marketplace

  • Design and messaging of the platform should incentivize and prime requesters to realistically consider workers’ abilities, needs, contexts, and limitations while designing and reviewing tasks.
  • No Illegal,unethical tasks should be asked.
  • If possible even rejected tasks should be awarded with lesser amount in order to value the turkers time.
  • Workers may encounter hard-to-reproduce technical problems. Or workers may unexpectedly object to some part of a task.
  • Requesters need to communicate with workers if problems arise, and give them tools with which to do so effectively, efficiently, promptly, and respectfully.
  • Platform designers should develop strong relationships with professional workers, understanding and valuing their paid and unpaid work, soliciting their advice on major design and operational decisions, and taking their advice seriously.
  • Cheaters should be punished for which a two-sided and somewhat complex reputation system is needed.
  • Workers and requesters are the most motivated and capable people available to solve one another’s problems, communication between them should be encouraged and easily available.
  • If reliable support — for both workers and requesters — requires paid staff, funding should be arranged, and tradeoffs and potential alternatives discussed with stakeholders.
  • Use technology to provide a better framework i.e to amplify the human relationships and communication to support and guide.
  • Platforms should be built in such a way so that they can promote trust among the people involved i.e requestors and turkers.
  • A next-generation crowd work market will help well-intentioned workers do what they already want to do — get paid fairly for doing good work.
  • Workers who are professional are involved in the large amount complete most of the work, and are more strongly invested in its smooth functioning than casual workers.
  • They also go out to teach their way to other normal workers.
  • Professional workers, because of their reliance on crowd work income and their participation in crowd worker communities, are both more able and more motivated to produce high quality work.

The future of crowd work

Vision of Future crowd marketplace

For the positive future ahead,

  • Worker considerations, such as motivation, feedback, and pay. These may be addressed by mechanisms to maintain reputation, provide better interaction with requesters, and increase motivation.
  • Requester considerations, such as coordination, task decomposition, and quality control. These may be addressed through workflow mechanisms including electronically mediated collaboration.
  • Distributed Computing and Organizational behaviour will enable better and professional work in near future.
  • A good framework is responsible for a better crowd market and is essential for a good management of shared resources and communication between turkers and requesters.
  • Workflow should be maintained with proper vision and motivation for the tasks to be completed efficiently and this should in a way inspire more and more workers.Hierarchy should be maintained.
  • Synchronous Collaboration , Quality Control , Motivation and Rewards promote a better crowd marketplace.
  • Better communication is necessary and platforms such as AMT should be responsible to enable it.
  • The present community’s observational, experimental, design and technical skills will play a vital role in shaping the future of crowd work and the next generation of workers.



  • The platform should be capable of showing the status of task on the platform itself, where the workers and requesters can collaboratively discuss, and only after the requester accepts the work and initiates payment, should the code(work done) be transferred to the requestor.
  • There should be “Trusted Requestors” and “Trusted Workers” which have earned this “trusted” tag by their history. While giving a task to a trusted worker, you can be sure that he will deliver. Similarly, while taking upon a task from a trusted requestor, you can be sure that he won’t scam you so you can work w/o any risk. This is particularly helpful in cases where the requestor has a strict deadline for his task and can’t afford non-delivery by the chosen worker.
  • Authenticity of workers can be enforced by allowing them to take up a considerably higher paid job only if they have been reviewed positively for a comparatively paid job.
  • Timely completion of jobs can be ensured if the platform provides facility to review part of task done after each deadline.

A transparent reviewing process can be ensured by keeping a small fraction of monetary fund, be reserved by the platform and released to either parties only after they complete the review process.

  • Prompt customer service helps prevent loss of users. The chances of a user abandoning the platform depends directly on the time it takes to resolve a query.
  • Skill level of any worker should be identified by a rating process, where both requesters and workers rate each other. This helps users to make future decisions better.
  • There should be enough random behaviour in the platform so as to minimize bot usage. Many people write scripts (e.g. check for new tasks and apply to all) which negatively impact the platform as a whole.

Workers tend to prefer a platform which is most favorable to them in terms of monetary value, i.e. which gives them the most earnings for same work. The amount charged by our platform should be kept as low as possible to attract more workers. Having more workers will ultimately result in having more requestors on the platform. (The platform’s success will be dependent on the user base.)

  • Restricting worker access to the jobs based on their past commitment, helps to segregate the marketplace, i.e., the jobs that require a particular set of skills can be accessed only by those workers who have successfully completed a task under that domain. This is identified by the review process, after each completion of task.
  • Since the most important factor is money, in order to maintain the integrity of the platform, there should be a check on whether each job posted by a requester is proceeds on the platform and not bypassing it. Only after every step is performed on the platform(right from acceptance of job applicants, their part-submissions and finally accept/reject), should the reserved money be released to either parties.

A standardized payment gateway has to be implemented to which all members of the community can adhere to.

  • Each member has to be uniquely identified(no fake profiles); there is a necessity of a mechanism to identify and eradicate fake profiles.
  • Workers tend to build a network via these platforms from the first few tasks and then contact the clients directly for more tasks outside the marketplace. There should be an incentive to keep both parties on the platform.


  • Monetary worth of jobs posted by any requester must increase progressively, in order to avoid scam. This ensures that highly paid jobs are put-up only by requestors who are trustworthy, i.e., they have shown their commitment to some workers prior to this deal.

To ensure quality of tasks, we can enforce an organised rating system.

  • Most of the tasks of the crowd-source platform(the middleman) need to be automatized to reduce commission rates to a bare minimum.

Tasks can be divided into multiple categories : hourly-basis, milestone-basis or complete. Different payment strategies can be decided by the requester depending on the type of work desired.

  • Conflict issues between requestors and workers should be managed by a powerful middleman, who is impartial to favour either.
  • We need an escrow for code too so that worker doesn't hand over the full code without getting paid. So, the worker hands full code to us and the client hands money to us too. We’ll give the client an option to check the quality of work without giving him the full code, say by remote login (ssh). Once satisfied, release both code and funds. This may be the “only” possible way to prevent scam..!
  • Initially, when the worker is new to the platform, he can be asked to qualify in a set of predefined jobs, that would be automatically graded. These tasks would be based on the skills, the worker claims to possess.

Problems faced by Requester and are never addressed :

  • Scaling up
  • Managing the complex API
  • Managing execution time
  • Ensuring quality of tasks

It is not good for the requestors as well as workers in either ways as both are dependent.