Milestone 8 Pumas Foundation2 InputOutputTransducers

From crowdresearch
Revision as of 07:58, 22 April 2015 by Saiphsavage (Talk | contribs) (Proposal Outline)

Jump to: navigation, search

Task Verification Gone Social


Challenges and Motivation

There is a need to have tasks improved and polished before they are shown on the site alive. However, it can become expensive to incorporate this verification step.
Central issues to task verification:

  • The costs involved in peer-reviewing tasks. Another person (or people), which is not a requester nor a worker has to review the task posted by the requester before releasing it to all workers.
  • Task verification can take a lot of time if there is not someone available to review the task or if the verification process itself is very thorough and detailed.
  • When tasks are being reviewed by a third party there might be a risk of tampering the task it self, due to a subjective interpretation or non-attachment to the work from behalf of the reviewer or even if the reviewer feels time-pressured he/she will not perform at her/his best.
  • What if the task is not improved but actually tweaked or transformed into another task due to an inaccurate interpretation from the reviewer? This affects directly the workers, as they might be performing a task correctly - due to the standard of the reviewed version - but when submitted they might their task rejected. This type of rejection arises from doing a different task to the one the requester originally posted.
  • If a task is being "interpreted" or "improved" by a reviewer then they have too much power over the situation, the tasks and the relationship between the workers and requesters.
  • The speed at which a task is reviewed is also under scrutiny. Requesters want their tasks to be taken as soon as they post them or at least start have some kind of feedback. Near immediate responses from reviewers would be ideal for requester.

Previous Work Done

Proposal Outline

We propose to turn these verification tasks into social activities, that people will do pro-bono.
People will do the tasks for free because they will be after the social interactions rather than the monetary compensations. The social activities will allow people to bond with their fellow co-workers and gain social capital.


We propose to create automatic methods that turn these tasks into fun social events, where workers & requestors can learn from each other, and build friendships. For instance, the system might recommend for all people who want their tasks to be verified to join a "midnight verification party!" People will all be online at the same to review each others tasks, while also listening to the same music, chatting, laughing and having a blast! :)

Proposal In Detail


[1] Eickhoff, Carsten, and Arjen de Vries. "How crowdsourcable is your task." Proceedings of the workshop on crowdsourcing for search and data mining (CSDM) at the fourth ACM international conference on web search and data mining (WSDM). 2011.

[2] Kulkarni, Chinmay, et al. "Peer and self assessment in massive online classes." Design Thinking Research. Springer International Publishing, 2015. 131-168.

[3] Dow, Steven, et al. "Shepherding the crowd yields better work." Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work. ACM, 2012.