Milestone 3 munichkindl

From crowdresearch
Jump to: navigation, search

Initial Brainstorm

The ideas you brainstormed, (at least 10 ideas for trust, and at least 10 ideas for power). Provide them in whatever format you want - diagrams, sketches, descriptions, or a combination (the wiki supports images, see here for instructions on uploading them).

We met in person for our brainstorm session and used flashcards, which we taped to the living room wall. As none of us has a nice handwriting, we decide to copy them instead of uploading pictures.

Trust-related Ideas

How might we help requesters to get high quality results?

  • Option for positive feedback to workers
  • Hidden minimum time requirement for tasks
  • End of task: Worker grades own work. If they grade themselves very good, punishment is harder if task is rejected.
  • Ability to pause a task and get feedback from requester if something is unclear
  • Tutorial on designing good tasks (content-wise, not technical)
  • Recording of mouse clicks, keystrokes and compare to ideal pattern
  • Split feedback to task categories (e.g. surveys, transcription etc.)

How might we ensure proper identities?

  • Collaboration with unis for approval of information
  • Uni login-system for login (via Shibboleth)
  • Require skill tests before registration / offer sample tasks for new workers to show ability
  • Ability to provide design samples / sketches etc. on profile
  • Show all or none own statistics on profile to requesters
  • New requesters have to accept each submission for their first 4-5 tasks
  • Providing IDs, address check by sending out real letters with verification code
  • Getting points for completing profile with certificates
  • Option for workers to promote successful tasks on profile
  • Option to link own account to other (social/business) networks

How might we ensure workers trust that they are going to be paid appropriately?

  • Minimum amount of payment for worker even with rejection
  • Maximum quota of tasks within each HIT (e.g. 20%) that can be rejected
  • If a requester rejects more submissionsn than a certain threshold, he still has to pay a small amount to everyone and there is no negative score on the workers profile
  • Entering minimum wage (worker) and maximum pay (requester) -> automatic match
  • Color-code payment offered depending on task-type average payment

Power-related Ideas

Who has the power to assess the quality of a submission?

  • When creating an HIT, requesters can decide whether they want submissions to be peer-reviewed
  • Workers can opt to get their finished task peer-reviewed before submission
  • Crowdgrading: Part of the crowd decides wheter a submission has to be accepted

How might we give workers the power to alter tasks?

  • Option to suggest variation of explanation in task (approval pending)
  • Let requesters promote valuable input of worker by giving them a 'like'

How might we enable someone without programming skills or funds for hiring it out to post a task?

  • Allow using and altering APIs of other users created for previous tasks as templates
  • Make interfaces more user-friendly
  • More templates

Dive Deeper into Specific Ideas

Trust-related Ideas

Power-related Ideas

Dark Horse idea