WinterMilestone 3 @ftw

From crowdresearch
Revision as of 08:54, 31 January 2016 by Aarongilbee (Talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Writeup for Winter Milestone 3 by team @FTW.

Brainstorm for the following


How to improve reputation system beyond Boomerang? (read this paper to learn more about Boomerang).

  • at scale, does it matter? the person will get new opportunities without a rating
  • use knowledge from boomerang to corral users into skill set groups
  • specialize: use knowledge from boomerang to give people shorter tasks rather than delays. it results in the same thing but maybe it will lessen the cognitive load.
  • use boomerang to establish foundational relationships between worker-requestor that progress to bigger things
  • show losers the door to another site(MTurk)
  • open opportunities for people to collaborate or lose it
  • allow people to see successes beyond cash flow
  • allow newcomers to become reviewers
  • enable proportionality --- people should not do exceptionally more than they can have done in the past. baby steps.
  • if a person doesn't complete a task, find task properties that led to failure (language, questions, quantity,...)
  • match task type set by requestor with the current intent of the user.
  • the design should get people to work together more incrementally and focus on implementation goals as an heuristic

How would reputation work for new comers - workers or requesters?

  • newcomers just enter, let them explore
  • have the system choose tasks that they cannot fail at and incrementally become more difficult
  • offer professional paths based upon skill that match with different tasks
  • let them self-rate and go through the system
  • requestors would need to be informed with case studies about what doesn't help workers
  • requestors should be exposed to the turkers good work practices
  • workers have coaches tied to skill, system knowledge
  • both should be introduced to the system
  • both should be introduced to a short video of the typical other
  • interactions should be left to questions only with experts to begin
  • exclusive newbie zones

How do workers find relevant work, and requesters find ideal worker?

  • workers are matched with jobs based on their extensive worker profile

- desired wage

- time availability

- perceived quality of work

  • requesters based on the profile matches of workers but also based on some other aspects that can be fine-tuned by requester

- time limit for task to be completed

- expected level of quality

- number of tasks/batches that can be completed by the same worker


How to make your voice heard?

  • numbers in mass
  • big bold fonts
  • funny pictures and short gifs
  • sound effects like chimes when the ignorer login
  • match the level of the problem with the level of urgency (one person cant speak to 1000 at one)
  • novelty of the issue factored with self-determined likelihood of failure and the severity of failure
  • put a rating system everywhere, everything, everyone
  • game system similar to stackexchange
  • provide a mailing list, newsletter, and blog
  • use golden eggs through communication channels to incentivize involvement

How to represent concerns/rights as workers and requesters?

  • automated rejection of work that meets requestor definition of spammers -- represented through the software as a regulatory mechanism
  • emojis which represent different levels of emotions coupled with hastags and @person to highlight problems
  • a simplified bill of rights for all, read before entering the system. scaled violations enforced.
  • acceptance of public outbursts is encouraged in forums.
  • publicize extreme cases
  • avoid forms of public humiliation, guilt mongering, shaming
  • promote rational micro conversations about isolated incidents with appropriate and agreed to actions
  • tie a rating system to direct social connections and have direct impact on access, responsibility, and privilege levels
  • squirrel avators give away rewards
  • identify those who seem to display appropriate regulatory behaviors through event logs and recruit them as employees.
  • be inspired by stack overflow...
  • notifications and updates that give enough time to others so that they can respond
  • provide enough time for any and all updates time to be resolved
  • consultations regarding development should be as broad as possible to include requestors, regulators, and workers
  • documentation and submissions regarding the governance should be publicly available
  • engagement with publics should cover multiple channels
  • concerns should be vetted through expert committees
  • traceable connections between updates in policies and known consequences
  • civil society, not ethics, should be monitored. ethics treated on a case-by-case basis
  • publish the details of algorithms used. always keep improving and changing to keep abusers behind. planned obsolescence.
  • the timelines must be publicly available and provided in response to any adverse situations
  • awareness of development efforts or updates or changes must be scaled to relevant stakeholders

Who has the power to post work?

  • power to post goes to all, power to raise up some
  • give a random person on a team the ability to post at random, surprise intervals.
  • give a worker who is dedicated community rights to post good ideas from a few.
  • no one has the power to post. it will be AI overlords.
  • sponsor the ability to post work
  • advertise the ability to post work to only those who can afford it
  • only those workers/requestors who talk within a community
  • the person who pays into the system
  • anyone who is credentialed
  • a physical neighborhood in manhattan or ethiopa -- the extreme rich and extreme poor are represented
  • only people who have a turker conversion... worker becomes requestor who knows what it's like to be a regular working person, like joe plumber

What would open governance look like on Daemo?

  • punish privately, award publicly, support directly and behind the scenes
  • personalization of work screens according to level (controlled accessibility and level ups)
  • public cafes in forums
  • physically local cafes
  • private discussion threads, which a publicly known (open secrets)
  • increased pay/discounts for those that get involved tactfully with governance issues for time compensation
  • published academic work on the subject (domain researchers get special access to data beyond the visible and interviewable)
  • holacracy, two people circles get things done
  • a task cannot be completed until two have accepted, even if they don't know each other
  • special invitations to join stanford crew at conferences
  • public yet exclusive meetings with core daemo administration

BIG IDEA for Reputation

What are the goals of the design? For example, Google's Android design goals are: delight me in surprising ways, simplify my life, and make me amazing (e.g., grant me special powers). Which aspects of your design reflect each goal? How does your design solution addresses the users' needs?

BIG IDEA for Representation

What are the goals of the design? For example, Google's Android design goals are: delight me in surprising ways, simplify my life, and make me amazing (e.g., grant me special powers). Which aspects of your design reflect each goal? How does your design solution addresses the users' needs?


  • they must be "dark": they must explore a space that is risky, radical, infeasible, and/or in a direction orthogonal to previously explored solutions. *They should feel slightly uncomfortable.
  • Second, they must be brainstormed after the more traditional ideas — you can't have a dark path without a traditional "light" path to contrast it against.
  • Third, they must be refined enough that they could be prototyped and objectively tested.

UX Flight Details Migrated