WinterMilestone 3 @ftw
Writeup for Winter Milestone 3 by team @FT.
- 1 Brainstorm for the following
- 1.1 Reputation
- 1.2 Representation
- 1.3 BIG IDEA for Reputation
- 1.4 BIG IDEA for Representation
- 1.5 DARK HORSE
- 2 User Experience Flight
Brainstorm for the following
How to improve reputation system beyond Boomerang? (read this paper to learn more about Boomerang).
- at scale, does it matter? the person will get new opportunities without a rating
- use knowledge from boomerang to corral users into skill set groups
- specialize: use knowledge from boomerang to give people shorter tasks rather than delays. it results in the same thing but maybe it will lessen the cognitive load.
- use boomerang to establish foundational relationships between worker-requestor that progress to bigger things
- show losers the door to another site(MTurk)
- open opportunities for people to collaborate or lose it
- allow people to see successes beyond cash flow
- allow newcomers to become reviewers
- enable proportionality --- people should not do exceptionally more than they can have done in the past. baby steps.
- if a person doesn't complete a task, find task properties that led to failure (language, questions, quantity,...)
- match task type set by requestor with the current intent of the user.
- the design should get people to work together more incrementally and focus on implementation goals as an heuristic
How would reputation work for new comers - workers or requesters?
How do workers find relevant work, and requesters find ideal worker?
- workers are matched with jobs based on their extensive worker profile
- desired wage - time availability - perceived quality of work
- requesters based on the profile matches of workers but also based on some other aspects that can be fine-tuned by requester
- time limit for task to be completed - expected level of quality - number of tasks/batches that can be completed by the same worker -
How to make your voice heard?
How to represent concerns/rights as workers and requesters?
- automated rejection of work that meets requestor definition of spammers -- represented through the software as a regulatory mechanism
- emojis which represent different levels of emotions coupled with hastags and @person to highlight problems
- a simplified bill of rights for all, read before entering the system. scaled violations enforced.
- acceptance of public outbursts is encouraged in forums.
- publicize extreme cases
- avoid forms of public humiliation, guilt mongering, shaming
- promote rational micro conversations about isolated incidents with appropriate and agreed to actions
- tie a rating system to direct social connections and have direct impact on access, responsibility, and privilege levels
- squirrel avators give away rewards
- identify those who seem to display appropriate regulatory behaviors through event logs and recruit them as employees.
Who has the power to post work?
- power to post goes to all, power to raise up some
- give a random person on a team the ability to post at random, surprise intervals.
- give a worker who is dedicated community rights to post good ideas from a few.
- no one has the power to post. it will be AI overlords.
- sponsor the ability to post work
- advertise the ability to post work to only those who can afford it
- only those workers/requestors who talk within a community
- the person who pays into the system
- anyone who is credentialed
- a physical neighborhood in manhattan or ethiopa -- the extreme rich and extreme poor are represented
- only people who have a turker conversion... worker becomes requestor who knows what it's like to be a regular working person, like joe plumber
What would open governance look like on Daemo?
- punish privately, award publicly, support directly and behind the scenes
- personalization of work screens according to level (controlled accessibility and level ups)
- public cafes in forums
- physically local cafes
- private discussion threads, which a publicly known (open secrets)
- increased pay/discounts for those that get involved tactfully with governance issues for time compensation
- published academic work on the subject (domain researchers get special access to data beyond the visible and interviewable)
- holacracy, two people circles get things done
- a task cannot be completed until two have accepted, even if they don't know each other
- special invitations to join stanford crew at conferences
- public yet exclusive meetings with core daemo administration
BIG IDEA for Reputation
What are the goals of the design? For example, Google's Android design goals are: delight me in surprising ways, simplify my life, and make me amazing (e.g., grant me special powers). Which aspects of your design reflect each goal? How does your design solution addresses the users' needs?
BIG IDEA for Representation
- they must be "dark": they must explore a space that is risky, radical, infeasible, and/or in a direction orthogonal to previously explored solutions. *They should feel slightly uncomfortable.
- Second, they must be brainstormed after the more traditional ideas — you can't have a dark path without a traditional "light" path to contrast it against.
- Third, they must be refined enough that they could be prototyped and objectively tested.
User Experience Flight
Reach out to people within Crowd Research community, who are not participating in this test flight and get them to use a specific feature of Daemo and get their feedback on it. You are free to reach out to people outside Daemo too, but they must have minimal crowd platform experience and don't share it publicly. We're not ready for huge traffic at this point. Report a synthesis of your findings along with your own experience. Your report should include 1) actionable item, one that can be implemented into Daemo quickly; 2) good-to-have long term vision item, one that you'd like to see at some point of time. There is no limit to how many people you work with, it can be 1, it can be 10, it can be none (means you reflect your own findings). Note: be respectful of others time, and don't push people for it - you can always report your own experience only.
Timing Results from 6 Daemo Tasks
|Task Description||Timings (sec)||Script|
|Create a Question: 5 Radio Button Response||45.0||File:TaskA1 Script.csv|
|Create a Question: create image labeling task - error||23.0||File:TaskA2 Script.csv|
|Create a Question: create image labeling task||34.9||File:TaskA3 Script.csv|
|Answer Question: 5 Radio Button Response||6.7||File:TaskB1 Script.csv|
|Answer Question: label the image task - error||10.8||File:TaskB2 Script.csv|
|Answer Question: label the image task||14.0||File:TaskB3 Script.csv|