Difference between revisions of "Crowdresearch:Winter Milestone 4 - Design (Test Flight) Alipta"

From crowdresearch
Jump to: navigation, search
(Needs from Milestone 2)
Line 10: Line 10:
== Needs from Milestone 2 ==
== Needs from Milestone 2 ==
We synthesized the main needs groups identified in Milestone 2 in the following table:
We synthesized the main needs identified in Milestone 2 below:
=== Worker and Requester Needs ===
=== Worker and Requester Needs ===

Revision as of 18:10, 25 January 2016

Note: The idea submissions for Milestone 2 have been archived at http://crowdresearch.meteor.com/

Due date (PST): 8:00 pm 31st Jan 2016 for submission, 12 pm 1st Feb 2016 for peer-evaluation.

This week, we will take the set of needs that we collectively identified in the previous milestone and use those insights to generate design ideas.

  • Youtube link of the meeting today: watch
  • Winter Meeting 3 slideshow: slides pdf

Needs from Milestone 2

We synthesized the main needs identified in Milestone 2 below:

Worker and Requester Needs

Please see this week's slides

Our synthesis

These needs boil down to two main issues: 1) task ranking and reputation, and 2) worker and requester representation (open governance).


  • How to improve reputation system beyond Boomerang? (read this paper to learn more about Boomerang).
  • How would reputation work for new comers - workers or requesters?
  • How do workers find relevant work, and requesters find ideal worker?


  • How to make your voice heard?
  • How to represent concerns/rights as workers and requesters? Who has the power to post work?
  • What would open governance look like on Daemo?

As we brainstorm, we should be thinking about solutions that holistically address these issues of reputation and open governance, not just surface fixes that get at micro-elements of the system.

Dive Deeper into Specific Ideas (Mandatory)

For the two topics - reputation and representation, pitch an idea for each, that you would like to pursue, and expand on them a bit further. In addition to describing the idea itself, make sure you also tell us:

  • What are the goals of the design? For example, Google's Android design goals are: delight me in surprising ways, simplify my life, and make me amazing (e.g., grant me special powers).
  • Which aspects of your design reflect each goal? How does your design solution addresses the users' needs?


For each of the 2 ideas (1 for reputation, 1 for representation), describe (using diagrams, sketches, storyboards, text, or some combination) the ideas in further detail.

Please create a separate wiki page for each of your ideas, so we can link to them individually. The title of the wiki page should be Milestone 3 followed by your team name and a description of the idea itself (ex: Milestone 3 YourTeamName TrustIdea 1: Automatic Pricing for Tasks based on Average Completion Time). Post a link to each of your trust-related ideas to http://crowdresearch.meteor.com/category/milestone-3-trust-ideas and a link to each of your power-related ideas to http://crowdresearch.meteor.com/category/milestone-3-power-ideas when done.

Dark Horse idea (Mandatory)

Now that you've identified some design directions you like, it's time to change tack and toss in a dark horse idea. A dark horse, in horse racing, is a contender who most people don't think will win, but may turn in an unexpectedly strong performance and produce a huge payoff. Dark horse ideas are intended to be something far out there or nearly impossible. In the best case, your dark horse ideas might end up winning the race. However, even in the worst case, they can give us tremendous design insight and prevent design fixation, where the design space shrinks too rapidly.

There are three requirements for dark horse ideas. First, they must be "dark": they must explore a space that is risky, radical, infeasible, and/or in a direction orthogonal to previously explored solutions. They should feel slightly uncomfortable. Second, they must be brainstormed after the more traditional ideas — you can't have a dark path without a traditional "light" path to contrast it against. Third, they must be refined enough that they could be prototyped and objectively tested. That is, it cannot be infeasible: it needs to be something that we could put in front of real people to see whether it would work.

If you have dark horse ideas that came up in your initial brainstorm, you can use that. If you're not satisfied, brainstorm some! Try using Powers of Ten and other techniques to push further and generate even more. After you brainstorm and sketch out dark horse ideas, choose one that you'd like to include among your set of two top candidates from before. Expand on your dark horse idea like you did in the previous section.


Describe your dark horse idea (using diagrams, sketches, storyboards, text, or some combination).

Please create a separate wiki page for your dark horse idea so we can link to it individually. Post the link on http://crowdresearch.meteor.com/category/milestone-3-dark-horse-ideas when done

Research Engineering (Test Flight)

Usability Experience (Test Flight)

Recommended Readings (Optional)

Coming up with good, novel visions and ideas is a crucial part of doing successful research, and reading other researchers' visions and ideas can help you come up with better ideas yourself. These readings discuss visions that crowdsourcing researchers have thought of related to future crowd marketplaces. This week's readings are optional and don't have a deliverable, but are highly recommended.

Design notes for a future crowd work market

Design notes for a future crowd work market - This is a Medium post written by researchers involved with Turkopticon in response to hearing about this research project. It discusses their vision for a future crowd marketplace, where workers are more involved in the management of the marketplace.

The future of crowd work

Kittur A, Nickerson J V, Bernstein M, et al. The future of crowd work. Proceedings of the 2013 conference on Computer supported cooperative work. ACM, 2013: 1301-1318. - This paper envisions a future crowd marketplace that emphasizes on workers' long-term development, and where people can be proud to be workers. It is a long paper. Feel free to focus on just the parts that particularly interest you.


Create a Wiki Page for your Team's Submission

Please create a page for your team's submission at http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_YourTeamName&action=edit (substituting in YourTeamName with the team name), copy over the template at Milestone 3 Template . If you have never created a wiki page before, please see this or watch this.

[Team Leaders] Submission or Post the links to your ideas until 18th March 11:59 pm

We have a service on which you can post the links to the wiki-pages for the individual ideas you generated, explore them, and upvote them.

Instructions for posting are at http://crowdresearch.meteor.com/posts/bXSNbqihjajASBQEL

Sign-up Instructions: Log in with either Twitter or Facebook. When it asks you to pick your username, pick the same username as your Slack, this will help us identify and track your contributions better.

There are 3 submission categories:

1- http://crowdresearch.meteor.com/category/milestone-3-trust-ideas where you can post links to the wiki pages for each of the 2 trust-related ideas you generated in the "Dive Deeper into Specific Ideas" stage

2- http://crowdresearch.meteor.com/category/milestone-3-power-ideas where you can post links to the wiki pages for each of the 2 power-related ideas you generated in the "Dive Deeper into Specific Ideas" stage

3- http://crowdresearch.meteor.com/category/milestone-3-dark-horse-ideas where you can post a link to the wiki page for your dark horse idea

Post links to your ideas only once they're finished. Give your posts titles which summarize your idea. Viewers should be able to get the main point by skimming the title ("Automatic Pricing for Tasks based on Average Completion Time" is a good title. "YourTeam TrustIdea 1" is a bad title).

-Please submit your finished ideas by 11:59 pm 18th March 2015, and DO NOT vote/comment until 19th March 12:05 am

[Everyone] Peer-evaluation (upvote ones you like, comment on them) from 12:05 am 19th March until 9 am 20th March

Post submission phase, you are welcome to browse through, upvote, and comment on others' ideas. We encourage you especially to look at and comment on ideas that haven't yet gotten feedback, to make sure everybody's ideas gets feedback. You can use http://crowdresearch.meteor.com/needcomments to find ideas that haven't yet gotten feedback, and http://crowdresearch.meteor.com/needclicks to find ideas that haven't been yet been viewed many times.

COMMENT BEST-PRACTICES: As on Crowdgrader, everybody reviews at least 3 ideas, supported by a comment. The comment has to justify your reason for upvote. The comment should be constructive, and should mention positive aspect of the idea worth sharing. Negative comments are discouraged, rather make your comment in the form of a suggestion - such as, if you disliked an idea, try to suggest improvements (do not criticize an idea, no idea is bad, every idea has a scope of improvement).

[Team Leaders] Milestone 3 Submissions

To help us track all submissions and browsing through them, once you have finished your Milestone 3, go to the link below and post the link:

Milestone 3 Submissions