Difference between revisions of "Milestone 4"

From crowdresearch
Jump to: navigation, search
 
(18 intermediate revisions by 3 users not shown)
Line 1: Line 1:
During this milestone we'll have you refine your ideas from the past milestone.
+
Flare and focus! Our goal this week will be to take the flare phase from last week, focus it around a few themes, and then use each others' ideas to flare out once more to identify the points of view we like.
 +
'''Due date (PST): 11:59 pm 25th March 2015 for submission, 9 am 27th March 2015 for voting and commenting on others' ideas.'''
  
* Youtube link of the meeting today: (Will be uploaded later)
+
* Youtube link of the meeting today: [http://www.youtube.com/watch?v=L-SpoUsowCE watch]
* Meeting 3 slideshow: (Will be uploaded later)
+
* Meeting 4 slideshow: [[:Media:Week4 Slideshow.pdf | pdf]]
  
 
== Themes from Milestone 3 ==
 
== Themes from Milestone 3 ==
  
Based on your submission for Milestone 3, we synthesized the following themes:
+
Based on [http://crowdresearch3.meteor.com your submissions for Milestone 3], we synthesized the following themes. Take a look at the high level patterns, since these will be the themes we'll cluster our brainstorm around.
  
=== Descriptions: How might workers+requesters work together help produce higher-quality task descriptions? ===
+
=== Task clarity: How might workers+requesters work together help produce higher-quality task descriptions? ===
Example ideas of this theme:
+
Examples:
 +
* [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_Pumas_PowerIdea_2:_lovePower New tasks go to workers first who improve the task before it goes live.]
 +
* [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_Triple_Clicks_PowerIdea_1:_Voting_on_HIT_design Voting on hit design]
 +
* [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_Team_Innovation_2_PowerIdea1:_Worker_Feedback_on_Tasks New tasks go into a holding pattern where they need to be "voted in” by workers]
 +
* [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_PixelPerfect_PowerIdea_2:_Task_Templates Task templates (can we do better than AMT’s templates?)]
 +
* [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_TuringMachine_TrustIdea_1:_Distribution_of_work_according_to_workers%27_interests Stronger categorization of work]
 +
* [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_Betzy_DarkHorseIdea:_Artificial_Turker Artificial turker that has to understand tasks before they go live]
  
1. [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_Pumas_PowerIdea_2:_lovePower New tasks go to workers first who improve the task before it goes live.]
+
=== Results: How might workers+requesters work together to produce higher-quality results? ===
 +
Examples:
 +
* [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_pentagram_TrustIdea_2:_Peer_Review_System Workers review other workers’ work (like on MobileWorks)]
 +
* [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_taskforce_poweridea1_qualitycontrolmanagers Could we create a crowd contractor who is in charge of each submission getting good results?]
  
2. [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_Triple_Clicks_PowerIdea_1:_Voting_on_HIT_design Voting on hit design]  
+
=== Resolution: How might we enable a fairer dispute resolution process and who's fault is it? ===
 +
Example:
 +
* [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_YourTeamName_PowerIdea_1:_Moderator_justification_for_rejected_hit Moderators can review if rejected work should be disputed] [http://crowdresearch.stanford.edu/w/index.php?title=Moderators_as_arbiters_of_disputes (similar idea here)]
 +
* [http://crowdresearch.stanford.edu/w/index.php?title=Dispute_resolution Dispute resolution process]
 +
* [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_pentagram_TrustIdea_1:_Priority_System Bad actors get hellbanned to a separate circle for a time being, or their tasks done more slowly]
 +
* [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_munichkindl_DarkHorseIdea_HangoutWithKetchup Force requesters to mediate previous problems before they can post new tasks moving forward]
  
3. [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_Team_Innovation_2_PowerIdea1:_Worker_Feedback_on_Tasks New tasks go into a holding pattern where they need to be "voted in” by workers]
+
=== Empathy: How might we build empathy between workers and requesters? ===
 +
Examples:
 +
* [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_taskforce_darkehorseidea_crowdbeers Meetings outside of the system to build trust]
 +
* [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_Pumas_TrustIdea_1:_empathy Make workers act as requesters, and visa versa]
 +
* [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_Opera_DarkHorse:Surprise_gifts Requesters can send gifts to workers]
 +
* [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_Team_Innovation_2_DarkHorseIdea:_Bonus_Pool_Payment_System A certain amount of requester costs are reserved to pay bonuses to workers]
 +
* [http://crowdresearch3.meteor.com/posts/XbuMSCMAM2XNxjM2W Humanizing worker profiles]
  
4. [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_PixelPerfect_PowerIdea_2:_Task_Templates Task templates (can we do better than AMT’s templates?)]
+
=== Transparency: How might we make payment clear and transparent? ===
 +
Examples:
 +
* [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_PixelPerfect_DarkHorseIdea:_Standardization_of_Task_Pricing Standardize task pricing]
 +
* [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_TestSet_DarkHorseIdea Required minimum wage? if you stay above the 15th percentile of “good work throughput”]
 +
* [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_ams_TrustIdea_2:_Checkpoint_System Checkpoints where you get reviewed and paid after every N tasks]
 +
* [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_Mustang_DarkHorse:_The_Kickstarter_Model Offer increased compensation to the first few people who take the task]
  
5. [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_TuringMachine_TrustIdea_1:_Distribution_of_work_according_to_workers%27_interests Stronger categorization of work]
+
=== Reputation: How might we design better reputation systems? ===
 +
Example:
 +
* [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_PixelPerfect_TrustIdea_2:_User_Rating_System Leveling up as a worker and requester: gives you better wages (Michael's thought there), first availability of tasks for workers, etc]. Also [http://crowdresearch.stanford.edu/w/index.php?title=Different_Levels_of_workers worker levels]
 +
* [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_ams_PowerIdea_1:_Bring_Top_Workers_Closer_to_Requesters Ranking top workers and making them most available]
 +
* [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_Professionals_PowerIdea_1:_Expose_worker_skills Workers rate requesters as they do tasks, used to learn matching for other workers.]
 +
* Tasks are first available to workers who match according to skill and performance
  
6. [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_Betzy_DarkHorseIdea:_Artificial_Turker Artificial turker that has to understand tasks before they go live]
+
=== Non-theme with interesting ideas ===
 
+
Examples:
=== Results: How might workers+requesters work together help produce higher-quality results? ===
+
* [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_RATH_Power_Idea_2:_Systemic_Democracy Democracy to decide on policy for the platform]
Example ideas of this theme:
+
 
+
1. [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_pentagram_TrustIdea_2:_Peer_Review_System Workers review other workers’ work (like on MobileWorks)]
+
 
+
2. [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_taskforce_poweridea1_qualitycontrolmanagers Could we create a crowd contractor who is in charge of each submission getting good results? (M). Related to crowd managers]
+
 
+
=== Resolution: How might we enable a fairer dispute resolution process? ===
+
Example ideas of this theme:
+
 
+
1. [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_YourTeamName_PowerIdea_1:_Moderator_justification_for_rejected_hit, also http://crowdresearch.stanford.edu/w/index.php? title=Moderators_as_arbiters_of_disputes Moderators can review if rejected work should be disputed]
+
 
+
2. [http://crowdresearch.stanford.edu/w/index.php?title=Dispute_resolution Dispute resolution process]
+
 
+
3. [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_pentagram_TrustIdea_1:_Priority_System Bad actors get hellbanned to a separate circle for a time being, or their tasks done more slowly]
+
 
+
4. [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_munichkindl_DarkHorseIdea_HangoutWithKetchup Force requesters to mediate previous problems before they can post new tasks moving forward]
+
 
+
=== Empathy: How might we build empathy? ===
+
Example ideas of this theme:
+
 
+
1. [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_taskforce_darkehorseidea_crowdbeers Meetings outside of the system to build trust]
+
 
+
2. [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_Pumas_TrustIdea_1:_empathy Make workers act as requesters, and visa versa]
+
 
+
3. [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_Opera_DarkHorse:Surprise_gifts, or also http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_Team_Innovation_2_DarkHorseIdea:_Bonus_Pool_Payment_System Requesters can send gifts to workers, or a certain amount of their costs are reserved to pay bonuses to workers]
+
 
+
4. [http://crowdresearch3.meteor.com/posts/XbuMSCMAM2XNxjM2W Humanizing worker profiles]
+
 
+
=== Transparent: How might we make payment clear and transparent? ===
+
Example ideas of this theme:
+
 
+
1. [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_PixelPerfect_DarkHorseIdea:_Standardization_of_Task_Pricing Standardize task pricing]
+
 
+
2. [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_TestSet_DarkHorseIdea Required minimum wage? if you stay above the 15th percentile of “good work throughput”]
+
 
+
3. [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_ams_TrustIdea_2:_Checkpoint_System Checkpoints where you get reviewed and paid after every N tasks]
+
 
+
4. [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_Mustang_DarkHorse:_The_Kickstarter_Model Offer increased compensation to the first few people who take the task]
+
 
+
=== Systems: How might we design better reputation systems? ===
+
Example ideas of this theme:
+
 
+
1. [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_PixelPerfect_TrustIdea_2:_User_Rating_System, also http://crowdresearch.stanford.edu/w/index.php?title=Different_Levels_of_workers Leveling up as a worker and requester: gives you better wages (M), first availability of tasks for workers, etc]
+
 
+
2. [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_ams_PowerIdea_1:_Bring_Top_Workers_Closer_to_Requesters Ranking top workers and making them most available]
+
 
+
3. [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_Professionals_PowerIdea_1:_Expose_worker_skills Workers rate requesters as they do tasks, used to learn matching for other workers Tasks are first available to workers who match according to skill and performance Skill categories on the platform]
+
 
+
=== Other: Interesting dark horse ideas outside of these themes ===
+
Example ideas of this theme:
+
 
+
1. Supply/demand curves for pricing — based on task classes
+
 
+
2. [http://crowdresearch.stanford.edu/w/index.php?title=Milestone_3_RATH_Power_Idea_2:_Systemic_Democracy Democracy to decide on policy for the platform]
+
  
 
== Recommended Readings ==
 
== Recommended Readings ==
Line 89: Line 66:
 
== Deliverables for Milestone 4 ==
 
== Deliverables for Milestone 4 ==
  
=== Read Ideas from Milestone 3 ===
+
=== Take inspiration from ideas in Milestone 3 ===
The first step to refining ideas is to become familiar with all the ideas that were generated by all the teams. Your first task is to read through and classify every idea generated into the themes that were mentioned above. You ''do not'' have to submit anything for this part. Create a list of as many ideas as you can read and classify them into the above themes.
+
Rather than brainstorming broadly as with last week, now we'll and use these ideas we've generated to inspire a focused brainstorm around these themes.  
  
=== Detailed Exploration of Ideas ===
+
As a team, choose three of the six theme areas that you are particularly interested in. Read through the ideas listed in the category.
From the above themes, choose 3 themes that you are most interested in and develop an idea that combines a subset of the ideas related to that theme from Milestone 3. You may use an existing idea from Milestone 3  and expand upon it by exploring it's feasibility.  
+
  
In Milestone 3, you had the opportunity to flare and consider many different ideas. Now, we have focused in on specific themes that have arisen from your ideas. The main purpose of this milestone is to flare again by generating ideas related to these themes. You are required to come up with 3 <strong>detailed</strong> ideas. Each idea must be from a different theme. And you may use at most one of the existing ideas from Milestone 3 and expand on it. The other two ideas must be original and should not ones that have already been presented in Milestone 3.
+
One of the best ways to have insights is to [http://groups.psych.northwestern.edu/gentner/papers/GentnerLoewensteinThompson03.pdf explicitly go through analogical reasoning], where you compare and contrast related cases. So, for each of the three themes you've chosen, talk with your team about:
 +
* What do these ideas tend to have in common?
 +
* How do these ideas tend to differ?
  
1. Create a succinct description of the problem that the theme is trying to address.
+
Use your insights from this conversation to generate design axes like we discussed in last week's meeting (see slide notes if you'd like). Write up a short paragraph (2-4 sentences) for each theme summarizing the similarities, differences, and axes inspired.
  
2. Summarize a few ideas related to this theme that try to address this problem from Milestone 3.
+
=== Flare and focus: generate ideas for the themes ===
 +
Now, for each of the three themes you've chosen, and armed with the knowledge from your colleagues' ideas and the "How might we" question that starts each theme, brainstorm with your team to generate more ideas related to that theme. Don't forget to do some dark horse brainstorming --- those ideas were often the most highly rated ones this week! Once you've generated a number of ideas for each theme, choose one idea per theme that you'd like to share with everyone. Ideally, your team will be inspired by the ideas listed above and generate new (and even better!) ideas. You may also want to take someone else's ideas from this past week and riff on them for your submission. Or, you may take one of your own previous ideas and iterate on it based on what you've learned.
  
3. Delve deep and explain your idea and how it can be used to solve the problem you described in your problem statement. Consider how your idea can be used. Mention it's limitations. Use charts, sketches, diagrams, tables and anything you think will help articulate your idea.  
+
As with last week, expand on the idea so others can understand it:
 +
* Create a succinct description of the problem that the theme is trying to address
 +
* In a sentence or two, credit the ideas that inspired or influenced your new idea.
 +
* Explain your idea and how it can be used to solve the problem you described in your problem statement. Consider how your idea will be used in context. Mention its limitations. Use sketches, charts, diagrams, tables and anything you think will help articulate your idea. Clarity and brevity are prized over detail.
  
 
== Submitting ==
 
== Submitting ==
Line 112: Line 94:
  
 
We have a service on which you can post the links to the wiki-pages for the individual ideas you generated, explore them, and upvote them.
 
We have a service on which you can post the links to the wiki-pages for the individual ideas you generated, explore them, and upvote them.
 
Instructions for posting are at http://crowdresearch.meteor.com/posts/bXSNbqihjajASBQEL
 
  
 
<strong>Sign-up Instructions: </strong> Log in with either Twitter or Facebook. When it asks you to pick your username, pick the ''same username as your Slack'', this will help us identify and track your contributions better.  
 
<strong>Sign-up Instructions: </strong> Log in with either Twitter or Facebook. When it asks you to pick your username, pick the ''same username as your Slack'', this will help us identify and track your contributions better.  
Line 119: Line 99:
 
There are 7 possible submission categories:
 
There are 7 possible submission categories:
  
1- http://crowdresearch.meteor.com/category/milestone-4-descriptions where you can post links to the wiki pages for each of the 2 description-related ideas you generated.
+
1- http://crowdresearch.meteor.com/category/milestone-4-descriptions where you can post links to the wiki pages for any description-related ideas you generated.
  
2- http://crowdresearch.meteor.com/category/milestone-4-results where you can post links to the wiki pages for each of the 2 results-related ideas you generated.
+
2- http://crowdresearch.meteor.com/category/milestone-4-results where you can post links to the wiki pages for any 2 results-related ideas you generated.
  
3- http://crowdresearch.meteor.com/category/milestone-4-empathy where you can post links to the wiki pages for each of the 2 empathy-related ideas you generated.
+
3- http://crowdresearch.meteor.com/category/milestone-4-empathy where you can post links to the wiki pages for any empathy-related ideas you generated.
  
4- http://crowdresearch.meteor.com/category/milestone-4-resolution where you can post links to the wiki pages for each of the 2 resolution-related ideas you generated.
+
4- http://crowdresearch.meteor.com/category/milestone-4-resolution where you can post links to the wiki pages for any resolution-related ideas you generated.
  
5- http://crowdresearch.meteor.com/category/milestone-4-transparency where you can post links to the wiki pages for each of the 2 transparency-related ideas you generated.
+
5- http://crowdresearch.meteor.com/category/milestone-4-transparency where you can post links to the wiki pages for any transparency-related ideas you generated.
  
6- http://crowdresearch.meteor.com/category/milestone-4-systems where you can post links to the wiki pages for each of the 2 systems-related ideas you generated.
+
6- http://crowdresearch.meteor.com/category/milestone-4-systems where you can post links to the wiki pages for any systems-related ideas you generated.
  
7- http://crowdresearch.meteor.com/category/milestone-4-other where you can post links to the wiki pages for each of the 2 other ideas you generated.
+
7- http://crowdresearch.meteor.com/category/milestone-4-other where you can post links to the wiki pages for any other ideas you generated.
  
Post links to your ideas <strong>only once they're finished</strong>. Give your posts titles which summarize your idea. Viewers should be able to <strong>get the main point by skimming the title</strong> ("Automatic Pricing for Tasks based on Average Completion Time" is a good title. "YourTeam Idea 1" is a bad title).
+
Post links to your ideas <strong>only once they're finished</strong>. Give your posts titles which summarize your idea. Viewers should be able to <strong>get the main point by skimming the title</strong> ("Leveling System for Workers Leading to Bettter Wages" is a good title. "YourTeam Idea 1" is a bad title).
  
 
-<strong>Please submit your finished ideas by 11:59 pm 25th March 2015, and DO NOT vote/comment until 26th March 12:05 am</strong>
 
-<strong>Please submit your finished ideas by 11:59 pm 25th March 2015, and DO NOT vote/comment until 26th March 12:05 am</strong>
 +
 
=== [Everyone] Peer-evaluation (upvote ones you like, comment on them) from 12:05 am 26th March until 9 am 27th March ===
 
=== [Everyone] Peer-evaluation (upvote ones you like, comment on them) from 12:05 am 26th March until 9 am 27th March ===
  
Post submission phase, you are welcome to browse through, upvote, and comment on others' ideas. We encourage you especially to look at and comment on ideas that haven't yet gotten feedback, to make sure everybody's ideas gets feedback. You can use http://crowdresearch.meteor.com/needcomments to find ideas that haven't yet gotten feedback, and http://crowdresearch.meteor.com/needclicks to find ideas that haven't been yet been viewed many times.
+
Post submission phase, you are welcome to browse through, upvote, and comment on others' ideas. We encourage you especially to look at and comment on ideas that haven't yet gotten feedback, to make sure everybody's ideas gets feedback.  
 +
 
 +
<strong> Step 1: </strong> Please use http://crowdresearch.meteor.com/needcomments to find ideas that haven't yet gotten feedback, and http://crowdresearch.meteor.com/needclicks to find ideas that haven't been yet been viewed many times.
 +
 
 +
<strong> Step 2: </strong> Once you find an idea of interest or less attended, please vote and comment upon it. Please perform this action from 3 to 5 ideas - this will help us balance the comments and votes. Please do not vote your team's idea. Once again, '''everyone''' is supposed to vote+comment, whether you're the team leader or not.  
  
 
<strong>COMMENT BEST-PRACTICES</strong>: As on Crowdgrader, <strong>everybody</strong> reviews at least 3 ideas, supported by a comment. The comment has to justify your reason for upvote. The comment should be constructive, and should mention positive aspect of the idea worth sharing. Negative comments are discouraged, rather make your comment in the form of a suggestion - such as, if you disliked an idea, try to suggest improvements (do not criticize an idea, no idea is bad, every idea has a scope of improvement).
 
<strong>COMMENT BEST-PRACTICES</strong>: As on Crowdgrader, <strong>everybody</strong> reviews at least 3 ideas, supported by a comment. The comment has to justify your reason for upvote. The comment should be constructive, and should mention positive aspect of the idea worth sharing. Negative comments are discouraged, rather make your comment in the form of a suggestion - such as, if you disliked an idea, try to suggest improvements (do not criticize an idea, no idea is bad, every idea has a scope of improvement).

Latest revision as of 09:47, 21 March 2015

Flare and focus! Our goal this week will be to take the flare phase from last week, focus it around a few themes, and then use each others' ideas to flare out once more to identify the points of view we like. Due date (PST): 11:59 pm 25th March 2015 for submission, 9 am 27th March 2015 for voting and commenting on others' ideas.

  • Youtube link of the meeting today: watch
  • Meeting 4 slideshow: pdf

Themes from Milestone 3

Based on your submissions for Milestone 3, we synthesized the following themes. Take a look at the high level patterns, since these will be the themes we'll cluster our brainstorm around.

Task clarity: How might workers+requesters work together help produce higher-quality task descriptions?

Examples:

Results: How might workers+requesters work together to produce higher-quality results?

Examples:

Resolution: How might we enable a fairer dispute resolution process and who's fault is it?

Example:

Empathy: How might we build empathy between workers and requesters?

Examples:

Transparency: How might we make payment clear and transparent?

Examples:

Reputation: How might we design better reputation systems?

Example:

Non-theme with interesting ideas

Examples:

Recommended Readings

Coming up with good, novel visions and ideas is a crucial part of doing successful research, and reading other researchers' visions and ideas can help you come up with better ideas yourself. These readings discuss visions that crowdsourcing researchers have thought of related to future crowd marketplaces. This week's readings are optional and don't have a deliverable, but are highly recommended.

Mason W, Watts D J. Financial incentives and the performance of crowds. ACM SigKDD Explorations Newsletter, 2010, 11(2): 100-108.

Dow S, Kulkarni A, Klemmer S, et al. Shepherding the crowd yields better work. Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work. ACM, 2012: 1013-1022.

Deliverables for Milestone 4

Take inspiration from ideas in Milestone 3

Rather than brainstorming broadly as with last week, now we'll and use these ideas we've generated to inspire a focused brainstorm around these themes.

As a team, choose three of the six theme areas that you are particularly interested in. Read through the ideas listed in the category.

One of the best ways to have insights is to explicitly go through analogical reasoning, where you compare and contrast related cases. So, for each of the three themes you've chosen, talk with your team about:

  • What do these ideas tend to have in common?
  • How do these ideas tend to differ?

Use your insights from this conversation to generate design axes like we discussed in last week's meeting (see slide notes if you'd like). Write up a short paragraph (2-4 sentences) for each theme summarizing the similarities, differences, and axes inspired.

Flare and focus: generate ideas for the themes

Now, for each of the three themes you've chosen, and armed with the knowledge from your colleagues' ideas and the "How might we" question that starts each theme, brainstorm with your team to generate more ideas related to that theme. Don't forget to do some dark horse brainstorming --- those ideas were often the most highly rated ones this week! Once you've generated a number of ideas for each theme, choose one idea per theme that you'd like to share with everyone. Ideally, your team will be inspired by the ideas listed above and generate new (and even better!) ideas. You may also want to take someone else's ideas from this past week and riff on them for your submission. Or, you may take one of your own previous ideas and iterate on it based on what you've learned.

As with last week, expand on the idea so others can understand it:

  • Create a succinct description of the problem that the theme is trying to address
  • In a sentence or two, credit the ideas that inspired or influenced your new idea.
  • Explain your idea and how it can be used to solve the problem you described in your problem statement. Consider how your idea will be used in context. Mention its limitations. Use sketches, charts, diagrams, tables and anything you think will help articulate your idea. Clarity and brevity are prized over detail.

Submitting

Create a Wiki Page for your Team's Submission

Please create a page for your team's submission at http://crowdresearch.stanford.edu/w/index.php?title=Milestone_4_YourTeamName&action=edit (substituting in YourTeamName with the team name), copy over the template at Milestone 4 Template . If you have never created a wiki page before, please see this or watch this.

[Team Leaders] Submission or Post the links to your ideas until 25th March 11:59 pm

We have a service on which you can post the links to the wiki-pages for the individual ideas you generated, explore them, and upvote them.

Sign-up Instructions: Log in with either Twitter or Facebook. When it asks you to pick your username, pick the same username as your Slack, this will help us identify and track your contributions better.

There are 7 possible submission categories:

1- http://crowdresearch.meteor.com/category/milestone-4-descriptions where you can post links to the wiki pages for any description-related ideas you generated.

2- http://crowdresearch.meteor.com/category/milestone-4-results where you can post links to the wiki pages for any 2 results-related ideas you generated.

3- http://crowdresearch.meteor.com/category/milestone-4-empathy where you can post links to the wiki pages for any empathy-related ideas you generated.

4- http://crowdresearch.meteor.com/category/milestone-4-resolution where you can post links to the wiki pages for any resolution-related ideas you generated.

5- http://crowdresearch.meteor.com/category/milestone-4-transparency where you can post links to the wiki pages for any transparency-related ideas you generated.

6- http://crowdresearch.meteor.com/category/milestone-4-systems where you can post links to the wiki pages for any systems-related ideas you generated.

7- http://crowdresearch.meteor.com/category/milestone-4-other where you can post links to the wiki pages for any other ideas you generated.

Post links to your ideas only once they're finished. Give your posts titles which summarize your idea. Viewers should be able to get the main point by skimming the title ("Leveling System for Workers Leading to Bettter Wages" is a good title. "YourTeam Idea 1" is a bad title).

-Please submit your finished ideas by 11:59 pm 25th March 2015, and DO NOT vote/comment until 26th March 12:05 am

[Everyone] Peer-evaluation (upvote ones you like, comment on them) from 12:05 am 26th March until 9 am 27th March

Post submission phase, you are welcome to browse through, upvote, and comment on others' ideas. We encourage you especially to look at and comment on ideas that haven't yet gotten feedback, to make sure everybody's ideas gets feedback.

Step 1: Please use http://crowdresearch.meteor.com/needcomments to find ideas that haven't yet gotten feedback, and http://crowdresearch.meteor.com/needclicks to find ideas that haven't been yet been viewed many times.

Step 2: Once you find an idea of interest or less attended, please vote and comment upon it. Please perform this action from 3 to 5 ideas - this will help us balance the comments and votes. Please do not vote your team's idea. Once again, everyone is supposed to vote+comment, whether you're the team leader or not.

COMMENT BEST-PRACTICES: As on Crowdgrader, everybody reviews at least 3 ideas, supported by a comment. The comment has to justify your reason for upvote. The comment should be constructive, and should mention positive aspect of the idea worth sharing. Negative comments are discouraged, rather make your comment in the form of a suggestion - such as, if you disliked an idea, try to suggest improvements (do not criticize an idea, no idea is bad, every idea has a scope of improvement).

[Team Leaders] Milestone 4 Submissions

To help us track all submissions and browsing through them, once you have finished your Milestone 4, go to the link below and post the link:

Milestone 4 Submissions