Winter Milestone 5

From crowdresearch
Revision as of 23:28, 9 February 2016 by Durimmorina (Talk | contribs) (Research Engineering (All))

Jump to: navigation, search

Due date (PST): 8:00 pm 14th Feb 2016 for submission, 12 pm 15th Feb 2016 for peer-evaluation

This week, we will accept proposals to pursue different aspects of the project, and start a design test run.

  • Youtube link of the meeting today: watch
  • Winter Meeting 5 slideshow: slides pdf
  • Youtube link of the task feed meeting 1 today: watch
  • Youtube link of the task feed meeting 2 today: watch
  • Youtube link of the task authoring meeting 1 today: watch
  • Youtube link of the task authoring meeting 2 today: watch

Research: Participate/watch additional meeting videos and submit a specific proposal (All)

Last week we asked you to pitch research ideas for the three themes - open gov, task ranking, task authoring. This week we ask you to choose at least one the themes below, and dive deep into it, with specifics. The goal is to propose a concrete and specific research project idea: one that we could execute in the coming months and submit to UIST or CSCW. This may take the form of a system or social infrastructure we build into Daemo, or an experiment or study we run on it. To express this idea as a concise research concept, write it in the form of an introduction section to a research paper - see the template Winter Milestone 5 Templates. The submissions that are most viable and popular will be set as our direction for our research project, and the submitters may be asked to help us lead it!.

a. Task authorship

Watch the video recordings from the meeting, below:

  • Youtube link of the task authoring meeting 1 today: watch
  • Youtube link of the task authoring meeting 2 today: watch

The typical narrative is that workers produce highly varying quality work. Nobody has studied the effect that requester quality has. Can we quantify the variance in requester quality, and introduce interventions to help improve it?

A specific idea raised in last week's submissions that gathered a lot of interest: could we run an experiment to demonstrate how much variance there is in requester quality for the same authoring task, vs. how much variation there is in worker quality? Task authorship

b. Task ranking

Requesters don't get ideal workers, and workers don't get ideal requesters - can we rank the relevant tasks, on the basis of reputation, skills, and other necessary aspects?

A specific idea raised in last week's submissions that gathered a lot of interest: could we design an task feed ranking interface and algorithm for Daemo? A combination of user-centered work and machine learning/data mining? Task ranking

c. Open gov

In current systems, worker and requester voices remain unheard; and the platform is run by a central organization with all control. Can we infuse the idea of open governance in Daemo?

A specific idea raised in last week's submissions that gathered a lot of interest: could workers form guilds for specific expertise areas, then run their own reputation and ranking operations? Like how doctors and lawyers administer their own tests for determining whether to license you? Open gov

Deliverable

We've synthesized some of the most popular ideas for each area. Grab at least one area (task authoring, task ranking, open gov), and develop it further into a concrete research proposal using the template - Winter Milestone 5 Templates! There are two type of proposals you can write - systems (where the novel contribution is a software system or platform to solve problems) and science (where the novel contribution is a phenomenon or an approach or a study to understand certain behavior that solves the problem). In your write up, follow the outline format below. Check out this really helpful paper.

Systems

After you choose one of the themes above, and decide to solve it through a system, write the system focussed introduction section using the outline below. You can see example introductions below.

Systems

Science

After you choose one of the themes above, and decide to solve it through a science, write the phenomenon focussed introduction section using the outline below. You can see example introductions below.

Science

Research Engineering (All)

This weeks main issues: #77 (this is a pretty big one), #660

announce in #research-engineering that you are working on a particular issue and please let the others know about the progress of the issues you are working on (so that we don't do duplicate work). You are encouraged to work together.

For any questions ping @aginzberg, @dmorina, and @shirish.goyal on Slack #research-engineering

Design (Test Flight)

DRI: @karolina and @michaelbernstein

Great work last week, its time to iterate on it. Watch the weekly meeting for lengthier discussion and refine the messaging system for a more complete mockup and storyboard. Something that we can go implement, rather than a very high level idea. You can look at this week's slide to see some of the top ideas that were pitched last week.

Use Balsamiq or any tool of your choice, and share your unique mockup/wireframe of the messaging system on Daemo.

Submission

Create a Wiki Page for your Team's Submission

Create a wiki page with the introduction section, diving deep into one the three themes (look at the template here: Winter Milestone 5 Templates). If you're participating in design test run, create another wiki and paste screenshots or mockup/wireframe files. If you have never created a wiki page before, please see this or watch this.

[Team Representative] Submission or Post the links to your ideas until 8:00 pm 14th Feb 2016

We have a [Reddit like service] on which you can post the links to the wiki-pages for the submissions, explore them, and upvote them.

Please use the same login avenue (Facebook, Twitter, or email address) as you’ve done in the past with Meteor​. This will help us identify and track your contributions better.

For newcomers joining Crowd Research, when it asks you to pick your username, pick the same username as your Slack. Please DO NOT forget to mention the milestone contributors' slackid below each wiki page.

On Meteor, there are 4 submission categories, 3 for research theme, and 1 for design test run.

1- [One of three mandatory] http://crowdresearch.meteor.com/category/task-rank where you can post a link to the wiki page for your task ranking proposal

2- [One of three mandatory] http://crowdresearch.meteor.com/category/task-author where you can post a link to the wiki page for your task author proposal

3- [One of three mandatory] http://crowdresearch.meteor.com/category/open-gov where you can post a link to the wiki page for your open gov proposal

4- [Test flight] http://crowdresearch.meteor.com/category/design-messenger where you can post mockup/wireframe of the Daemo messenger.

Give your posts titles which summarize your idea. Viewers should be able to get the main point by skimming the title ("Automatic Suggestion for Tasks based on Average Completion Time" is a good title. "YourTeam TrustIdea 1" is a bad title).

-Please submit your finished ideas by 8:00 pm 14th Feb 2016, and DO NOT vote/comment until then

[Design Test Flight]

For your Messenger Design Wiki submissions, please create Google Slides showing the flow of your designs and link it in your Wiki page. For example, see this slide show

When you create the "Share" link, make sure to click “Anyone with the link can view"

[Everyone] Peer-evaluation (upvote ones you like, comment on them) from 8:05 pm 14th Feb until 12 pm 15th Feb 2016

Post submission phase, you are welcome to browse through, upvote, and comment on others' ideas. We encourage you especially to look at and comment on ideas that haven't yet gotten feedback, to make sure everybody's ideas gets feedback. You can use http://crowdresearch.meteor.com/needcomments to find ideas that haven't yet gotten feedback, and http://crowdresearch.meteor.com/needclicks to find ideas that haven't been yet been viewed many times.

COMMENT BEST-PRACTICES: Everybody in the team reviews at least 3 ideas, supported by a comment. The comment has to justify your reason for upvote. The comment should be constructive, and should mention positive aspect of the idea worth sharing. Negative comments are discouraged, rather make your comment in the form of a suggestion - such as, if you disliked an idea, try to suggest improvements (do not criticize an idea, no idea is bad, every idea has a scope of improvement).