Difference between revisions of "Main Page"

From crowdresearch
Jump to: navigation, search
(Summer meetings and slides)
(Summer milestones)
Line 247: Line 247:
 
* [[Summer Milestone 27]] - 11:59 pm 06th December 2015 for submission and for voting + commenting on others' submissions.
 
* [[Summer Milestone 27]] - 11:59 pm 06th December 2015 for submission and for voting + commenting on others' submissions.
  
 +
* [[Summer Milestone 28]] - 11:59 pm 13th December 2015 for submission and for voting + commenting on others' submissions.
  
  

Revision as of 18:44, 7 December 2015

Welcome to the Crowd Research project wiki, we're a group of five hundred designers, engineers, crowd workers, and crowd requesters from around the world who are building Daemo - a self-governed crowdsourcing marketplace.

NEWS:

  • We'll invite applications for the winter session, aspiring researchers and hackers apply in December 2015. We will open up the forms soon. To help you get started and understand what we've done so far, we've created this -> "Introducing Crowd Research Initiative and Recap Fall15" (recap and overview page), where you can learn more about this project, our progress and plans!
  • Our first version (Daemo) is live on daemo.stanford.edu.
  • We're semi-finalists (top 40 of 1,000+) at the Knights Challenge, check out our proposal and video for the brief overview of Daemo.
  • Our ACM UIST'15 got accepted, read it for more details.

Some research projects are too big and too important to tackle alone. Sometimes, we need to team up.

At Stanford’s Computer Science department, we've observed that people who are aiming to get research experience or launch their research career will often fall into an expertise valley. Undergraduates are assigned extremely tightly scoped activities within research projects, getting little room for creativity. Then these folks get into PhD programs, have literally the entire space of human knowledge to explore, and don’t have enough scaffolding to make quick progress.

We're going to create a crowdsourced research team to tackle both these challenges together. We’ll gather as many talented folks as we can get, and work to build out that intervening bridge between tightly-scoped work and open-ended exploration. It will have far more flexibility than a typical research experience, but with a focused goal where we can bring each other back on track each week and nobody gets lost.

About the project

About Daemo - a self-governed crowdsourcing marketplace

Whether you need help gathering data, labeling machine learning training examples, running experiments, or transcribing audio, today we use crowdsourcing platforms such as Amazon Mechanical Turk. However, these platforms are notoriously bad at ensuring high quality results, producing respect and fair wages for workers, and making it easy to author effective tasks. It’s not hard to imagine that we could do better.

This research will be a complete design, implementation, launch, and evaluation of a new crowdsourcing platform. What would it take to create an effective marketplace? One where workers have more power in the employment relationship, or could take additional responsibility for the result quality? How might we design such a market? Could we launch it and become the new standard? This research in human-computer interaction will involve a combination of design thinking, web development, and experimental design. This is far more ambitious than your typical project. It’s an entire marketplace design question. Thus, we’re banding together to solve it.

About Crowd Research initiative

Crowd Research is an experimental initiative by professor and students at Stanford to recognize the potential of upcoming student researchers and exploring the possibility of massive research collaboration between the two. It is a first of its kind research-at-scale initiative in the world. We're a group of about 500 students, professionals and researchers from all around the world, working together by engineering, building, designing and making Daemo a reality. We call ourselves, "Stanford Crowd Research Collective". The project is organized by Prof. Michael Bernstein at Stanford HCI and his students.

Participant contribution

Well, first, there’s creating a crowdsourcing market that becomes the new standard. This could lead to a far better future for crowdsourcing and crowd work, and millions of people could eventually use it. It’s research, of course, so there’s always risk it might not work out — but if we knew it would work, it wouldn’t be research!

Second, we’ll be planning papers to top-tier conferences based on our work. If you are considering an MS or especially a PhD program, being a heavily contributing author on a paper can greatly improve your chances. How much you contribute to the project will determine author order. Last, I really do hope to build relationships with a diverse range of researchers.

Summer meetings and slides

See Archives for winter and spring meeting. If you're just getting started, consider going through this page: Introducing Crowd Research Initiative and Recap Fall15.

Summer Meeting 1: Onboarding summer researchers and recap

  • Mid week meetings : Getting started with research and code
  • Youtube link of the office hour meeting : watch
  • Youtube link of getting started with code: watch

Summer Meeting 2: Reflecting on getting started and open-gov + micro+macro

Summer Meeting 3: Focussing on open-gov + critiquing micro+macro

Summer Meeting 4: Focussing on micro+macro, open gov, UIST paper and research engineering

Summer Meeting 5: Focussing on micro+macro, open gov, UIST paper and research engineering

Summer Meeting 6: Focussing on micro+macro, UIST paper and research engineering

Summer Meeting 7: UIST paper+poster

Summer Meeting 8: Celebrating UIST 2015 submission, what's next?

  • Youtube link of the meeting today: watch

Summer Meeting 9: Find, fix, verify

  • Youtube link of the meeting today: watch
  • Summer Meeting 9 slideshow: pdf
  • Leadership slideshow: pdf
  • Usability study 7 slideshow: pdf

Summer Meeting 10: Moving forward

Summer Meeting 11: Gathering momentum

Summer Meeting 12: Puzzle pieces falling into place

Summer Meeting 13: End to end

  • Youtube link of the meeting today: watch
  • Summer Meeting 13 slideshow: end to end

Summer Meeting 14: Boomerang

  • Youtube link of the meeting today: watch
  • Summer Meeting 14 slideshow: boomerang

Summer Meeting 15:

  • Youtube link of the meeting today: watch
  • Summer Meeting 15 slideshow: pdf

Summer Meeting 16:

  • Youtube link of the meeting today: watch
  • Summer Meeting 16 slideshow: pdf

Summer Meeting 17:

  • Youtube link of the meeting today: watch
  • Summer Meeting 17 slideshow: pdf

Summer Meeting 18:

  • Youtube link of the meeting today: watch
  • Summer Meeting 18 slideshow: pdf

Summer Meeting 19:

  • Youtube link of the meeting today: watch
  • Summer Meeting 19 slideshow: no or minimal slides

Summer Meeting 20:

  • Youtube link of the meeting today: watch
  • Summer Meeting 20 slideshow: pdf

Summer Meeting 21:

  • Youtube link of the meeting today: watch
  • Summer Meeting 21 slideshow: pdf

Summer Meeting 22:

  • Youtube link of the meeting today: watch
  • Summer Meeting 22 slideshow: pdf

Summer Meeting 23:

  • Youtube link of the meeting today: watch
  • Summer Meeting 23 slideshow: coming soon/did not exist

Summer Meeting 24:

  • Youtube link of the meeting today: watch
  • Summer Meeting 24 slideshow: coming soon/did not exist

Summer Meeting 25:

  • Youtube link of the meeting today: watch
  • Summer Meeting 25 slideshow: coming soon/did not exist

Summer Meeting 26:

  • Youtube link of the meeting today: watch
  • Summer Meeting 26 slideshow: coming soon/did not exist

Summer Meeting 27:

  • Youtube link of the meeting today: watch
  • Summer Meeting 27 slideshow: coming soon/did not exist

Summer Meeting 28:

  • Youtube link of the meeting today: watch
  • Summer Meeting 28 slideshow: coming soon/did not exist

Summer milestones

All timings in PST (California time)

See Archives for winter and spring milestone wikis and see what's coming up next: calendar.

  • Summer Milestone 1 - 11:59 pm 28th May 2015 for submission, 12 pm 29th May 2015 for peer-evaluation.
  • Summer Milestone 3 - 11:59 pm 11th June 2015 for submission and peer-evaluation. Please note the deadline changed to Thursday midnight.
  • Summer Milestone 4 - 11:59 pm 18th June 2015 for submission, 12 pm 19th June 2015 for voting and commenting on others' prototypes.
  • Summer Milestone 5 - 11:59 pm 25th June 2015 for submission, 12 pm 26th June 2015 for voting and commenting on others' prototypes.
  • Summer Milestone 12 - 11:59 pm 14th August 2015 for submission and for voting + commenting on others' submissions.
  • Summer Milestone 13 - 11:59 pm 21st August 2015 for submission and for voting + commenting on others' submissions.
  • Summer Milestone 14 - 11:59 pm 28th August 2015 for submission and for voting + commenting on others' submissions.
  • Summer Milestone 15 - 11:59 pm 4th September 2015 for submission and for voting + commenting on others' submissions.
  • Summer Milestone 16 - 11:59 pm 11th September 2015 for submission and for voting + commenting on others' submissions.
  • Summer Milestone 17 - 11:59 pm 18th September 2015 for submission and for voting + commenting on others' submissions.
  • Summer Milestone 18 - 11:59 pm 25th September 2015 for submission and for voting + commenting on others' submissions.
  • Summer Milestone 19 - 11:59 pm 9th October 2015 for submission and for voting + commenting on others' submissions.
  • Summer Milestone 20 - 11:59 pm 18th October 2015 for submission and for voting + commenting on others' submissions.
  • Summer Milestone 21 - 11:59 pm 25th October 2015 for submission and for voting + commenting on others' submissions.
  • Summer Milestone 22 - 11:59 pm 1st November 2015 for submission and for voting + commenting on others' submissions.
  • Summer Milestone 23 - 11:59 pm 8th November 2015 for submission and for voting + commenting on others' submissions.
  • Summer Milestone 24 - 11:59 pm 15th November 2015 for submission and for voting + commenting on others' submissions.
  • Summer Milestone 25 - 11:59 pm 22nd November 2015 for submission and for voting + commenting on others' submissions.
  • Summer Milestone 26 - 11:59 pm 29th November 2015 for submission and for voting + commenting on others' submissions.
  • Summer Milestone 27 - 11:59 pm 06th December 2015 for submission and for voting + commenting on others' submissions.
  • Summer Milestone 28 - 11:59 pm 13th December 2015 for submission and for voting + commenting on others' submissions.


Infrastructure

Infrastructure - homepage for Infrastructure related efforts, contains separate milestones.

GettingStarted - getting started for infrastructure related efforts.

BranchingStrategy - to help you get started with GitHub.

General weekly plan

We’ll meet weekly over videochat and lay out our goals for the next week. At the end of the week, you’ll submit what you’ve been working on. Your peers and a Ph.D. student here at Stanford will peer critique the work, and we’ll talk about the best stuff each week in our meeting. The sky’s the limit.

I’m sure we’ll adjust this as we go. Because, this entire crowdsourced research idea is a bit of a research project in itself, too.

  • Monday evening 8 pm PST: Prof meeting with participants and milestone set for the next week (over Google Hangout on Air)
  • Monday after meeting - Sunday midnight PST: participants work on their milestones (~6 days)
  • Post Sunday midnight PST - Monday noon 12 pm PST: peer-evaluation by the participants (~12 hours). This is occasional.
  • Monday evening 8 pm PST: Prof meets based on the input from RAs and top submissions. Participants receive their next milestone and a feedback survey after every meeting.

Skill requirements

In research? None. Anybody who is smart and dedicated can help us envision the future of crowdsourcing and articulate how it might play out.

In terms of skills, there are many different ways that you can participate. If you want to contribute design skills, having a portfolio of past work would be helpful. If you are a CS major or enjoy programming, you’d likely need to have completed an introductory programming course sequence to succeed. We’re currently building infrastructure and implementing foundations using Djanjo, Angular.js, PostgreSQL, REST framework, so knowledge or experience in these areas would be extremely helpful. If you have experience in social science methods (e.g., surveys, qualitative work, designing controlled experiments), there will be lots to do as well to help us make sure we’re creating the right thing.

For everyone, a class in human-computer interaction (such as Scott Klemmer’s HCI Online, which you can complete as prep) will be a huge leg up.

Resources

  • Slack - used for chat and discussion
  • Meteor - used for voting, evaluation and comments, for open-gov open-gov
  • Github - used for collaborating the development process
  • Relevant Work - used to save relevant articles, links and papers
  • Forums - discussions about this project on external sites. Note that all official announcements and communications will occur via Slack and email.
  • Resources - used to index all platforms and resources as we evolve
  • Archives - older meetings, slides and milestones
  • Badges - tutorial for badges, list of them and how to use -> Bonus.ly
  • Current Pagerank - Current pagerank, as computed based on the badge-giving on bonusly (will be used to set author order)

Want to share some other resources? Create a wiki page, and post it at Resources.

About this wiki

You need to be logged in to edit this wiki and view private pages. If you already have an account but cannot login, try resetting your password. If you need an account, please check your email (login credentials were emailed out at the start of the project), and if you can't find the email, contact Geza (@geza on slack), stating your email, and you will be emailed your login details.

All pages on this wiki are public by default - so anyone visiting this site can see them.

Any page which ends with (private) will be private - so you need to be logged in to see them. See example.

Contact us

Find us on Slack! @rajanvaish or @geza or @ranjaykrishna or @michaelbernstein. Alternatively, you can email us at crowdresearch@cs.stanford.edu.