Difference between revisions of "Winter Milestone 10"
(→Open Gov and Design) |
|||
| (3 intermediate revisions by one other user not shown) | |||
| Line 4: | Line 4: | ||
* Youtube link of the meeting today: [http://www.youtube.com/watch?v=CDZwFWCv7yg watch] | * Youtube link of the meeting today: [http://www.youtube.com/watch?v=CDZwFWCv7yg watch] | ||
| − | * Winter Meeting 10 slideshow: | + | * Winter Meeting 10 slideshow: [[:Media:03-14-Milestone9.pdf| slides pdf]] |
== Task Feed == | == Task Feed == | ||
| Line 49: | Line 49: | ||
== Task Authoring == | == Task Authoring == | ||
| − | + | '''Watch last week's task authoring meeting''' [https://www.youtube.com/watch?v=BqhlhKQwGzo here] | |
| − | ''' | + | '''Rewind of Monday's meeting:''' We ran our second pilot study on the Dark Matter (marijuana) task. It produced useful insight! We realized that the requester variance was mainly coming into play with the tasks that are the most difficult. The obvious ones, everyone gets right. So, we started cleaving off the lowest quartile of tasks by average worker accuracy, and saw bigger variation across requesters. We went back and re-analyzed our original pilot data, and saw the same effect with some of the tasks (e.g. football action prediction). |
| − | ''' | + | '''Milestone''' |
| − | '''By Wednesday | + | '''By Wednesday''' Find a task that is difficult, where in the answer is not obvious. |
| − | '''By | + | We're thinking that Minnesota and Macalister paper on Arafat Gold might be the right one to go with. |
| + | So in addition, please read the following Macalister paper [http://www-users.cs.umn.edu/~bhecht/publications/goldstandards_CSCW2015.pdf paper] | ||
| + | |||
| + | Message '''@catherine.mullings''' with any questions and stay up to date with '''#taskauthoring''' channel. | ||
| + | |||
| + | '''By Thursday/Friday''' we will analyze the results and hold a Google Hangout. Stay tuned! | ||
The analysis for the study will be formally written [https://docs.google.com/document/d/1B-gSbAn3atDvV8qZtsRWrJ0CdKhbMDO7B3dgFK_yZyo/edit?usp=sharing here on Google doc]. | The analysis for the study will be formally written [https://docs.google.com/document/d/1B-gSbAn3atDvV8qZtsRWrJ0CdKhbMDO7B3dgFK_yZyo/edit?usp=sharing here on Google doc]. | ||
Latest revision as of 08:38, 17 March 2016
Due date (PST): 8:00 pm 20th March 2016 for submission, 12 pm 21st March 2016 for peer-evaluation.
This week, we will refine methods and systems proposed last week further:
- Youtube link of the meeting today: watch
- Winter Meeting 10 slideshow: slides pdf
Task Feed
Great work last week!
This week we are continuing our push for the UIST submission. Our goals for this week are the following:
- Finish of our timer/data collection mock - @sehgalvibhor.
- Design review of taskfeed system - @christopher.diemert @hizai @mathiasburton
- Finish task time backend:
- Handle time scaling factor @dmorina.
- Finish backend update of Boomerang:
- Use the new data from the rejection rate to modify the cascade: @aginzberg.
- Design Study 2: extended Boomerang for requesters
- Start Sharelatex
- Draft introduction and related work
- Prepare system for studies
- Handle various treatments
- Put many tasks on the platform: protocol to come
We will be having a very brief hangout on Tuesday at 10AM Pacific to divide up work and discuss any questions about this week's milestones.
With the internal deadline on April 3rd, we are going to need all hands on deck for the next month. Whether you are a developer, designer, writer, thinker, etc. there will always be something for you to work on. Please pick up anything and give a yell in the channel so everyone knows who is working on what and so that we can help and guide each other. As always, feel free to reach out to @aginzberg or @dmorina individually with any questions.
Topic background
Check out our system proposal, mainly first few pages and comments to catch up on what we've been working on and thinking about.
The task feed hangouts from previous weeks:
- Youtube link of the task feed meeting 1: watch
- Youtube link of the task feed meeting 2: watch
- Youtube link of the task feed meeting 3: watch
Michael's synthesized needs:
- to find new tasks that will maximize income (reduce uncertainty in payment, rejection, maximize certainty in what will be asked of me and how quickly I can do it)
- to find new tasks that fit my expertise profile
- to refind old requesters' new tasks, since I know I like them
In addition...
- to identify tasks I can do on my own time
- to learn new skills
Task Authoring
Watch last week's task authoring meeting here
Rewind of Monday's meeting: We ran our second pilot study on the Dark Matter (marijuana) task. It produced useful insight! We realized that the requester variance was mainly coming into play with the tasks that are the most difficult. The obvious ones, everyone gets right. So, we started cleaving off the lowest quartile of tasks by average worker accuracy, and saw bigger variation across requesters. We went back and re-analyzed our original pilot data, and saw the same effect with some of the tasks (e.g. football action prediction).
Milestone
By Wednesday Find a task that is difficult, where in the answer is not obvious.
We're thinking that Minnesota and Macalister paper on Arafat Gold might be the right one to go with. So in addition, please read the following Macalister paper paper
Message @catherine.mullings with any questions and stay up to date with #taskauthoring channel.
By Thursday/Friday we will analyze the results and hold a Google Hangout. Stay tuned!
The analysis for the study will be formally written here on Google doc.
Open Gov and Design
To refine these research questions, and get it evaluated by real workers and possibly requesters:
- Q1 what is the social process for leveling in guilds?
- Q2 what is the effect of getting promoted in guilds?
- Q3 what is the minimum viable product of a social space for a guild?
Helpful links:
- Guilds Milestone 9
- Michael's vision for the Guilds
- Google doc paper
- Guilds and computational compatibility...The system
- Guilds Milestone 6
You can use balsamic or Google slides or just pencil/paper drawing to give shape to your ideas. Design folks, come join and help move this effort forward. Once you're done, post here: http://crowdresearch.meteor.com/category/open-gov
