Difference between revisions of "Winter Milestone 5 Templates"

From crowdresearch
Jump to: navigation, search
(System details)
(Evaluation plan and results)
Line 93: Line 93:
 
2) or in a fact database (Figure 5).
 
2) or in a fact database (Figure 5).
  
=== Evaluation plan and results ===  
+
=== Evaluation method and results ===  
 
We deployed Twitch publicly on the web and attracted 82
 
We deployed Twitch publicly on the web and attracted 82
 
users to install Twitch on their primary phones. Over three
 
users to install Twitch on their primary phones. Over three

Revision as of 00:33, 9 February 2016

Please use the following template to write up your introduction section this week.

Science

System

We're going to borrow introduction section from this paper as an example: Vaish R, Wyngarden K, Chen J, et al. Twitch crowdsourcing: crowd contributions in short bursts of time. Proceedings of the 32nd annual ACM conference on Human factors in computing systems. ACM, 2014: 3645-3654. Please note how this section was divided into different parts.

Specific problem being solved (not just crowdsourcing, but getting into specifics)

Mobilizing participation is a central challenge for every crowdsourcing campaign. Campaigns that cannot motivate enough participants will fail. Unfortunately, many interested contributors simply cannot find enough time: lack of time is the top reason that subject experts do not contribute to Wikipedia. Those who do participate in crowdsourcing campaigns often drop out when life becomes busy. Even seemingly small time requirements can dissuade users: psychologists define channel factors as the small but critical barriers to action that have a disproportionate effect on whether people complete a goal.

Motivation

Despite this constraint, many crowdsourcing campaigns assume that participants will work for minutes or hours at once, leading to a granularity problem where task size is poorly matched to contributors’ opportunities. We speculate that a great number of crowdsourcing campaigns will struggle to succeed as long as potential contributors are deterred by the time commitment.

Introducing the concept, the high level insight

To engage a wider set of crowdsourcing contributors, we introduce twitch crowdsourcing: interfaces that encourage contributions of a few seconds at a time. Taking advantage of the common habit of turning to mobile phones in spare moments, we replace the mobile phone unlock screen with a brief crowdsourcing task, allowing each user to make small, compounded volunteer contributions over time. In contrast, existing mobile crowdsourcing platforms (e.g., [12,16,22]) tend to assume long, focused runs of work. Our design challenge is thus to create crowdsourcing tasks that operate in very short time periods and at low cognitive load.

The system

To demonstrate the opportunities of twitch crowdsourcing, we present Twitch, a crowdsourcing platform for Android devices that augments the unlock screen with 1–3 second volunteer crowdsourcing tasks (Figure 1). Rather than a typical slide-to-unlock mechanism, the user unlocks their phone by completing a brief crowdsourcing task. Twitch is publicly deployed and has collected over eleven thousand volunteer contributions to date. The system sits aside any existing security passcodes on the phone.

System details

Twitch crowdsourcing allows designers to tap into local and topical expertise from mobile users. Twitch supports three unlock applications: 1) Census envisions a realtime people-centered world census: where people are, what they are doing, and how they are doing it. For example, how busy is the corner café at 2pm on Fridays? Census answers these questions by asking users to share information about their surroundings as they navigate the physical world, for example the size of the crowd or current activities (Figure 1). 2) Photo Ranking captures users’ opinion between two photographs. In formative work with product designers, we found that they require stock photos for mockups, but stock photo sites have sparse ratings. Likewise, computer vision needs more data to identify high-quality images from the web. Photo Ranking (Figure 1) asks users to swipe to choose the better of two stock photos on a theme, or contribute their own through their cell phone camera.

3) Structuring the Web helps transform the written web into a format that computers can understand. Users specify an area of expertise — HCI, the Doctor Who television series, or anything else of interest on Wikipedia — and help verify web extractions relevant to that topic. Each unlock involves confirming or rejecting a short extraction. In doing so, users could power a fact-oriented search engine that would directly answer queries like “heuristic evaluation creator”. After making a selection, Twitch users can see whether their peers agreed with their selection. In addition, they can see how their contribution is contributing to the larger whole, for example aggregate responses on a map (Figure 2) or in a fact database (Figure 5).

Evaluation method and results

We deployed Twitch publicly on the web and attracted 82 users to install Twitch on their primary phones. Over three weeks, the average user unlocked their phone using Twitch 19 times per day. Users contributed over 11,000 items to our crowdsourced database, covering several cities with local census information. The median Census task unlock took 1.6 seconds, compared to 1.4 seconds for a standard slide-to-unlock gesture. Secondary task studies demonstrated that Twitch unlocks added minimal cognitive load to the user. Our work indicates that it may be possible to engage a broad set of new participants in crowdsourcing campaigns as they go about their day or have a few spare moments. In the following sections, we introduce twitch crowdsourcing in more detail and report on our public deployment and field experiments.