Milestone 3 Betzy

From crowdresearch
Jump to: navigation, search

Initial Brainstorm

The ideas you brainstormed, (at least 10 ideas for trust, and at least 10 ideas for power). Provide them in whatever format you want - diagrams, sketches, descriptions, or a combination (the wiki supports images, see here for instructions on uploading them).


Workers need to be able to quickly find tasks they'd want to work on:

• Workers provide a profile upon signup with skill sheet and interests. System automatically matches available HITs that meet criteria described in worker’s profile. (Automatic suggestions)

• Search options: - Graph search: Workers can filter HITs by certain search terms much like on Facebook. (example: “Labelling, Duration > 5 min, mobile-friendly) - Search tree: Workers can choose a number of criteria from a list they are interested in and filter the list of available HITs. (view sketch 1)


Workers need to feel they are being fairly compensated for their work:

• Workers create a pricing schematic that shows common prices for certain types of tasks that can be reviewed by requesters when they are pricing their HITs. (Establishment of “minimum wage” by users -> users shape the market)

• Paid Test Turkers who go through system to weed out “bad” or even spam requests. (Service provided by platform as part of customer satisfaction guarantee)

Workers need to feel like they are treated fairly and respectfully, and have a voice in the platform:

• Workers may send rejected work and resulting answer from requester to neutral 3rd party reviewer who evaluates whether or not the worker was being treated unfairly. Should the requester have made a mistake by rejecting the work provided, he must pay the “review fee” to the 3rd party reviewer and compensate the worker for his time/work.

• Give worker option to redo a HIT after rejection within a certain time frame. ( Make-up HIT)

Workers need to be able to expose their skills so they can get work they are qualified for and advance their skills:

• General tests available on the platform to evaluate a worker’s skill set. The test results are then added to the worker’s profile and can help filter search results and help pass “minimum requirements” hurdle of a task.

• Provide requester reviews and work portfolio in worker profile. For i.e. design work, a worker could post link to portfolio website to his worker profile.

Workers need to be confident that they understand the goal of the task, and quickly:

• Include simple multiple choice question at the beginning of the task after reading the task goal. (What is the goal of this task: a) Cute bunny noses b) Herbivores. Are they as nice as we think? c) The Gulf War) A simple congruency test with a predefined answer given by the requester, could help determine whether or not a worker understood the task at hand.

• Ability to quickly contact users who are working on same/similar HITs and ask them questions if one is unclear on what the goal of a task is.


Requesters need to get their HITs completed (quickly / correctly):

• Requester includes a tutorial or “example HIT” in the task description to verify that users understand what they are asked to do.

• Requester includes quantifiable/verifiable question into the HIT to make sure he isn’t being fed spam for answers.

Requesters need to be able to trust the results they get:

• System automatically prechecks answers according to criteria provided by requester. (When designing the HITs, requesters include what type of answers he/she is expecting, i.e. yes/no, animal name, number, verb… If this type of answer is not provided, i.e. “Yes” is entered in a field that requires an integer, the system marks the HIT as rejected)

• A requester fills out x-amount of HITs him-/herself and the system automatically checks for congruency with answers provided by workers.

Requesters need to have workers who have the appropriate skills and demographics do their tasks:

• When requiring a certain demographic to participate, the system could automatically only allow users to participate who match this criterion in their profile and/or only allow users in a certain geographical location, which is known by the IP Address.

• System remembers HITs/tasks workers have worked on in the past and checks if any of them required a certain skill set/ demographic. If a worker has already worked on a HIT as a “Caucasian male between the ages of 35 and 45” he will not be allowed to work on a HIT requiring a “Hispanic woman between the ages of 50 and 60.”

Requesters need to be able to easily generate good tasks:

• Requesters are given a tutorial on designing the optimal HIT upon signing up and can even take “classes” taught by either other requesters or workers paid for by HITs or actual money.

• Requesters can view similar tasks by other requesters and can ask these for help through an included forum.

Requesters need to price their tasks appropriately:

• A pricing schematic created by the workers themselves acts as a guide for the requesters when they are pricing their tasks. The system could warn them automatically when they have underpriced a task, reminding them that this could result in bad answers.

• Example HIT could be posted with the instruction to give pricing recommendation. The worker giving their recommendation could be paid a “pricing fee” or be rewarded with x-amount of HIT to complete.

Requesters need workers to trust them:

• Workers should be able to see which workers have already done tasks for a specific requester and have the ability to contact these workers in a forum asking them on their experience with this requester.

• Requester profile showing a rating provided by users to make requesters more personable.

Dive Deeper into Specific Ideas

For each of the 4 ideas (2 for trust, 2 for power), describe (using diagrams, sketches, storyboards, text, or some combination) the ideas in further detail.

Trust-related Ideas

Power-related Ideas

Dark Horse idea

Describe your dark horse idea (using diagrams, sketches, storyboards, text, or some combination).