WinterMilestone 3 UX Flight@ftw

From crowdresearch
Jump to: navigation, search

UX Flight Tasks

This section of the UX Flight focused on 2 tasks expected to be among the most commonly performed tasks. Daemo is a crowd task sourcing platform that enables requestors to post tasks for workers to complete. As the design implies when compared to MTruk, the platform is specialized for micro-tasks, which are those individual tasks to be completed within 60 seconds. This page consists 2 reports, one focused integrating the results from a simulated human and human performing 2 tasks from the 2 major roles found within the system. This is followed by the results of a cognitive walk-through of more general system level tasks performed by both roles.


Usability Role Specific Tasks

CrowdresearchMap-tasks.png

Timing Results from 6 Daemo Tasks

Task Description Simulated Expert Timings (sec) Script
Create a Question: 5 Radio Button Response 45.0 File:TaskA1 Script.csv
Create a Question: create image labeling task - error 23.0 File:TaskA2 Script.csv
Create a Question: create image labeling task 34.9 File:TaskA3 Script.csv
Answer Question: 5 Radio Button Response 6.7 File:TaskB1 Script.csv
Answer Question: label the image task - error 10.8 File:TaskB2 Script.csv
Answer Question: label the image task 14.0 File:TaskB3 Script.csv

File:6 Simulated Daemo Tasks.csv

Requestor - Create a Question: 5 Radio Button Response

In this task, a simulated human and a real human designed a 5 radio button response question. The requestor asked a question, which was then followed by a set of radio buttons. The simulated human, an artificial expert, was created with a GOMS-KLM model and repeated for this task. For the most part the creation of the response question was smooth. However the requestor demonstrated expert level difficulties with the system. The errors presented by the user, a demonstrated in the youtube video include the following.

Time in Video (m:ss) Error Correction
0:10 User applies CTRL+A shortcut that highlights entire page, rather than the text box content. Relies on mouse here and consecutively to highlight designed text. Automated clearing out of tips upon mouse click
0:50, 0:58, 1:06 User chases text cursor with mouse from Option 3 to Option 4, after Option 4 box loads prematurely. Keep the text cursor in the uppermost and cleared option.
1:40 User searches for what to do next Move the publish, save, and delete options to bottom of screen. The user needs to be led downward in this layout.
2:00 (speed enhancement) User moves moves across screen to publish task. Move the publish, save, and delete options to bottom of screen to minimize mouse motion.

Youtube Video of Human Performing Task

File:TaskA1 Script.csv

Worker - Answer Question: 5 Radio Button Response

Nothing to add.

Youtube Video of Human Performing Task

File:TaskB1 Script.csv

Requestor - Create a Question: create image labeling task

In this task, a simulated human and a real human designed an image labeling task. The requestor posted an image from the web, as per Daemo's current functionalities and asked workers to label the image. The simulated human, an artificial expert, was created with a GOMS-KLM model and repeated for both a task with and without an error. The error being that the requestor forgot to add a text box input for a requestor to insert the answer. In this case, the error can be identified by the time spent, number of motor operations the requestor performed on the task and the number of input options on the "authoring task" page.

Comparison-of-Requestor-Tasks.jpg

Possible solutions to this issue include binding any number of Daemo's input options with the insertion of an image container. Further exploration identified that this same problem exists with audio container as well. Potentially the first word of the question typed by the requestor can be used to match the input device with the media container. For example, please "rate" this image would best go to a closed/constrained input such as a slider, drop down menu, or radio button. In contrast, tasks requesting please "describe" this image point towards open input method such as a text box.

Youtube Video of Human Performing Task Successfully

File:TaskA3 Script.csv

Youtube Video of Human Performing Task with Error

File:TaskA2 Script.csv

Worker - Answer Question: label the image task

In this task, a simulated human and a real human designed an image labeling task. The requestor posted an image from the web, as per Daemo's current functionalities and asked workers to label the image. The simulated human, an artificial expert, was created with a GOMS-KLM model and repeated for both a task with and without an error. The error being that the requestor forgot to add a text box input for a requestor to insert the answer. The worker in response cannot submit a successful task for pay due to the error/

From the workers side, the error can be identified by the time spent, number of motor operations the requestor performed on the task, the number of input options on the "authoring task" page, and most strongly by the use of the "skip" button.

Comparison of Worker Tasks.JPG

Youtube Video of Human Performing Task Successfully

File:TaskB3 Script.csv

Youtube Video of Human Performing Task with Error

File:TaskB2 Script.csv

General Usability Tasks

The purpose of this test is to learn how a number of participants interact with Daemo.com on the desktop version. The final evaluation would be based on the performance and user satisfaction in 3 areas: Functionality, Ease of use, Features and Aesthetics. This usability test was conducted from two Master of Digital Experience Innovation candidate at Waterloo University. Vincent a 24 year old male with a bachelor of graphic design, and Leanne, a 31 year old female with a bachelor in Digital Marketing. The users were instructed to vocally state their thoughts during the test.

General Task: Logging into Daemo

Participant Key Observation Success
Vincent Did not really face a problem. But was surprised by the placement of the login option. Yes
LeAnn After clicking on the Signin button, she seemed puzzled that nothing new appeared. She kept clicking on it and did not scroll down for sometime. Eventually she tried out new things and noticed it. She noted that if the color of the background section was different as opposed to the white page background, she was more likely to spot it much earlier. Yes

Result:The Login to be moved right beneath the Signin and Register buttons. The participants had difficulty Logging in, the Login credential box was located far from where they expected.

General Task: Loading a Profile Picture into Daemo

Participant Key Observation Success
Both Both of the users navigated to the Profile page easily. They kept clicking on the action button and expected to find an Edit Your Profile option. They tried right clicking on the picture as well, but nothing seemed to work. No

Result:An Edit Your Profile option to be added to the Actions bar.

General Task: Navigating to Homepage from Profile Page

Participant Key Observation Success
Both Both expected a click on the Daemon Logo on the top left corner would direct them to the Home page, but it did not happen.

Vincent proceeded to changing the URL in order to go to the Home Page.

No

Result: The Logo should navigate the user to the Home page.

General Task: Integrate Google Drive into Profile

Participant Key Observation Success
Both Integrating google drive to Daemo resulted in the page crashing. I asked the participants to try it with different browsers, but it was the same case every time. No

Result: Right now does not seems to function.

Conclusion

Participant Key Observation
Vincent It is still in the Prototype stage so it is really hard to tell how the final product would look.
LeAnne It was really confusing to me, and by just looking at the website I can not tell what is it about.

In a long run, when the product is ready to be launched, it would be essential to be able to view other user's profiles, in order to know more information about both your competitors and the requester.

Contributers to this page

@anotherhuman @mahsa @vlado