Hashem Elezabi- Usability Evaluation
I tested the site in a holistic manner, following both Jakob Nielsen's ten usability heuristics and my common sense. Other than trying out different functionalities of the site, I created an Essay Proofreading project and recorded my observations/notes. Only I tested the site, so this is a one-user usability test. I am sure I may mention a point which was already mentioned in other posts, but I will include all my notes and I am sure we will aggregate all the points from all contributors into one great output eventually!
The Usability Heuristics
Jakob Nielsen's ten usability heuristics:
- Visibility of System Status
- Match between System & World
- User Control & Freedom
- Consistency & Standards
- Error Prevention
- Recognition Rather than Recall
- Flexibility & Efficiency of Use
- Aesthetic & Minimalist Design
- Help Users Recognize, Diagnose, & Recover from Errors
- Help & Documentation
Issues and Proposed Solutions
The following are the results of my evaluation in the form of separate problems and their respective suggested solutions. They are ordered according to severity (on a 1 to 4 scale where 1 is "low priority" and 4 is "must fix", as Professor Bernstein said), starting from most severe (4) to least severe (1).
Issue 1: (Severity: 4)
There is no form of help to familiarize the user with the website and explain what each part of the website is for. This is an obvious problem and I'm sure people already have it in mind, but mentioning it here won't do harm. Heuristic(s) not satisfied: Help & Documentation
These solutions are not mutually exclusive, i.e. more than one of them can be implemented at the same time.
- We can make a tour of the website once a user registers. Such a tour should be simple, somewhat short, and easy to understand, with minimal use of words that are still fully descriptive and understandable. This tour would maybe be interactive, and would show the user around the website with arrows or something similar. I recently used a website that had a very pleasant, two-minute tour of this kind: .
- We can add a help button that either (1) takes the user to a Documentation (or similar) page or (2) displays some arrows and brief info about parts of the website (like in Apple Keynote and in iMovie, for example).
- We can create an introduction video that would explain the website and its parts in a way similar to the tour.
Issue 2: (Severity: 4)
It is not clear whether the user is a requester or worker. Can a user be both? If so, we need to add functionality that differentiates wholly between the Requester part of the account and the Worker part of the account. (this is a self-evident issue as well; it's about one of the core foundations of any crowdsourcing platform)
I have no previous experience with crowdsourcing platforms, but I am guessing that it is possible for a user to be both a requester and a worker. According to this, we need to give the user the option to choose whether to be a requester, a worker, or both while he/she is registering. When the user logs in, he/she can access the worker part of his/her account or the requester part, and the user can navigate between both parts.
Issue 3: (Severity: 4)
As a requester, I can't access a list of my projects that I submitted. When I press My Projects, I see a list of projects available for a worker. I submitted a project, but then I found it among other projects in My Projects, not in a list of my own projects which I submitted. We also need to add functionality for the requester to edit the projects he/she submitted; currently, it is impossible for a requester to do so. Heuristic(s) not satisfied: User Control & Freedom
This is a simple problem. We need to add a button that opens a requester's list of projects. This can maybe be done, and the My Projects confusion issue can be resolved, after Issue 2 is resolved.
Issue 4: (Severity: 4)
A user creating a project is not able to add multiple different tasks, as far as I understand. If multiple different tasks will be implemented as milestones, this is not implemented yet, since a user can only set the first milestone. Heuristic(s) not satisfied: User Control & Freedom
We can implement the milestones and enable the user to set a number of them if a project is that complex. I also thought of an idea that may help get rid of the confusion with 'task' and 'project' and with 'micro' and 'macro' (just a preliminary thought) (this is not exactly related to this issue). The idea is mentioned toward the bottom of this page.
Issue 5: (Severity: 3)
In Create Project, under No. 3 (Milestones), it is not made clear what is expected of the requester to upload. Also, the mention of the spreadsheet (in the third option) indicates a certain expectation, but I for example don't understand what the spreadsheet is used for or what it means in this context, so we need to clarify that. In addition, the phrase "worker details" is vague, but maybe I am missing something so I don't get it. Heuristic(s) not satisfied: Match between System & World, Help & Documentation
We can add a sentence explaining what files the user should upload here. For example: "Add files needed for completion of the project here. For example, if you are creating an Essay Proofreading task, upload your essay." Also, we can explain what is meant by "worker details".
Issue 6: (Severity: 3)
In Create Project, under No. 4 (Design Task), the design board is a great idea but it seemed intimidating. When I saw it, I was a bit discouraged from continuing since it is not that user-friendly. Heuristic(s) not satisfied: Flexibility & Efficiency of Use
I actually liked the idea of a board where the user can drag items that turn into formatting options, but it may not suit all users, and even though I liked it, I might prefer an easier way of designing a task. I wasn't there when the team thought of this idea, so I may be missing something (for example, I don't know why the ability to put options a worker should choose from was added). I am only talking about what I saw as a user. We can keep the board but make it a lot easier for a user to use. We can add examples and photos that show how such a board can/should be used. Also, I remember the HTML editor or Rich Text editor (something like that) that is present in a number of websites like Stackoverflow, Coursera, etc., and maybe we can use this (just speculation; I have little experience in task creation and crowdsourcing platforms :))
Issue 7: (Severity: 3)
There are multiple parts in Create Project where the user needs to input a number or something. In these cases, leaving it as free response causes errors and confusion. Heuristic(s) not satisfied: Error Prevention, Consistency & Standards
Add a mechanism that lets the user choose a number (from a drop-down menu), etc. Another way is to indicate the format the user should use when entering the info. For example: "Enter a number followed by the period of time. For example: enter '20 minutes' or '2 days'."
Issue 8: (Severity: 3)
There is confusion surrounding the expected time of completion of tasks and also confusion surrounding milestones. In my test, I chose a time I expect the task to be finished within, but after I submitted it was written that the expected time for the task is 1 minute. First, I'm not sure that's talking about the first milestone or the whole task. Second, the time isn't the one I entered, although I entered it as "30 minutes" and that may not have been recognized (see Issue 7). Heuristic(s) not satisfied: Visibility of System Status
We should make the first milestone process clearer, and explain clearly of what the timing asked for is. These explanations should be shown throughout the procedure of Create Project.
Issue 9: (Severity: 3)
The user creating the project is able to get to later steps in Create Project before completing the current one (which includes required fields). Unless this is created purposefully, it doesn't make sense. Heuristic(s) not satisfied: Error Prevention
We should make the later steps faded (i.e. can't be pressed on) until the current step is done, and then the user can go on to the next step in an ordered manner. Of course, the user can go back; this is a great feature and it satisfies the User Control & Freedom heuristic too.
Issue 10: (Severity: 3)
In Create Project, under No. 1 (Project Category), the categories are clearly different from each other but the user can choose more than one category. With the current list of categories, I don't think it is possible for any task to include more than one category of these, unless we implement complex projects/tasks that include different things to be done.
If we won't implement the idea of complex projects/tasks, we can simply make the category list of the Radio Button type (i.e. only one option can be chosen) instead of the Checkbox type where many options can be chosen.
Issue 11: (Severity: 2)
Once I login, I see a page that can be described as the Home page. If I navigate to any other part of the website, I can't return to that Home page. Heuristic(s) not satisfied: User Control & Freedom
We can simply add a button in the left navigation panel (where all the buttons are), or maybe at the top, that goes to that Home page.
Issue 12: (Severity: 2)
In my test, I submitted the project but it didn't tell me that the project was submitted successfully. Heuristic(s) not satisfied: Visibility of System Status
We can simply make the website tell the user that the project has been submitted successfully after the user submits it, and do that in all similar situations. As a user, I get assured when a website tells me the thing I've been doing has been completed successfully.
Issue 13: (Severity: 2)
Users might not be familiar with the terms "Microtasks" and "Macrotasks", so we need to familiarize users with them. This is an obvious problem and we need to consider it with all terms, but I mentioned those two terms especially, since they are very important in a crowdsourcing platform. Heuristic(s) not satisfied: Match between System & World, Help & Documentation
Maybe as part of the help/guidance we will provide as a solution to Issue 1 (tour, documentation, etc.) we can define both these terms and give examples for each.
Issue 14: (Severity: 1)
The user doesn't know what his/her username is. After I signed up, it asked for my email or my username. I entered my email and was able to login, but I didn't even see a username so I don't know anything about it.
Add one more input field when the user registers: the username. Or maybe we don't even need a username, in which case we should omit the "or username" part in "Enter email or username".
Issue 15: (Severity: 1)
In Create Project, under No. 2 (Project Details), we need some guidelines regarding the title of the project. Heuristic(s) not satisfied: Help & Documentation
Explain what the title should contain and include an example. For example: "Enter a short, descriptive title here. The title should name the task you want accomplished. For example, if you want your history essay proofread, the title should be 'Two-page History Essay Proofreading'."
Issue 16: (Severity: 1)
In Create Project, under No. 2 (Project Details), we need some guidelines regarding the end goals of the project. Heuristic(s) not satisfied: Help & Documentation
Explain what the end goals should contain and how they should be written, and include an example. For example: "Summarize the results you expect from the work. For example, if you want your history essay proofread, your end goals should be 'I want the essay without any grammar or spelling mistakes, and without any changed meanings. If you need to change any meaning, contact me first'."
Ideas and Suggestions
The following are some ideas and suggestions that are not solutions to problems; they are just ideas for improvement, or suggestions for solving an issue that is not mentioned above because it is not that concrete.
- There was discussion about the two options (single-instance or multiple-instance task) that appear in Create Project under No. 2 (Project Details) and whether/how they should be changed. I was also slightly confused about the idea of "Project" vs. "Task". An idea occurred to me that may clear the confusion around the micro and macro or single-instance and multiple-instance tasks, and the confusion around "Project" and "Task". I must point out that there is no significant confusion and that's why I didn't put this as an issue above. Still, I thought I would put my idea here (I mentioned it on Slack too). The idea is: Since both micro and macro tasks can each be either single-instance or many-instance tasks, we can remove the choice of single-instance vs. many-instance task from Create Project. Instead, we add another option: Create Task. A task is a single instance task, while a project is a many-instance task. If the user selects Create Task, he/she chooses between micro and macro and creates the task (which is one, single task). If the user selects Create Project, he/she chooses between micro and macro as well and creates the many-instance task (thus the name project). This also gives users the option of quick task creation if the user wants to add a small or quick task (the Create Task option would take less time to complete than the Create Project option would). I think this also makes things clearer, and distinguishes between "Project" and "Task" in a clear way. However, this may be different from what team members may have had in mind, or may affect other things. Also, the current situation isn't problematic, so we may not need such an idea, but I thought I would share it.
- The "My Profile" and "Logout" buttons can be put at the top bar.
- This is a purely engineering-related consideration and is very trivial: when I was logging in, I pressed Enter because I'm used to pressing Enter to login in almost all websites (instead of moving the cursor and pressing Login), but nothing happened. We can add the functionality that enables users to press Enter to login.