User Testing: A Step-by-Step Guide, Templates and Examples
In this handbook, we cover everything related to conducting usability research and testing the usability of a website, app, or prototype. We take a closer look at what user research entails, why it is important, what methods there are, and how you can get started with it.
What is usability?
Usability or user-friendliness is the extent to which a user can use a product in his own way without obstacles, doubts, or questions. With a user-friendly product, the use of the product is smooth and the user does not experience frustration.
Definition of usability
The ISO 9241-11:2018 standard provides the following usability definition:
“Usability is the degree of effectiveness, efficiency, and satisfaction of a defined group of users to achieve a specific goal in a specific context.”
Usability consists of three parts:
- Effectiveness. The extent to which users can use the product to achieve their goal(s).
- Efficiency. The effort that users have to make to achieve a certain goal.
- Satisfaction. This refers to the thoughts of the user while using the product. This includes the user's opinion about the ease of use, the added value of the website or app, and its applicability.
What is usability testing?
In usability or user testing, users perform tasks on a website, app, or prototype while being filmed as they continuously think out loud.
Because the user provides constant feedback, it becomes clear why certain functionalities do or do not work, which translates into a list of concrete action points to make the customer experience (even) better.
PS: Usability testing is also referred to as user testing.
Example user test script
Curious what kind of questions you can ask during a usability test? Then download the sample usability test questionnaire.Download user test scripts
Real user test example
Conducting usability tests
For every phase in the development process of a website or app, testing with real users is of great value. The five biggest benefits are:
- Discover what users encounter. Convenience serves people, now more than ever. When users want to order a product, but the ordering process does not work, they order it from the competitor with three clicks. That leaves the owner of the webshop with one question: why did the user abandon its shopping cart? Through user testing, you discover the frustrations of your user audience that would normally go unnoticed, providing you with the advantage of knowing who your user is, how they want to receive your product, and how you need to provide it.
- A better end product with prototype testing. Save time and money by conducting usability testing during the development process. Is it clear what they can do with the app or website? What do they encounter? What would they use it for? By collecting and processing this feedback during the development process, you build a product that better suits the end user.
- Create empathy for the end user. It often occurs that a team that has been working on a project for a long time, find it increasingly difficult to empathize with the end user, because they know the product inside out as an expert and no longer a first time user. User testing forces you to see the product through the eyes of the end user, which gives you a lot of valuable information for the rest of the development process.
- Valuable addition to quantitative data. Tools such as Google Analytics and Hotjar show what users do, but not why they do it.. A heatmap can show you that users are dropping out during the ordering process, but not why users are still considering. Usability testing removes that uncertainty and provides insight into how the ordering process can be improved.
- A better product for everyone. Whether you have just launched your product, or have been in the market for some time, through testing you can find out to what extent the product meets users' expectations. Is something missing in the interface? Do we lose money by having inefficiently arranged the ordering process?
Benefits of a better UX
The ultimate goal of usability testing is to make the experience for the user as good as possible. The usefulness of a better user experience or user experience (UX) has been demonstrated by countless studies. Improving the user experience generally brings the following five benefits:
- Reduced workload for customer service employees. With a good user experience, the website or app answers questions that users normally have.
- Loyal customers generate more sales. A good user experience ensures more satisfied customers who are less likely to switch to the competitor. Depending on the changes made, a better UX can increase sales by more than 100%! Would you like to know more about the relationship between user testing and conversion ? Download our whitepaper.
- Competitive advantage. By really putting yourself in the shoes of the user, you can further strengthen your position compared to the competition.
- Risk reducing. When you put a lot of money and time into the development of a website or app, you want to make sure that users don't drop out because they can't figure it out.
- Progress can be tracked. Characteristic of a good user experience is that usability tests are carried out structurally. By keeping track of usability metrics via, for example, the System Usability Scale, you can measure the progress of a website or app.
The difference between usability testing and user testing
You may have noticed that different terms are used for conducting usability research and user research. The most common variants are user testing, usability testing and user testing.
Some UX researchers argue that the primary purpose of user testing is to find out if potential users have a need for a particular product or service. In that case, you focus more on validating a problem instead of testing the usability of a website or prototype. Through in-depth interviews and market research you try to find out whether your concept / idea solves a problem. Are you sure about that? Then the next step is to test your concept .
Tip: Do you want to validate whether your business idea really solves a problem? Then we recommend that you read the book The Mom Test .
If user testing is concerned with the needs of potential users, then usability testing is performing user testing.
But… To be honest, it remains a gray area and it is often possible to determine from the context what it is about. If you pay attention to that, you will see that nine times out of ten user testing refers to performing user tests. So… use whichever term you like best.
User testing, usability testing or user testing - it often all comes down to the same thing.
Four methods of user testing
Usability research can be performed in different ways. When someone talks about usability research, they almost always talk about performing usability or user tests. Did you know that usability research has been conducted since the 1980s?
The research methods for usability research can be distinguished on two dimensions. The first dimension is the location of the usability study: online or in a usability lab. The second dimension concerns the way in which the research is conducted: with or without a moderator.
In a usability lab
By taking the tests in a usability lab, you have maximum control over the equipment that is used and you can quickly intervene when there are technical problems. The main drawback is that the user is not in their natural environment, which affects the answers and makes users less honest.
It also costs more time and money to recruit users and have them travel to the usability lab. User testing in a usability lab is almost always accompanied by a moderator who interacts with the tester / respondent during the test.
Remote / Online
A remote user test means that the tester can participate in the usability research online. In practice, this often means that the user test takes place from home or at the office. The main advantage of this is that the user takes the test in their natural environment, which leads to fairer feedback and makes it easier to reach difficult respondent profiles.
Another added benefit of remote user testing? It is often many times cheaper than in a usability lab. Where a test in a lab costs on average between €5,000 and €10,000, this can be done from €650 via an online platform such as User Sense. A world of difference!
This usability research method is also known as moderated user testing. The moderator guides the test session from start to finish. The test can take place both face-to-face and online. When the test takes place online, the moderator uses an online tool to communicate with the users.
The biggest advantage is that a moderator can ask in-depth questions and thus gather more insights. On the other hand, an (inexperienced) moderator can influence the results of the research with leading questions.
If this happens, it will have a negative effect on the reliability of the results - something you logically want to avoid.
Moderated usability research is most suitable if:
- You want to test and validate a concept or idea
- You want to test a wireframe, mock-up or non-interactive prototype
- You want to make a day out of testing, with the entire (customer) team watching the execution of the user tests
Of course, it doesn't make sense to have testers travel to a usability lab and then not talk to them. This also means that this variant of usability research almost always takes place online. This variant is therefore called remote unmoderated usability testing (URUT) .
The biggest advantage of this variant of usability research is that the tester cannot be influenced by a usability researcher during the execution, which means that the tester exhibits its most natural behavior during the user test. In our experience, this often leads to fairer feedback and greater willingness to test, because the tester can perform the usability test at a time of his/her choosing.
Unmoderated remote usability research is most appropriate if:
- You want to improve an existing app or website based on qualitative research (= user testing)
- You want to test a clickable prototype so changes can be made before the app is built
- You want to conduct international usability research
This variant of user testing is almost always carried out via a platform such as that of User Sense. Are you looking for a tool to perform user testing? Then we advise you to take a good look at how the quality of the testers is guaranteed and the size of the pool of the country in which you want to test.
How much does usability research cost?
The costs for conducting usability research start at € 650 and can sometimes rise to € 10,000. The price of the research depends on the chosen research variant, the location of the research and whether respondent recruitment is necessary. In general, UX research via a platform such as User Sense is almost always cheaper than on-site studies. You can check our pricing here.
Get started with (remote) usability testing
Starting usability research? Great plan! If we can give you one tip: make sure you take enough time to set up the research. The more precisely you formulate the research questions and draw up the test plan, the more useful the results will be. Good luck!
Step 1: Data Deep Dive
Do you want to test an existing website? Then you get the most out of the usability research by including the data you have already collected. Google Analytics, Hotjar or Clarity are an excellent starting point for this.
Questions you can ask yourself are:
- Which pages have the highest bounce rate?
- Which type of device has the lowest conversion rate?
- Where do the biggest conversion leaks seem to be?
Now that you have mapped out the bottlenecks of the website, you can organize the usability research in such a way that you find out what the biggest obstacles are.
No existing website, but want to test an app or prototype? Then there is a good chance that you have little quantitative data to draw from. That doesn't have to be a problem, as long as you think carefully about the research questions.
Tip: No clear picture of what the target group looks like? Then use Google Analytics to better map out your target group. This helps you to test with respondents who most closely match your target audience.
Step 2: Formulate research questions
The next step is to formulate a clear main question and sub-questions. Drawing up clearly defined main and sub-questions helps you to get the most out of the usability research.
This has a number of advantages:
- It helps you maintain focus
- You assure yourself that the research provides answers to the questions that have the most impact
- Main and sub questions are easy to communicate and help you get buy-in from stakeholders
Step 3: Choose the right usability research method
Now that it is clear what exactly you want to research, you can look at which methodology is best suited.
Do you want to test a concept or idea? Or get feedback on a mock-up or non-clickable prototype? Then you are often best off with moderated usability research.
Step 4: Determine the respondent profile and sample size
You prefer to conduct usability research with respondents who match the target group. It is therefore important to have a good idea of what your target group looks like.
Example respondent profile:
- Respondents must be between 30 and 40 years old
- Respondents must have a higher than average income
- Respondents must have booked an intercontinental flight in the past year
Need help recruiting respondents? Take advantage of our respondent pool.
In general, we recommend that customers test with a minimum of five testers per device type. You can read more about respondent numbers for usability research in this article.
Step 5: Compose the usability test plan or script
Now that it is clear what you want to get from the user research and with whom you want to conduct the research, it is time to make the implementation concrete. To do this, you draw up a usability test plan, in which you record the tasks that testers must perform.
This way you ensure that testers do not stray too much during the research and that you receive feedback on aspects that are most important to you as a company.
Do you find it difficult to create a user test plan? In our whitepaper we help you on your way with a number of slides.
Step 6: Conduct usability research
It's finally time: you can start carrying out the usability research. In the case of moderated test sessions, this means that you can start recruiting respondents and then schedule the test sessions with them.
Step 7: Analyzing research results
Would you like to analyze the results of the usability test sessions yourself? Then make sure you take enough time for this step. A lot of time is spent reviewing the user tests, taking notes, and tracing patterns.
Because analyzing the results of usability research takes a lot of time, many companies choose to outsource the analysis. We also help companies with this.
The advantage of performing user tests via a tool is that you can also easily collect a number of quantitative usability metrics, such as:
- System Usability Scale (SUS)
- Net Promoter Score (NPS)
- Single Ease Question (SEQ)
- Task-to-completion rate, time-on-task and error rate
Step 8: Create a UX research report
Perhaps the most satisfying step is to summarize the most important findings in a UX research report, so that you can answer the main and sub-questions, and it immediately becomes clear how you can improve the UX and conversion.
Tip: Are you going to present the research results? Then make sure you generate a highlight video of the most interesting fragments from the test sessions. This immediately makes it clear why something needs to be improved!