Why should you conduct a user test?

What is a user test?

User tests are used both when developing solutions or revising existing concepts.

User tests originate from qualitative research, focusing on observational studies and interviews to generate insights and help you understand the human needs of those who need you most.

User testing focuses on testing people and their behaviour when they consume or use a solution. User tests ensure that customer preferences are included in the process when solutions are designed.

But wait… do you have to interview the whole world?

No—when we conduct user tests, the rule of thumb is that if you test 5–8 people, you will, with a high degree of certainty, uncover around 75% of the errors and pitfalls that may exist.

User testing also goes hand in hand with usability, as the purpose of conducting user tests is fundamentally rooted in the need to optimise the user experience of a given solution.

*If you simply want the formula/framework to conduct a user test yourself, skip down to the section “The method”; otherwise, enjoy the read.

So when—and who on earth—conducts these user tests?

In principle, anyone can conduct a user test, and you have almost certainly taken part in one without even thinking about it—for example, when you occasionally grab a free sample at Bilka while shopping. Such a user test has almost certainly measured several of your preferences, such as:

• What do you taste first?
• What do you go back for a second time?
• Do you buy the product when you leave the stand?
• What do you say about the product before, during, and after the experience?

These parameters reveal, for example, “your willingness to invest”, “your taste preference”, “the product’s visual performance”, and “the product’s functional ability to meet needs”.

So think about that the next time you grab a free meatball from the deli’s tasting tray. Nothing in this world is free.

User tests are therefore always conducted with the purpose of gaining insight into how people interact with a product, and how well the design object manages to meet people’s many needs.

Case – User test of coffee preferences

Coffee is an international product that most people are familiar with… You can probably already imagine how a freshly brewed cup of coffee smells and perhaps even recall the warmth of the cup in your hand, as well as the feeling of indulgence right as you take the first sip (that is, if you are the type who drinks coffee at all).

So what should the first sip taste like? I will not even begin to generalise, as this is a subjective preference that can vary across international cultures, products, and coffee rituals.

So if, as a business, you want to investigate which flavour notes most people on Funen prefer, you can approach the study in a dual way to optimise the coffee’s taste experience.

From quantitative data reports =Statistics might show that people on Funen prefer plant-based milk in their instant coffee, but why is not revealed.

From qualitative user tests =User tests help us understand why a consumer chooses as they do, and also map when a behavioural consumption pattern can be optimised.

Methods

How do I approach a user test, and what should I prepare?

In short, it is recommended to choose a method (there are countless approaches) and then find a handful of participants based on the method’s framework.

In addition, you need an open mind and be ready to acknowledge that you almost certainly do not know the outcome of the test in advance.

If you are responsible for the user test, you should at a minimum ensure that the test addresses the following points:

• Selection of user testing method
• Set up the user test, plan which tasks will be completed, and the expected time required.
• Recruitment of test participants (random selection or segmented groups)
• Running the tests, including data collection via notes and test video.
• Information about practicalities for your test participants, as well as rewards.
• A GDPR compliance signature document for the data collection.
• Preparation of a report with recommendations, issues, screenshots, and possibly mock-ups.
• Follow-up meeting with stakeholders to present the data and recommendations

A small handful of method examples (described using the coffee analogy) can be found in the sections below.

The cowboy test

The name comes from the idea of a cowboy duel, where you must identify your target in record time and react in a split second with your best shot.

It is also called “quick and dirty” testing, but the purpose is the same: to let people interact with your product/design for 1, 5, or 10 minutes to quickly see how it is received and whether users can figure out how to use/read the design.

In the coffee example, you could set up 15 differently shaped mugs/cups/glasses in front of the consumer and ask them to pour a cup of coffee.

This quickly provides insight into the selection process and whether the increased number of options helps or hinders the user in choosing.

For this type of test, it is recommended to record the action, as reactions can be so fast that it is difficult to capture all impressions.
With this test, you get answers to a series of clearly defined questions with open-ended responses, which can help you understand how readable the product is—or whether it should be redesigned.

The think-aloud test

This test builds on the cowboy test and is based on evaluating the product while the respondent/consumer continuously shares their experience.

The respondent is instructed to keep speaking out loud and put words to all their thoughts, considerations, expectations, and frustrations throughout the entire process, so that all insights can be included in optimising a future design.

If you are testing a coffee drinker, you could ask the person to start by brewing their coffee and, along the way, share which ritual they follow until the coffee is ready to drink.

Afterwards, you can test while they drink the coffee to gain another layer of insights that can help us understand, for example, whether the coffee is too hot, cold, bitter, or sweet for them. This could take the form of questions such as: what do you think about the taste or the temperature? You could also ask, “Why do you leave your spoon in the cup?” or “What do you associate the smell and taste with?”

With this test, you gain a new understanding of the interface, usability, and user motivation within a group of respondents.

Expert review

This is based on experts’ understanding, thoughts, considerations, expectations, and frustrations regarding the product.

Here, you would have other coffee experts assess the product experience and, to a greater extent, provide recommendations with direct improvement suggestions for the final design.

For example, it could be a coffee expert from Espresso House or from the coffee shop Bryg in Vejle—both places are home to people who live and breathe coffee.

An expert is involved because they see and experience the full palette and highlight the smallest details as potential improvement elements—for example, by recommending a specific handling of the product, with statements such as: “the coffee would have been less bitter if you used 60-degree water instead of 90 degrees” or “the coffee would smell better served in a U-shaped mug rather than a V-shaped glass because it best suits beans of type X”.

With this test, you raise the quality of the product, the interface, and usability through an expert quality-control check, which often aims to lead to a seal of quality.

The A/B test

This is also known as a split test, and the method differs from the above methods, as it often produces a more quantitative result.

The purpose is to test based on a division into two groups—hence the name split test—where one group receives the original design while the other group receives the new one.

However, you can also split by looking at the design and, for example, asking a group of respondents whether they would rather buy coffee bag A or coffee bag B solely based on the design.

The method is therefore most often used on graphical UI elements—for example, CTA buttons in an email, where you want to determine which visual layout has the best effect on the recipient.

The purpose of the method is that you (much like a marketing benchmarking method) evaluate the graphical elements based on measurable parameters, such as: “How long did respondents take before they found the right button?” or “Which design got the most people to click the button?”

In essence, the method is simply to test multiple combinations on different groups to assess which combination/design works best.

Division of responsibilities

What is my responsibility when a user test is to be conducted:

Pre-phase:

• Review previous reports for recommendations and issues.
• Set up the user test and define which tasks are to be solved.
• Prepare a contract that explains the GDPR aspects of the study (that the test participants are OK with you storing the data).
• Prepare the tool/element to be tested and bring reporting tools.
• Design a report logbook with: issues, screenshots, and possibly mock-ups.
• Book a location for the test itself.
• Find test participants and recruit based on relevance to your product or issue. (Remember: 5–8 user tests reveal approx. 75% of all problems).

Administration:

• Set up the test room, including video equipment.
• Review GDPR and data handling.
• Conduct the test with 30–60 minutes per respondent (including test videos).
• Reward your test participants.

After the session:

• Process the insights and video material.
• Discuss these in a data group, or hold a workshop to support each other in the data processing.
• Prepare a report with recommendations, issues, screenshots, and possibly mock-ups for improvements.
• Follow-up meeting presenting recommendations, as well as dialogue about the report’s insights and recommendations for improvements.

The goal

So whether you want to conduct a user test to identify pitfalls, errors, and shortcomings, or to increase your awareness and validate which solutions your customers prefer, user testing is an excellent tool for validating the success rate of designs with real people.

At Morningtrain, this is used as an extremely effective way to gain insights from users’ digital behaviour and to test websites for optimal purchase flows and the functionality of information placement.

News, you actually wants to read

Join newsletter