Running a usability test on a small budget

A usability test means watching real people try to do real tasks with your website or app, and noting where they struggle. It does not need a lab, special software or a research agency. A laptop, a video call and five well-chosen participants will show you most of the obvious problems in an afternoon.

This guide sets out a low-cost method based on guidance from the Nielsen Norman Group (NN/g) and the GOV.UK Service Manual: who to recruit, how to write tasks, how to run a think-aloud session, and how to turn what you see into fixes.

Why five users is usually enough

In Why you only need to test with 5 users, Jakob Nielsen sets out a model he built with Thomas Landauer. Each extra participant finds fewer new problems than the one before, because they start repeating what earlier participants hit. On that model, the article says a study with five participants finds about 85% of the usability problems.

The more useful advice in the same article is about budget. If you can afford 15 participants, Nielsen suggests three rounds of five rather than one round of 15. You fix what the first round found, then test again, and the second round finds problems the first set of issues was hiding.

Two caveats matter. The model only holds for comparable users: if you have two distinct groups, such as buyers and sellers on a marketplace, the article suggests three or four people from each. And a five-person test is qualitative. It tells you what goes wrong and why, not what percentage of users will be affected. NN/g’s comparison of quantitative and qualitative testing puts quantitative studies at often more than 30 participants.

Plan the test in an hour

The GOV.UK guidance on moderated usability testing recommends agreeing three things before you book anyone: your research questions, the types of user you need, and the parts of the service to focus on. On a small budget, keep each short.

• Research questions. Two or three, such as “Can new visitors find delivery costs before checkout?”
• Participants. Real or likely customers. Friends and colleagues know too much about your business to behave like strangers.
• Session length. GOV.UK says sessions usually last 30 to 60 minutes, and advises no more than six one-hour sessions a day with at least 15 minutes between them.
• A discussion guide. A short script with your introduction, the tasks in order and a few closing questions, so every participant gets the same test.

For cheap recruitment, try your customer email list, a post on your social channels, or people who recently contacted support. A modest voucher is a common thank-you. Screen with two or three questions so you get people who match your customers.

Write tasks that don’t give away the answer

Tasks are where most home-made tests go wrong. Marieke McCloskey’s NN/g article on turning user goals into task scenarios gives three rules: make each task realistic, make it something the participant does rather than describes, and avoid clues.

A clue is any word that matches a label on screen. “Use the Track my order link to find your parcel” tests reading, not usability. “You ordered a teapot last week and it hasn’t arrived. Find out where it is” tests whether your design helps. The same article warns against going too vague: give enough detail (a date, a price limit, a product type) for the participant to act without asking you what you mean.

GOV.UK adds that several small tasks in a logical order work better than one large one.

Run a think-aloud session

Nielsen calls thinking aloud the single most valuable usability method. Participants use the product while saying what they are thinking: what they are looking for, what they expect, what confuses them. His summary of the method is three steps: recruit representative users, give them representative tasks, and let them do the talking.

A simple running order for a 45-minute session:

1. Welcome (5 minutes). Explain what will happen, ask for consent to record, and say clearly that you are testing the website, not them. GOV.UK recommends this line.
2. Practice (2 minutes). Ask them to think aloud while doing something unrelated, such as finding the weather for tomorrow.
3. Tasks (30 minutes). Read each task, then stay quiet. If they fall silent, prompt with “What are you thinking?”
4. Wrap-up (8 minutes). Ask what was hardest, what they expected that wasn’t there, and thank them.

The hardest part is not helping. When someone is stuck, the urge to point is strong. GOV.UK’s advice is to mostly watch and listen, and step in only to follow up on something interesting, to probe with an open question, or to rescue someone who is completely stuck. Nielsen notes that poorly timed prompts can change behaviour and make findings unreliable.

A worked example

An example (illustration only). A two-person firm that sells bookkeeping packages to sole traders wants to know whether visitors understand its pricing page. The founder writes three tasks:

1. “You’re a self-employed plumber who files one tax return a year. Find the package that fits you and how much it costs per month.”
2. “You want to know whether VAT returns are included. Find out.”
3. “You’ve decided to go ahead. Get as far as you can without paying.”

She recruits five sole traders from a local business group, runs 40-minute video calls over two days, and records each one with permission. A colleague takes notes. Four of the five pick the wrong package because the package names describe the firm’s internal tiers rather than the customer’s situation. Two can’t tell whether VAT is included. Nobody has trouble with the sign-up form.

She renames the packages after customer types, adds a short “What’s included” list under each price, and tests again with five new people a fortnight later.

Turn observations into fixes

Hold a short debrief straight after the last session while memories are fresh. List each problem you saw, how many participants hit it, and how badly it stopped them. Rank by severity: a problem that blocked a task beats one that only slowed someone down. Fix the top few, then test again.

Invite colleagues to watch live or view clips. Nielsen lists persuasiveness as a benefit of thinking aloud: watching a real customer fail tends to end internal debates faster than a slide does.

Common mistakes and when not to use this method

• Testing with people who already know the product well.
• Writing tasks that repeat the words on the buttons.
• Asking “Do you like it?” instead of watching what people do.
• Explaining the design when a participant is confused, which hides the problem you came to find.
• Reporting “60% of users failed” from five sessions. The sample is far too small for percentages.
• Testing once and never again.

A small think-aloud test is the wrong tool if you need numbers to compare two designs, track usability over time or justify a large budget. Those questions need a quantitative study, analytics or an A/B test. It is also not a way to discover what product to build in the first place; interviews and field research do that better. For finding out why people struggle with something that already exists, even as a paper sketch, it is hard to beat for the money.