The old way
- Run the test with clients or colleagues attending as passive observers.
- Detailed notes taken by skilled observer while facilitator runs the test
- When testing is finished, detailed analysis over a number of days and a report is created
- Report presented to client a week or more later along with video clips, heatmaps.
The problems with this approach is that it can feel too drawn out, clients are so detached from the reported problems that they don’t feel they own them.
This means it can be a struggle to justify changes to programme owners or developers and, because they weren’t involved and didn’t read the report, you find it hard to get your team aligned and agreed on the problems to solve and how to best solve them.
So, this creates a gap between insight and action.
A better approach to user testing
At Each&Other we’ve developed a method of user testing that helps close this gap by:
- Creating a shared understanding of user needs among your whole team
- Gives ownership of the solution to the whole team
- Gets fast, actionable insights
Better together: the invite list
We invite in eight to ten clients with the following roles:
- Product managers
- Designers, QA
- Subject Matter Experts
- Directors, manager from other departments
Working it out together
We operate with a simple framework where we analyse and synthesise the data together.
- See & hear
1. See & Hear: observations
Before starting our tests, we print out the screens from the site or application and put them up on the wall (as they appear as part of a flow).
After every test session we get into a group for a 10-15 minute de-brief and write our observations on post-its and place them on the printed screens. Very quickly, we get to see where most of the action happens. We avoid inferring or concluding anything on the basis of one test. We stick to what we saw and what what we heard – just the facts. Then it’s onto the next test.
2. Think: Analysis and synthesis
After we’ve finished testing for the day, we do a daily round-up moving from what we saw and heard, to what it might mean and what we must do.
We start with a simple “dot exercise”, where everyone gets to vote on what they think are the most important issues. Finally, we open it up for discussion. Hat tip: the K-J Technique.
3. Do: A design workshop
Finally, we work together to identify the right problems and generate some initial sketches of how we might best solve them.
The secret sauce: Meaningful roles
Team members are not here to passively observe. Instead, we assign them meaningful roles during the day’s testing. They’ll facilitate a test, lead a post-test de-brief or prioritise the issues in our daily round-up.
Benefits of this approach
- Shared understanding: recognition of the users’ context and the obstacles they face with the current design
- Owning the problem. Clients who have been through this process feel they have a far greater understanding and ownership of the problems in their product.
- Better insights, faster. Many teams are working with Lean or Agile where design can often be squeezed. This approach gets better insights faster and gets everyone bought into the problems that need solving.
Finally, it’s important to note that this isn’t the only way we run user testing. There’s still a place for a more formal and measured approach, particularly around strategic and formative testing.
Illustrations by Tom Cunningham