How to get faster, better insights from user testing

The old way

Run the test with clients or colleagues attending as passive observers.
Detailed notes taken by skilled observer while facilitator runs the test
When testing is finished, detailed analysis over a number of days and a report is created
Report presented to client a week or more later along with video clips, heatmaps.

The problems with this approach is that it can feel too drawn out, clients are so detached from the reported problems that they don’t feel they own them.

This means it can be a struggle to justify changes to programme owners or developers and, because they weren’t involved and didn’t read the report, you find it hard to get your team aligned and agreed on the problems to solve and how to best solve them.

So, this creates a gap between insight and action.

A better approach to user testing

At Each&Other we’ve developed a method of user testing that helps close this gap by:

Creating a shared understanding of user needs among your whole team
Gives ownership of the solution to the whole team
Gets fast, actionable insights

Better together: the invite list

We invite in eight to ten clients with the following roles:

Product managers
Developers
Designers, QA
Subject Matter Experts
Directors, manager from other departments

Working it out together

We operate with a simple framework where we analyse and synthesise the data together.

See & hear
Think
Do

1. See & Hear: observations

Before starting our tests, we print out the screens from the site or application and put them up on the wall (as they appear as part of a flow).

After every test session we get into a group for a 10-15 minute de-brief and write our observations on post-its and place them on the printed screens. Very quickly, we get to see where most of the action happens. We avoid inferring or concluding anything on the basis of one test. We stick to what we saw and what what we heard - just the facts. Then it’s onto the next test.

2. Think: Analysis and synthesis

After we’ve finished testing for the day, we do a daily round-up moving from what we saw and heard, to what it might mean and what we must do.

We start with a simple “dot exercise”, where everyone gets to vote on what they think are the most important issues. Finally, we open it up for discussion. Hat tip: the K-J Technique.

3. Do: A design workshop

Finally, we work together to identify the right problems and generate some initial sketches of how we might best solve them.

The secret sauce: Meaningful roles

Team members are not here to passively observe. Instead, we assign them meaningful roles during the day’s testing. They’ll facilitate a test, lead a post-test de-brief or prioritise the issues in our daily round-up.

Benefits of this approach

Shared understanding: recognition of the users’ context and the obstacles they face with the current design
Owning the problem. Clients who have been through this process feel they have a far greater understanding and ownership of the problems in their product.
Better insights, faster. Many teams are working with Lean or Agile where design can often be squeezed. This approach gets better insights faster and gets everyone bought into the problems that need solving.

Finally, it’s important to note that this isn’t the only way we run user testing. There’s still a place for a more formal and measured approach, particularly around strategic and formative testing.