Show me the numbers – the curse of UX Research


Jason WalshJournalist

Designers have a problem: they are failing to communicate effectively. It’s ironic, really, given that design is fundamentally about communication, but when designers talk to businesses, the value of their work is often seen only at the level of pixel pushing.

There’s nothing wrong with a beautiful design of course. In fact beautiful designs should be encouraged – but products also need to work, and in order to work they need to be grounded in how people interact with them. It appears self-evident, then, that user research has a key role in development. And yet, businesses struggle with it.

Consider the scenario: a user experience (UX) designer putting together a proposal for a chief marketing officer (CMO) of a software company – and this company has problems. Its site, the sole way it converts potential customers into actual customers, is vastly underperforming.

They're juicing paid search and hammering SEO, so they're getting traffic, but it's not converting. They're drowning in analytics and Hotjar data, so they know what is happening on the site. But something just isn't clicking.

This should be a textbook case illustrating the need for qualitative research, uncovering the why behind customer behaviour. After all, this CMO needs help. But the researcher finds the CMO sceptical. He looks at a proposal, looks at the activities planned, looks at the proposed outputs – and he just doesn’t get it.

He pushes back, asking one simple question: how do you measure the success of user research? He wants numbers, numbers, numbers.

20 years into UX as a discipline and this scenario is not only plausible, it is an example of an actual recent encounter. So if user research can answer the question of what is going wrong, or even why something is going right, why is spending on it still met with raised eyebrows?

The product design lifecycle

It seems to be a problem particular to the domain of software and services: where does user research fit into the product design lifecycle? By contrast, product design, perhaps due to higher costs associated with manufacturing and distribution, demands close attention is paid to how people interact with products.

Nevertheless, Billy Kennedy, research principal at Each&Other, sayssay the commonalities need to be stressed.

“When you as a business decide to go ahead and design, develop, deliver a new product, be that a website, an app, a service, whatever it might be, there are a couple of fundamental questions right up front that you would probably want to answer, just to make sure that you're actually designing and developing the right thing,” he said.

The most fundamental question of all, naturally, should be: does this product or service meet a need? A client may think the answer is an obvious yes, but with the landscape littered with failed products it does need to be asked. After that, how users are interacting with the product needs to be examined in detail, again with a single, clear goal in mind: understanding what is really happening.

However, given budgetary constraints, research can be given short shrift, so the value the research process will bring needs to be clearly communicated.

“All businesses have a set budget for what they can actually build and deliver in a given quarter, year, or other period of time, whatever it might be. There are obviously choices to be made there around how they spend that budget, and ensuring that they are delivering best value for their business. And usually that means answering real user needs, which means driving engagement with users, driving purchases, driving intent, all those different types of things,” Kennedy said.

Therefore, from a fundamental point of view, user research can help define metrics by which to understand the performance of a product or service. In other words, given that grounding the offering is essential for a successful product or service, user research can be this grounding, by asking three questions that work to reduce risk. Are we in the right space? What do users need? And what are our competitors doing?

There is a risk of arrogance here. The goal of research should be to guide and develop the product or service, its introduction to the market and its iteration over time. On the other side of the equation, while designers’ clients certainly know their business best, this can lead them to think that they already know everything they need to know.

Talking business

But, returning to the scenario outlined at the beginning of this blog post, clearly that is not the case as businesses often struggle to get the results they expect. As such, it remains the responsibility of designers to solve the problems that they face. So, if the questions research asks are essential and focussed on need, then it is equally the case that the form of its answers must also meet a need.

This means one thing: designers must learn to speak the language of business. From a user research perspective, this means taking both simple and complex key performance indicators (KPIs) for the research process to investigate and showing how they can be met. In other words, user research should address the numbers businesses deal in.

This, Kennedy says, can provide an additional benefit in the form of organisational efficiency by bringing something concrete to the table.

“It's very typical, particularly in larger organisations, for decisions to go around the houses multiple times, basically in a circular fashion, where the decisions are being made on personal intuition, preference, all these different things,” he said.
This is in spite of the fact that the actual answer, the confidence a business needs, can be built by finding out what do the actual users do, how do they behave, and what do they think. Research can deliver precisely this.

“That will many times give you the answer that internal stakeholder discussion just simply can't,” he said.

However, how this is communicated is essential. Explaining how research can propel a project forward by defining the right KPIs will give confidence to management as it is a case of speaking a language they understand.

As a result, a thorough design process, founded in user research will set a number of clear goal:

  • Reduce the risk of product failure
  • Reduce the work and eliminate, as much as possible, re-working of the product or service
  • Help make better informed decisions based on quantitative data

Putting those together results in the one thing every business wants with its offerings: competitive advantage. That is the real message designers need to communicate.

Subscribe to our newsletter

Copyright © 2023 Each&Other Ltd.

Registered in Ireland. No. 545982

Privacy Policy