How do we measure UX impact?

I got into the field of customer-centred design as a usability consultant back in 2004. As a usability consultant, you do a lot of testing - and you measure.

You measure task completion rate, the time it takes to complete a task, the number of errors, and the perceived satisfaction.

Coming from a cognitive & behavioural science background, this made a lot of sense to me initially. We have some hypotheses (our product will make our customer's lives easier through XYZ), and we want to measure or validate them.

But the longer I worked in the industry and the more user research and testing I did, the more I questioned UX measurements.

For one: the bosses didn't really care about some satisfaction scales from rounds of testing with only a handful participants. They wanted quantitative research, big and "reliable" data. Something that would let them say: 80% of our customers really like it.

That's kind of why we started to add all these detailed measurements to usability tests: task completion rate or time spent could give us some numbers. But what do these numbers really tell us? If someone spends more or less time on something - does that mean that he enjoys it more or less?

​

I like numbers. I understand their importance. And here are two things I've learned when ti comes to measuring UX:

1. When it comes to measuring something so subjective as a "customer experience", we need to combine quantitative with qualitative numbers. Quantitative data can give you some hints about the "what." Qualitative data will help you to understand the "why" behind the "what."
2. Don't just go with "standard KPIs." And don't track data just because you can. Instead, be clear about your objective. What is it that I want to improve with my product? What is the promise I give my customers? Then, define the measurements you need to validate your assumptions. If you promise to speed up the process for your customers, you might want to measure the task completion rate. If you promise accuracy instead of speed, you want to collect customer satisfaction rates.

​
​
​

Over to you

For your next project, go beyond your standard metrics (page visit, bounce rate...). Instead, set a clear goal of what you want to see improved. Come up with 2-3 success criteria and focus on those for a while.

​
​

​
​