Choose the Right UX Metrics to Show Business Impact
How many designers have heard one of the following statements as direction?
“Needs to be simple and usable”
“Great customer experience”
“Easy and intuitive”
“Make it fun and engaging”
“It’s about design-thinking”
These phrases roll off the tongue, as if user comprehension, learning, performance, and satisfaction were wishes to be granted. Of course, we want all of these things in our products and work hard to do so with user-centered design. However, in the age of measurement and optimization, what specifically do these phrases mean? How will you measure the achievement of those ideas? How will you know if they make a difference for the users?
A crappy product can have a wonderful user experience or a wonderful product can have a wonderful user experience, but what may be unknown for designers is true impact. We know a great UX solves problems. UX helps make everything better for our poor users out there in the world, but how? Makes our brand pop! Make our product delightful! Yeah, but, specifically, how? What is the result of all your effort?
And, how do directional statements map to a company’s business goals? If the product is simple and usable, so what? Does it mean more sales or profit?
- “Needs to be simple and usable” = efficiency?
- “Great customer experience” = customer satisfaction?
- “Easy and intuitive” = customer satisfaction?
- “Make it fun and engaging” = loyalty?
- “Show more design-thinking” = innovation?
Many in the design community are grasping with the question — what should I measure? What are the right metrics to measure for a UX? For product design? And then, how do I measure it to show a difference?
You can choose any metric as the quantitative answer to any question you ask, but choosing the right metrics depends on the question you want answered, the story you want to tell. This article is about choosing metrics that tell the story of the business impact of your design. This method helps you tell a story that starts and ends with “The design led to these specific business results, and here’s how that work contributes to the company goals and strategy.”
But first, let’s start at the beginning… Why measure design at all?
If you’ve been to business school or any business class, or perhaps even just read an article about business, Dr. Peter Drucker’s name is no doubt a part of the reading. He’s said many wise things about business, but a few quotations stick out as the reason why we need to measure what we design:
- “What is measured gets improved.”
- “Management is doing things right. Leadership is doing the right things.”
Picking the right metrics and what to measure is about paying attention to what’s working and what’s not –for the users and the business. It’s about having the evidence of success success and opportunities for improvement. Clear measurement is the language of that specificity we need to show the impact of design on the business. It means we have a fact-based story to tell on why we need more design resources and the business reason for design’s existence.
We need to show how your (team’s) design makes an impact on your company. And to do this, you need to look at what your company is trying to achieve. That will set us on the course for the right UX metrics.
What metrics do we choose? The ones that map to your company’s business goals
Many managers I’ve worked for have what I call the “shiny object syndrome.” They shift their attention to different initiatives every quarter and successes and opportunities are lost because no one was doing what Dr. Drucker said. Measurement keeps us all honest by helping us focus on the right things. These right things are what impact a company’s business goals.
All businesses have goals that they need to measure against and report on to their board, shareholders, and stakeholders. They come from the top leadership down. You have probably attended a company “all hands” meeting or deleted an email about company business goals and current progress to those objectives and key results (OKRs) or key performance indicators (KPIs) like sales at key accounts, growth rates, market penetration, or brand awareness. These goals are what your company is investing in and must achieve and regularly report on if they are to stay focused deliver any shareholder return.
Examples of company goals are:
- Growth: New markets, new ways to market, new products, new products to new markets, and acquisition to increase revenue
- Profitability & Efficiency: Faster, cheaper, better to increase profit
- Customer Service & Retention: Happier customers are loyal and valuable over time
These are the “what” a company is focused on. They are not strategies — strategies are the “how” a company achieves its goals and there are many, many other articles written about them. Design is a strategy, as well as low prices, innovation, new markets, low prices, and improved quality.
As a designer, it’s time to pay attention to your company’s goals instead of deleting the quarterly email or making fun of corporate-speak at the next meeting. The business goals are what you need to care about in your day-to-day job and not just in your yearly performance review. Knowing what your company leadership cares about means you can focus your work, and pick the right metrics and measurement methods for your design. If you work for an agency, then you care about your client’s business goals. If you don’t know, ask.
Understand and focus on the company goals, and you have the technique to determine the right metrics.
Here are some examples:
“Our excellent results represent a terrific start to 2017, with revenues up 22% versus the first quarter of 2016 and 24% on a constant currency basis.
We clearly continue to benefit from our ongoing investments in product innovation and have great momentum in our new businesses across Alphabet,” said Ruth Porat, CFO of Alphabet.
Alphabet’s goal is growth, which they hope to achieve through innovation and new products.
“Our first-quarter results again demonstrated that customers value a high-quality network experience,” said Chairman and CEO Lowell McAdam of Verizon.
“To build on our loyal customer base and the third-party recognition we have received for network leadership, we extended our wireless and fiber network capabilities, began offering an unlimited pricing option and expanded our opportunities in new markets. We’re executing on strategies to capture future growth and create long-term shareholder value.”
Verizon’s goals are customer loyalty and growth through quality and expansion into new markets.
Bonus: If the term “managing up” has ever been a mystery to you, it’s now solved. The “up” is your boss’s goal, which is the company’s goals. Focus your work on measuring how it contributes to the company’s goals, and you’ll always have a meaningful story to tell about your work to your boss. In turn, she can tell her bosses (unless she’s the CEO, then she should promote you).
Pause for reflection: What are your company’s goals?
Now let’s translate those business goals into UX metrics
Let’s get down to the nitty-gritty of determining the metrics for your specific design. The chart below breaks down corporate goals into what they most likely mean for your department and team. They focus on two areas where UX is needed: marketing and product. These two areas have existing models and metrics that are common vocabulary related to business goals, so we are not inventing a strange language.
So how do we determine what to measure? Here’s how…
Here are a few examples:
Uber CEO Travis Kalanick shared his business’s goal to Business Insider. “Our intention is to make Uber so efficient, cars so highly utilized that for most people it is cheaper than owning a car.” His goal for Uber is about efficiency. With this one statement, marketing and product teams have an obvious goal, a focus for their efforts, and a clear indication of what to measure.
If I were the Director of UX director at Uber, I would do two things immediately after hearing this statement:
1. Look at my list of projects and prioritize all those that increase efficiency
2. Find data on or measure the following to set a benchmark:
- Efficiency → Time to task completion
- Profitability → Conversion rate /task completion rate for the most profitable action on the product
According to Investopedia, PayPal finally revealed their master plan: “Pay with Venmo,” which would allow customers to use Venmo at retailers, and pay for merchandise on their phones instead of using cash or credit cards.”
To determine the right UX metrics that aligned to business goals of awareness/acquisition for new customers, I would measure:
- Conversion rate/task completion rate of paying on a phone at a retailer for first-time users
So let’s measure and show that business impact!
We have our focus (company goal) the right metrics (UX metrics that tie to a company goal), and now what? Now we gather two sets of data. To show impact, you need to show change — an improvement or a decline —of the before and after. The “before” is a benchmark of what the UX is like for your users today. The “after” is what we’ve accomplished — the change — with that new UX.
And when I say, “your users,” what I really mean is your company’s target audience. That’s your population. For example, if the company goal is growth and the metrics are user acquisition, then you really care about measuring new users. If you focus on existing customers, your measurements and UX reflect the impact for current user growth. It’s the key audience’s actions and behaviors that matter most to benchmark and measure.
Some naysayers will say, “But how do you know your design is the only thing that contributed to improvement? There are so many things going on that we can’t measure.”
100% attribution or causation is very rare anywhere outside a science lab or controlled experiment. You can do a few things to make sure as much noise as possible is drowned out in your analysis. First, you’ve chosen a few key metrics to measure that directly relate to the business goals and UX strategy. That’s a start — you do not measure things that don’t matter (i.e. the noise). Also, it’s also about timing.
Gather the benchmark measurement 90-days before launch and measure for change 90 days after. You’ll have a clear idea of how much the UX improved on those key metrics for the user. That timeframe will show an “evening out” of any unusual spikes like an unexpected press release or event. If you’re in the prototyping phase, run two usability tests, one that’s a benchmark with current site or product and one that mimics real-world tasks and usage on your prototype design.
In either case, you can quantify subjective measures for satisfaction by using a Likert scale of 1 to 7 to indicate level of satisfaction, a binary measurement on likelihood to purchase (yes or no) or the 10-point Net Promoter Score (NPS). While you can’t infer averages from these numbers, you can count how many people scored the highest value before and after and show an improvement that way.*
For example, when I created the redesign strategy for a large corporate website, engagement was a UX goal because the website needed to qualify sales leads (that was marketing’s business goal). The marketing department invested huge amounts of money in asset creation as a way to quality sales leads. However, in a content analysis, my team learned that those assets were not being accessed or used, most likely due to poor page design and UX. Therefore, I measured key metrics before and after redesign. We saw a 63% increase in total asset downloads, with a 31% increase in case studies, which was a direct result of our design strategy.
Next, it’s about sample size. If your product is out in the market, you have larger sample sizes to gather data. The larger the sample size, the better, but any amount over 30 for your target audience or segment, is a good place to start (see Central Limit Theorem on why).
If your product is not out in the market, you’re looking for indicators that show you’re going in the right direction. Here, you’re looking for patterns and the sample size of 8–12 holds up, per our friend Dr. Jakob Nielsen. For example, if all eight participants in a usability test are able to complete the task quickly and rank it high in satisfaction, you can be assured that your design is going in the right direction to achieve that business goal.
Here’s a table of recommended data collection techniques by sample size and phase of a project.
Finally, let’s telling the story of the impact
Picking the right UX metrics gives you evidence of your design’s business impact. It aligns what you do with what the company does. It provides a focus of measurement for resource prioritization and investment. Now you have something to share with leadership and your colleagues. How do you do that? With pride and a bit of savvy. But first, a story…
When I started work as an associate brand manager fresh out of business school, I quickly learned from my colleagues that if the president of the division asked me, “How are things going?” he was not after a recount of my weekend. He was asking me, “How’s your brand’s business?” The correct response was a recount of last week’s shipments and if there was an improvement or decline and why that happened. Those metrics I shared laddered up to numbers he was responsible for when reporting how the division was meeting its goals. In that answer, I told a story and proved to him that I knew the impact of my work to the top line of his business.
By focusing on UX metrics that map to business goals, you now have the story of impact and results. You have the one line that you can share in the elevator when a VP asks you, “How are things going?” You can answer, “Just launched a new release… pretty excited about it. We found that the new UX increased user satisfaction by 36%*.” You have a conversation starter, which will hopefully lead to a raised eyebrow and curiosity and lead to sharing out how what you do helps the company achieve its goals. You also give a company leader a talking point to share out when their boss says, “We need to make this easy to use.”
— — -
- You can show a percentage change even if you collected numbers on a Likert. Count the number of users who ranked their satisfaction extremely likely (far side of the scale) in before and after studies. Then, calculate % change of that count — keeping denominators constant in the ratio.
______
Check out “How to Define User Value with UX Research” to make sure your product is truly valuable to users.
If you like this story, make sure to visit my profile and follow me. You’ll get more articles on user value, showing the business impact of UX, and career advice.
Want to see this topic in person? I’ll be sharing a more updated version at World Usability Congress 2021 in Graz, Austria.
