Principle six: Measure what matters

One of the most oft-cited principles of business is that what gets measured gets managed.

But what happens when you measure the wrong things?

If your organization's measures of success are based on assessing output and overall adherence to scope, schedule and cost, then you’re measuring the wrong things. If you want to measure the right things, measure outcomes, not outputs.

Although measuring activity can provide a useful way to identify anti-patterns and opportunities for improvement, they are otherwise worthless as a true measure of success because they are uncorrelated to either customer or business value. Measuring outcomes, however, can be much more useful because they are designed to directly correlate to value, and they also embrace the inherent unknowability and unpredictability of most digital endeavours. Working in this way has a lot of upside. Outcomes not only accept that the right solution cannot be known upfront, but they also provide teams with the autonomy, responsibility and ownership needed to solve the right problem the right way, using hypothesis-driven envision/explore methods to find the sweet spot between customer value and business value.

This approach is in stark contrast to the output-based approach, which requires teams to deliver a predetermined solution based on sweeping assumptions (that are virtually guaranteed to be incorrect), and that don’t change even as teams uncover significant amounts of hidden complexity. An outcome-based approach, however, measures what matters by incrementally assessing quantitative and actionable metrics that signal value — whereas the task-driven approach often measures value at the end of the process, when it is far too late to change course.

For example, when using an outcome-based approach the goal shouldn’t be to deliver a check-out feature, because that’s just an output. Instead, the goal should be to learn how to improve revenue (i.e. business value or impact) using a series of hypothesis-driven experiments to iteratively improve the conversion rate (the outcome) using a series of usability enhancements (the outputs) to improve the customer experience during the payment process.

Similarly, from a digital transformation perspective, transformation success shouldn’t be measured by how many teams received training or how many groups have adopted new processes or tools, because again, these are just outputs. Instead, success should be measured by incremental outcomes such as improved time-to-market or increased customer retention that drives broader business goals, such as increased revenue or new customers.

Ultimately, learning to measure what matters is about a mindset shift to focusing on approaching business objectives differently. This shift has been captured beautifully by the author and consultant Henrik Kniberg in the illustration shown in Figure J below.

Figure J: Example of an outcome based mindset from Henrik Kniberg

But this shift can be challenging for many established organizations. Their established lines of business are primarily output- and activity-based, with measures of success that are oriented to maximizing the production, distribution or sale of physical rather than digital products.

The difference, of course, is that digital products are never truly done and can be endlessly evolved to adapt to change. Success in digital is mainly a matter of determining the outcomes you want to create and what the measures of success are needed, followed by a series of iterative and incremental experiments to help you get there.

Making the shift to outcomes over outputs can be a radical change for many established organizations, but it is possible to get there by focusing on three key questions:

1. What are the human behaviors that drive results we want?

2. How might we influence the behavior we want?

3. How will we know when we are successful?