(In this final installment of our two-part conversation on innovation, Harvard Business School Professor Stefan Thomke breaks down why companies that do a better job of combining software and controlled experiments make better decisions, create better customer experiences and generate bigger financial returns than the rest of the pack. Read part 1 here.)
Innovation expert Stefan Thomke argues that the "best guess" approach to innovation is changing fast as digital leaders conduct numerous online, controlled experiments and engage millions of users faster than ever before.
"What many of these organizations have discovered," says Thomke, "is that an experiment-with-everything' approach to innovation offers a bigger payoff and a considerable competitive advantage."
And if that's the case, what can you do to help your organization capitalize on this experimentation trend?
The critical success factor, says Thomke, is to create an 'experimentation organization' that masters the science of testing and puts the discipline of experimentation at the center of the innovation process.
In the past, it could take years for companies to develop this kind of game-changing capability. Today, the most successful brands have figured out how to do it by combining the speed and power of custom software with the rigor of controlled experiments.
Read and enjoy.
Appian: One of the big takeaways from your new book (Experimentation Works: The Surprising Power of Business Experiments) is thatthe velocity of business experiments is what really drives innovation. You argue that digital leaders conduct thousands of experiments a year to stay on top. So, is being able to quickly run controlled business experiments an existential challenge in the digital economy? Does it rise to that level of importance?
Thomke: Very much so, especially in the online space. The reality is that when it comes to user behavior, we honestly don't really know what works. I mean, it's very difficult to predict. And even the experts will admit that they're often wrong.
In Bing's case, they found that somewhere between 10 and 20% of the experiments that they tried worked. The rest either had no impact at all or generated results that were opposite of what was expected.
Company-wide, Microsoft has a rule of thumb that just a third of the experiments they run will have some measurable impact on a metric that they care about. Another third will have no effect at all. And a third will have the opposite effect of what was anticipated. I think the numbers are comparable for other companies as well.
So, it requires, I think, for managers to have some humility (laughter). You know, when it comes to online behavior, the reality is that I'm much more likely to be wrong than right.
Don't Freak out Over Failure
Appian: But fear of failure is an innovation roadblock at many traditional companies.
Thomke:Yes, but you have to be willing to accept that (failure) and live with it. Failure is part of the game. So the question is not 'how do I figure out how to be right more often?' The challenge is more about velocity and 'how can I get the right answer more quickly?'
Appian: So, how can senior execs at large, traditional companies leverage the concept of experimentation when fear of failing may be a big deal in the organizational culture? How do you overcome that fear to compete against digital disruptors?
Thomke: First of all, they have to understand and accept the fact that experimentation is the engine of innovation. If you don't take advantage of that, it's like driving a car without an engine. It doesn't get you very far. You know, it's not a new insight that many large institutions are hesitant to test and learn.
In their bestselling book "In Search of Excellence",Tom Peters and Roger Waterman said that:
Many big institutions would rather analyze and debate rather than try something out. These companies tend to be paralyzed by fear of failure. So, part of challenge is a cultural problem that needs to be addressed.
https://youtu.be/U3Dw5XQTyUc
Appian: So, what are digital leaders doing right that other companies are doing wrong?
Thomke: The most successful big tech companies have turned themselves into what I would call experimentation organizations. They're doing it with platforms, governance structures, culture and so on. And as you rightly pointed out, failure is just part of the game.
Appian: But failure has such a negative connotation.
Thomke: Failure is a loaded word and the biggest takeaway is that you need to fail a lot more. But we should make the distinction between failing and making a mistake.Mistakes create no new or useful information. So, if Amazon builds yet another warehouse and things go wrong, that's not terribly useful information.
It just means that somebody wasn't paying attention. Because from an operational standpoint, Amazon's done this kind of project many times before, so it should be just like working down a checklist. But that's different than failure.
Failure is about addressing a question. It's about learning something new and taking certain risks.
The Six Laws of Innovation
Appian: So, how do you determine if failure is beneficial or not?
Thomke: The key is whether there was a learning objective involved and you learned something new. As a manager, you want to minimize the number of mistakes you make. But you also want to create an organization where making mistakes is okay.
Appian:I came across an article you wrote a while back that talks about the six principles of solving business innovation problems. Can you give our audience a quick overview of these principles?
Thomke:Sure...
- 'Front-loading' problem solving:When you're innovating, getting information early is much more valuable than getting it late because the cost and time to make a change increases exponentially over time. So, what you want to do is experiment in a way that as much information as possible is revealed early on. For example, you could build a system prototype or have customer interaction early on. The problem is that many companies find out way too late what works and what doesn't work. It's not just about learning what works but also what doesn't work, so you don't invest more resources in it. It's one of those situations where if I've got bad news, I want to know it now. Good news can wait.
- Experiment often: It's about frequency and velocity because of the digital revolution and all of the platforms out there. The cost of experimenting has been reduced by orders of magnitude. So, we're reaching a point where experiments are virtually free. Sometimes people hesitate to run experiments because they have to get a budget and it's too complicated. You want to get to a point where the marginal cost of experimenting is virtually free.
- Integrate new and traditional technology: You know we often get excited about new technologies and want to throw out the old. But you have to be careful because sometimes old technologies have advantages over new technologies. So, managing the transition means managing both. This happens in the space of advanced modeling, simulations and prototype testing. You get the biggest bang by managing both together.
- Organize for rapid experimentation:That gets at organizing for velocity. If you have lots of company interfaces and sign-off procedures, it can get in the way of innovation. So, you want to organize around speed, for example, by having experimentation teams where different functions are represented on the teams so that you don't have to climb over walls in your organization. You want to empower people and give them a fast track and get rid of bottlenecks.
- Fail early and often: This goes back to the idea of front-loading the innovation process. You want to fail a lot but you want to do it early because you want to find out when things go wrong before you make big commitments.
- Manage projects as if they were experiments:Look at experimentation as different units of measurement. You can look at experimentation from a project level where you have a hypothesis and you can quickly learn and pivot. I think the whole movement around Agile follows a similar philosophy. The big idea is that you want to test your hypotheses faster which allows you to iterate more quickly and get to the result more quickly. This is a way more productive way to get things done.
https://youtu.be/k2vRbsY5aAI
Fast Experimentation Changes Everything
Appian:Finally, let's touch on your previous book: "Experimentation Matters". Clayton Christensen is like the godfather of 'disruptive innovation. When a big thinker like Christensen calls your book the most important book on innovation he's ever read, that's a huge deal. So, what motivated you to write the book?
Thomke: I looked around and saw that there was a revolution underway in research and development (R&D) and innovation. New technologies were driving down the cost of experimentation so dramatically.
And, so, I wanted to write a book to show people how to take advantage of these new technologies in R&D and product development and also develop some principles around that. And to go back to an earlier question, I think the same revolution is underway right now in the online world. We're now seeing, for example, a revolution in the way decisions get made in companies especially around online user experiences.
Today, you can go out and instantly test an idea on millions of users and customers to see if it works or not. That's phenomenal. It changes everything.
Appian: Which is also a big takeaway from your latest book.
Thomke:Yes, inExperimentation Works: The Surprising Power of Business Experiments,I show how online and offline companies are benefiting from these amazing changes.
Appian: There's a ton of research out there showing that companies are now prioritizing customer experience as a competitive advantage.
Thomke: Exactly, look at the companies that are doing this well. Look at the Amazons and the Googles and the Bookings.com and what you'll see is that customers are benefiting from this revolution as well.
Whether you're talking about search, or travel or online purchasing, chances are you're participating in an experiment.But as a result, these companies are able to make the customer experience better and better every single day.
Appian: Which dovetails with your argument for experimenting with customers to create value.
Thomke: That's right.Now imagine that you're trying to compete against companies that are really good at this. You have no chance because they'll just out-experiment you.
The Future of Innovation
Appian:Before we end, I wanted to get your expectations for the future. Which big trends are on your radar for 2020 and beyond?
Thomke:Oh no, you're asking me for predictions. I won't make any predictions but let me answer your question this way. I studied artificial intelligence (AI) back in the 1980s. So, it's amazing to see the massive growth of data and neural networks.I think we'll see the continuing evolution of 5G and data transmission. I also think we'll see exponential advances in the evolution of data and AI.
The big question is how will we bring these technologies together and how will they change the way companies do business?
I also think the scientific method which can be traced back to 1620 when Francis Bacon wrote about it is going to revolutionize modern business management.
That's why I feel so strongly about experimentation because experiments drive the scientific method. It's going to change the way we make decisions and it's also going to force us to rethink the way we run our companies.
The tech trends we're seeing right now including AI, big data, digital platforms and the ability to reach customers more rapidly are just going to elevate the importance of experimentation even more.
Nobel Laureate Richard Feynman said it best:
'It doesn't matter how beautiful your theory is. And it doesn't matter how smart you are. If your theory doesn't agree with experiment, it's wrong.'
That's pretty simple, isn't it?
