(In this final installment of our two-part conversation on innovation, Harvard Business School Professor Stefan Thomke breaks down why companies that do a better job of combining software and controlled experiments make better decisions, create better customer experiences and generate bigger financial returns than the rest of the pack. Read part 1 here.)
Innovation expert Stefan Thomke argues that the "best guess" approach to innovation is changing fast as digital leaders conduct numerous online, controlled experiments and engage millions of users faster than ever before.
"What many of these organizations have discovered," says Thomke, "is that an experiment-with-everything' approach to innovation offers a bigger payoff and a considerable competitive advantage."
And if that's the case, what can you do to help your organization capitalize on this experimentation trend?
The critical success factor, says Thomke, is to create an 'experimentation organization' that masters the science of testing and puts the discipline of experimentation at the center of the innovation process.
In the past, it could take years for companies to develop this kind of game-changing capability. Today, the most successful brands have figured out how to do it by combining the speed and power of custom software with the rigor of controlled experiments.
Read and enjoy.
Appian: One of the big takeaways from your new book (Experimentation Works: The Surprising Power of Business Experiments) is thatthe velocity of business experiments is what really drives innovation. You argue that digital leaders conduct thousands of experiments a year to stay on top. So, is being able to quickly run controlled business experiments an existential challenge in the digital economy? Does it rise to that level of importance?
Thomke: Very much so, especially in the online space. The reality is that when it comes to user behavior, we honestly don't really know what works. I mean, it's very difficult to predict. And even the experts will admit that they're often wrong.
In Bing's case, they found that somewhere between 10 and 20% of the experiments that they tried worked. The rest either had no impact at all or generated results that were opposite of what was expected.
Company-wide, Microsoft has a rule of thumb that just a third of the experiments they run will have some measurable impact on a metric that they care about. Another third will have no effect at all. And a third will have the opposite effect of what was anticipated. I think the numbers are comparable for other companies as well.
So, it requires, I think, for managers to have some humility (laughter). You know, when it comes to online behavior, the reality is that I'm much more likely to be wrong than right.
Appian: But fear of failure is an innovation roadblock at many traditional companies.
Thomke:Yes, but you have to be willing to accept that (failure) and live with it. Failure is part of the game. So the question is not 'how do I figure out how to be right more often?' The challenge is more about velocity and 'how can I get the right answer more quickly?'
Appian: So, how can senior execs at large, traditional companies leverage the concept of experimentation when fear of failing may be a big deal in the organizational culture? How do you overcome that fear to compete against digital disruptors?
Thomke: First of all, they have to understand and accept the fact that experimentation is the engine of innovation. If you don't take advantage of that, it's like driving a car without an engine. It doesn't get you very far. You know, it's not a new insight that many large institutions are hesitant to test and learn.
In their bestselling book "In Search of Excellence",Tom Peters and Roger Waterman said that:
Many big institutions would rather analyze and debate rather than try something out. These companies tend to be paralyzed by fear of failure. So, part of challenge is a cultural problem that needs to be addressed.
Appian: So, what are digital leaders doing right that other companies are doing wrong?
Thomke: The most successful big tech companies have turned themselves into what I would call experimentation organizations. They're doing it with platforms, governance structures, culture and so on. And as you rightly pointed out, failure is just part of the game.
Appian: But failure has such a negative connotation.
Thomke: Failure is a loaded word and the biggest takeaway is that you need to fail a lot more. But we should make the distinction between failing and making a mistake.Mistakes create no new or useful information. So, if Amazon builds yet another warehouse and things go wrong, that's not terribly useful information.
It just means that somebody wasn't paying attention. Because from an operational standpoint, Amazon's done this kind of project many times before, so it should be just like working down a checklist. But that's different than failure.
Failure is about addressing a question. It's about learning something new and taking certain risks.
Appian: So, how do you determine if failure is beneficial or not?
Thomke: The key is whether there was a learning objective involved and you learned something new. As a manager, you want to minimize the number of mistakes you make. But you also want to create an organization where making mistakes is okay.
Appian:I came across an article you wrote a while back that talks about the six principles of solving business innovation problems. Can you give our audience a quick overview of these principles?
Appian:Finally, let's touch on your previous book: "Experimentation Matters". Clayton Christensen is like the godfather of 'disruptive innovation. When a big thinker like Christensen calls your book the most important book on innovation he's ever read, that's a huge deal. So, what motivated you to write the book?
Thomke: I looked around and saw that there was a revolution underway in research and development (R&D) and innovation. New technologies were driving down the cost of experimentation so dramatically.
And, so, I wanted to write a book to show people how to take advantage of these new technologies in R&D and product development and also develop some principles around that. And to go back to an earlier question, I think the same revolution is underway right now in the online world. We're now seeing, for example, a revolution in the way decisions get made in companies especially around online user experiences.
Today, you can go out and instantly test an idea on millions of users and customers to see if it works or not. That's phenomenal. It changes everything.
Appian: Which is also a big takeaway from your latest book.
Thomke:Yes, inExperimentation Works: The Surprising Power of Business Experiments,I show how online and offline companies are benefiting from these amazing changes.
Appian: There's a ton of research out there showing that companies are now prioritizing customer experience as a competitive advantage.
Thomke: Exactly, look at the companies that are doing this well. Look at the Amazons and the Googles and the Bookings.com and what you'll see is that customers are benefiting from this revolution as well.
Whether you're talking about search, or travel or online purchasing, chances are you're participating in an experiment.But as a result, these companies are able to make the customer experience better and better every single day.
Appian: Which dovetails with your argument for experimenting with customers to create value.
Thomke: That's right.Now imagine that you're trying to compete against companies that are really good at this. You have no chance because they'll just out-experiment you.
Appian:Before we end, I wanted to get your expectations for the future. Which big trends are on your radar for 2020 and beyond?
Thomke:Oh no, you're asking me for predictions. I won't make any predictions but let me answer your question this way. I studied artificial intelligence (AI) back in the 1980s. So, it's amazing to see the massive growth of data and neural networks.I think we'll see the continuing evolution of 5G and data transmission. I also think we'll see exponential advances in the evolution of data and AI.
The big question is how will we bring these technologies together and how will they change the way companies do business?
I also think the scientific method which can be traced back to 1620 when Francis Bacon wrote about it is going to revolutionize modern business management.
That's why I feel so strongly about experimentation because experiments drive the scientific method. It's going to change the way we make decisions and it's also going to force us to rethink the way we run our companies.
The tech trends we're seeing right now including AI, big data, digital platforms and the ability to reach customers more rapidly are just going to elevate the importance of experimentation even more.
Nobel Laureate Richard Feynman said it best:
'It doesn't matter how beautiful your theory is. And it doesn't matter how smart you are. If your theory doesn't agree with experiment, it's wrong.'
That's pretty simple, isn't it?
Appian is a software company that automates business processes. The Appian Platform includes everything you need to design, automate, and optimize even the most complex processes, from start to finish. The world's most innovative organizations trust Appian to improve their workflows, unify data, and optimize operations—resulting in better growth and superior customer experiences.