Intelligent Failures vs. Costly Mistakes: Navigating the Innovation Paradox

We’re trained from a young age to want to avoid it at all costs. Our successes are met with praise, rewards, and belonging, while our failures can bring condemnation, shame, and isolation. We associate failure with losing. Despite the rebel appeal of failure parties, or failure resumes, it’s hard for most of us to buy into the sunny rhetoric, and the managers we work for rarely help.

From Nobel laureates to elite athletes: The universality of failure

And yet, Nobel-winning scientists, elite athletes, and innovators in every field fail frequently.

Without their failures, their successes would have been impossible. What do these resilient game changers know that most of us don’t? How do they manage the fear of failure to win tournaments and make game-changing scientific breakthroughs? How do they bounce back from the failures they experience?

To begin with, they understand that not all failures are created equal. They eschew the kind of failure that comes from not trying, and they know not to take personally the failures created by a combination of complexity and bad luck.

They’re willing to work hard to prevent the failures that can be prevented. It is this discernment that helps them embrace the good kind of failure: the kind that really is intelligent.

How can you tell whether a failure is intelligent?

Here’s a four-criterion rubric I’ve developed through my research in organizations across industries and around the world: It (1) takes place in new territory (2) in pursuit of a goal, (3) driven by a hypothesis, and (4) is as small as possible. Because they bring valuable new information that could not have gained in any other way, intelligent failures are praiseworthy indeed.

Exploring the unknown: The essence of intelligent failure

No company can innovate without them. They are also the bread and butter of pioneers in fields ranging from science to sports to elite cuisine.

Intelligent failures, which play a vital role in expanding knowledge and achievement in every field, are, technically notpreventable. Why? Because they occur in uncharted waters where there is simply no way to know in advance what will happen.

When knowledge is not yet developed; you have no choice but to experiment to figure out what works (and what doesn’t). In contrast, when a playbook for how to get a desired result exists, use it!

Neither reinventing the wheel nor ignoring a tried-and-true recipe is admirable; the former wastes resources and the latter leads to preventable failures.

These can rightly be considered wasteful.

The delicate balance between innovation and stagnation

But, intelligent failures are indeed valuable; they bring the new knowledge needed to advance further. Moreover, the only way to avoid them is to never experiment, which creates another kind of risk: the risk of stagnation and eventual obsolescence.

New territory in our lives and organizations takes various forms. For instance, three years ago, none of us had ever personally experienced a global pandemic.

The devastating 1918 influenza outbreak provided only a partial playbook for navigating Covid-19 safely, given the dramatically different global society in which it occurred. Unsurprisingly, therefore, many actions taken to address the novel threat produced failures.

Fortunately, some actions led to astonishing successes. That’s the nature of novelty. But a new-to-the-world event is not the only type of new territory that matters.

Say you want to pick up a new skill or expertise to fuel your career. It’s new for you, and you should expect some intelligent failures as you build competence. And every company depends on today’s innovation projects for tomorrow’s revenues.

If you’re not failing, you’re not journeying into new territory – whether as an individual or as an organization. In this sense, “fail fast, fail often” has its place – it’s how innovation and discovery happen in every industry and it's a source of discovery and adventure in every life.

While it’s easier to stay in your comfort zone, progress and adventure are rarely found there. Nor are prize winning scientists, Olympic gold medalists, or successful entrepreneurs.

Learning from expert fail-ers

I think of the people who best model intelligent-failure principles as “elite failure practitioners” (EFPs) – my deliberately provocative term to emphasize the centrality of failure in the journey to high achievement.

The EFPs I’ve profiled in my writing are diverse in every way.

There’s Jocelyn Bell Burnell, an astrophysicist from Northern Ireland who discovered the first radio pulsars in 1967 and James West, an African American inventor at Bell Labs (historically a haven for EFPs) who revolutionized microphones, holding more than 250 patents for his various inventions.

Another is Rene Redzepi, the pioneering Danish chef whose restaurant was voted the best in the world an extraordinary five times.

Consider celebrated American entrepreneur Sara Blakely, founder of apparel company Spanx, who famously endured countless setbacks on her path to becoming the youngest ever billion-dollar entrepreneur.

Or, what about Barbe-Nicole Ponsardin Clicquot, born in France in 1777 and suddenly widowed at the age of 27?

Instead of giving up on the tiny struggling business she had recently started with her husband, Barbe-Nicole persisted through staggering business and technical failures for years, ultimately building Veuve Clicquot, one of the most successful and enduring companies in the wine industry.

Clicquot’s technical innovations revolutionized the production of Champagne, converting the originally cloudy liquid into the sparkling clear wine we know today.

Each of these pioneering thinkers and do-ers experienced far more failures in their lives and careers than the average person. And this, I would argue, is the not-so-secret sauce in their success.

Innovation across centuries: Consistent traits of pioneers

EFPs span fields – science, technology, cuisine, business – and their stories span centuries. But I have found that they have three things in common: they are relentless experimenters, driven by curiosity, who’ve made friends with failure.

West, for example, was one of those kids who couldn’t stop asking questions. To his parents’ chagrin, he often sought to answer them himself by experimenting.

Fortunately, he survived when, trying to fix a broken radio he found in his 1930s farmhouse, he plugged it in to the electrical outlet in a ceiling light fixture only to discover his hand stuck to the ceiling – an early failure that only reinvigorated his desire to figure out how electricity worked.

Redzepi worked hard to make his laboratory a safe place for wild-eyed culinary experiments, rewarding rather than bemoaning failed dishes his team of young chefs produced behind closed doors. Making friends with failure wasn’t easy at first, but once he understood its necessity, he, like so many other leading innovators, became a passionate advocate of failing well.

It’s not that any of these pioneers like to fail; their understanding of its necessity drives them forward.

How to put failure to work

An instinctive aversion to failure and concern about its social stigma holds many of us back. It’s impossible to calculate the wasted time and resources created by our reluctance to take the smart risks that lead to progress.

When we go out of our way to avoid failing, we rob ourselves of discovery, accomplishment, and meaningful relationships with colleagues. But once you appreciate the distinction between intelligent failures and the rest, you can start to put the insights and practices of expert fail-ers to work in your team or company.

And, by learning to fail intelligently – in new territory, pursuing goals we care about, with thoughtful, small experiments – we can also live fuller, more adventurous lives.

Eager for more?

Are you intrigued by Amy's research into intelligent failure? Her latest book THE RIGHT KIND OF WRONG: The Science of Failing Well is on sale now!

In our upcoming newsletter, we'll share some actionable insights on this topic. Subscribe below to have it delivered straight to your inbox.