Most ideas fail, and that's fine

On Camus and A/B tests

I: Failure everywhere

Let’s start with the facts: Most ideas fail.

In the United States, around 50% of new businesses fail within 5 years. Only a third survive to celebrate their 10th birthday. The numbers are almost identical in the European Union, so it’s not some weird American quirk.

In tech, most new software features don’t add value. Success rates in gold-standard A/B tests hover around 10–20%:

Success rates of new software features in A/B tests. Source

Maybe you think, “Oh, that’s only because these tests are mostly trivial changes like tiny font adjustments.” However, the numbers are similar for high-stakes experiments, such as trials for new drugs. Only 10–15% of drugs in Phase I trials are eventually approved by the FDA.

Source

It’s harder to estimate success rates in science. The problem is that we can only read about scientific results that get published (either in journals or preprints). As a result, most failures go unrecorded.

However, success rates in science also seem low. A recent study by Christopher Allen and David M. A. Mehler measured the success rate in a sample of registered reports. With registered reports, the paper is peer-reviewed before the results are known. The report is then conditionally accepted for publication, regardless of the actual findings.

Allen and Mehler found that, for registered reports, more than half of the hypotheses fail:

Source

II: Why think about this at all

These data points are all pretty depressing. Most new businesses, drugs, software, and scientific ideas fail. That’s before we start thinking about all the failures in our personal lives, from marriages that end in divorce to diets that don't produce a lasting benefit. Why get ourselves down thinking about this?

Several reasons.

First, good data is good. If you’re considering starting a business, it’s prudent to consider baseline failure rates and plan accordingly.

More importantly, these failure statistics help us calibrate our world models.

For example, if you open LinkedIn, you’ll be bombarded with posts of folks getting promoted, landing a dream job, or closing a multimillion-dollar deal. We all intuitively understand that social media is not an accurate depiction of the world. However, these statistics highlight just how inaccurate it is.

Contrary to social media posts, the reality is that we mostly fail and fail and fail.

For me, this realization is liberating. We failed? That’s okay. Airbnb, big pharma, and all these smart scientists mostly fail, too.

I’m not advocating for recklessness. If we can reduce the likelihood of failure or make failure less costly, we should do that. For instance, there’s deep wisdom in the Silicon Valley adage of “fail fast.” Quick experiments make failure a lot less costly. However, even if we follow all the best practices, our failure rate won’t be zero.

III: You got this, Sisyphus?

At this point, I’m tempted to conclude with something like, “Yes, failure is common, but each failure brings us one step closer to success.” Finish on a positive note, you know.

But I feel that wouldn’t be right. If the success probability for a new business is 20%, then even if you founded five startups, you would still have roughly a 33% chance of all of them failing. That’s still a big chance of failure.

There’s also a deeper philosophical question: Suppose you never reach success, whatever your definition. Does that mean it was all in vain?

I’m reminded of Albert Camus’ discussion of Sisyphus. In this age-old myth, Sisyphus is condemned by the gods to roll a rock to the top of a mountain. Once Sisyphus is almost done, the rock inevitably falls back, and Sisyphus must start from scratch.

Imagine you came to Sisyphus and said, “Dude, you got this. Sure, the rock rolled down a couple of times, but you’re bound to get this eventually.” That seems foolish.

Camus takes a very different approach and writes:

The struggle itself toward the heights is enough to fill a man’s heart.

For me, that’s much more inspiring than shallow positive psychology. Failure is extremely common. Even if we follow all the best practices, we’re still destined to fail. However, as long as we cherish the journey, as long as we find meaning in rolling that rock up the mountain, or at least smile at the absurdity of it all, that’s fine.

And if we have done that, then we haven’t really failed.

---

---