The Known Unknowns of Churn Prediction

By Gabe Weisert September 16, 2020

     “Reports that say that something hasn’t happened are always interesting to me, because as we know, there are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns—the ones we don’t know we don’t know. And if one looks throughout the history of our country and other free countries, it is the latter category that tend to be the difficult ones.

-U.S. Secretary of Defense, February 12, 2002

 

Churn is fundamentally difficult to predict, as Zuora’s Chief Data Scientist notes in the introductory chapter to his excellent new book “Fighting Churn with Data.”  It remains an eternally tricky problem, despite the fact that most subscription companies are drowning in analytics and AI platforms that promise to heroically optimize every business process under the sun. The problem is that subscription businesses are built on subscribers, and those subscribers are humans, and humans can be messy and idiosyncratic on even their best days.

Think about the last time you dropped a subscription service. It was probably not the result of a long process of deliberation and cost-benefit analysis. You probably weren’t assiduously tracking your consumption rates for months and months in order to arrive at a sober, data-based decision to pull the cord. You probably just saw something on your credit card bill that you realized you weren’t using. Or maybe you saw an ad for something better.

Aha, but what if your subscribers are companies instead of people, you might ask? Surely organizations aren’t liable to the same kinds of vagaries and inconsistencies as sorry old individuals? And if you’re seriously asking that question, you’ve probably never worked at a company before. S#%t happens all the time. Zombie SaaS services can linger for months, completely undetected. Companies can get bought. Companies can go under. The list goes on.

These are the known unknowns of churn prediction. These are the scenarios that we know are out there, and are happening all the time, but are still impossible to predict despite all of our fancy analytics platforms. Carl Gold contributes several more known unknowns:

“Consider the following additional pieces of information that would help to predict churn if you knew them, but you probably never will:

  1. A customers’ ability to pay is usually unknown. Equivalently for free (feed or ad-supported) products, the amount of time and attention the user makes available for this type of content overall is unknown.
  2. The switching costs to an alternative service or means of achieving the same end is also very often unknown, as are any alternative solutions the subscriber already employs (including but not limited to competitive products.)
  3. Utility or enjoyment of the service is a fundamentally subjective experience that varies from individual to individual, even under the exact same circumstances. This is especially important for consumer services, for which churn is particularly difficult to predict. Businesses tend to use a more rational cost-benefit analysis when deciding on renewing or cancelling a service, but even this will be influenced by subjective elements.”

And that’s just the stuff that we know that we don’t know about! What about the dreaded unknown unknowns — the stuff that we don’t know that we don’t know? Like, say a global pandemic? Yes, there is historical precedent, but I’m guessing that most boards and management teams didn’t see that one coming.

I grieve to report that the brutal truth on churn doesn’t stop there. If predicting churn is hard, preventing churn is even harder. Why? Because as Gold astutely notes, “In order to prevent churn in a long term and reliable way a service must actually improve either the benefit delivered by the service or reduce the cost incurred from using the service.” You have to constantly improve. You have to keep offering more for less. You have to keep happily surprising your subscribers. You have to win them every day.

But let’s end on a hopeful note. This is just the first chapter, which you can download for free (without an email!) here. The vast majority of this book is aimed at offering concrete, prescriptive ways to help minimize churn. A lot of it requires a data science background to understand, but a lot of it doesn’t. I will be providing my reactions to the latter stuff on a periodic basis.

Fighting Churn With Data,” by Carl Gold. Buy it for your team!