Predictive Value

2015.07.22   prev     next

STEVE Jobs came back to Apple in 1997. Bringing him back was a near-bankruptcy, last-ditch, desperation move. He immediately set about getting Apple away from death’s door, then improving its constitution over time. And it worked: Year after year, for the past eighteen years, Apple’s health steadily advanced, leading to today’s supreme fitness. Sadly, Jobs’s own health did not hold out, and four years ago he stepped down, before dying several weeks later. But he knew for years that his end was approaching, and so put a lot of effort into ensuring that Apple would do well without him. And indeed the past four years have seen only a smooth continuation of Apple’s growth.

We have an old saying at Apple. It’s: “The best way to predict the future is to invent it.” — Phil Schiller

Whether by revenue, profits, assets, or accounts, Apple’s last eighteen years look like nothing so much as a simple, exponential rocket launch. Apple currently is setting new definitions of corporate vigor: Last holiday saw the highest quarterly profits of any company, in any business, in any quarter, ever, and Apple looks on track to set the all-time record for annual profits by any company in any business. And while Apple’s stock price has been a wild ride (as stock prices generally are), it is nonetheless roughly fifty times as valuable as it was in ’97.

Coincidentally, 1997 is also the year that Harvard business professor Clayton Christensen rose to fame with his best-selling book, The Innovator’s Dilemma, which launched an eighteen-year career in promoting his “disruption theory.” Christensen’s theory is multi-faceted — perhaps too multi-faceted; see my previous comments on his “drawers” metaphor — and to the extent that its claims can be pinned down, fellow Harvard professor Jill Lepore has written the definitive article about the problems with that. Here, I would like to focus on the could-have-been scenario. How would we view Christensen and his theory if he had taken a different attitude toward what is now the most successful corporation on Earth?

The Pro

Imagine that for the past eighteen years, whenever Christensen had been invited to comment on a new product from Apple, he had said something to the effect that, “I think this product will be enormously successful. I think it will seriously disrupt entrenched companies in its field, even companies that haven’t, up to this point, been thought of as Apple competitors. My theory strongly indicates this.” While a few times he might have been wrong with such a prediction (but only for products like Ping that were quickly forgotten), most of the time he would have been extremely correct.

Also imagine that many times over the ’97-to-’15 stretch, he had said something like, “A few years from now, Apple will be doing substantially better than it’s doing now.”

To which the typical interviewer might incredulously ask, “Really? Because they’re doing unbelievably well now. They’re going to be doing even better in a few years?”

“My theory predicts that they will.”

“A lot of tech analysts are saying that Apple has peaked, that it can’t go any higher, that it can only go down from here.”

“Yep; they’re wrong.”

After eighteen years of statements like that, Christensen today would look like a super genius. I would say he’s a super genius. I would believe it. I might be right.

People would be tripping all over themselves to give their money to Christensen to invest, and to ask what he thinks about anything and everything. Of course, he wouldn’t need any of those people’s money if he had scraped together as much as possible in ’97 and invested it in Apple (plus some ongoing investment in the subsequent years) — today he would be independently wealthy many times over.

The Con

Of course, none of that happened. I’m not aware that Christensen ever invested in Apple (or any other company that did very well), but I understand he makes a reasonably comfortable living from his Harvard salary, book royalties, the lecture circuit, and the like. Until Lepore’s New Yorker article, he got to say what he wanted about business theory pretty much unopposed — as Lepore herself notes:

Most big ideas have loud critics. Not disruption. Disruptive innovation as the explanation for how change happens has been subject to little serious criticism ...

Which is odd, because Christensen has spent the past eighteen years saying things about Apple that are almost the complete opposite of my above-described could-have-been: To take his long-running opinion as gospel, one would have thought that Apple’s products would be flops at worst, or tepid, temporary successes at best, and that any given year of the last eighteen was probably Apple’s lucky-run peak, to be followed swiftly with failure and downfall. Apple’s business model is in trouble; Apple’s market situation is cause for worry; Apple needs our prayers; maybe Apple can figure out a way to survive; is there hope for Apple? Etc.

Despite all that, his reputation held until Lepore’s exposé just a year ago, and even since hasn’t seen any other high-publication criticism.

Prediction

Christensen has made a writing/teaching career out of his theory of disruptive innovation. In case after case, he shows what happened to some particular companies competing in some particular business, during some particular range of years. And because these descriptions have some obvious similarities, they seem to comprise a coherent theory, with predictive value. They seem to be explaining why the things that happened had to happen that way.

The importance of this predictive value is not lost on Christensen’s main disciple, Medallia’s James Allworth (Exponent #8):

There’s an implicit suggestion in [Lepore’s] article that [Christensen’s theory is] not predictive. I’m not entirely sure that— in fact, that’s probably framing it too gently — I believe disruption is predictive. And it’s predictive because — how is the best way to describe this — it’s predictive because people are predictable. It’s predictive the same way that capitalism is— the way people behave in a capitalist society, by and large, is predictable. You put people inside a large organization with a profit motive, particularly if it’s a successful organization, and there are incentives for them to do certain things.

And those patterns play out time and time again. And that’s why disruption is— I find it such a useful frame, or useful lens to look out into the future because when you see that pattern, and you understand the reason for it, like— and that’s what good theory does: It drives to the reason why something happens. It’s not corellative; it’s causal. It’s like, what causes what to happen and why. When you see the pattern and you understand why, then it’s actually a very useful mechanism to be able to make— to have insight on the future. Whether you’re outside a company looking in, like we are in a lot of instances, or whether you’re a manager inside a company deciding what action to take in the future.

Retrodiction is the activity of showing that a theory could have predicted events that already have occurred, if that theory had been applied before those events occurred. Christensen’s Innovator’s Dilemma performed several acts of retrodiction, not prediction.

A good theory should be able to retrodict, yes, but it must also be able to predict events that have not yet occurred. This is critical, because successful retrodiction might be based on selective or distortive application of the theory to make it retrodict events that the theory’s practitioner already knows to have happened. But to predict events that have yet to occur (or not occur), the practitioner must rely entirely on the true powers of the theory. For all the Lepore-exposed faults of his numerous retrodictions, we would happily forgive them if the theorizer was making successful predictions. So — besides his eighteen years of Apple goofs, what else has Christensen tried to predict?

A year-and-a-half ago on FaceBook, Christensen looked forward a nice, round five years into the future:

I predicted that 50% of all high school classes will be taught online by 2019. What do you think?

His target year now is just three-and-a-half years away, so we don’t have too terribly long to wait to find out just how laser-accurate this prediction proves to be.

Although Christensen has made this and other long-term predictions that conveniently can’t be verified any time soon (e.g. half of U.S. universities will be bankrupt fifteen years from 2013), he thankfully has graced us with a few attempts to predict the near future — a few months or a few years out. That should be easier, not harder, yet oddly enough, he seems to fall flat on his face when he attempts it. Lepore reminds us of his famous prediction of iPhone failure (which I’ve already talked enough about, last November). She also tells us about his investment venture:

The theory of disruption is meant to be predictive. On March 10, 2000, Christensen launched a $3.8-million Disruptive Growth Fund, which he managed with Neil Eisner, a broker in St. Louis. Christensen drew on his theory to select stocks. Less than a year later, the fund was quietly liquidated: during a stretch of time when the Nasdaq lost fifty per cent of its value, the Disruptive Growth Fund lost sixty-four per cent.

Lepore makes a point of mentioning that the Nasdaq lost 50% of its value, but the DGF lost substantially more than that. The implication is that perhaps it would be unfair to blame Christensen for the majority of his fund’s losses, just the additional portion that went beyond the overall Nasdaq’s collapse.

But then, what if, while the Nasdaq was falling 50%, Christensen’s DGF had lost only 40%? If I was a DGF investor, I would still very much wish I had never heard of Christensen and his DGF, and had instead kept my money in an ordinary savings account. A theory that purports to guide investors had better be able to tell a good investment from a bad one — not a bad investment from a worse one. Couldn’t Christensen have told his would-be partners, “This doesn’t feel like a good time to be playing the tech market. I’m going to delay my DGF for a year or two and see how things look then. Stay in touch.”

As damning as the failure of the DGF is, an even more damning observation is that since he shut it down — fifteen years ago — Christensen apparently has not started a new one. Why not figure out what went wrong with the theory the first time, fix it, then start a new fund based on the newly corrected theory? Skittish investors? So what; invest your own money. Or just pretend-play the market with a fake-but-public account (I’m sure Eisner would be happy to oblige) until you’re confident it’s work­ing?

No word of any new fund in fifteen years means that Christensen knows his theory doesn’t have predictive value. And if it can’t predict anything that would help investors, how can it predict anything that would help companies, or businesspeople? Is his theory good for anything but “explaining” things that have already happened? Understanding the past implies at least some predictive ability of the future. If a seeming explanation of the past imparts no predictive powers for the future, then the explanation is an illusion. It’s a way to make people feel good, to give them a warm fuzzy from seeming to understand why things happened the way they did. This good feeling manifests from the appearance of security: If I understand why the past had to happen the way it did, then I might be able to dodge calamity, and steer toward prosperity, in the future.

Being the Theory

How could a theory seem to explain the past so well, yet perform so badly at seeing even the very near future? Let’s suppose that Christensen didn’t really have a theory at all (or not a correct one). How could he make it appear that he did? A few techniques can be highly effective:

• cherry picking — Discuss cases that support the theory, while ignoring cases that don’t.
• distortive interpretation — Misdescribe the cases under study so that they appear to support the theory, when in reality they either don’t support it or actually refute it.
• flexible explanation — Make the theory so flexible that, if applied correctly, it can be used to “explain” whatever happens.

Much harder to detect than any one of the above three strategies would be a combination of two or even all three of them. If the theory has just enough flexibility, then only a little bit of cherry picking and distortion would be necessary to round out the theory’s apparent compatibility with known events of the past.

Do I think that Christensen is purposely lying to cover up his lack of a theory? Perhaps not. Another possibility is that he really believes his theory, but thinks that some degree of deceit is necessary to promote it, to make it sufficiently convincing. Yet another possibility is that he really has no idea what he’s doing: He genuinely believes that his theory is true, and doesn’t even think he is engaging in any kind of subterfuge to convince others. He himself might be blissfully unaware of the combination of deceptive techniques at work in his own practice.

Lepore’s best-selling book, The Secret History of Wonder Woman, tells the true story of William M. Marston, a Harvard PhD and university professor who, many years before creating Wonder Woman, invented the lie detector (today professionally called the “polygraph”). The detector seemed to perform spectacularly well in his hands, but when others could not corroborate his results, he was discredited and forced out of academia. In subsequent years when reporters and other visitors asked about his lie detector, he said, “You’re looking right at it, or at him.” (chapter 19)

When people asked Marston, “Where’s the lie detector?” he liked to say, “I’m the lie detector!” (chapter 20)

Christensen and his defenders variously described Lepore’s critique of disruption theory as “personal,” “strangely personal,” a “personal attack,” a “very personal attack,” critical in a “mean way,” and containing “meanness.” This despite the fact that her article said nothing of Christensen’s personal history, his relationships, his religion, his personality, his demeanor, his style, or his appearance. It concerned only the very public phenomenon of Christensen’s theory, and the specific claims of that theory. So why the personal reaction?

Simple: When the person is the theory, criticism of the theory feels like criticism of the person. If in fact there’s no real, objectively usable theory, then an attack on the theory is, in a way, an attack on the person. Clay Christensen doesn’t have a business theory. He is the theory.

Update 2015.08.05 — From Jacquie McNish’s & Sean Silcoff’s Losing the Signal — The Untold Story Behind the Extraordinary Rise and Spectacular Fall of BlackBerry (ch. 15):

[In 2010, co-CEO Mike] Lazaradis had a blueprint inspired by Clayton Christensen’s acclaimed 1997 management book, The Innovator’s Dilemma. The Harvard professor argued that for established companies to succeed against disruptive competitors, they had to empower small, cloistered teams. These autonomous groups, unsullied by the parent company’s set ways, would develop disruptive technologies of their own and could eventually subsume other parts of the organization. It was tumultuous but necessary to stay at the forefront of innovation.

I heard that worked really well.

Update 2015.09.15 — MIT study of Christensen’s theory of disruptive innovation finds that “the theory is not very explanatory, it’s not very predictive, and it can only be narrowly applied.”

Update 2016.02.14 — “not to mention a correct one” changed to “or not a correct one”

Update 2017.07.04 — Schiller quote added

See also:
The Old-Fashioned Way
&
Apple Paves the Way For Apple
&
iPhone 2013 Score Card
&
Disremembering Microsoft
&
What Was Christensen Thinking?
&
Four Analysts
&
Remember the iPod Killers?
&
The Innovator’s Victory
&
Answering the Toughest Question About Disruption Theory
&
Predictive Value
&
It’s Not A Criticism, It’s A Fact
&
Vivek Wadhwa, Scamster Bitcoin Doomsayer
&
Judos vs. Pin Place
&
To the Bitter End

prev     next