The Undoing of Disruption

A little after 8 a.m. on a Tuesday last September, Clayton Christensen walked on stage at the convention center in Orlando. Concert-caliber spotlights circled the room. Tall and slightly stooped, dressed in a dark suit, white shirt, and bright-orange tie, he placed a legal pad on the lectern and looked up at the audience of 5,000. The title of his talk filled two large screens: “A View of the Future of Higher Education.” If there is a cult of Clay, this felt like a house of worship.

The crux of his speech, delivered at Educause, the higher-ed-technology trade show, can be summed up in two words: online learning. As the technology improves, he told the packed room, its impact — and market share — will grow. Christensen has predicted that half the colleges in the United States could face bankruptcy in the next 15 years. He half-jokingly asked the audience to pray for Harvard Business School, where he is a professor, and which has so far not heeded his advice to “disrupt” itself. “The low end today,” he warned, mentioning outfits like the University of Phoenix, “is the leader of tomorrow.”

It’s been 20 years since Christensen introduced the theory of disruptive innovation, and it’s taken hold as a cautionary tale of why good companies, institutions, and business models fail. Simply put, a disruptive innovation is a typically inferior invention that eventually crowds out better products because it’s cheaper, more convenient, and, for many customers, good enough. Then it transforms the entire industry.

Disruption is nothing less than the mechanism at work behind the boom-bust churn of capitalism, according to Christensen. To true believers, it’s something more: a gospel of progress for the Internet age, a democratic force that makes more things — education, air travel, health care — more accessible to more people.

The idea has made Christensen one of the most influential business thinkers in the world. The Economist declared his 1997 book, The Innovator’s Dilemma, among the six best business books published in the past half-century; it’s been cited endlessly. Christensen has twice topped the Thinkers50 ranking, a biannual who’s who of business gurus. The concept of disruption has become commonplace, the rare academic insight that breaks free of the scholarly literature.

Yet for years critics grumbled, with perhaps a touch of envy, that disruption was little more than a buzzword, and that Christensen’s ascent was undeserved. They cheered when, in the summer of 2014, The New Yorker published an essay by Jill Lepore, a Harvard historian, taking him to task for sloppy, credulous work. “The handpicked case study, which is Christensen’s method, is a notoriously weak foundation on which to build a theory,” she wrote. His sources are “often dubious,” she continued, noting an instance when his praise for a company seemed based only on a book published by a foundation established by that company’s founders. “This is akin to calling an actor the greatest talent in a generation after interviewing his publicist,” Lepore wrote. By the last of her 6,000 words, Christensen appeared to be a man who cherry-picks his evidence — and picks poorly.

Harry Lewis, a former dean of Harvard College, expressed delight at seeing a “Harvard snake oil salesman debunked by another Harvard colleague.” David B. Sicilia, a business historian at the University of Maryland at College Park, said, “If Christensen is not able to muster a good defense, this could be the story of the meteoric rise and fall of a Harvard superstar.”

Christensen fired back, calling The New Yorker article a “criminal act of dishonesty.” A Forbes magazine writer wondered whether Lepore’s critique was a misdirected manifestation of academic insecurity: “Was Lepore unconsciously projecting onto Christensen and his theory her own well-justified anxiety, panic, and fear about the ‘disruption’ of the humanities itself at Harvard?”

Christensen’s theory was dinged, but not quite disrupted.

But now a new paper, the most extensive test yet of Christensen’s theory, may prove more difficult to dismiss. Andrew A. King, a professor at the Dartmouth College business school, and Baljir Baatartogtokh, a graduate student at the University of British Columbia, spent two years digging into disruption, interviewing scores of experts, trying to determine whether 77 of Christensen’s own examples conformed to his theory, studies involving big names like Ford, McDonald’s, and Google, along with lesser-known makers of blood-glucose meters and blended plastics. Only a tiny minority — 9 percent — fit Christensen’s criteria. Disruption is real but rare, King and Baatartogtokh conclude, which suggests that it’s at best a marginally useful explanation of how innovation happens.

King says he’s not out to take down Christensen, although that may be what he’s done. Instead, he wants to prove a point. “A theory is like a weed,” King says. “Unless it is pruned back by empirical testing, it will grow to fill any void.”

Disruption’s oft-told origin story goes like this: In 1989, Christensen decided to get a Ph.D. at Harvard Business School. He was 38 and had been a Mormon missionary, a Rhodes Scholar, a White House Fellow, a consultant, and CEO of an industrial-ceramics company. For his dissertation, he set about trying to establish why good companies go bust.

He looked for answers in the disk-drive industry, where failure is fast and frequent. He detected a pattern: Each time a new generation of disk drives hit the market, they were dismissed by the leading manufacturers because the new drives were smaller, offered less storage and lower profit margins, and didn’t meet the needs of existing customers. But before long, those cheaper, newer drives would conquer the market — a process that repeated itself again and again.

In The Innovator’s Dilemma, Christensen transformed this industry-specific insight into a universal theory of innovation. His explanation of why successful companies are vulnerable to disruption was counterintuitive: Because they are well-managed. The CEO of a successful disk-drive manufacturer had every reason to ignore the new, inferior formats. Doing so, however, would contribute to his company’s demise. Take another example: The University of Iowa has good reason to ignore for-profit competitors, but is that wise? How much will online learning improve in the years to come?

Christensen’s book landed just as business executives were eyeing the Internet with both fascination and dread. It was cited admiringly by Steve Jobs and called “absolutely brilliant” by Michael Bloomberg. According to USA Today, it was passed among CEOs “like a Victoria’s Secret catalog in a boys’ locker room.”

In 1997, Andrew Grove, then CEO of Intel, invited Christensen to the company’s headquarters, in Santa Clara, Calif. The following year, Intel introduced the Celeron chip, a microprocessor for cheap PCs. It was a hit. Forbes put Christensen and Grove on its cover and declared Christensen “hotter than a cafe latte among the technoscenti.” Kim B. Clark, dean of the Harvard Business School at the time, recalls the atmosphere of excitement: “Disruption took off like a rocket ship. It was this combination of a great idea perfectly suited to the time and validated by one of the most respected CEOs in the country.” Disruption has been flying high ever since.

Christensen has published nine (almost all co-written) books, though few peer-reviewed studies. He has applied disruption to primary education (Disrupting Class), higher education (The Innovative University), health care (Innovator’s Prescription), even how to lead a more meaningful life (How Will You Measure Your Life?).

“Disrupter” has become an accolade, and an industry has grown up around it. Fortune publishes a 40 under 40 list: “They’re all disrupting — and they’re all just getting started”; Inc. celebrates the “rising stars who are disrupting industries, making millions and building successful companies”; Forbes ranks “young disrupters, innovators, and entrepreneurs”; Vanity Fair has honored the “News Disrupters” shaping the future of media. When the University of Southern California unveiled a new undergraduate program in arts, technology, and innovation, the announcement declared: “The degree is in disruption.”

What happens when a theory is transformed into a globe-spanning explanation of nearly everything? Cue the backlash, which may have reached its apogee last October when The Atlantic announced the creation of an app that will replace “disrupt” with “bullshit” in your web browser.

Christensen has said that he regrets the name he gave his theory, telling the editor of the Harvard Business Review that he never thought people would “flexibly use an idea, twist it, and use it to justify whatever they wanted to do in the first place.”

Still, Christensen has made eccentric claims on his theory’s behalf. Among the areas he’s singled out as ripe for disruption: conflict resolution, the environment, politics, terrorism, and the military. In a co-authored article titled “Disrupting Hell,” he writes:

The article was published by something called the Disruptor Foundation. Christensen is a co-founder. The foundation holds an annual event, the Tribeca Disruptive Innovation Awards. Honorees include the former Fox News opinionator Glenn Beck and the Korean pop star Psy. (Remember “Gangnam Style”?)

In 2003, Christensen co-founded a consulting company, Innosight, now with about 100 employees. In 2007 he opened Rose Park Advisors, which applies the theory of disruption to investment opportunities. (His son Matthew is CEO.) There is also the Clayton Christensen Institute for Disruptive Innovation, a think tank that, according to its website, is “dedicated to improving the world through disruptive innovation.” (His daughter Ann is president.) Christensen commands more than $40,000 per speech.

Kim Clark, the former Harvard Business School dean, is now president of Brigham Young University-Idaho. He first met Christensen, together with Mitt Romney, in an intro economics course at BYU in 1970. They remain close: Christensen is Kim B. Clark Professor of Business Administration at Harvard, and Clark’s tenure at BYU-Idaho is singled out for praise in Christensen’s The Innovative University.

Asked if Christensen is at all to blame for how the theory has been distorted, his old friend thinks it over for a while. “Clay bears some responsibility for having lost control of his idea,” Clark finally says. “But it wasn’t like he sat down with a yellow pad and weighed the pros and cons of, say, going into health care. For him it was like, ‘We’ve got this problem, and I’ve got this idea that can illuminate this problem.’ " After a brief pause, he adds: “Should Clay have stuck to business? There’s an argument for that. But it wouldn’t have been Clay.”

Last year’s meeting of the Academy of Management, the leading association of business-school professors, drew 10,000 scholars from 89 countries to Philadelphia. On the rainy afternoon of the first day, a few dozen attendees gathered in a harshly lit lecture hall at Drexel University to discuss disruption. After 20 years, how does the theory hold up?

Andrew King spoke first. He projected a slide listing example after example of supposedly disruptive innovations culled from Christensen’s work: online stockbrokers, the University of Phoenix, email, Southwest Airlines, Honda motorcycles. He’d begun to interview experts in each industry, he said, and in instance after instance the facts didn’t align with the theory. His research was preliminary, but the implication was clear: Disruption is neither broadly applicable nor very predictive.

The other speakers were less adversarial. One sprinted through an analysis of technological change in the robotics industry. Another talk — “It’s Time for Gene Therapy to Get Disruptive!” — lamented the failure of pharmaceutical companies to adopt cheaper methods for treating inherited diseases like hemophilia. The session concluded with the dean of a new school of entrepreneurship touting her own derring-do: “You study disruptive innovation,” she said. “I’m doing it.”

As King listened, he grew visibly agitated. Jill Lepore’s essay had been mentioned only once, innocuously and in passing. No one had engaged with his findings. As the session came to an end, he sprang from his seat. “This is pathetic!” King yelled, silencing the room. “Somebody wrote an article saying a central theory in our field is wrong, and no one says anything?”

Afterward, at a nearby bar, reaction to King’s outburst was muted, amounting to: Well, we already kinda knew that disruption doesn’t quite hold up. A year later, King remains perplexed. “If they already knew, then why didn’t they publish it? Why didn’t they rein in the theory? Why didn’t they engage in the conversation?” Why, in short, has one of the most influential ideas to emerge from a business school in the past few decades received so little scrutiny from scholars?

What’s so striking about King and Baatartogtokh’s new study is that it evaluates disruption on its own terms, drawing examples of disruption from Christensen’s first two books. The results, published this month in MIT Sloan Management Review, found that while many cases exhibit some elements of the theory, only seven of 77 fit all of the facets. Christensen argues that upstarts displace market leaders “nearly every time,” but King and Baatartogtokh found that this was true in only two-thirds of the cases. And in the vast majority of even those, the circumstances did not track with the theory.

Some of the experts interviewed by King and Baatartogtokh noted several problematic assumptions embedded in the theory. They questioned whether, for instance, the goal of every business and organization is, as Christensen has put it, to “maximize shareholder value.” A higher-education expert, asked about Christensen’s contention that community colleges are a disruptive innovation, said that the “access mission of community colleges often runs counter to what presidents or other leaders might do to cut costs or improve completion outcomes.” The expert added: “That makes it not such a great example for the theory [because] as a mission-driven institution, they are responsible to the public and a higher calling.”

Other experts took issue with Christensen’s claim that organizations under threat by potentially disruptive competitors frequently possess the ability to respond but are too complacent to do so. For example, Christensen has described Concord Law School, an online institution owned by Kaplan University, as a disruptive force in legal education. He’s also criticized the weak response of more-established law schools. But as King and Baatartogtokh argue, those law schools are hamstrung by the American Bar Association’s restriction on the number of hours of online courses that law schools can offer without losing their accreditation.

“Following simple theories or using quick analogies may provide a sense of certainty,” King and Baatartogtokh write, “but they are no substitutes for careful, fundamental analysis of the nature of competition and the sources of competitive advantage.”

Christensen has said that the “value of a theory is assessed by its predictive power.” But King and Baatartogtokh argue that Christensen’s belief that disruptive innovations almost always win has led him to make poor predictions. They cite a 1995 article in Harvard Business Review in which Christensen and a co-author predicted that the 1.8-inch disk drive was poised to upend the industry. An executive at Conner Peripherals, a hard-drive manufacturer, wrote a letter to the editor explaining why Christensen was wrong. Christensen said the executive was making “exactly the mistake we warn against.” That executive has been proved right: 2.5-inch and 3.5-inch drives dominate the industry, while the 1.8-inch is no longer produced. In 2003, Christensen predicted that ultrasound would “disrupt” radiation imaging, but the opposite has occurred. He also had a wrongheaded view of the iPhone, as Lepore and others have pointed out. “The prediction of the theory would be that Apple won’t succeed with the iPhone,” Christensen said in 2007. “History speaks pretty loudly on that.”

It was the late 1990s when King first took an interest in disruption. With a co-author, Christopher Tucci of the École Polytechnique Fédérale de Lausanne, in Switzerland, he crunched the same disk-drive data that Christensen used in The Innovator’s Dilemma. They tested several hypotheses about market outcomes, all of which should have been true according to Christensen’s theory. In each instance the theory didn’t hold up.

King and Tucci presented their findings at a conference in 1999. King recalls sitting at a restaurant soon after and a well-known figure in the field approached, shook his hand, and said, “You’re the guy who burst Christensen’s bubble.” But it didn’t turn out that way. “We wrote a couple of papers, which we had to tone down a little bit because of the referees,” says Tucci. The paper — working title: “Wrong. Wrong. Wrong.” — was too polemical, they were told. When it finally appeared in Management Science, in 2002, the article had been smothered in theory and jargon. The published title: “Incumbent Entry Into New Market Niches: The Role of Experience and Managerial Choice in the Creation of Dynamic Capabilities.” As Brent Goldfarb, an associate professor of management at the University of Maryland business school and friend of King, says, “You have to look really hard to realize King and Tucci slaughtered Christensen.”

Around the same time, Stanford University Press published an obscure book with an attention-getting appendix. From Silicon Valley to Singapore: Location and Competitive Advantage in the Hard Disk Drive Industry, by David G. McKendrick, Richard F. Doner, and Stephan Haggard, was almost complete when the authors decided to comment on Christensen’s theory, then just coming into vogue. They pointed out that contrary to his analysis, most leading disk-drive companies, despite being late to adopt new technologies, did not fail, while most start-ups did, despite being early to market with their supposedly disruptive innovations. The authors’ conclusion: “The facts do not support the theory.”

There is a Rorschach-like quality to disruption. The confusion is traceable in part to the vagaries of failure. Goldfarb likens the collapse of a company or business model to the demise of a romantic relationship. “Thousands of behaviors contribute to that outcome,” he says. To nail it down empirically, “you’d have to run experiments in which you go back in time and behave differently and look at different outcomes. Obviously, you can’t do that.” That leaves a lot of room for debate. As Goldfarb puts it, “Christensen can always create a story in which he’s right based on some interpretation of the events.”

If there is someone on the Harvard Business School faculty who rivals Christensen in star power, it’s Michael Porter. So observers took note when, in 2002, he suggested in an interview that disruption is a distorted lens through which to view the world. “If one reads much management literature, one would think that disruptive technologies are coming at us every day,” Porter said. “I think that disruptive technologies that are successful in displacing established leaders are extremely rare.” (Lepore worked in Porter’s office at Harvard in the mid-1980s.)

In the eyes of some, the best test of disruption began on March 10, 2000, when Christensen and a stockbroker from St. Louis launched the Disruptive Growth Fund. The timing was inauspicious. Within a year, the Nasdaq plummeted 50 percent. The Disruptive Growth Fund lost 64 percent and was shuttered by February. Lepore and others cite this as evidence that Christensen’s theory has little of the predictive power he claims for it. As Erwin Danneels, an associate professor of management at the University of South Florida, puts it, “What better way to test a theory than to have its originator (the most knowledgeable person) pick stocks he expects to do well, based on his theory?”

Since its collapse, Christensen has distanced himself from the fund, claiming that he had no role in picking the stocks. “That money was put in the market by somebody who is not Clayton Christensen,” he said last June. That is at odds with previous news coverage, including a June 2001 article in Businessweek: “Nearly all the stocks were chosen by Christensen based on his theory that companies that develop innovative products — ‘disruptive technologies’ — can topple market leaders.” According to filings with the Securities and Exchange Commission, both Christensen and his partner were responsible for “primary day-to-day management of the Fund’s portfolio.” Not until a November 13, 2000, filing was Christensen redefined as a “consultant” who “does not make any portfolio investment decisions.”

Twenty years after the theory of disruptive innovation was born, scholarly criticism of it remains oddly muted. According to Joshua S. Gans, a professor at the University of Toronto’s school of management and author of The Disruption Dilemma, forthcoming from MIT Press, academics have “generally picked and chosen what they liked about Christensen’s account and not delved in more deeply to the phenomenon.”

Meanwhile, Christensen has been embraced by a public eager to hear from the oracle of innovation.

I met Christensen backstage at his Orlando speech. Beset with medical problems, he moves like a man uncomfortable in his own body. “All my joints ache horribly all the time” he said. “Sometimes it’s so intense I can’t think.” He occasionally struggled to find or pronounce the right word, the result of a stroke in 2010. As we walked, people approached to shake his hand, chat, snap a photo. We eventually sat down at a small table in a quiet room.

How does it feel to be on the defensive? “It’s different,” he said. Since the publication of Lepore’s article, even in friendly venues — a Harvard Business Review forum, a tech conference called Disrupt SF, an interview with Business Insider — Christensen has been asked about her essay. His chief line of defense has been to challenge her credibility: “She clearly lied about things,” he told me, without citing specifics.

“I’ve been mad maybe three times in my whole life,” Christensen said. Reading Lepore’s essay was one of those times. He left a note on her office door; they exchanged emails. He even asked her to co-write an article. (She said no.) He asked to meet, but Lepore decided that would be unproductive after Christensen denounced her in an interview. “I’ve since written a couple of emails and gone over once and just waited and waited in front of her office.” He paused. “Christianity says that if you love people who hate you, you’re better off.”

Christensen has traditionally maintained a peculiar stance toward his critics, smothering them with gratitude while never quite engaging their critiques. He likes to say that theory-building “is a process, not an event,” and that the key to strengthening a theory is to find anomalies — phenomena the theory can’t explain. (A hand-carved wooden sign above his office door reads: “Anomalies Wanted.”) Some people find this pose of open-mindedness irksome: What use is criticism if all challenges to disruption are only part of an effort to strengthen the theory?

Asked if he could be convinced that his theory is wrong, Christensen said: “You know the series on TV called Cold Case? When you find a phenomenon that the theory can’t account for, you need to put that in a cold case. After you have three or four cold cases, then some young researcher has to look at the theory and say, ‘There is something fundamentally wrong here.’ "

“So at some point there could be so many anomalies that the theory is undermined?”

“Yeah,” he said.

“But right now the locker of cold cases isn’t full?”

“No,” he said.

Christensen knows that his reputation among some scholars is distinctly lightweight. “I feel it when I walk into a room. No one is saying it to me. But I feel it.”

He is unimpressed with the trend in scholarship toward sophisticated statistical modeling of ever more complicated ideas that are, in his view, increasingly irrelevant to business executives. “Any piece of data, you open it up, and you get this horrific set of estimations, arguing, compromising, fudging. You take all of that subjectivity and you hide it in the numbers,” he said. “Those guys think this is hard data and it’s rigorous.” He shook his head. “They’ve hijacked rigorousness.”

Andrew King and Clayton Christensen, who have known each other since graduate school, are in many ways opposites: King is sarcastic, mischievous, and self-assured, while Christensen is earnest and self-deprecating — he refers to his books as “my junk” — and has a reputation as a sage-like figure who combines the intellectual prestige of Harvard with the self-help bromides of a motivational speaker. King is a number cruncher, while Christensen is a storyteller.

They met this spring in Christensen’s office. King requested the meeting to discuss his preliminary findings. He didn’t want to blindside Christensen. They agreed to keep the details of their conversation private. King will say only that it wasn’t argumentative. Christensen declined an invitation to respond in MIT Sloan Management Review, where he sits on the editorial advisory board. His assistant said he wasn’t available to comment on the King-Baatartogtokh study.

That paper, titled “How Useful Is the Theory of Disruptive Innovation?,” does not take up a more far-reaching question: Why has a mostly untested theory persisted and proliferated for 20 years?

“That is a bigger issue,” King says. “There is widespread confusion about proper epistemology: What is the right way of knowing? What role do scholars play in ensuring that what gets promulgated and taught has strong evidence behind it? That problem goes far beyond Christensen.”

How far beyond is suggested in a forthcoming paper by King and Brent Goldfarb. They looked at 300 studies published in top management journals from 2003 to 2012. According to their analysis, forthcoming in Strategic Management Journal, around one-third of the studies have false or inflated findings. The problem isn’t outright deceit, they argue, but the tendency — known as apophenia — to detect seemingly meaningful patterns in random data.

Goldfarb sees apophenia at play in disruption. “Christensen saw a pattern and created a theory around what he perceived, and everything he saw afterward fit that pattern, even when it did not,” he says. “All the evidence suggests that Christensen genuinely believes his theory. All the evidence also suggests that he doesn’t know how to reform his theory in the face of new evidence.”

Goldfarb is quiet for a moment. Then he says, “That’s really hard to do if you have so much at stake. And he does.”

Evan Goldstein is editor of The Chronicle Review.