Guest post by Jim Manzi, founder and Chairman of Applied Predictive Technologies, and the author of Uncontrolled: The Surprising Payoff of Trial-and-Error for Business, Politics and Society.
I often criticize social scientists for making overly-aggressive claims for understanding causality in complex systems by building regression and other pattern-finding models. This is not evidence of some unique weakness of social scientists The same thing happens in business all the time, but business analyses tend not to be published for obvious reasons. A good example of one that has been published is in the current Harvard Business Review. This matters a lot, because HBR holds a unique position as the most important serious business publication in America.
Anne Marie Knott, professor of strategy at Washington University's Olin Business School, has written an article called "The Trillion-Dollar R&D Fix." The article proposes a new measurement of R&D effectiveness: RQ. In her words, RQ is a measure of "how effective your company is at R&D."
What is so striking to me about this article is how unvarnished Knott is in claiming that she has discovered a tool to do exactly what I say is so hard: make useful, reliable and non-obvious predictions for the effect of interventions in social systems. She writes that "Using standard regression analysis, the calculation tells us in a very precise way how productive each of the inputs is in generating output. It tells us, for instance, how much a 1% increase in R&D spending would increase a firm's revenue." Knott asserts that RQ allows the management if a company "to see how changes in your R&D expenditure affect the bottom line and, most important, your company's market value." She even names names: providing a table of what she thinks each of the top 20 public corporations in America should have spent on R&D, and how much more each would be worth if they followed her recommendations.
For example, Knott claims that she knows that Apple would have maximized its market value by spending $9.5 billion on R&D in 2010 They actually spent $1.8 billion. That's a fairly incredible claim. She thinks that what is generally conceded to be a management team that is pretty savvy about innovation underspent on R&D by more than 400 percent -- Apple ought to have quintupled its R&D spending in 2010. As another example, Knott claims that Dow Chemical could have roughly doubled its total market capitalization by increasing its R&D spending by 10 percent. That's a lot of money for them to leave on the table. And a very easy fix.
Knott claims that if just the top 20 American corporations had followed her recommendations, they would have collectively increased their market capitalization by more than $1 trillion. Consider this assertion for a moment The current total market capitalization of the top 20 U.S. public companies is a little over $4 trillion. Knott claims that she has outsmarted the entire system of management teams, investors, equity analysts, hedge funds, large-scale private equity firms and everyone else who is trying to change management practices to increase share price, and knows how to increase the total value of the most-closely followed companies in the world by almost 25 percent....by building a regression model using publicly-available data.
If you could rely on Knott's predictions, you could raise capital, buy these companies, change R&D spending in line with her model, and then sell them again at an enormous profit. You could start with Dow, because you know how to double its share price.
Maybe Knott has discovered an incredible, remediable market inefficiency, and somebody is about to get very, very rich. Or maybe there's a problem with her model.
The HBR article describes the calculation of RQ conceptually, and references a journal article (ungated) in which Knott describes the mechanics of it. In it, effectiveness in managing R&D is explicitly analogized to IQ, and refers to what is normally termed among businesspeople a competence for managing the R&D function. Knott contrasts her theory with an existing body of research on the topic:
Theories of innovation typically assume that firm R&D behavior is endogenously determined by industry conditions. If all firms in an industry share these conditions, and behave optimally then in equilibrium all firms should have identical R&D investment. Accordingly increases in R&D beyond the optimum should decrease market value. However the empirical record consistently demonstrates the opposite. Firms with higher R&D investment have higher market value.
We proposed that the inconsistency between theory and empirics stems from the assumption of homogenous firms. If instead firms have heterogeneous R&D elasticities (IQ), then a) the optimal levels of investment will differ across firms (firms with higher IQ invest more), and b) the market value per dollar of investment will differ across firms (firms with higher IQ have higher value per R&D dollar). This gives the empirical finding of increasing market value for increased R&D spending theoretical grounding: It is not that investing more in R&D increases market value, it is that higher IQ yields both higher returns (and market value) and therefore stimulates greater investment.
I'm sure she's correct that there are a lot of academic analyses that assume all firms have equal competence in R&D. Such studies may have some utility for some purposes. But the idea that competence in research management varies across firms is a belief that is universally held among relevant senior executives. I mean this literally. I doubt you could find three COO / CEO level executives among all large public U.S. companies who spend significant amounts on research that disagree.