Addicted to Data: How an Obsession With Measuring Can Hurt Businesses

The Gospel of Metrics says, "if you can't measure it, you can't manage it." But what if some companies are measuring themselves to death?

buriy/Shutterstock

Here's one thing I love about plumbers: whenever I hire one, they stick to the plumbing. Not once has a plumber fixed my kitchen sink, only to follow up with a credit card offer. No teaser rates, no plumber points, no "convenience checks." Not even a customer satisfaction survey. They simply do their job and collect their fee. It makes me wish dealing with larger companies were that simple.

Take for example the pre-authorized credit card offers that incessantly arrive in the mail. Every weekend, I spend a few minutes opening, shredding, and recycling the week's accumulated offers. This routine is especially galling because many of the offers come from companies I have a relationship with. As with the plumber, I hire these companies to do a job for me (one that has nothing to do with credit cards). But unlike the plumber, these companies don't seem to understand their role in my life.

Most of us call these unsolicited offers "junk mail." The industry prefers the euphemism "direct mail." Within marketing circles, this kind of tactic is known for being highly measurable. Outside of marketing, it is known for being highly annoying. (I'd suggest that these two attributes are not mutually exclusive.)

Complaining about junk mail is hardly novel. But "Junk Mail Thinking" is not limited to credit card offers. Junk mail thinking is metric-oriented thinking, and it pervades the business world, stemming from an almost religious devotion to measurement. An entire generation of managers has been brought up in the Church of Measurement, whose catechism is: "If you can't measure it, you can't manage it." It seems like an innocent enough idea. But as uncontroversial as it sounds, a dogmatic devotion to measurement can create problems. Those problems begin with a few simple truths:

Some things are easier to measure than others. It is easy to measure how many people respond to a credit card offer. It is much harder to measure the cumulative frustration that these tactics inspire among the thousands who don't respond. But, the fact that something is hard to measure doesn't mean that it isn't real. Unfortunately, we tend to fall back on things that are easy to measure over taking on an initiative that might bring real value to users. And since nothing is easier to measure than income, it's no wonder that customers of measurement-centric companies end up feeling "nickel and dimed." But financial focus isn't the only flaw in the measurement mindset.

Measurement can impose a hidden cost on the customer. In Physics, there's a phenomenon called the Observer Effect. Often confused with the Heisenberg Uncertainty Principle it refers to the idea that observing a phenomenon can change the phenomenon itself. This is also the case with market research. Consider the call to a customer service department that ends with the question: "Are you willing you take a brief survey?" As with junk mail, these questions add friction to the experience, imposing a cost (in this case, time) on the customer. I always wish that these phone surveys ended with a question like "On a scale of 1 to 5, how irritating do you find this survey?" As satisfying as it might be to answer that question, there is a much more significant, systemic issues lurking inside the Church of Measurement.

Measurement culture tends to trade long-term value for short-term gains. The online magazine Salon recently found itself in a situation I call the Measurement Trap. Amid the recession, it began moving away from original reporting, relying instead on news aggregation to generate traffic. Traffic is of course, the lifeblood of an online magazine. But, as is often the case in the Church of Measurement, the short-term metric (traffic) ended up taking a toll on a harder-to-measure, but more important idea: brand value. As Salon editor-in-chief Kerry Lauerman recently told the Nieman Journalism Lab

I remember we had aggregated a Charlie Sheen story, and I saw it tweeted a lot. It wasn't a really interesting essay, just the latest news breaking. I was watching all of our peers -- either before or after us -- tweet the exact same story. I thought, 'This is how it ends. This is grim. We're all just sort of regurgitating the same thing over and over again."

How did Salon escape the Measurement Trap? Tellingly, writes Lauerman, it happened when founder David Talbot, returned to the magazine as CEO, giving the staff free reign to "work longer on stories for greater impact, and publish fewer quick-takes that we know you can consume elsewhere."

It often takes a charismatic leader to shepherd companies out of the Measurement Trap -- or to prevent them from ending up there in the first place. By definition, those of us who question the Church of Measurement often lack the cold, hard facts to back up our case (they're hard to measure). Instead, we must rely on hypotheses and anecdotal evidence. Luckily, these hypotheses are often hard to argue with, to the point of sounding banal. "Happy customers will be repeat customers" is one I'm fond of advocating. And while I may not always have the data to prove it, I think it explains why my plumber has never offered me a credit card.

>