A day in which two big Obama polls have been released (NYT poll here and Pew poll here) seems like a decent enough excuse to revisit the question of the value of political polls. Mark Blumenthal, Ed Kilgore and John Sides (twice!) have all been kind enough to write detailed responses to my original thoughts on the subject. And I'm pretty sure they are winning the argument. Nonetheless, I don't think we're at Waterloo just yet, and I want to respond to some of their points.
This is dredging up an old controversy, so here is a quick recap: I wrote a short piece for the Atlantic a few weeks ago ("Get Rid of Polls") that piece made three basic points: (1) we should want our democratic institutions to operate according to pre-established mechanisms, not random and constant quasi-referenda; (2) Lots of polls are wrong or misleading or insufficient snapshots of "actual" preferences; and (3) Present opinion polls can affect future opinion polls, due to information cascades. (Which offends my romantic sense of how and why opinions should change.) In a follow-up I added a fourth point, which was mostly a desperate attempt at burden-shifting: Even if you disagree with the three points above, what's the affirmative case for polling?
Well, Mark and John and Ed responded to my three points and answered my question and more. So, in no particular order, here's a few thoughts on the issues they raise [looking back over it after writing, I think the debate probably boils down to #'s 4, 6 and 7]:
1. Let me make one big concession at the start. I wouldn't vote for a law than banned polls, and were I dictator of the universe I wouldn't want to outlaw them. (I'm a fan of free speech; I'd get that first amendment tattoo in a heartbeat; etc.) The position that I would feel more comfortable defending is something like "polls are on balance a bad thing." Or, even more milquetoasty: "polls are not the best use of newspaper resources."
2. I hope that goes along way towards addressing the main thrust of Ed Kilgore's argument -- namely, that I am weirdly in favor of reducing the total amount of information that exists in the public sphere. I think it's fair to say, with Ed, that there should be a strong presumption in favor of putting additional information in the public sphere. But I think that's very different from saying the current level of information is optimal or desirable.
3. I think there is an easy extension of points #1 and #2: Producing information isn't free. A dollar that the New York Times spends on a poll is a dollar that it isn't spending on a Baghdad bureau or a congressional beat reporter. Well, that's not really how tradeoffs work. But the general point -- you have to measure something against its opportunity cost -- is one I agree with and one that is relevant here. So one question to consider might be this: Could the resources that the Times and the Post spend on polls be better spent elsewhere?
4. I am happy concede John Sides' empirical point that there's little evidence that politicians are "buffeted to and fro by the winds of the capricious public," as expressed in polls. I am also willing to concede that politicians do in fact use polls "to find out how best to sell their ideas." But I'm not sure what this proves. If politicians become more adept at selling their ideas to the public, does that benefit the public? That sounds like a question with a complicated answer. On the other hand, I feel reasonably confident saying that it is not the purpose of the media, at least as traditionally understood, to provide a consulting service by which politicians can hone their rhetorical skills.
5. John Sides also has a lot of helpful stuff to say about information cascades, but I'm not sure what to make of his conclusion -- namely, that "the role of polls [in influencing future polls and outcomes] is unclear." I agree that it would be difficult to prove how a poll in the present affects a poll in the future. (Or at least I'm not sure how you'd ever tease causation away from correlation.) But it would surprise me if they did not. We all have preferences for all sorts of things that are popular -- neighborhoods, fashions, music and so on -- which will in turn make them more popular. These preferences are endogenous or interdependent or whatever you want to call them. I think it would be strange to discover that political preferences are any different.
(A sidenote: One quality of many situations in which choices are interdependent -- in which my choice depends on your choice and your choice depends on Michael's choice so forth -- is that they have multiple equilibria. The mere fact that we all end up living in neighborhood x or drinking at bar y or supporting candidtate z is not evidence that we are best off doing so.)
6. On the affirmative case for polls: Sides and Blumenthal both point to the value of accountability. in Sides' words, "we should know whether policymakers and policy are acting in accord with public opinion." Ack. I feel a bit dense saying this, but I really don't understand why we should want to know this at all. I am intensely interested in whether or not policy is acting in accord with my preferences. And so, presumably, is everyone else. But I can't figure out why any individual should care about how well policies line up with aggregate preferences. (John or Mark [or someone!]: please be patient and explain this to me once more.)
(One possible exception to my above confusion is this: I might have an individual preference for policies that line up with aggregate preferences. But, you know, I don't. And I don't think most people do, either.)
7. A related point: I am happy to concede, as per #4, that politicians might be influenced by polls. But even if you could prove that polls helped bring political behavior more in line with public opinion, I wouldn't totally know if that were a good or a bad thing. The wooly-headed paternalist in me thinks that the public doesn't always know what's best for itself. And the hard-headed-fan-of-The-Myth-of-the-Rational-Voter in me thinks that the incentive structure in democracy leaves something to be desired. It seems to me that the best case for democracy is a moral one ("people deserve a system that takes their preferences seriously") and not an instrumental one ("democracy will produce the best policies"). So while I'm happy to celebrate elections as a check on political power and a moment of collective expression, I'm less eager to celebrate polls as a tool for relentlessly enforcing the public will. (Of course, the fact that the two sides of the previous sentence are in obvious tension makes me realize that I need to think about this more.)
8. I think the killer point that John and Ed and Mark all make is that there's really no reason for me to believe the world would be better off -- less misleading, more responsive to "actual" preferences, whatever -- if we got rid of polls. There's lots of other disingenuous noise floating around out there. Which is probably true. But I guess I reserve the right to complain about that stuff, too.
Gaak, this was a lot longer than I thought it would be when I started writing.
We want to hear what you think about this article. Submit a letter to the editor or write to firstname.lastname@example.org.