African-Americans, Prop 8, and the beguiling art of polling

Before we get started on this, I want to say that this a layman's attempt to discuss a really complicated issue. Folks who know this stuff are more than welcome to point out the flaws. Alright, let's go..

So, in between bouts of Warcraft, and a family trip to Dive Bar to watch the playoffs, I talked to some folks who were smarter than me in regards to polling, Prop 8 and black folks. The biggest takeaway was that journalists/bloggers/writers/pundits need to be a lot more careful when deploying polling data. We should be even more careful when deploying numbers about minority communities, if only because of the sample sizes. And we should be especially careful about drawing broad conclusions based on one exit poll.

There are many reasons to doubt that Prop 8 garnered 70 percent approval in the black community, and there are many more reasons to doubt that African-Americans were the ones who killed the gay marriage in California. The first thing to understand is the methodology at work. Exit Pollsters select a group of random precincts which reflect, as accurately as possible, the demographics of the state. Then they approach every nth voter as they leave the polls. Then they do some phone polling to account for absentee balloting. Then, as the returns come in, they weight the results to match the actual vote count, as well as the actual demographics of the polling area.

Almost every step introduces the possibility of error. For instance, census data is the only reliable means for understanding the racial makeup of a particular precinct. But how do you know that the racial makeup matches the makeup of eligible voters? Of registered voters? Thus it's possible to get an unrepresentative precinct.

"The precinct may or may not be representative," said Patrick Egan, NYU professor and author of the latest Prop 8 study. "A pure sample picks random people out out of hat. But an exit poll is like picking a selection of people and putting into different hats, and then trying to pick out of those hats."

Then there's actual execution. David Moore, former VP of Gallup, that refusal rates for exit polls can run as high as 50 percent.  "When you have small sample sizes in the case of minorities, exit polls aren't very good predictors," said Moore. "There are so many people who refuse to participate, that you have a response rate problem--and then people who do respond are different than those who don't."

Polling, in general, is subject to some margin of error. But as sample size decreases, the chance of error increases. The CNN Exit Poll estimated that 10 percent of California's vote was African-American. Most professionals I talked to this weekend thought that was really high number. (African-Americans make up 7 percent of California's population--and they tend to be a younger 7 percent, when compare to whites. A ten percent share would mean they were overrepresented by somewhere around 30 percent.) But let's say that 10 percent number is correct--it means that the (weighted?) sample size for African-Americans was about 224 people.

Now, these problems are generally true of all exit polls, and polling in general. I'm not writing this to impugn anyone's credibility. Opinion pollsters know that these are the hazards built into an imperfect science. So how do they get around them? Typically you look at a group of polls (as a lot of us did during this election) and you try to get some range of where things are. That's exactly what did not happen in the reporting around Prop 8. All of our analysis and blogging was based on a single poll. Think of it this way--What if we had just one poll for every state during the general, as opposed to people like Nate Silver looking at multiple polls? How different would our impressions be?

Here's what gives me confidence in this latest study. Patrick Egan and Ken Sherrill didn't simply do another poll. Well, they did commission DBR to do another poll, but they didn't stop there. They compared DBR's poll to three other polls taken close to and after the election, and the exit poll. Then after that, they used Goodman's regression to analyze census data and precinct returns. Then after that, they used Gary King's EZI software in much the same way. In other words, instead of employing a single method (an exit poll) to analyze the Prop 8 vote, they used several.

With two exceptions, the polling put black support for Prop 8 in the 50 percent range. The two exceptions were SUSA poll which put at black support at an implausible 41 percent (Oh, if only it were true.) and the exit poll which put black support at 70 percent. Moreover the two population analysis put black support in the high 50s range also. In other words, the results corroborated each other, making them imperfect, but highly likely.

There's more. Last week I noted that black support for Prop 8, among weekly church attendees, was actually lower than other ethnic groups. Timothy Kincaid noted, it was higher among those who don't attend church weekly. Doesn't that mean, as Timothy argued, that religion really didn't change anything? Well no. First, when you start slicing the pie to that level, you're down to a really small sample size, and in fact, the Support for Prop 8 among church-goers only spans about eight points. Second, those who "don't attend church weekly" is a very large group of people ranging from those who go to church monthly to militant atheists. One guess as to which group is more represented among blacks.

Lastly, Egan argues--and I agree--that blacks and Latino Democrats are, on the whole, more socially conservative. They share a lot of the social beliefs of conservative whites, and that's reflected in proposition votes. But unlike conservative whites, blacks and Latinos have other issues that exert a strong pull. Latinos, for instance, may regard the GOP as anti-immigrant. Blacks may regard the GOP as racist. Both may regard the GOP as anti-worker, or anti-poor. Thus you have a group of people who aren't particularly socially liberal, but still ID themselves as liberal or as Dems. Blacks, in the main, may oppose gay marriage--but they aren't going to support a Republican because of it.

I think that last point is at the heart of the outrage over Prop 8--we've really underestimated and simplified what it means to call yourself a liberal or a Democrat. For a lot of black and browns it means pro-common man, but doesn't necessarily mean pro-gay rights. That, for me, is really the lesson in all this data. What we need (the we being socially liberal blacks) are people in the black church, who know the church (that disqualifies yours truly) and can engage other black people in the debate. We have a very powerful incentive to do so--our silence is killing us. But there's also a moral component, an attempt to put us on the right side of history. Gay marriage is coming. Who amongst us will want to tell their grand-kids where they stood?