The Geography of Partisan Prejudice
In early March, The Atlantic published a guide to the most—and least—politically open-minded counties in America. Amanda Ripley, Rekha Tenjarla, and Angela Y. He teamed up with PredictWise, a polling and analytics firm, to create a ranking of counties in the U.S. based on partisan prejudice (or what researchers call “affective polarization”).
The results were surprising in several ways, they found. “In general, the most politically intolerant Americans, according to the analysis, tend to be whiter, more highly educated, older, more urban, and more partisan themselves.”
In its analysis, PredictWise used multilevel regression with post-stratification (MRP), a method that has lots of promise in estimating public opinion within states from national survey data. However, this method should be used with great care and attention to detail. In particular, the method has been shown to produce estimates that are highly variable when sample sizes are small, as demonstrated in this 2017 paper by Matthew K. Buttice and Benjamin Highton.
In my view, a substantial issue with the methodology is that the sample size used in this study (N=2,000) indeed seems too small to detect real differences in partisan prejudice between states, much less differences between counties. This might explain why levels of partisan prejudice appear to differ so greatly between counties in North Carolina and South Carolina. It seems highly unlikely that partisan prejudices respect the Carolina border in such a way, yet that is exactly what the results suggest. Consider further that if states are sampled in proportion to population, something like 60 North Carolinians were likely included in this sample. That doesn’t seem nearly enough to make any strong claims about the state as a whole, much less the 100 counties the state is composed of, without making some grand statistical assumptions. Even with a method such as MRP, it’s unlikely there’s enough coverage here to generalize findings to all 3,007 U.S. counties. I’d be interested to see confidence intervals around the estimates of partisan prejudice for both states and counties—I suspect they are quite wide. The data produced here are at best interesting “guesses” derived from small data, and the level of confidence in these results seems largely overstated.