We'd probably have more productive conversations if we just agreed that its not hugely useful to ask whether someone like Haley Barbour "is" a "racist," or to reflexively read that accusation into every criticism involving race. Then we could focus more narrowly on what ought to be a relatively uncontroversial proposition: That his misguidedly sanguine view of the Citizen's Councils reflects a lamentable (and perhaps self-serving) ignorance of the uglier aspects of his own state's history, and that we should expect our elected officials to be better informed.
This is exactly right. But I've come to believe that reason this doesn't happen is quite simple--deploying the racist card is not a misunderstanding, it's an act of intellectual dishonesty. There are some conservative and liberal places where writers and commenters genuinely disagree. But there are many more where they are just playing out loyalties.