Charles Johnson explains his concerns here. He's particularly right about the kind of proto-fascist love of violence against "the other" that you see pulsating in the writing of, say, Michael Goldfarb or Robert Stacy McCain.
I think proto-fascism is a better term than neo-fascism. Cheney and Bush respected the outer limits of constitutional democracy. They obeyed a Supreme Court ruling that struck down their maximalist views of their own inherent power as the executive branch. They left office after an election. They are not fascists. But they do see the executive branch as a kind of fascist element within a democratic polity, an element that can simply ignore the law or hire lawyers to twist it into meaninglessness, an element that has the inherent power to seize anyone, citizen or non-citizen, in the US or not the US, detain them without due process and torture them, in the name of national security, meaning any government response to "active threats" of terrorism.
This proto-fascist tendency, proven chillingly in the last week as Cheney Republicans like Stephen Hayes called for the torture of the undie-bomber, is what worries me. It is the embrace of raw violence against the defenseless - not within the constraints of just war, but outside all constraints except victory against an 'evil' enemy.
I do think partisanship has clouded conservative eyes on this question. I don't think many on the right have yet absorbed the full ramifications of what Cheney asserted and what the GOP now holds as its view of the power of government - i.e. total power over the individual, to the point of torture, in the name of national security.
That's why, in my judgment, Obama is essential. He is the barrier between us and a form of fascism, imbued with utter moral certainty, that now animates the core of the GOP. Until that core is defeated, real conservatives need to keep their distance from this kind of authoritarian thrill.