...as you might expect, a policy turn undertaken during a period of emergency will eventually produce diminishing returns - as Steven Levitt puts it, "the two-millionth criminal imprisoned is likely to impose a much smaller crime burden on society than the first prisoner" - even as it imposes substantial moral costs. And precisely because the tough-on-crime approach was largely vindicated by events, it's extremely difficult for elected officials to walk back from some of the dubious practices that have grown up around it - like, say, the possibly cruel-and-unusual use of long-term solitary confinement.
This political dynamic explains why the chances for effective prison reform probably depend on Nixon-to-China conservatives, who can put the credibility the Right has built up on law and order to good use. (It wouldn't hurt if conservatives were willing to champion some alternative approaches to crime reduction as well.) But they probably also depend on crime rates staying flat, or falling - and in the current downturn that may be too much to hope for.
I'm less certain that the "tough on crime" approach has been "largely vindicated" by events--mostly because I think a large part of the events include the moral costs, and the real costs to communities where alarming numbers of men are under the watch of the state. One should consider the numbers here--blacks make up a third of all drug arrests, and black men are 12 times as likely to be imprisoned on a drug conviction. Four in Five of these arrests were for possession, not sale. Perhaps this is because the drug epidemic has run rampant through black communities, but probably not. The difference in illicit drug usage is slight (9.5 percent of blacks have used illicit substances, 8.2% of whites). Those are the sort of numbers that feed an intense distrust of the justice system in many black communities. I think Ross (though I can't be sure) sees the ends justifying the means. But the means are disproportionately born by people who live far away from those "Nixon to China" conservatives.
This is more than theory for me. Ten years ago, my college friend Prince Jones was followed by a cop from Prince George's county Maryland, into the District, and out into the suburbs of Virginia, where he was going to see his young daughter and girlfriend. The police officer was allegedly looking for a drug dealer--a short man with long dreads. Prince was about 6'3 and wore a low caesar. The officer and Prince ended up in a confrontation, merely yards away from the home of Prince's girlfriend. He produced no badge, just a gun and a claim that he was a cop. Prince didn't believe him (and without a badge, I wouldn't have either) and rammed the guy's car. The cop shot Prince eight times, killing him.