What happens when politicians paint by the numbers? ... when smart people use advanced statistical formulas to cut vital public services based on academic assumptions? It's a great day to ask these questions, as President Obama and his staff are trying to contextualize news that less money under two Homeland Security grant programs will be flowing to New York City. (Overall, New York's getting more money.) Joe Flood has written a book about the city's fetish for efficiency in the 1970s, and how its overreliance on smart guys and computer formulas turned out be a disaster, especially when it came to the withdrawal of fire protection from poorer neighborhoods.
Marc Ambinder: Your thesis here is that the best and the brightest and their technocratic ways failed the City of the New York in ways that are still being felt today.
Flood: One of the big appeals of using numbers to understand complex problems is getting counterintuitive results, which by definition go against common sense. After all, why spend all the time and money on a study that will only tell you what you already suspected? And we love surprising, counterintuitive takes on the world--it's one of the reasons we like reading Freakonomics, or Malcolm Gladwell's work or Michael Lewis' Moneyball, where the numbers reveal that the chubby guy who strikes out a lot is actually incredibly valuable to a baseball team.
Those are the kind of results the city hired RAND to produce, and that's what they got. According to the models, they could close busy fire companies in fire-prone areas without much impact on overall service. For the city, that meant saving money, focusing budget cuts in politically weak areas and supposedly not losing much fire protection. That was exactly what everyone wanted to hear, and they ran with it. It just happened to be wrong.
It wasn't the first time it had happened and certainly wasn't the last. Think of Robert McNamara's Pentagon during Vietnam, or Wall Street computer models alchemy that turned no-money-down mortgages into AAA-rated securities. Modelers usually come from a purely scientific background, but things like finance and government services aren't sciences--there's a human element. The models will never be perfect, and people who make and use them need to take a humble approach and second guess their own work. Bill James, one of the heroes of Moneyball, once said something along the lines of, "Any new metric should tell you 80% what you already knew, and 20% what you didn't. Less than 20% and it's not very useful, more than 20% and there's probably something wrong with the numbers." That "more than 20%" part is where RAND got caught. They believed their own studies, the fire department and mayor's office believed in RAND (and in the political usefulness of RAND's findings) and no one bothered to question whether the results were too good to be true.
MA: It makes sense that if fire houses close down in fire prone areas, fires will be more frequent and harder to fight. Why was common sense ignored? What's RAND's explanation today for what happened back then?
Flood: Interviewing the RAND researchers was a really interesting and sometimes frustrating experience. The short answer is that they're convinced that nothing was wrong with the studies. I keep bringing up financial ratings agencies but it's very similar to their explanation of how so many bonds their computers rated AAA ended up failing. When I asked how a model could recommend closing one of the busiest fire companies in the city--in the whole world, really, because nowhere was burning like New York City--they'd cite a litany of equations and talk about how consistent the R-squares were, things like that. In effect, they were saying, "The Emperor is wearing clothes, because our calculations tell us he must be." One model actually says that traffic has absolutely no effect on how quickly a fire engine travels through New York City. Firefighters just laugh and shake their heads when I mention that. But I couldn't get anyone from RAND to admit that there might have been something wrong with that model.
MA: Do you mean to indict the Bloomberg administration for trying to be good managers? Are you arguing that a certain amount of flexibility must be built in to data-driven standards and benchmarks?
Flood: Well I think there are a lot of ways to define "good" managers. We have a tendency to think of "good" managers as self-assured efficiency experts with an Ivy League MBA, a McKinsey pedigree, and a hard drive full of charts, graphs and regression analyses. Some good managers are like this. But sometimes a great manager is a gruff fire captain who speaks in expletive-laced Brooklynese. For a mayor or anyone else in a position of oversight, it's all about recognizing that different people, with different expertise, styles, and approaches, work in different situations, balancing experience and street-level knowledge with analytical ability. Over the last ten years New York City made an enormous bet--a bet that record Wall Street profits would keep coming, that the need for government-subsidized office space and luxury apartment buildings would keep growing, that the good times would keep rolling, because all the studies and numbers from the management consultants and development experts said they would. So the city spent billions of dollars on new stadiums for the Mets, Yankees and soon-to-be Brooklyn Nets, granted massive subsidies for new headquarters for investment banks so poorly run they had to be bailed out. New York tried to spend billions more to bring the Olympics here based on some studies that said it would help the economy. Ask the people of Greece how that's working out. (The Olympic campaign, by the way, was spearheaded by Dan Doctoroff, who became Bloomberg's right-hand man, and by Jay Kriegel, the chief of staff to Mayor John Lindsay and a big part of bringing RAND and its style of problem solving to City Hall in the 1960s and 70s.)
Now New York is spending billions of dollars every year in debt service for these big projects, and losing just as much in real estate tax abatements. In the meantime, thousands of teachers are about to be laid off, and with 11 attempted terrorist attacks on New York City since 9/11, police budgets are being slashed and 20 fire companies are about to be closed. Don't get me wrong, I think Bloomberg has done a lot of great things for this city and he's been willing to make politically unpopular decisions to get the city budget in line. But part of being a good manager is not risking huge sums of money out of overconfidence in the numbers. And I wonder whether data-drive studies aren't sometimes a political tactic for pushing through whatever initiatives the mayor and his staff want. Take the big Atlantic Yards stadium project for the Nets in downtown Brooklyn, which is deeply unpopular with the surrounding community and will require tearing up homes, businesses and roads in what's already one of the busiest, most economically successful parts of Brooklyn. The argument for the project was some early studies indicating that it would create jobs and bring in a lot of tax revenue--studies where it looks like some of the numbers may have been fudged, by the way. But the latest numbers from the Independent Budget Office say the project will probably lose money for the city. The Bloomberg credo is supposed to be that if you can't make your case with numbers, you don't have much of a case. If you follow that logic, the project should be dropped. But the state and the city want this project built and are moving forward, the latest numbers and economic projections be damned.
Disclosure: Joe is an old pal of mine.
We want to hear what you think about this article. Submit a letter to the editor or write to firstname.lastname@example.org.