Just How Green Is the New Green Cities Index?

Part two of a two-part series on a recently announced set of sustainability rankings. Air quality, water, and more.

Earlier this week I reviewed the overall, land use, and transportation components of the new "Green City Index," which rates and ranks 27 cities in the U.S. and Canada on various environmental criteria (and which is sponsored by Siemens). Today I consider the rest of the report.

As noted in the previous post, all ratings and rankings of places with respect to environmental issues are imperfect at best, for the reasons I stated. In this case, you certainly have to wonder about the methodology for a study that classifies Los Angeles among the leaders for air quality (more on that issue below). But such studies, though flawed, still can be useful as starting points for serious discussion about issues.

CO2, energy & buildings

These categories are (or should be) very closely related. Buildings, along with transport, provide a large majority of overall urban carbon emissions, so one would expect CO2 performance to correlate closely with building and transportation performance. In addition, since most of our energy consumption comes from the combustion of fossil fuels, CO2 performance should also correlate closely with energy performance.

That is not what the results suggest, however. Vancouver is the only city to make the top five in all three categories. Otherwise, there was no overlap at all among the top five in CO2 and buildings. Los Angeles was the only city other than Vancouver to make the top five in both CO2 and energy, and San Francisco was the only city other than Vancouver to make the top five in both energy and buildings.

Here's a likely reason why: Although the CO2 scoring considered carbon emissions per unit of GDP and per person (both valid measures), the "energy" scoring looked only at electricity consumption as a quantitative measure, leaving out natural gas and vehicle fuels (both carbon emitters) among sources of energy. The quantitative "building" scoring, for its part, was based only on the number of LEED-certified buildings per capita. Given that certified buildings remain a small fraction of overall buildings in all cities, this was more an indication of policy or aspiration than of environmental performance. In addition, two-thirds of the "building" ratings were based on qualitative evaluations of policy measures. (Subjective policy evaluations composed a third of the scores for CO2 and energy.)

So I suggest taking some grains of salt, as they say, with these results:

• CO2 top five: Vancouver, Miami, New York City, Los Angeles, Ottawa.
• Energy top five: Denver, Boston, San Francisco, Vancouver, Los Angeles/Toronto (tie).
• Buildings top five: Seattle, San Francisco, Washington, Pittsburgh, Vancouver.

I am very surprised to see Miami and Los Angeles in the top five for CO2, considering the amount of transportation emissions generated in each. The report doesn't explain L.A.'s high rating other than with respect to policy, but notes that its CO2 performance has improved further since 2002, when the study's comparative CO2 data were collected. Miami's good performance is attributed to a lack of industry in the city and reliance on newer, more efficient power plants than used by other cities.

Among global comparisons, the report states that the 22 U.S. cities in the study, on average, emit approximately twice as much CO2 per person as the five Canadian cities in the study, as well as over three times as much CO2 as European cities, and approximately four times as much as Asian cities studied in similar Siemens-sponsored research.

Water

This category measured water consumption per capita (important, but an order of magnitude more important in arid regions), "share of non-revenue water shortage leakages," and subjective evaluations with respect to policies to protect a city's "main water sources" and stormwater management. 

Maybe they were hampered by a lack of data sources, but this looks like a particularly weak evaluation to me. Wouldn't it be more relevant to measure drinking water purity or pollutant loads in city waterways? Wouldn't water consumption be more relevant if expressed in ratio to levels of precipitation?

• Top five: Calgary, Boston, New York City, Minneapolis, San Francisco.
• Bottom dwellers: Detroit, Montreal, Cleveland, Washington, Philadelphia.

New York's strong showing is attributed to very low water consumption per capita, which I suspect is related to far fewer lawns and gardens to irrigate per capita. (The report's authors relate it solely to policy.) Calgary scored well in all categories. Among the losers, I was curious about Philadelphia, given that the city is becoming an "industry leader" (along with Seattle) in the U.S. for green stormwater management. This gets a mention in the report but must not have counted for much; the biggest negative seems to be aging water infrastructure that generates a relatively high number of leaks.

The report notes that residents of the 27 North American cities studied consume, on average, about twice as much water as do residents of other parts of the world. 

Waste

This appears to be another weakly measured category, with the only quantitative measurement being the percentage of municipal solid waste recycled. Since municipal waste is but a fraction of overall waste, one wonders how indicative this is of a city's true performance. (I'll grant that it is relevant as an indication of policy.) The other half of the scoring was based on subjective evaluation of waste management policy.

• Top five: San Francisco, Seattle, Los Angeles, Toronto, Minneapolis.
• Bottom five: Detroit, Cleveland, Pittsburgh, St. Louis, Miami.

San Francisco was awarded a perfect score of 100, with a very high rate (77 percent) of municipal waste recycling, along with strong policy that mandates recycling and composting.  Detroit was given zero points.

Air

On the surface, it is hard to quarrel with the quantitative measurements used:  emissions per capita of nitrogen oxides, sulfur dioxide, and fine particulate matter. I also like that the objective data make up 75 percent of the score, leaving only 25 percent to qualitative policy evaluations.

• Top five: Vancouver, San Francisco, New York, Sacramento, Los Angeles (?).
• Bottom five: St. Louis, Detroit, Pittsburgh, Houston, Calgary.

Huh? The air quality in highly rated Los Angeles is notoriously awful. The American Lung Association rates it the worst metropolitan area in the country for ozone pollution (derived from nitrogen oxides) and the second-worst for particulate pollution. A story written by Margot Roosevelt for The Los Angeles Times posits some reasons:

Unlike parts of the East and Midwest, where coal-fired power plants are a primary source of toxic pollution, Southern California's chemical stew is the product of tailpipe emissions from cars and diesel pollution from trucks, trains and ships linked to the ports of Los Angeles and Long Beach. Oil refineries, manufacturing plants and residential wood burning also are significant contributors.

So, although the criteria appear sound, there is something seriously wrong with a methodology that gives a high ranking for air quality to the most polluted city in the country. Nevertheless, the Green Cities report praises the city's "impressive performance," "all the more impressive when ranked against other large-area cities." There isn't the slightest hint that other data suggest quite the contrary. Incidentally, Sacramento -- another top-ranked city in the Index -- was found by the Lung Association to have the fifth-worst ozone pollution in the country.

At least the report notes, correctly, that "nitrogen oxides and particulate matter emissions [per person] decline noticeably with density."

Environmental governance

As might be expected, this category was based entirely on qualitative evaluations.

• Top cities: Denver, New York, Washington (all tied for first place with the maximum 100 points each), Seattle, five cities tied for fifth.
• Worst: St. Louis, Detroit, Cleveland, Toronto, three cities tied for fifth worst. 

The researchers were impressed with the Greenprint Denver program, Mayor Bloomberg's leadership in New York (although the report makes no mention of the excellent PlaNYC framework), and Washington's overall environmental strategy. In contrast to those perfect scores, the evaluators awarded a paltry 5.6 points to St. Louis, which apparently has a sustainability plan in the works but not yet fully realized. I don't have a lot to say about this category, since it is so subjective and I am unfamiliar with the particulars in most of the cities.

*

So, in the end, what to say? After spending a lot of time with this report, I am more convinced than ever that city rankings of any sort raise as many questions as they answer. Even at NRDC, we have struggled with finding the right criteria to measure in our Smarter Cities evaluations (which highlight best performers but no longer include numerical rankings). Any metric one selects is going to leave something important out, and at worst you can end up with bizarre findings such as the high ranking of badly polluted Los Angeles for clean air. When the results are counterintuitive, the authors should at least acknowledge the dissonance and offer some explanation.

Still, Siemens (please note my disclaimers about affiliations in the previous post) deserves credit for giving serious attention to the subject of urban sustainability, which the corporation does in a number of ways in addition to this study. The report does bring some important issues to light if one focuses less on the rankings and more on the best practice exemplars sprinkled throughout.

This post also appears on NRDC's Switchboard.
Images (top to bottom): iwona_kellie/flickr, Johnida Dockens