It's The Computer Code, Silly

In trying to unravel what the real story is behind climategate - not the hysterical conspiracy theories but the skeptic's valid concerns about corrupted data and the bias of groupthink - John Tierney is, as always, worth reading. But more and more readers and bloggers are fingering, so to speak, the computer code which is the raw data for the CRU results. That code itself seems deeply problematic, which is not so much a measure of wilful malfeasance as of translating vast amounts of uneven and varied historical data into readable consistent form by scientists already looking for a pattern. In that process, the temptation to massage what is unmassageable may be real - and human (as scientists, for all their pretensions, are). A reader writes:

The really interesting story in Climategate is in the computer code and related files from CRU.  See HARRY_READ_ME.txt online at  What the code and the Read_me file tells us:

(1) CRU's data files of weather-related information gathered at worldwide sites were (are?) a hopeless mess. 

(2) The programmer applied arbitrary adjustments to the data (he says so himself) to get the desired results.

Perhaps this is an isolated case of incompetence, and other researchers have good data and programs.  Perhaps the research created at CRU was somehow good research, despite appearances.  Or perhaps at least some of these scientists are fooling themselves.

Richard Feynman, the brilliant physicist, in his commencement address (PDF) at CalTech related a story of a famous scientist whose published result turned out to be a little bit off.

As he says,

"It's interesting to look at the history of measurements of the charge of the electron, after Millikan.  If you plot them as a function of time, you find that one is a little bigger than Millikan's, and the next one's a little bit bigger than that, and the next one's a little bit bigger than that, until finally they settle down to a number which is higher.  Why didn't they discover that the new number was higher right away?  It's a thing scientists are ashamed of--this history--because it's apparent that people did things like this:  When they got a number that was too high above Millikan's they thought something must be wrong--and they would look for and find a reason why something might be wrong.  When they got a number closer to Millikan's value they didn't look so hard.  And so they eliminated the numbers that were too far off, and did other things like that."  

Feynman knew that learning not to fool yourself was one of the hardest parts of becoming a scientist. 

Scientists are as prone as anyone to taking part in "informational cascades," particularly when they are being funded by granting agencies that reward those who continue in an established line of inquiry (can you imagine funding going to a scientist who found a climate counter-trend?) and when they are trying to publish in peer-reviewed journals whose editorial staff refuse to consider papers that do not come up with the expected results.

2006-2011 archives for The Daily Dish, featuring Andrew Sullivan

Join the Discussion

After you comment, click Post. If you’re not already logged in you will be asked to log in or register with Disqus.

Please note that The Atlantic's account system is separate from our commenting system. To log in or register with The Atlantic, use the Sign In button at the top of every page.

blog comments powered by Disqus


How a Psychedelic Masterpiece Is Made

A short documentary about Bruce Riley, an artist who paints abstract wonders with poured resin


Why Is Google Making Skin?

Hidden away on Google’s campus, doctors are changing the way people think about health.


How to Build a Tornado

A Canadian inventor believes his tornado machine could solve the world's energy crisis.


A New York City Minute, Frozen in Time

This short film takes you on a whirling tour of the Big Apple


What Happened to the Milky Way?

Light pollution has taken away our ability to see the stars. Can we save the night sky?

Just In