DOE Responds to Complaints About Its Commitment to Open Government

After a journalist called attention to the difficulty of finding documents about the prospective Yucca Mountain nuclear repository last week, the Department of Energy responded in near-record time. Cammie Croft, director of New Media & Citizen Engagement, addressed the concerns in a reasonable blog post on Energy.gov. She pulled together a specific list of Yucca Mountain documents and explained the general problem that governmental bodies have:

One of the biggest challenges for federal websites - including Energy.gov -- is managing the millions of PDFs the government has online. That challenge existed before our redesign and still exists today. The problem is that unless older PDFs are correctly metatagged with relevant keywords, they may not show up in search results. The Energy.gov of today, however, is much better than what was offered before. And it's getting better every day as we migrate additional documents and Departmental office websites to the new platform. Within a couple days of the concern being raised, we were able to quickly elevate additional Yucca Mountain documents, update their metadata and make them more findable. These documents were always available -- and with improved metadata and a dedicated landing page for context, they are now more search friendly.

While we are incredibly proud of the new Energy.gov platform, there are Energy Department program office websites and subsequent documents that just aren't part of the platform yet. To improve the availability and transparency of our information, we're in the process of migrating the remaining program office content into the system - but this process takes some time.

If you've ever tried to manage a huge amount of content, you will feel some sympathy for the government's position. Managing tons of stuff that wasn't created digitally is a huge problem. Attaching the correct metadata is very difficult and sometimes the OCR doesn't work well for full-text searching. It's rough.

On the other hand, if you've ever tried to find a government document and been unable to, it's easy to find Croft's position unacceptable. It seems like the DOE should have a better system for dealing with the documents with with it has been entrusted.

I have to wonder if there is some way that these two problems could actually cancel each other out. If journalists and other interested parties could systematically identify those documents they want most, the DOE could prioritize their efforts better. Back when I was at Wired, we tried to create something like this with a wiki. It sort of worked, but not really. While it was nominally a crowdsourcing project, I was the primary member of the crowd.

Who knows, maybe a system that combined a subReddit with some librarians and journalists would work?