Data Dilemmas: What We Don't Know About School and Work

Collecting rigorous, in-depth data related to education and labor issues should be a federal priority.

When it comes to some important questions about education and workforce trends in America, chasing down an accurate answer can be a bit dizzying.

Here's an example: There's a lot of work being done these days around college completion. This is a real and important issue. Helping more students finish school would be good for them and good for the economy. So, you might wonder, how many college students never finish school? Well, that depends on who you ask and what you count.

Try opening a new window on your browser and see what you can find. I'll meet you back here in a few hours with some aspirin.

Better yet, I'll save you the headache: check out this college completion website put together by The Chronicle of Higher Education. The series of infographics featured at the top of the site show that of the 4.3 million freshmen who started college in 2004, 1.2 million aren't counted and 2.1 million didn't officially graduate (but that doesn't mean they did or didn't). So out of 4.3 million, we only have a handle on what happened to 1 million. Wow.

Delve a little deeper into the related articles and resources compiled by the Chronicle, and you will learn more about who's counted and who's not in data compiled by the National Center for Education Statistics. To be counted, a student has to graduate within six years of starting school, go to school full-time, and attend a single school (personally, I know a lot of college graduates who fall outside at least one of these parameters).

As a number of commentators have pointed out, including Sara Lipka in this Chronicle article, according to this method, President Obama and Governor Romney would not be counted as having completed college because they both changed schools as undergraduates.

Like a lot of data-collection efforts, this way-off-the-mark method of measuring college completion threatens to hold us hostage to outdated methodology, and it's a good example of how something that sounds like a pretty basic question (hey, let me just hop on the Web and find out!) can be maddeningly hard to answer.

Since the federal government's data in this area is clearly falling short, others are trying to pick up the slack, and the Chronicle site also explores other ways of trying to assess completion rates, including using data from the Voluntary System of Accountability. Which is, you know, voluntary. (You may need that aspirin after all).

It is possible, of course, to fix this kind of thing, and sometimes the federal government gets it right. For instance, up until very recently, coming up with accurate data about high-school graduation and dropout rates was headache-inducing, too. Different states determined the rates in a variety ways, creating an apple-and-orange situation. Not only was the data over the place, but people would cherry pick data to paint either the best or worst possible picture, depending on their agendas.

As explained in this piece by education reporter Kavitha Cardoza for WAMU public radio, the federal government is now requiring states to follow a standardized method: "The new methodology, which requires tracking students individually from 9th grade all the way through 12th grade to determine if they graduate, means states across the country will be able to compare their graduation rates accurately."

Despite such signs of progress, the role of the federal government as the central keeper of data seems to be in jeopardy. Exhibit A: In May, the House voted to eliminate the American Community Survey, a rolling survey of 3 million Americans that is administered by the U.S. Census.

In response to this development, economic researcher E.J. Reedy spoke to the importance of the ACS on the Kauffman Foundation's Growthology blog, saying in part:

The American Community Survey was a pioneering effort when it was put in place a decade ago and is now our main avenue of estimating current trends in our population. It is the ultimate public good in that these data are best collected by government sources (governments around the world do similar things) and then fed back to households and businesses in a variety of ways. These data would be more expensive, less reliable, and ultimately only available to a few if the government was not the one collecting them.

...why don't other groups band together to fund these types data, such as Chambers of Commerce or even private foundations? It is a fair question, but one more appropriate for smaller types of activities. There are no institutions in place, other than the government, to allow the scale of cooperation necessary to collect data of this scale, quality, and public-use.

Fortunately, the Senate won't pass the House bill. However, a Senate bill has been introduced that would make the ACS voluntary, which is nearly as bad. This would greatly undermine its statistical accuracy and create more work for the Census, since the current sampling is randomized and those chosen to take the survey must participate.

The proposed Senate bill is unlikely to pass, but the prevailing attitude toward the ACS on Capitol Hill is indicative of a yawning gap between what the federal government could and should be doing on the data front and what Congress seems willing to fund. In the age of big data, our elected leaders seem to be thinking small, and looking backward instead of forward.

In a 2010 overview describing cross-sector efforts to get federal agencies to generate better economic data related to small businesses and entrepreneurship (the Kauffman Foundation's primary focus), Reedy commented that in many ways, the U.S. is "better equipped to measure the economy from the 1950s rather than the 2010s." This also sounds like a good assessment of NCES college completion numbers.

No, the federal government isn't he only source of data collection and analysis. And of course a lot of very useful workforce-related data are tracked by the Department of Education, Department of Labor, and many other federal agencies. But too often, the data isn't as useful as it could be. We certainly shouldn't be undermining our current data collection efforts. Instead, we should be making them more robust and more relevant to understanding our role and assessing our progress in the 21st-century global economy.

Granted, data isn't sexy. It doesn't hold a lot of popular appeal. Nobody is going to run for president on a "More data! Better data!" platform. And as Oscar Wilde reminds us, "The truth is rarely pure and never simple." Gathering good data is challenging. Making sense of data is even more so, especially in complex social contexts.

Nonetheless, we need to get at the truth the best we can, using the best data possible, so that policymakers, educators, philanthropists and community leaders can make informed decisions. So I'll say it, even if I don't have the stump: "More data! Better data!"