One interesting project recently launched by the Education Trust, a nonprofit group that advocates for education reform, compares the graduation rates of close to 1,500 colleges and universities. The comparison assumes that a school is adding considerable value if it graduates more of its students than would be expected given their high school records and socioeconomic background, and adding little if it admits a bumper crop of high-achieving kids and then graduates them at a below-average clip. It's not a perfect metric: accumulating credit hours and earning passing grades isn't equivalent to actual learning, and a school can easily improve its graduation rate by grading more generously. But comparing graduation rates with actual learning measures should prove interesting.
Two other value-added initiatives may soon be able to provide such measures. In the fall of 2006 the Center of Inquiry in the Liberal Arts, at Wabash College, plans to initiate a longitudinal study of 5,000 students at sixteen institutions. Researchers will use existing standardized tests along with in-depth interviews to examine the students' development of problem-solving abilities and their inclination to learn, cultural sensitivity, leadership, and moral character. They hope their findings will help reveal which teaching conditions are most conducive to learning and whether initiatives such as study abroad, service learning, and diversity programs are effective.
At the same time, the Collegiate Learning Assessment Project (of which I am a co-director) has created two types of tests that evaluate students' ability to articulate complex ideas, examine claims and evidence, support ideas with relevant reasons and examples, sustain a coherent discussion, and use standard written English.
The first, called a "performance task," provides students with a mini-library of diverse documents, such as letters, memos, summaries of research reports, newspaper articles, photographs, diagrams, tables, charts, and interview notes or transcripts. Students are asked to identify the strengths and limitations of alternative hypotheses, points of view, and courses of action. An example:
A catfish with a grotesque mutation is caught in Paradise Lake, the source for the local water supply. Local press coverage has the village buzzing. Mayor Carp has asked you and some others in your community to serve on a panel to investigate this matter. You are provided with the following documents:
*a newspaper article that contains a picture and description fothe fish and the opinion of a recognized expert as to its source
*an editorial by an environmental activist
*a radio interview with a biologist who teaches at a nearby college
*a State report with the results of water testing and other investigations regarding Paradise Lake
*a map of the area
*an article about similar fish "catches" from ECO, a journal focusing on issues of clean air and safe water
Using these data sources, pelase prepare a memo to the chair of the panel regarding (1) your analysis of the strength and limitations of various explanations for finding such a fish in Paradise Lake and (2) your recommendations regarding what should now be done about this situation and your reasons for these recommendations
All the tasks demand similar skills: students must weigh, organize, and synthesize evidence from different sources; distinguish rational from emotional arguments and fact from opinion; analyze data; deal with inadequate, ambiguous, or conflicting information; spot deception and holes in the arguments of others; recognize what information is or is not relevant to the task at hand; and identify additional information that might help to resolve issues.
Each performance task is set in the context of a broad academic field, such as science and engineering, business, the social sciences, or the arts and humanities. But a student should be able to respond adequately to a task without having specialized knowledge in the particular field. Indeed, students do as well on CLA performance tasks drawn from other fields as they do on those related to their own majors.
The second test, of analytic writing, requires two essays: a forty-five-minute "Make an argument," in which students either support or reject a position on some issue; and a thirty-minute "Break an argument," in which they consider the validity of someone else's reasoning.
"Make an argument" asks students to react to an opinion—for example, "There is no such thing as 'truth' in the media. The one true thing about information media is that they exist only to entertain." They can address the issue from any perspective they wish, as long as they provide support for their views. "Break an argument" might present students with a passage like this:
A well-respected professional journal whose readership includes elementary school principals recently published the results of a two-year study on childhood obesity. (Obese individuals are usually considered to be those who are 20 percent above their recommended weight for height and age.) This study sampled 50 schoolchildren, ages 5-11, from Smith Elementary School. A fast food restaurant opened near the school just before the study began. After two years, students who remained in the sample group were more likely to be overweight relative to the national average. Based on this study, the principal of Jones Elementary School decided to confront her school's obesity problem by opposing any fast food restaurant openings near her school.
Rather than agree or disagree with the position, students must discuss how well reasoned they find the argument by considering the soundness of its logic.
Approximately 19,000 students from 134 colleges and universities participated in the CLA through May of 2005. In this academic year an additional 100 institutions and more than 30,000 students are participating. They come from a national sample of colleges, universities, and community colleges—private and public, large and small, more selective and less selective. Results are being aggregated at the institutional level, to permit comparisons across institutions and to determine how well individual schools are doing.
The findings to date are illuminating. After controlling for admissions selectivity, the CLA shows that which school a student attends does make a difference. Assuming that these initial results hold up, and that learning differences can be attributed to variations in campus culture, curricula, and pedagogy, the next step will be to create case studies of the schools that achieve the best results, and conduct follow-up studies of those that make changes to improve learning outcomes. The ultimate goal is not to create a new college ranking (though some will be tempted to use our findings for that end) but to let colleges and universities share their successes the way doctors and hospitals do.
Some cautionary notes are warranted. Value-added assessment tells us only how schools are doing in relation to their competitors, not what absolute standards of excellence they should be setting. Nor should it be allowed to crowd out other measures—particularly affordability and equity—that bear on how we judge a school's quality. Finally, there are many things we cannot yet measure accurately—and some aspects of "quality" will always remain elusive.
Nonetheless, value-added assessment offers an excellent place to start, and a chance for higher education to demonstrate that "faith-based" answers about quality are no longer acceptable. This country has always looked to higher education to take the lead in innovation, and to define, seek, and demand excellence from its students. Today's academy should be satisfied with nothing less.