The Measure of Merit

Nicholas Lemann on Testing and Meritocracy

August 22, 1995

The following is the transcript of a live online conference with Nicholas Lemann as it appears in The Atlantic Monthly Online on the America Online network.

Hosted, with an introduction, by Wen Stephenson

What is the measure of merit? How have educational testing and the idea of meritocracy shaped American society in the twentieth century?

Is it necessary -- or possible -- to gauge an individual's merit on the basis of his or her scores on a standardized test such as the SAT?

How does standardized testing affect the arguments for and against affirmative action?

Welcome to The Atlantic Monthly Auditorium. These questions and others are the subject of tonight's conference. Our guest is Nicholas Lemann, national correspondent for The Atlantic Monthly and the author of our August cover story, "The Structure of Success in America." In this article and its follow-up, "The Great Sorting," which is forthcoming in The Atlantic's September issue (on sale this week), Lemann offers the first inside look at the origins of the Educational Testing Service (ETS) and its Scholastic Aptitude Test, universally known as the SAT. In the course of this story Lemann also explores the history of meritocracy in the United States and how the idea of meritocracy has shaped American society in the twentieth century.

As Lemann writes in the August issue of The Atlantic Monthly, "ETS and its tests are a national obsession, deeply worked into the fabric of middle-class life." Not only has ETS generated an entire test-prep industry and influenced the development of elementary and secondary education in this country, but its tests -- the SAT, GRE, LSAT, MCAT, and GMAT -- have become fixtures of the culture. They stand at the center of American higher education and, along with its growth, have had a profound effect on the development of American society in the past half century, having become an essential part of what the sociologist Robert K. Merton has termed "the structure of opportunity."

With the exceptions of race and gender, to which it is closely related, no subject is more sensitive for Americans today than the structure of opportunity -- that is, the way in which we sort ourselves in the struggle for success. And, as the country has witnessed in recent months, issues of race, class, gender, and opportunity converge in the highly charged debate over the future of affirmative action policies in the workplace and in public education.

Today, the very concept of meritocracy -- selecting a society's leaders solely on the basis of merit -- raises questions as to how we determine, how we *measure*, merit. Even the creators of standardized tests, now used universally in college and graduate-school admissions, admit that they are not "objective." As their critics have long argued, and as defenders of affirmative action maintain, standardized tests are in many important ways culturally biased.

In much the way that IQ tests and the notion of general intelligence have been largely discredited (witness the firestorm of controversy ignited by last year's *The Bell Curve*), so the SAT, itself modeled on IQ tests, has come under fire for narrowly measuring just one ability -- the ability to get good grades in school, a skill often dependent upon whether one has had the benefit of a good primary and secondary education, something greatly influenced by socio-economic factors.

Nick Lemann is recognized nationally as an expert on issues of race and class in America. His article "What Happened to the Case for Affirmative Action?" in the New York Times Magazine (June 11, 1995) traced the history of affirmative action and asked where its liberal supporters have gone as the policy has come under attack. His recent book *The Promised Land* told the story of the Great Migration of African Americans from the rural South to the urban North in the middle of this century, one of the major transformative events in American history.

Tonight, you can ask him how he thinks another transformation -- the creation of an entirely new elite made up of skillful test-takers -- will influence American society and politics for generations to come.

WenSatl: Welcome back, Nick. We're glad you could join us tonight for another of these online gigs.

LemannAtl: Thanks, glad to be back.

WenSatl: You were last here to talk about your book The Promised Land, back in February. I understand you're at work on a new book.

LemannAtl: I have been for two or three years. It's on meritocracy in the US, and I'd like to think of it as being in the homestretch phase.

WenSatl: Well, I've got a question to get us started tonight. This morning on the way to work I was confronted by an ad for a prominent test-prep course. It said they would guarantee higher scores on the SAT, GRE, LSAT, GMAT, and MCAT -- all ETS tests. What does the presence of this ad, located on a train running between Harvard and MIT, say about the "structure of opportunity" in this country?

LemannAtl: It says that people perceive test scores as the gateway to opportunity, and that they believe test prep works, at least well enough that raising your scores somewhat will have a payoff. All of which universities (and ETS) will pretty much deny, but I suspect the people taking these courses haven't lost their minds. Stanley Kaplan has been bought by the Washington Post Company and is poised to become a really big business.

WenSatl: All right. Here's another one, something I've thought about a bit lately. The terms race and class are often conflated, as though "race equals class." But we know that not all blacks in this country are poor, and not all the poor are black. Poverty comes in all colors, including white. So in regard to testing, fairness, and equal opportunity, do you think one or the other, race or class, should take priority?

LemannAtl: It isn't totally illogical to use race not class. First, race is a pretty good proxy for class. Not perfect but there aren't very many Vernon Jordans out there. Also, if one reason for affirmative action is to put out potential fires in the society, it's race not class that's the divisive issue in America, historically speaking. We didn't fight the Civil War over class. Finally, I question how accepted class-based affirmative action really would be in operation. One researcher I know at ETS constructed a model for adjusting SAT scores on the basis of class--not race--and ETS told him it wouldn't support the research.

WenSatl: This might be a sensitive question. I'm not sure how you feel about divulging the answer: What are your SAT scores?

LemannAtl: I'm sure I'll get asked that a lot as I get into the book-tour phase, and I haven't formulated a policy on it. So let me ask the questioner: Why should I reveal?

WenSatl: Fair enough.

WenSatl: A member of the audience is a reader of Kurt Vonnegut, and asks: Nick, have you read the Kurt Vonnegut short story depicting a dystopia in which everyone is made to be truly equal? Smart people get electric shocks sent through their brains every so often in order to disrupt their thoughts; athletic people are laden with weights to slow them down, etc. I'm curious to know your thoughts about it.

LemannAtl: Yes, I've read it, and it's a fascinating story. But, the main point is, this is NOT what's happening in America today. I say in the Atlantic story that we live in the most IQ-sorted society in the world. We do an incredibly efficient job of cutting the high scorers out of the herd and giving them special training. Affirmative action, a fairly minor phenomenon, serves to elevate a few people rather than bring down the super achievers. So what we have is the exact reverse of the Vonnegut story.

WenSatl: You mention in your August Atlantic Monthly cover story that the ACT is ETS's main competitor so here's a question from the audience that wants to know more about ACT: Does the ACT measure the same things as the SAT (without defining those "things")? Or something else?

LemannAtl: Good question. The ACT was started by a man in Iowa named E.F. Lindquist (who also invented the "Iowa Tests" that my kids take in elementary school), in a spirit of rebellion against the SAT. The ACT was supposed to be a combined achievement/aptitude test, whereas the SAT was pure aptitude, and it was supposed to be used mainly by state universities for placement/guidance, rather than by elite schools for selection. A much more democratic vision. But, the scores correlate very highly, it turns out, and most universities now treat them as interchangeable and let you take either one.

WenSatl: This next question, from AIRSCOTT, ties in with one of your main points in your Atlantic pieces: Do you think that our nation's elite--its politicians, pundits, business executives, etc.--would be comprised of different people if the SAT didn't exist. In other words, have the SAT and other standardized tests actually changed the composition of our society's leadership?

LemannAtl: Yes, but. The SAT has changed the composition of those fields that use it and other tests as the screen for entry, that is, mainly, law, medicine, MBA, and academia. But there are still lots of lightly screened fields, like corporate management, entrepreneurial business, and entertainment, where the advent of the SAT hasn't made nearly as much difference.

WenSatl: R Moze has a specific question about the current version of the SAT. I didn't realize this was true: Why were analogies eliminated from the SAT? Isn't the ability to make comparisons between word meanings vital to learning in a higher education setting?

LemannAtl: Check me on this, but I think they eliminated the antonyms and kept the analogies.

WenSatl: Ah. Here's one from an audience member: How will the authentic testing movement impact student assessment vs. standardized testing?

LemannAtl : Basically for political reasons. Testing critics don't like the antonyms--partly because they produce a higher racial difference than the other sections of the test. So, I'd guess the ETS reasoning is, no antonyms, less criticism.

WenSatl: You can disregard that last question. I cut you off. Here's another to do with race and affirmative action:

LemannAtl: I'd like to answer it.

WenSatl: You've written that "numerical, education-based meritocracy" has been "bad for blacks." How does that observation stack up against arguments made by conservative black intellectuals, such as Shelby Steele, against affirmative action on the grounds that it's "bad for blacks"?

LemannAtl: I'm now going to answer two questions. First, authentic assessment. Psychometricians basically don't like it. It's complicated, expensive, and produces lower validity coefficients than the traditional tests. So it's another political issue. ETS is very eager not to appear intransigent, and has officially embraced authentic assessment. But if there were no outside pressure for it, then I think they'd just go back to the old way. Now, on affirmative action, Shelby Steele's position is that it would be better for blacks if there were many fewer in meritocratically distributed billets, but those who had them "really deserved" to be there. I was saying that it would be bad for blacks just because you'd see a huge decrease in the black presence in white collar America.

WenSatl: This question really cuts through the theory to the real-life experience of the SAT: Should a person be judged on the basis of one test that is stressful, long, boring and composed by scholars?

LemannAtl: Well....Let me give you the psychometrician's position. All the test is designed to do is predict academic grades in the short term. If you find it boring, then you're likely to find college courses boring. It's composed by scholars, but college course are taught by scholars. Etc. In other words, they let themselves off the hook of the issue that the tests really do use one ability to screen for a wide range of roles in the society.

WenSatl: But what about the fact that some people just don't react well to stress?

LemannAtl: You can't win with the "test anxiety" argument. Every question on every test is correlated with college grades. The better each question correlates, the more likely it is to be used. They will answer you by saying, all we claim is that you get more predictive power over college grades by looking at the test score than by not looking at the scores. They're not totally predictive, and you can't adjust the scores to account for test anxiety. Instead the answer is to take them with a grain of salt.

WenSatl: A large grain, perhaps. Here's a grad-student question for you: Have you taken and/or examined the GRE? It fills a different niche, but I view it as SAT-plus, with the same pros and cons. Your thoughts?

LemannAtl: Agree. It's basically a more advanced aptitude test, rather than being fundamentally a test of what you learned in college.

WenSatl: I agree. This audience member takes the issue into the workplace: Have you seen the request for SAT scores on some job applications? Is this just a lazy way of filtering resumes? If so, doesn't the SAT gain even more credence?

LemannAtl: Yes. I think it's a really bad idea, and it's possibly even illegal (see the Supreme Court Griggs v. Duke Power decision).

WenSatl: You may have heard of the writers referred to in this next question ;-): Isn't the SAT more or less a neutral vessel. It seems to me that its potential for good and bad resides in those who put it to use. A leader who combined the views of Peter Brimelow and Charles Murray might use the SAT to cutoff immigration and sterilize people. But in the hands of someone more benign, the SAT is a useful sorter. Thoughts?

LemannAtl: The SAT, as I said, is a predictor of first-semester college grades. It's very good at finding people who could do brilliantly well in college, wherever they may be, and bringing them to the attention of the best universities. That's benign. To use it as a screen for all opportunity is NOT benign. I agree that the real problems inhere in society not in the tests themselves. For that reason, I don't have that much hope for test reform.

WenSatl: Let's bring the discussion into the realm of current political events.

WenSatl: The next questioner wants to know something about the situation in California: The California Civil Rights Initiative on the 1996 ballot would eliminate "preferential treatment" on the basis of race, gender, ethnicity, etc., in all state education and employment. If standardized tests favor certain socio-economic groups, wouldn't their use in admissions constitute "preferential treatment"?

LemannAtl: Well, that's the obvious grounds for a legal challenge against the proposition if it passes. The Regents already have pretty much eliminated preferences. Their position would be that the tests themselves are race neutral and it's not following the results strictly that's preferential. Interestingly, UC didn't even use standardized tests until about 30 years ago, by the way.

WenSatl: Here's someone who has read your Atlantic Monthly cover story, and is interested in history: In your Atlantic cover story for August you cite a 1943 Atlantic article by James B. Conant, titled "Wanted: American Radicals," calling for expansion of opportunity through public education. What did Conant mean by "radical" then? Do you think the present situation calls for a radical response? If so, what would radical mean in 1995?

LemannAtl: Good question. In 1995 it's inconceivable that a leading Establishmentarian of the Conant variety would call himself a "radical," so I'm not quite sure what it would mean. Gingrich sometimes calls himself a radical--the word has become the province of right-wing populists who hate the meritocratic upper class. Anyway, getting back to Conant, the key point is AMERICAN radical, by which he meant, not a Marxist. He wanted to shake up American society in order to fend off the Communist threat better.

WenSatl: Well, it looks like our time is up.

WenSatl: We might have time for one more question. Are you game?

LemannAtl: Sure

WenSatl: Here goes. How do you think the rise of multiculturalism to predominance in our colleges and universities will affect the future of standardized testing and admissions policies? Do you think changes to the status quo would pose a threat to tenured and/or tenure-seeking proponents of multicultural academic politics?

LemannAtl: The multicultural wave began in the late 60s, rose through the 70s, and is now, in my view, receding. In the future you're going to see the proponents of multiculturalism steadily losing ground.

WenSatl: That's all we have time for tonight. Thanks again, Nick, for joining us. And thanks everyone for coming.

LemannAtl: Thanks, it was fun.