The Measure of Merit
Nicholas Lemann on Testing and Meritocracy
August 22, 1995
The following is the transcript of a live online conference with Nicholas
Lemann as it appears in The Atlantic Monthly Online on the America Online network.
Hosted, with an introduction, by Wen Stephenson
What is the measure of merit? How have educational testing
and the idea of meritocracy shaped American society in the twentieth century?
Is it necessary -- or possible -- to gauge an individual's merit on the basis of his or her scores on a standardized test
such as the SAT?
How does standardized testing affect the arguments for and against affirmative
action?
Welcome to The Atlantic Monthly Auditorium. These questions and others are the
subject of tonight's conference. Our guest is Nicholas Lemann, national
correspondent for The Atlantic Monthly and the author of our August cover
story, "The Structure of Success in America." In this article and its
follow-up, "The Great Sorting," which is forthcoming in The Atlantic's
September issue (on sale this week), Lemann offers the first inside look at the
origins of the Educational Testing Service (ETS) and its Scholastic Aptitude
Test, universally known as the SAT. In the course of this story Lemann also
explores the history of meritocracy in the United States and how the idea of
meritocracy has shaped American society in the twentieth century.
As Lemann writes in the August issue of The Atlantic Monthly, "ETS and its
tests are a national obsession, deeply worked into the fabric of middle-class
life." Not only has ETS generated an entire test-prep industry and influenced
the development of elementary and secondary education in this country, but its
tests -- the SAT, GRE, LSAT, MCAT, and GMAT -- have become fixtures of the
culture. They stand at the center of American higher education and, along with
its growth, have had a profound effect on the development of American society
in the past half century, having become an essential part of what the
sociologist Robert K. Merton has termed "the structure of opportunity."
With the exceptions of race and gender, to which it is closely related, no
subject is more sensitive for Americans today than the structure of opportunity
-- that is, the way in which we sort ourselves in the struggle for success.
And, as the country has witnessed in recent months, issues of race, class,
gender, and opportunity converge in the highly charged debate over the future
of affirmative action policies in the workplace and in public education.
Today, the very concept of meritocracy -- selecting a society's leaders solely
on the basis of merit -- raises questions as to how we determine, how we
*measure*, merit. Even the creators of standardized tests, now used universally
in college and graduate-school admissions, admit that they are not "objective."
As their critics have long argued, and as defenders of affirmative action
maintain, standardized tests are in many important ways culturally biased.
In much the way that IQ tests and the notion of general intelligence have been
largely discredited (witness the firestorm of controversy ignited by last
year's *The Bell Curve*), so the SAT, itself modeled on IQ tests, has come
under fire for narrowly measuring just one ability -- the ability to get good
grades in school, a skill often dependent upon whether one has had the benefit
of a good primary and secondary education, something greatly influenced by
socio-economic factors.
Nick Lemann is recognized nationally as an expert on issues of race and class
in America. His article "What Happened to the Case for Affirmative Action?" in
the New York Times Magazine (June 11, 1995) traced the history of affirmative
action and asked where its liberal supporters have gone as the policy has come
under attack. His recent book *The Promised Land* told the story of the Great
Migration of African Americans from the rural South to the urban North in the
middle of this century, one of the major transformative events in American
history.
Tonight, you can ask him how he thinks another transformation -- the creation
of an entirely new elite made up of skillful test-takers -- will influence
American society and politics for generations to come.
WenSatl: Welcome back, Nick. We're glad you could join us tonight for another
of these online gigs.
LemannAtl: Thanks, glad to be back.
WenSatl: You were last here to talk about your book The Promised Land, back in
February. I understand you're at work on a new book.
LemannAtl: I have been for two or three years. It's on meritocracy in the US,
and I'd like to think of it as being in the homestretch phase.
WenSatl: Well, I've got a question to get us started tonight. This morning on
the way to work I was confronted by an ad for a prominent test-prep course. It
said they would guarantee higher scores on the SAT, GRE, LSAT, GMAT, and MCAT
-- all ETS tests. What does the presence of this ad, located on a train running
between Harvard and MIT, say about the "structure of opportunity" in this
country?
LemannAtl: It says that people perceive test scores as the gateway to
opportunity, and that they believe test prep works, at least well enough that
raising your scores somewhat will have a payoff. All of which universities (and
ETS) will pretty much deny, but I suspect the people taking these courses
haven't lost their minds. Stanley Kaplan has been bought by the Washington Post
Company and is poised to become a really big business.
WenSatl: All right. Here's another one, something I've thought about a bit
lately. The terms race and class are often conflated, as though "race equals
class." But we know that not all blacks in this country are poor, and not all
the poor are black. Poverty comes in all colors, including white. So in regard
to testing, fairness, and equal opportunity, do you think one or the other,
race or class, should take priority?
LemannAtl: It isn't totally illogical to use race not class. First, race is a
pretty good proxy for class. Not perfect but there aren't very many Vernon
Jordans out there. Also, if one reason for affirmative action is to put out
potential fires in the society, it's race not class that's the divisive issue
in America, historically speaking. We didn't fight the Civil War over class.
Finally, I question how accepted class-based affirmative action really would be
in operation. One researcher I know at ETS constructed a model for adjusting
SAT scores on the basis of class--not race--and ETS told him it wouldn't
support the research.
WenSatl: This might be a sensitive question. I'm not sure how you feel about
divulging the answer: What are your SAT scores?
LemannAtl: I'm sure I'll get asked that a lot as I get into the
book-tour phase, and I haven't formulated a policy on it. So let me ask the
questioner: Why should I reveal?
WenSatl: Fair enough.
WenSatl: A member of the audience is a reader of Kurt Vonnegut, and
asks: Nick, have you read the Kurt Vonnegut short story depicting a dystopia in
which everyone is made to be truly equal? Smart people get electric shocks sent
through their brains every so often in order to disrupt their thoughts;
athletic people are laden with weights to slow them down, etc. I'm curious to
know your thoughts about it.
LemannAtl: Yes, I've read it, and it's a fascinating story. But, the main point
is, this is NOT what's happening in America today. I say in the Atlantic story
that we live in the most IQ-sorted society in the world. We do an incredibly
efficient job of cutting the high scorers out of the herd and giving them
special training. Affirmative action, a fairly minor phenomenon, serves to
elevate a few people rather than bring down the super achievers. So what we
have is the exact reverse of the Vonnegut story.
WenSatl: You mention in your August Atlantic Monthly cover story that the ACT
is ETS's main competitor so here's a question from the audience that wants to
know more about ACT: Does the ACT measure the same things as the SAT (without
defining those "things")? Or something else?
LemannAtl: Good question. The ACT was started by a man in Iowa named E.F.
Lindquist (who also invented the "Iowa Tests" that my kids take in elementary
school), in a spirit of rebellion against the SAT. The ACT was supposed to be a
combined achievement/aptitude test, whereas the SAT was pure aptitude, and it
was supposed to be used mainly by state universities for placement/guidance,
rather than by elite schools for selection. A much more democratic vision. But,
the scores correlate very highly, it turns out, and most universities now treat
them as interchangeable and let you take either one.
WenSatl: This next question, from AIRSCOTT, ties in with one of your main
points in your Atlantic pieces: Do you think that our nation's elite--its
politicians, pundits, business executives, etc.--would be comprised of
different people if the SAT didn't exist. In other words, have the SAT and
other standardized tests actually changed the composition of our society's
leadership?
LemannAtl: Yes, but. The SAT has changed the composition of those fields that
use it and other tests as the screen for entry, that is, mainly, law, medicine,
MBA, and academia. But there are still lots of lightly screened fields, like
corporate management, entrepreneurial business, and entertainment, where the
advent of the SAT hasn't made nearly as much difference.
WenSatl: R Moze has a specific question about the current version of the SAT. I
didn't realize this was true: Why were analogies eliminated from the SAT? Isn't
the ability to make comparisons between word meanings vital to learning in a
higher education setting?
LemannAtl: Check me on this, but I think they eliminated the antonyms and kept
the analogies.
WenSatl: Ah. Here's one from an audience member: How will the authentic testing
movement impact student assessment vs. standardized testing?
LemannAtl : Basically for political reasons. Testing critics don't like the
antonyms--partly because they produce a higher racial difference than the other
sections of the test. So, I'd guess the ETS reasoning is, no antonyms, less
criticism.
WenSatl: You can disregard that last question. I cut you off. Here's another to
do with race and affirmative action:
LemannAtl: I'd like to answer it.
WenSatl: You've written that "numerical, education-based meritocracy" has been
"bad for blacks." How does that observation stack up against arguments made by
conservative black intellectuals, such as Shelby Steele, against affirmative
action on the grounds that it's "bad for blacks"?
LemannAtl: I'm now going to answer two questions. First, authentic assessment.
Psychometricians basically don't like it. It's complicated, expensive, and
produces lower validity coefficients than the traditional tests. So it's
another political issue. ETS is very eager not to appear intransigent, and has
officially embraced authentic assessment. But if there were no outside pressure
for it, then I think they'd just go back to the old way. Now, on affirmative
action, Shelby Steele's position is that it would be better for blacks if there
were many fewer in meritocratically distributed billets, but those who had them
"really deserved" to be there. I was saying that it would be bad for blacks
just because you'd see a huge decrease in the black presence in white collar
America.
WenSatl: This question really cuts through the theory to the real-life
experience of the SAT: Should a person be judged on the basis of one test that
is stressful, long, boring and composed by scholars?
LemannAtl: Well....Let me give you the psychometrician's position. All the test
is designed to do is predict academic grades in the short term. If you find it
boring, then you're likely to find college courses boring. It's composed by
scholars, but college course are taught by scholars. Etc. In other words, they
let themselves off the hook of the issue that the tests really do use one
ability to screen for a wide range of roles in the society.
WenSatl: But what about the fact that some people just don't react well to
stress?
LemannAtl: You can't win with the "test anxiety" argument. Every question on
every test is correlated with college grades. The better each question
correlates, the more likely it is to be used. They will answer you by saying,
all we claim is that you get more predictive power over college grades by
looking at the test score than by not looking at the scores. They're not
totally predictive, and you can't adjust the scores to account for test
anxiety. Instead the answer is to take them with a grain of salt.
WenSatl: A large grain, perhaps. Here's a grad-student question for you: Have
you taken and/or examined the GRE? It fills a different niche, but I view it as
SAT-plus, with the same pros and cons. Your thoughts?
LemannAtl: Agree. It's basically a more advanced aptitude test, rather than
being fundamentally a test of what you learned in college.
WenSatl: I agree. This audience member takes the issue into the workplace: Have
you seen the request for SAT scores on some job applications? Is this just a
lazy way of filtering resumes? If so, doesn't the SAT gain even more
credence?
LemannAtl: Yes. I think it's a really bad idea, and it's possibly even illegal
(see the Supreme Court Griggs v. Duke Power decision).
WenSatl: You may have heard of the writers referred to in this next question
;-): Isn't the SAT more or less a neutral vessel. It seems to me that its
potential for good and bad resides in those who put it to use. A leader who
combined the views of Peter Brimelow and Charles Murray might use the SAT to
cutoff immigration and sterilize people. But in the hands of someone more
benign, the SAT is a useful sorter. Thoughts?
LemannAtl: The SAT, as I said, is a predictor of first-semester college grades.
It's very good at finding people who could do brilliantly well in college,
wherever they may be, and bringing them to the attention of the best
universities. That's benign. To use it as a screen for all opportunity is NOT
benign. I agree that the real problems inhere in society not in the tests
themselves. For that reason, I don't have that much hope for test reform.
WenSatl: Let's bring the discussion into the realm of current political
events.
WenSatl: The next questioner wants to know something about the situation in
California: The California Civil Rights Initiative on the 1996 ballot would
eliminate "preferential treatment" on the basis of race, gender, ethnicity,
etc., in all state education and employment. If standardized tests favor
certain socio-economic groups, wouldn't their use in admissions constitute
"preferential treatment"?
LemannAtl: Well, that's the obvious grounds for a legal challenge against the
proposition if it passes. The Regents already have pretty much eliminated
preferences. Their position would be that the tests themselves are race neutral
and it's not following the results strictly that's preferential. Interestingly,
UC didn't even use standardized tests until about 30 years ago, by the way.
WenSatl: Here's someone who has read your Atlantic Monthly cover story, and is
interested in history: In your Atlantic cover story for August you cite a 1943
Atlantic article by James B. Conant, titled "Wanted: American Radicals,"
calling for expansion of opportunity through public education. What did Conant
mean by "radical" then? Do you think the present situation calls for a radical
response? If so, what would radical mean in 1995?
LemannAtl: Good question. In 1995 it's inconceivable that a leading
Establishmentarian of the Conant variety would call himself a "radical," so I'm
not quite sure what it would mean. Gingrich sometimes calls himself a
radical--the word has become the province of right-wing populists who hate the
meritocratic upper class. Anyway, getting back to Conant, the key point is
AMERICAN radical, by which he meant, not a Marxist. He wanted to shake up
American society in order to fend off the Communist threat better.
WenSatl: Well, it looks like our time is up.
WenSatl: We might have time for one more question. Are you game?
LemannAtl: Sure
WenSatl: Here goes. How do you think the rise of multiculturalism to
predominance in our colleges and universities will affect the future of
standardized testing and admissions policies? Do you think changes to the
status quo would pose a threat to tenured and/or tenure-seeking proponents of
multicultural academic politics?
LemannAtl: The multicultural wave began in the late 60s, rose through the 70s,
and is now, in my view, receding. In the future you're going to see the
proponents of multiculturalism steadily losing ground.
WenSatl: That's all we have time for tonight. Thanks again, Nick, for joining
us. And thanks everyone for coming.
LemannAtl: Thanks, it was fun.
Copyright © 1995 The Atlantic Monthly. All rights reserved.
|