The Case for SAT Words

On several occasions in the past year, David Coleman, the president of the College Board, which administers the SAT, has suggested that changes need to be made to the vocabulary tested on the exam.  In a talk he gave at the Brookings Institution a year ago, Coleman remarked, “I think when you think about vocabulary on exams, you know, how SAT words are famous as the words you will never use again? You know, you study them in high school and you’re like, gosh, I’ve never seen this before, and I probably never shall.” Coleman co-opted an old criticism of the SAT, that it forced students to learn difficult vocabulary that is useless for much of anything other than scoring well on the exam. He went on, “Why wouldn’t it be the opposite?  Why wouldn’t you have a body of language on the SAT that’s the words you most need to know and be ready to use again and again? Words like transform, deliberate, hypothesis, right?”

Jim Montoya, the College Board’s Vice President of Higher Education, in an interview on NPR, reiterated Coleman’s criticism of “SAT words.” Asked why the SAT is “always [testing] the word 'unscrupulous',” Montoya replied, “Yes, you're right. It's one of those words people identify as an SAT word. All I can say is that as we move forward, one of the things we want to make absolutely certain of is that the vocabulary that students are expected to know will be vocabulary that they will be able to use as college students, and which will be valuable to them.”

Coleman’s comments on vocabulary provoked little response at the time, although one commentator accused him of “sending a message that devalues language.”  The notion that some words are not worth knowing is bound to raise the hackles not only of people who love the wealth and power of all kinds of words, including fancy ones, but also of people who know just how much importance educational experts attribute to what they call “lexical richness,” or a large and diverse vocabulary.  Coleman just so happens to be both of those kinds of people, and he understands that the question is less what vocabulary students should learn vocabulary than how they should learn it. 

The College Board’s unfortunate remarks make it sound like the choice is to decide which of two classes of words to teach and test—the technical (e.g., “hypothesis”) or the writerly (e.g., “unscrupulous”)—neither of which is particularly likely to appear in ordinary conversation. The real choice, though, is between treating vocabulary as part of a strong education that incorporates a lot of reading in all the disciplines or teaching vocabulary explicitly with word lists, flashcards, quizzes, and high stakes standardized tests.  The surprising news is that the College Board seems to be adopting the first view, even if it has done a poor job in saying so.

***

There is a simple explanation why the SAT does not test words like transform and hypothesis.  They are too easy.  The linguist Dennis Baron, who runs The Web of Language, suggested that it is inevitable that some SAT words are on the obscure side. Baron said, “If [the College Board’s] goal is to [test] words that only the most massively memorizing kid is going to know. . . then [it is] going to have to reach.  It’s like the spelling bee,” although he acknowledged that the SAT’s words are not as extreme as those used by the Scripps National Spelling Bee, which itself added a vocabulary test this year. 

The College Board has to preserve the difficulty of the Reading section of the SAT, because it a norm-referenced exam: It ranks students by percentiles along a bell-curve distribution.  As a result, the exam must be designed so that only a small minority of test takers will be able to answer over 90 percent of the questions correctly.  In contrast, most state standards assessment tests are criterion-referenced exams.  They set a standard for test-takers to meet, with the hope that all of them will do so.  Everyone cannot get an 800 on the SAT Reading section, but all the students in the nation could test at or above grade level on a Common Core State Standards assessment exam; indeed, that is the aim of the Common Core. The difference between the SAT and standards tests lies, ultimately, in what they are designed to do.  State assessments are designed to identify failing students, while the SAT has historically been used to identify the most successful ones, those who can get into the relatively small fraction of selective schools among the nation’s 6,000-plus colleges and universities.

When Coleman and Montoya talk about the testing the words students “most need to know,” they make the SAT sound like a criterion -referenced exam, designed to shape teaching and curriculum.  Instead of talking about measuring a student’s potential for academic success, they make it sound like the exam should play a role in shaping that potential.  This is most likely not a mistake. Before Coleman became the president of College Board in 2012, he played an instrumental role in creating the Common Core Standards. It looks like he is now trying to bring the College Board’s work in line with the Common Core. In order to understand Coleman’s remark about SAT words, Georgia Scurletis wrote at Vocabulary.com, we need to understand how the Common Core thinks about vocabulary, and, in particular, about different classes of vocabulary.

The Common Core Standards are unambiguous about the value of vocabulary.  They state, “The importance of students acquiring a rich and varied vocabulary cannot be overstated.” They are just as clear about the importance of reading for this acquisition: “vocabulary acquisition eventually stagnates by grade 4 or 5 unless students acquire additional words from written context.”  The Common Core is no enemy of language, but an advocate for its study within its natural habitat.  It is a friend of reading.

The same cannot be said for the SAT.  From the exam’s inception in 1926 and almost without interruption, the SAT has relied heavily on testing vocabulary, perhaps most notoriously in its antonym questions, which asked examinees to pick the best antonym of a stem word from among five choices.  Since the mid-‘90s, the test has moved away from such contextless questions, which were particularly rewarding to those who memorized the dictionary definition of vast numbers of words.  The problem has not gone away, however.  Ben Zimmer, the executive producer of Vocabulary.com, suggested that “a lot of what might be considered the traditional SAT words might have been a little esoteric and also very specific in their meaning,” which meant that “you just learned this strange-sounding word and its meaning and you’re done,” rather than learning the kinds of words that take on different meanings in different contexts.

On the current SAT, vocabulary is tested in two ways: within the context of reading comprehension passages and in discrete sentence completion questions. Sherral Miller, Vice President of Assessment Development at College Board, reported in an email that both the reading comprehension passages and sentences used in the sentence completions are drawn from published material; they are not written for the test.  College Board does not use a word list for developing questions. The words they test, Miller wrote, “are all important words to know in academic and even civic life; words are not used if they are extremely rare or precious, overly discipline-specific, etc.”  Questions are reviewed by high school and college teachers to determine their “relevance” and “appropriateness,” and their difficulty is assessed by field-testing with students.  Miller did not explain what the criteria are for determining what is appropriate, relevant, difficult, or important to academic and civic life.

By far the most interesting revelation in Miller’s emails is:

Vocabulary in the new SAT will focus on multiple meaning words and phrases that ask examinees to determine their meaning based on the context in which they are used.   Testing to see if the examinee knows the one and only one meaning of a word will no longer be tested in the new SAT.  Rather, we will be testing students’ understanding of the meaning of words in context.

Describing the current test, Miller explained that “passage-based items [i.e., Reading Comprehension questions] assess multiple-meaning, context-dependent vocabulary” while Sentence Completions, Miller explained, “require vocabulary knowledge but also knowledge of sentence structure and logic.” The shift in emphasis on the new SAT will likely require a significant change to the Sentence Completions format, if not its outright elimination, since a single fill-in does not lend itself to testing different senses of a word.  The change might also affect the level of vocabulary tested. Currently, the context-dependent, passage-based questions test easier vocabulary, probably because asking students to tease apart different meanings of a word like baroque, which has been tested in Sentence Completions, is simply too difficult.  If the SAT does hold on to questions involving difficult vocabulary, they might be even harder than those on the current test.

***

Why was vocabulary tested in the first place on the SAT? It is not just the SAT, of course, that tests vocabulary, but a host of school assessment exams and IQ tests.  Vocabulary tests are easily administered, and the size of a person’s lexicon correlates surprisingly well with all kinds of desirable qualities. In a piece published earlier this year, E. D. Hirsch, the founder of the Core Knowledge Foundation, argued that the “key to increasing upward mobility is expanding vocabulary” since “there’s no better index to accumulated knowledge and general competence than the size of a person’s vocabulary.” Zimmer, too, said, “The scope of one’s vocabulary correlates quite well with general educational achievement and career success as well.”

And why is vocabulary such a good index? Is it only an index?  Is the size of a person’s vocabulary like the brightness of a parrot’s plumage, a form of display meant to signal fitness?  Or is a lexicon more like a beak, a tool that the stronger and sharper it is the greater the chances are for its owner to thrive?  Does knowing words show you are smart or make you smart?

If word knowledge plays the same role that plumage does, then studying vocabulary will do no more to cultivate intelligence than waxing and dying a parrot will add to its health. (Although a color and wax job might dupe a gullible bird into mating in the same way that using “SAT words” might make a person look intelligent without being so.)  If, on the other hand, vocabulary knowledge is itself a mode of intelligence, then it makes good sense to invest significant resources to expand the vocabulary of children and perhaps adults, too.

James Lee, a professor of psychology at the University of Minnesota whose research examines the genetic basis of intelligence, provided some background to explain why vocabulary might be one of our best indicators of a person’s general intelligence and not just an indication of a person’s socioeconomic class, upbringing, and, education.  These elements clearly play a role in determining what and how many words a person knows, as demonstrated in the influential work of Betty Hart and Todd Risley, who showed that educational inequalities among nine-year-olds from different socioeconomic correlated closely with the sheer number of words they heard in their first three years of life.  For Hart and Risley, words are beaks.

In classical IQ science, words are feathers because, without denying the importance of exposure to words, there is more to vocabulary acquisition and knowledge than exposure. In older, behaviorist models of word acquisition, children were believed to learn words simply through association. A kid hears the word “ball” in the presence of said object enough times, and he links the sound to the concept. 

Nowadays, Lee explained, thanks to the work of Paul Bloom and others, many psychologists think that word learning “is actually a quite rational process, and not , to concepts through repetition; they “parse what they hear into segments and use all the intellectual skills at their disposal to figure out what these individual words mean.”  It is a small cognitive marvel every time a child or an adult for that matter learns a new word, which is why a large vocabulary can be a sign of an innate intellectual ability. Lee warned that much more study remains to be done on this subject. It seems likely that, given an equal playing field, differences in innate intelligence probably account for some differences in the sizes of individuals’ lexicons, so, yes, vocabulary can serve a plumage function. The problem is that we are a long way from a society in which all children grow up in rich linguistic environments, and the insidious effects that appear to be a product of this particular kind of poverty suggest that words are also beaks that allow us to think more efficiently and with greater complexity about our world.

***

There are two fundamental ways difficult words make us smarter: They bundle concepts together so that they can be recalled and mentally manipulated. And they help us cut fine distinctions in our thinking about the world.  Some difficult words chunk and other chisel. Unfortunately, the same qualities that allow these words to contribute to our intelligence also make them more difficult to master and, thus, less frequently used in speech and writing, which only makes them harder to learn.

Borrowing the concept of chunking from cognitive psychology, Hirsch argued in “A Wealth of Words,”

Words are fantastically effective chunking devices. Suppose you put a single item into your working memory—say, “Pasteur.” So long as you hold in your long-term memory a lot of associations with that name, you don’t need to dredge them up and try to cram them into your working memory. The name serves as a brief proxy for whatever aspects will turn out to be needed to cope with your problem.

Of course, this chunking only works once you have mastered all the concepts that the word bundles together.  “Topology” would not strike most mathematicians as difficult, but that is only because they are immersed in their particular domain of knowledge. 

Someone looking in the dictionary at a definition of topology as it is used in math will almost certainly fail to understand its meaning, since he does not know the concepts the word brings to bear. What makes the word valuable to a mathematician is what makes it difficult for the layperson to learn the word.  It is not simply that he does not encounter it enough; when chunking words are difficult it is because what they refer to is difficult.  The way to master them is to master the domain in which they are found.  Gaining such knowledge entails gaining such words.

The same is not typically true of the other class of difficult words, the chisels.  They tend to be much more promiscuous, showing up in all kinds of domains, but tending perhaps to more writerly ones.  Their aim is not to synthesize, but to analyze.   The linguist Geoffrey Nunberg suggested that these kinds of difficult words make us smarter because they allow us to “carve up conceptual space more precisely.” Consider the example of “unscrupulous.”  The Microsoft Word thesaurus provides as synonyms for “unscrupulous” “dishonest,” “corrupt,” “dodgy,” “immoral,” and “ruthless.”  These words occupy a similar semantic space, but they are not equivalent to each other. One of the keys to speaking and writing eloquently is the power to select the most appropriate chisel for the context. You need to know, for instance, that it is appropriate to call the behavior of many mortgage lenders unscrupulous, but not to call a small child’s lie to his mother unscrupulous, since we do not expect young children to live according to principles. You would not find this distinction out, however, if you simply memorized the definition at Merriam-Webster.com:  “not honest or fair; doing things that are wrong, dishonest, or illegal.”

Knowing when to use unscrupulous depends not just on repeated textual encounters with the word as it is actually used, but on repeated encounters with similar words, with the whole semantic family of synonyms and antonyms for unscrupulous, since it is through them that a person can learn how similar words are different. The literary scholar and former Oxford Professor of Poetry Christopher Ricks suggested that the SAT “is stupid about what words are like.  [W]ords get their meaning and have their force like iron filings, from a whole field of them,” which means that understanding a word means encountering a field of meaning.  “If learning new vocabulary words in theory is a question of carving up conceptual space more precisely,” Nunberg said, “that can only come with reading and acquiring the conceptual distinctions.” 

The problem with semantic families is that they tempt the brain to take a shortcut, to go for the more familiar word and forget the less familiar one, particularly if someone does not see the difference between the two. The College Board might be right that most people will never use “SAT words” again after they learn them for the SAT, but that probably has much less to do with the words themselves and more with the way students learn them.  “Unscrupulous” is only useless if you don’t appreciate why you would choose it over “dishonest” or “corrupt.”

Chunking words and chiseling words present a serious problem for the College Board as it reconsiders its approach to testing vocabulary.  Miller wrote in her email that the current SAT “does not directly test domain-specific words” because “doing so would privilege those candidates who had content knowledge in that domain.” That would seem to rule out that the difficult chunking words, unless the College Board changes its policy on domain-specific words, which it well might in years to come, with the Core Curriculum in place. Flexible, chiseling words that represent more nuanced synonyms of simpler concepts (e.g., meretricious and cheap) present a different problem.  After suggesting that knowing a word like “arcane” requires us not just to know its dictionary definition but to know its difference from “obscure” and “recondite, ” Nunberg said, “the College Board cannot ask you a question in which knowledge of the difference between “arcane” and “recondite” will be important to the answer.” The distinction is too fine and too open to challenge.

***

If difficult vocabulary does disappear from the SAT, what will the consequences be for the language as it is written and spoken in this country? I suspect they will be negligible, given how little long-term benefit there is to flashcard-style studying.  In an age when more and more people are carrying around multiple dictionaries on their phones and their reading devices, the notion of memorizing hundreds or even thousands of words could come to seem antiquated, like knowing how to use a slide rule or being able to spell. When I brought this up to Dennis Baron, he explained that often when new technologies with pedagogical impacts appear, such as the word processor’s spell-checker, the calculator, or the pencil with a built-in eraser, there are heated debates over whether the machines will prevent students from learning what seem like essential skills, such as long division, spelling, or taking the time to figure out exactly what you are going to write before you write.  These technologies are not only assimilated; they eventually become essential to education. “Now instructors complain if students don’t run the spell-check,” Baron said.

Technology could end up making it much easier for students to explore the meanings of difficult words within a meaningful context rather than a vacuum. Baron admitted there is some difference—the tablet dictionary will not work if you have to look up every third word. But the spellchecker is subject to much the same principle, and there can be little doubt that while it has had a negative effect on individuals’ abilities to spell it has had a positive effect on our society’s ability to do so. It is probably not a coincidence that the National Spelling Bee has become a live television event in the age when fewer people need to know how to spell. Perhaps this will be the fate of vocabulary, thanks in part to the new SAT.  We will turn on ESPN to marvel at schoolchildren defining “rodomontade,” “byzantine,” and, perhaps, “unscrupulous.”