Harvard's Impossible Personality Test

New revelations from a lawsuit against the university capture the difficulty of comparing thousands upon thousands of talented teenagers.

Brian Snyder / Reuters

Every year, Harvard’s admissions officers are charged with whittling a batch of 40,000 applicants down to a bare-bones selection to fill the institution’s roughly 1,600 freshman seats. Those officers can’t just set a high bar for test scores and GPAs to arrive at that selection; far too many candidates boast those qualifications. Plus, even if that arithmetic were possible, Harvard prides itself as an institution where students learn from each other in addition to their professors. Building each freshman class is, in turn, both an art and a science—one that effectively requires admissions officers to exercise a combination of number-crunching and intuition.

An organization representing Asian Americans who were at some point rejected from Harvard is suing the school for that art-and-science approach. The group, Students for Fair Admissions (SFFA), recently reignited tensions around that approach with a new legal filing that it says confirms suspicions that the university’s admissions practices discriminate on the basis of race. The documents, filed on Friday in federal court by SFFA, could influence admissions policies at universities across the country; the case is slated to go to trial in October.

One of the most striking revelations pertains to Harvard’s consideration of applicants’ soft skills—things like “likability,” “helpfulness,” “integrity,” and “courage”—in determining their acceptance. Despite boasting higher test scores, better grades, and stronger extracurricular resumes than applicants of any other racial group, Asian American applicants consistently received lower rankings on those personality traits, according to a statistical analysis conducted on behalf of SFFA of more than 160,000 student records. This emphasis on personality, the analysis concludes, significantly undermined otherwise-qualified Asian Americans’ chances of getting in.

The news media have largely fixated on whether the evidence contained in the filing is the “smoking gun” needed in the longstanding crusade to crack down on admissions practices at the country’s premier institution of higher learning, practices that until now the school hadn’t disclosed to the public. Whether the evidence is incriminating isn’t clear, but it does reveal how tricky it is to assign value to personality traits in college admissions. In offering a glimpse into how the emphasis on those traits might favor some races over others, the filing points to a puzzle whose solution could be integral to the quest to get affirmative action right: Is it possible to define a characteristic as intangible and subjective as good personality in a way that protects students against people’s deep-seated, and often subconscious, biases?

“However you measure these things—whatever you do [in the admissions process]—is going to privilege one group and disadvantage another group,” said Natasha Warikoo, an associate professor of education at Harvard who wrote the book The Diversity Bargain, which explores the role of race at elite universities. “There’s no such thing as a perfect admissions system that leads to ‘a meritocracy.’” Warikoo wasn’t involved in this lawsuit.

SFFA’s filing includes a report by Peter Arcidiacono, a Duke University economist, that delves into the process by which Harvard rates applicants in five main categories: academic, extracurricular, athletic, personal, and overall. Arcidiacono shows that, after narrowing down applicants to those with the strongest objective academic qualifications, Asian Americans were far more likely than blacks or Hispanics to receive a low personality score from admissions officers. The analysis juxtaposes the Asian Americans’ personality scores from admissions officers with the relatively high personality ratings given by alumni interviewers, as well as with the relatively positive feedback provided to Harvard by applicants’ teachers and counselors, for those same students.

Arcidiacono concludes that this apparent personality-score phenomenon leads to what he describes as “the Asian-American penalty”; if that so-called penalty were removed, he contends, Asian Americans’ admission rate would increase from 5.2 percent to 19.2 percent. Is it possible to justify the glaring discrepancies identified by Arcidiacono? Should Harvard admit one out of every five Asian American applicants to ensure its admissions practices are “fair”?

Arcidiacono’s data certainly indicate a correlation between an applicant’s race and her personality rating—among the four main racial groups, Asian Americans have the lowest share receiving a top personality score even though, for all racial groups, applicants with top academic qualifications are most likely to receive top personality ratings. But establishing a causal relationship between students’ race and their personality scores is tricky.

In its 2016 decision on Fisher v. University of Texas, the U.S. Supreme Court confirmed that race can be used as one of many factors in admissions, reiterating the notion that a diverse student body is educationally beneficial. In assessing applicants based on a nebulous suite of academic and non-academic qualities, Harvard’s policy makes it all but impossible to draw a statistically—and legally—airtight conclusion about why one applicant was chosen over another. Further complicating matters is the ambiguity Harvard’s admissions office assigns to the “personal” category. While the other categories come with detailed and straightforward criteria, a vague rubric determines applicants’ personality rating; “outstanding” personal skills secure an applicant a top score, for example, and being “bland or somewhat negative or immature” gets her a middling one.

Harvard has long argued, as it does in its own expert analysis filed Friday, that diversity is inherent to its broader mission in justifying its “holistic” approach to admissions. In pursuit of this mission, admissions officers every year are charged with constructing a freshman class that features students who express an array of academic interests; who represent a range of cultural, socioeconomic, and geographic backgrounds; who bring a breadth of sought-after personality traits, like humor and sensitivity, that may be hard to define but often have intuitive appeal.

Against this backdrop, it’s easy to chalk up the correlation between Asian Americans’ racial identity and their relatively low personality scores to mere coincidence. And that’s essentially what Harvard has done in response to any critique of its admissions practices. The basis of SFFA’s filing was an internal investigation conducted by Harvard in 2013 that found that being Asian American puts applicants at a disadvantage. But as the school alludes in its latest court filing, such findings—the 2013 analysis for its part focused on the hypothetical impact of an academics-only policy—amount to little more than an unintended byproduct of what one official, according to Inside Higher Ed, described in an internal memo as “relative trade-offs.” Were the institution to judge students only on objective academic metrics—a very rare practice among selective U.S. colleges and universities—the share of Asian Americans in a given class would more than double, yet that of blacks and Latinos would be significantly reduced. An academics-only approach, the thinking goes, would undermine Harvard’s ability to be the institution it strives to be.

“The reality is that Asian Americans don't do as well on” certain measures that Harvard prioritizes when building a class, said Warikoo, who’s of Indian descent. ”We tend to live on the coasts, are less likely to be recruited athletes or legacies, more likely to want to study medicine,” and so on. Determining whether that’s a fair outcome of Harvard’s emphasis on applicants’ non-academic qualities, she said, “depends on what we think is important for admission—how does the system of selection further the university’s goals?”

Efforts to arrive at a consensus on that issue have fallen flat. Yet, regardless of the merits of SFFA’s lawsuit (and of Harvard’s response), there’s compelling evidence that the emphasis on character traits disproportionately undermines Asian Americans’ admissions prospects. And given how subjective personality is, implicit biases against that racial group could explain the trends Arcidiacono identified; Asian Americans, for example, are consistently regarded as more “foreign” than other racial groups and are widely stereotyped as being reserved. Perhaps these hypothetical biases, and the “trade-offs” that hypothetically feed them, are worth scrutiny that could advance the affirmative-action debate past the standard ideological arguments. Few would argue that an individual’s worth stops at her success in school—that limiting an evaluation of her strengths and weaknesses to easily quantifiable metrics is a productive means at ensuring Harvard and other schools provide a “world-class education.” But SFFA’s latest filing brings into sharp relief the difficulty of relying on metrics that extend beyond the classroom and in turn necessitate some degree of gut instinct.

Warikoo in our conversation broached the idea of a lottery system into which Harvard would feed all applicants who meet certain criteria, establishing weights for particular categories. Not only could that system help reduce the consequences of the current approach to students’ personalities, it could also free up resources that could, say, instead be invested in financial aid; according to Harvard’s own Friday court filing, a team of roughly 40 officers votes on each final freshman admissions decision at the school.

But while they could spark discussions about alternatives to the use of race in achieving socioeconomic equity, neither SFFA’s case against Harvard nor a similar probe by the Justice Department is likely to result in a simple rethinking of resource-allocation. Nicole Ochi, an attorney with Asian Americans Advancing Justice, said she fears SFFA’s suit could have far-reaching negative consequences by prompting other higher-education institutions across the country to roll back the extent to which they consider race in admissions or eliminate the practice altogether. Research has already shown that cases like this one have a chilling effect on affirmative action: One study, for example, found that the rate of “very competitive” schools, including private institutions, that factor race into admissions has plummeted in recent decades, dropping from 75 percent in 1994 to 47 percent in 2014. If the court rules that Harvard’s policy is unconstitutional, Ochi argued, “then that will be the death knell for affirmative action everywhere, throughout the country.”

That prospect makes Ochi and other supporters of affirmative action all the more frustrated with Harvard’s reliance on such a vague means of evaluating applicants’ personalities, and with the school’s failure until now to publicly disclose exactly how character traits factor into that evaluation. For advocates of affirmative action, these omissions allowed SFFA to conflate the consideration of race in admissions with discrimination against Asian Americans—even though there’s little hard proof that allowing colleges to consider race in itself disadvantages them.

“I’m willing to entertain the notion that there implicit bias is at work in the personal score that results in some discriminatory impacts against Asian Americans,” Ochi said, “but I do not think that that provides evidence that the consideration of race in the admissions process is causing that—especially since that score doesn’t consider race.” In other words, Ochi and others contend that if something is indeed amiss in Harvard’s admissions office, the source of the problem is implicit bias. It isn’t affirmative action.