Surveys Dramatically Underestimate Homophobia

Using a technique for eliciting confessions, researchers find there are more people who don't like gays—and who have had same-sex experiences—than commonly measured.

"Secrets," Luc Tuymans, 1990 (WikiPaintings)

Americans are much more homophobic—and much more likely to have had a same-sex experience—than surveys done using traditional methods reveal, according to a new National Bureau of Economics Research working paper.

Ohio State University Professors Katherine Coffman and Lucas Coffman and Boston University School of Management Professor Keith Marzilli Ericson used a survey research method known as "veiled" reporting to test an array of statements on hot-button issues such as same-sex marriage, homophobic beliefs, and history of same-sex experiences. Unlike traditional anonymous online-survey methods designed to elicit true answers on stigmatized beliefs or practices, this method groups questions so that respondents don't have to answer them directly.

For example, in a normal survey, a person would be asked to answer yes or no about a statement. But in veiled reporting, statements like "I consider myself a heterosexual" are grouped with a bunch of statements like "I remember where I was the day of the Challenger space shuttle disaster," and respondents are asked to say how many statements in the group are true, without having to specify which ones they are saying yes to. A gap between the responses of a control group answering directly and those answering through the veiled-questioning system has been shown to exist for questions where there is a "social desirability bias"—which is to say, where people are inclined to give what they think is the socially acceptable answer.

The authors tested whether questions about sexuality showed "evidence of social desirability bias even when asked in a self-administered, computer-assisted survey."

Their findings: Yes, even in 2012, and "even under extreme privacy and anonymity."

The veiled method increased self-reports of non-heterosexual identity by 65 percent, same-sex sexual experiences by 59 percent, and directionally same-sex attraction by 9.4 percent. We combine all own-sexuality questions into an index, and find that the Veiled Report treatment significantly raises the number of sensitive answers overall.

The veiled method also increased the measured rates of anti-gay sentiment. Respondents were 67 percent more likely to express disapproval of an openly gay manager at work, 71 percent more likely to say it should be legal to discriminate in hiring on the basis of sexual orientation, 22 percent less likely to support the legality of same-sex marriage, 46 percent less likely to support adoption by same-sex couples, and 32 percent less likely to state they believe homosexuality is a choice. We again combine all the opinion questions into an index, and find that the Veiled Report treatment significantly raises the overall number of intolerant answers. Taken together, these results indicate that both non-heterosexuality and anti-gay sentiment are substantially underestimated in existing surveys.

Those results would seem to be mutually reinforcing and suggest that people know that there is more anti-gay bias than is generally admitted these days and so are extremely reticent to admit to having had same-sex experiences, especially if they do not identify as gay or lesbian.

The sample was not a representative one and so can't be taken to show the full extent of opinion or behavior in these areas—only that traditional survey methods aren't picking them up accurately, even among a fairly young and not wildly conservative sample. Those surveyed had a median age of 26, and less than 16 percent said they were Republicans.

Because they were measuring stigma, the authors postulated that "under reporting of non-heterosexuality in the Direct Report treatment ... should be larger for demographic groups with social norms that are perceived as less LGBT-friendly: Christians, older respondents, and Black/African Americans." And that's in fact what they found:

Among Christians in our sample, the Veiled Report condition raises reports of non-heterosexuality by 13 percentage points (from 8 to 21 percent) in Question 1 and same-sex sexual experience s by 14 percentage points (from 11 to 25 percent) in Question 3, compared to the Direct Report. These are increases of 163 and 127 percent, respectively. Among participants with no religious affiliation, the Veiled Report treatment produces much smaller differences in these questions.

In short, though surveys show growing support for LGBT rights, there's still plenty of bias below the surface.