The best model used just five of the methylation marks, and correctly classified the twins 67 percent of the time. “To our knowledge, this is the first example of a biomarker-based predictive model for sexual orientation,” Ngun wrote in his abstract.
The problems begin with the size of the study, which is tiny. The field of epigenetics is littered with the corpses of statistically underpowered studies like these, which simply lack the numbers to produce reliable, reproducible results.
Unfortunately, the problems don’t end there. The team split their group into two: a “training set” whose data they used to build their algorithm, and a “testing set,” whose data they used to verify it. That’s standard and good practice—exactly what they should have done. But splitting the sample means that the study goes from underpowered to really underpowered.
There’s also another, larger issue. As far as could be judged from the unpublished results presented in the talk, the team used their training set to build several models for classifying their twins, and eventually chose the one with the greatest accuracy when applied to the testing set. That’s a problem because in research like this, there has to be a strict firewall between the training and testing sets; the team broke that firewall by essentially using the testing set to optimize their algorithms.
If you use this strategy, chances are you will find a positive result through random chance alone. Chances are some combination of methylation marks out of the original 6,000 will be significantly linked to sexual orientation, whether they genuinely affect sexual orientation or not. This is a well-known statistical problem that can be at least partly countered by running what’s called a correction for multiple testing. The team didn’t do that. (In an email to The Atlantic, Ngun denies that such a correction was necessary.)
And, “like everyone else in the history of epigenetics studies they could not resist trying to interpret the findings mechanistically,” wrote John Greally from the Albert Einstein College of Medicine in a blog post. By which he means: They gave the results an imprimatur of plausibility by noting the roles of the genes affected by the five epi-marks. One is involved in controlling immune genes that have been linked to sexual attraction. Another is involved in moving molecules along neurons. Could epi-marks on these genes influence someone’s sexual attraction? Maybe. It’s also plausible that someone’s sexual orientation influences epi-marks on these genes. Correlation, after all, does not imply causation.
So, ultimately, what we have is an underpowered fishing expedition that used inappropriate statistics and that snagged results which may be false positives. Epigenetics marks may well be involved in sexual orientation. But this study, despite its claims, does not prove that and, as designed, could not have.