If the Supreme Court’s landmark decision on same-sex marriage is a slippery slope, where does it lead?

The most common—and often ridiculed—hypothetical that critics of the ruling cite is polygamy. If a marriage between two men or two women is suddenly a constitutional right, why can’t a group of people decide to wed? Yet to Justice Samuel Alito, the consequences of the high court decision go far beyond marriage. In a lengthy and revelatory interview with Bill Kristol of the Weekly Standard published Monday morning, Alito suggested that the concept of “liberty” envisioned in Justice Anthony Kennedy’s majority opinion could lead to a parade of potential horribles, and some of them might be just as concerning to progressives as they are to conservatives:

And there are many other conceptions. The Court’s conception, I said in this opinion and I believe to be true, is a very postmodern idea; it’s the freedom to define your understanding of the meaning of life. Your—it’s the right to self-expression. So if all of this is on the table now, where are the legal limits on it?

If a libertarian is appointed to the Supreme Court, is it then proper for the libertarian to say, “Well, I think that there is a right to work less than the minimum wage? I think there is a right to work as many hours as I want without being limited by the government. I think I have the right to build whatever I want on my property irrespective the zoning laws and so forth.”

If a socialist is appointed to the Supreme Court, can the socialist say, “I think that liberty and the 14th Amendment means that everyone should have a guaranteed annual income or that all education through college should be absolutely free,” or whatever. There’s no limit.

The entire interview runs more than an hour, but it’s worth watching for Alito’s extended commentary on the marriage case and the rare light it sheds on a justice who generates significantly less discussion than his conservative colleagues on the bench (particularly Chief Justice Roberts, Antonin Scalia, and Kennedy, the frequent swing vote). More predictable in his voting than Roberts or Kennedy, and less colorful than Scalia, Alito has sat on the Court for nearly a decade but is perhaps best known to the general public for shaking his head and mouthing “Not true” when President Obama criticized the Citizens United decision during his State of the Union address in 2010. (One noteworthy tidbit from the interview is that Alito remembers his father personally drawing up new legislative districts for New Jersey after a 1964 Supreme Court ruling created the standard of “one person, one vote”; the Court will consider that principle anew next term.)

Alito expands significantly on his dissent in the marriage case (one of four total), which touched on his slippery-slope concerns but focused more on the traditional definition of marriage and fears that the ruling would be used to “vilify” those who support it. If there’s a theme to Alito’s voting record on the court, it’s that he’s no libertarian. As he explained to Kristol, he was on the opposite side of the slippery-slope argument in a pair of earlier First Amendment cases, when he dissented from the high court’s expansive reading of the freedom of speech. In one case, Alito was the lone dissenter in a decision striking down a federal law banning the sale of so-called “crush” videos depicting animal cruelty. And he was again alone in opposing the Supreme Court’s 8-1 ruling in favor of Westboro Baptist Church, the anti-gay group that protests at military funerals. In the interview, Alito explained that those restrictions were “peripheral” to the core “political speech” principle of the First Amendment—unlike, say, the question in Citizens United of whether financial contributions constituted political speech, a decision that he emphatically defended.

The same-sex marriage case, Alito said, turned on the definition of the word “liberty” in the Fourteenth Amendment. Historically, the Court had limited liberty to rights “that were deeply rooted in the traditions of the country.” But Kennedy’s marriage ruling, he lamented, “threw that out.”

We are at sea, I think. I don’t know what the limits of substantive liberty protection under the Fourteenth Amendment are at this point.

Yet as Michael Dorf notes, Alito’s desire to rely on historical interpretation is no more than an ideological choice itself. “The conservatives’ use of history is highly selective,” said Dorf, a professor of constitutional law at Cornell University. He pointed out that it was Alito and his fellow conservatives on the Court who threw out the Court’s historical interpretation of the Second Amendment in finding an individual right to bear arms. They similarly had little use for history in their recent approach to affirmative action, he said. “Why do the conservatives, including Justice Alito, say nary a word about the historical understanding of race-conscious government programs when they strike down affirmative action programs?” Dorf asked. “Could it be because their own ideological preferences make what they would find there uncomfortable?”

In his interview with Kristol, Alito went on to suggest more worrisome implications for the Court's marriage ruling, predicting that the “hotly-debated issue” would linger.  “It raises questions of legitimacy, it raises practical questions because the more the Court does this sort of thing, the more the process of nomination and confirmation will become like an election,” he said. “It will become like a political process.” As Alito well knows, it's hard to see how the Senate confirmation process could become any more politicized than it has been in the last decade. (On the other hand, imagine President Hillary Clinton trying to persuade a Republican Senate to confirm her nominee to replace, say, a retiring Antonin Scalia.)

The more interesting question that Alito raises is whether the Court has actually expanded the definition of liberty to the degree he suggests it has. It's similar to the one Jeffrey Rosen posed before the marriage decision in contemplating Kennedy’s “right to dignity.” He noted that in opposing Kennedy’s 2003 opinion in the Lawrence v. Texas case invalidating sodomy laws, Scalia said the ruling “decrees the end of all morals legislation.” Alito is going even further by suggesting that a ruling ostensibly limited to social policy could threaten economic regulations as well. It seems like quite a reach, but if he’s right, the Supreme Court’s ruling last month—already considered a landmark—would be far broader than almost anyone could imagine.