U.S. Supreme Court justices don’t typically appear on late-night TV, so Justice Stephen Breyer’s visit to the Late Show with Stephen Colbert could have been something special. Instead, it felt like a missed opportunity.

Breyer, fresh off one of the Supreme Court’s most historic terms in decades, received second billing to Emily Blunt, who promoted her upcoming action film. The justice’s slow, measured tone didn’t really gel with Colbert’s bantering interviewing style. Breyer didn’t get a chance to discuss his new book, which ponders the Supreme Court’s evolving (and controversial) relationship with foreign legal thinking. Questions about jurisprudence or recent decisions were neither asked nor answered. But Breyer had a chance to give a convincing argument against cameras in the courtroom, and he emphatically defended the Court’s collegiality from Colbert’s hints of acrimony.

Journalistic interviews with the justices are increasingly common and often good, but rarely great; New York magazine’s 2013 conversation with Antonin Scalia is a delightful exception. All nine of them speak often in public, although their audiences are usually law schools, state bar associations, judicial conferences, or similar law-related organizations. Some justices occasionally make unconventional appearances, like Sonia Sotomayor’s visits to The Daily Show with Jon Stewart and Sesame Street. But lawyers talking to lawyers is the norm.

That could change, however. UC Irvine law professor Rick Hasen provided some interesting data about their public appearances in a recent study. All of the current justices have been more public than their predecessors, save for Justice Arthur Goldberg, who spoke frequently about anti-Semitism during the 1960s. Scalia and Ruth Bader Ginsburg tend to receive more press for their remarks, but Breyer actually holds the record for the most public appearances by a justice since 1964.

Are these public appearances good for the Court? Judge Richard Posner, a popular federal appellate judge in the Seventh Circuit, disapproves of what he calls the justices’ “public intellectual” activities. At the same time, he doesn’t see them as a threat to the Court’s legitimacy, since institutional support for the Court remains relatively high. Besides, he adds, few Americans can even identify individual justices. Colbert also observed before his chat with the justice that 3 percent of Americans know who Breyer is. The rest, he joked, confuse him with Mr. Burns from The Simpsons, with whom the justice shares a vague resemblance.

But by my count, Breyer has elevated two significant legal issues within the past year: the death penalty, in his dissent last term renouncing it in Glossip v. Gross, and—in his new book—the globalization of American law. Breyer also appeared alongside Justice Anthony Kennedy at a House Appropriations Subcommittee hearing on the federal judiciary’s budget in March. There, Breyer criticized mandatory-minimum sentences and Congress’s piecemeal approach to sentencing. Kennedy also criticized mass incarceration and solitary confinement more broadly.

The nine justices are—for better or worse, whether you agree with them or not—America’s most important public intellectuals. Breyer and Kennedy’s remarks in March seem salient amid the current national debate over mass incarceration, but they’re not new arguments for either justice. Breyer told the subcommittee he has frequently criticized mandatory minimums before. Imagine if he had critiqued it for a national TV audience a decade ago. Imagine if he had been asked about it now.