2. Richard Johnson
You wouldn't think having sex with your wife could cause a scandal, but if it was the 1820s, you were white, and that wife was black, it was shocking enough to hurt your career. Johnson, a senator and the ninth vice president of the United States, openly kept one of his slaves as his common-law wife and even publicly acknowledged his two children with her. While his constituency wasn't bothered by this at first, as word of his situation spread, his career took a hit—and he lost his seat in the Senate.
Johnson pointed out that he was far from the only politician to have a relationship with a slave, and he defended his honor as being better than others who were secretive about their affairs, saying, "Unlike Jefferson, Clay, Poindexter, and others, I married my wife under the eyes of God, and apparently He has found no objections." Despite his love for his wife, Johnson never freed her and she was his slave until she died.
3. John Eaton
Peggy O'Neale was just 17 when she married 39-year-old Navy purser John Timberlake. While John was away at sea for months at a time, Peggy would host prominent politicians in her Washington home. Then John died on one of his voyages and Peggy found herself a young widow with two children to support. Thankfully, Senator John Eaton, an old friend, was there to help her pick up the pieces. The two married almost immediately after her husband's death.
While today getting remarried so quickly might raise a few eyebrows, in the 1800s this was just not done. There were strict rules on mourning, and waiting less than a year before getting hitched again indicated a ferocious sex drive or the existence of a previous affair. The O'Neale-Eaton marriage scandalized the women of Washington, who made sure their husbands knew just how to feel on the matter. In 1829, President Jackson tried to show support for the couple by making Eaton his Secretary of War. But by 1831 the scandal had engulfed Jackson's administration, and all but one member of his cabinet resigned. All because Eaton married a pretty young widow too quickly.
4. James Henry Hammond
Over the course of a quarter century, from 1835 to 1860, James Hammond was a member of the House, a senator, and governor of South Carolina. However, he only spent a total of six of those 25 years holding office, due mostly to his questionable sex life. In college, Hammond had a gay affair with a friend (which is documented in a series of explicit letters kept at the South Caroliniana Library), and rumors of this followed him his whole life. When he got older, he had relationships with four of his own nieces; in his diaries, he blamed the girls for coming on to him. When these relationships came to light, Hammond had to leave the national scene for 13 years before his reputation recovered enough to allow him to get reelected; the girls had their reputations tarnished forever, and never married.
A version of this post originally appeared on Mental Floss, an Atlantic partner site.