A reader writes:
Don't forget James Buchanan, our 15th president. Lifelong bachelor, lived with William Rufus King, and at least a couple people at the time referred to them as the "two queens." This could play into the taboos you were talking about; people are more willing to associate Buchanan with being gay because he's one of the worst presidents in American history.
Another writes:
Buchanan lived with King for 15 years prior to King’s election as vice president under Franklin Pierce. King died before being inaugurated, and Buchanan was so bereft, some have suggested, that President Pierce sent him to the Court of St. James to be away from the sad memories connected with the house the two bachelors had shared together.
More details from Wikipedia:
Buchanan and King's close relationship prompted Andrew Jackson to refer to King as "Miss Nancy" and "Aunt Fancy", while Aaron V. Brown spoke of the two as "Buchanan and his wife". Further, some of the contemporary press also speculated about Buchanan and King's relationship. Buchanan and King's nieces destroyed their uncles' correspondence, leaving some questions as to what relationship the two men had, but surviving letters illustrate the affection of a special friendship, and Buchanan wrote of his communion with his housemate.
Buchanan wrote in 1844, after King left for France, "I am now solitary and alone, having no companion in the house with me. I have gone a wooing to several gentlemen, but have not succeeded with any one of them. I feel that it is not good for man to be alone; and should not be astonished to find myself married to some old maid who can nurse me when I am sick, provide good dinners for me when I am well, and not expect from me any very ardent or romantic affection."