In her widely lauded speech at the Democratic National Convention on Monday, Michelle Obama reflected on the remarkable fact of her African American family living in the executive mansion. “I wake up every morning in a house that was built by slaves. And I watch my daughters, two beautiful, intelligent, black young women, playing with their dogs on the White House lawn,” she said.

On Tuesday, Fox News host Bill O’Reilly discussed the moment in his Tip of the Day. In a moment first noticed by the liberal press-tracking group Media Matters, O’Reilly said this:

As we mentioned, Talking Points Memo, Michelle Obama referenced slaves building the White House in referring to the evolution of America in a positive way. It was a positive comment. The history behind her remark is fascinating. George Washington selected the site in 1791, and as president laid the cornerstone in 1792. Washington was then running the country out of Philadelphia.

Slaves did participate in the construction of the White House. Records show about 400 payments made to slave masters between 1795 and 1801. In addition, free blacks, whites, and immigrants also worked on the massive building. There were no illegal immigrants at that time. If you could make it here, you could stay here.

In 1800, President John Adams took up residence in what was then called the Executive Mansion. It was only later on they named it the White House. But Adams was in there with Abigail, and they were still hammering nails, the construction was still going on.

Slaves that worked there were well-fed and had decent lodgings provided by the government, which stopped hiring slave labor in 1802. However, the feds did not forbid subcontractors from using slave labor. So, Michelle Obama is essentially correct in citing slaves as builders of the White House, but there were others working as well. Got it all? There will be a quiz.

O’Reilly’s comments were a small line at the end of the show, so one shouldn’t make too much of them. But the riff is notable for the way it resuscitates two common threads of apology for slavery: First, that human bondage didn’t play as large a role in American history as its critics might have you believe; and second, that it wasn’t as bad as they’d have you believe, either.

Obama’s invocation of the slaves who built the White House was rhetorically powerful because the building is a symbol of the United States; to say slaves built it is to imply that they built the United States. Pointing out that it was not only slaves who built it (in essence, debunking an assertion that Obama did not make) undercuts the point, and in turn connects to the long tradition of arguing that however bad slavery may have been, it was only a small, regional phenomenon in the South. But as my colleague Ta-Nehisi Coates has written, slavery was the engine of the entire antebellum American economy: “In the seven cotton states, one-third of all white income was derived from slavery. By 1840, cotton produced by slave labor constituted 59 percent of the country’s exports.” Coates quotes historian David Blight, who wrote, “Slaves were the single largest, by far, financial asset of property in the entire American economy.”

More immediately controversial was O’Reilly’s statement that slaves working at the White House “were well-fed and had decent lodgings provided by the government.” The idea that slavery was, if not a good thing, not such a bad thing overall has a long, if hardly venerable, intellectual history.

This defense of slavery as essentially benign runs, for example, through Senator John C. Calhoun’s 1837 speech defending the “peculiar institution” from the criticism of northerners. In a speech in which he labeled slavery “a positive good,” the South Carolinian contended that enslaved blacks actually had it good:

Compare his condition with the tenants of the poor houses in the more civilized portions of Europe—look at the sick, and the old and infirm slave, on one hand, in the midst of his family and friends, under the kind superintending care of his master and mistress, and compare it with the forlorn and wretched condition of the pauper in the poorhouse.

Some 20 years later, another senator from South Carolina, James Henry Hammond, made the same point. Speaking to Northerners, he contended:

The difference between us is, that our slaves are hired for life and well compensated; there is no starvation, no begging, no want of employment among our people, and not too much employment either. Yours are hired by the day, not cared for, and scantily compensated, which may be proved in the most painful manner, at any hour in any street in any of your large towns. Why, you meet more beggars in one day, in any single street of the city of New York, than you would meet in a lifetime in the whole South.

Of course, that wasn’t quite all of it. Hammond added that the manifest inferiority of blacks further not only justified but practically sanctified the endeavor:

We do not think that whites should be slaves either by law or necessity. Our slaves are black, of another and inferior race. The status in which we have placed them is an elevation. They are elevated from the condition in which God first created them, by being made our slaves. None of that race on the whole face of the globe can be compared with the slaves of the South. They are happy, content, uninspiring, and utterly incapable, from intellectual weakness, ever to give us any trouble by their aspirations.

It is not hard to find contemporary accounts from slaves who were neither happy nor content—and whose eloquent testimony accounts to their inspiration, capability, and intellectual powers. If many whites like Hammond were unable to see this, however, others were not so blinkered. As O’Reilly noted, Michelle Obama’s predecessor as first lady, Abigail Adams was living in the White House at the time when slaves were building it, and she recorded her observations of those working on landscaping the grounds.

“The effects of Slavery are visible every where; and I have amused myself from day to day in looking at the labour of 12 negroes from my window, who are employd with four small Horse Carts to remove some dirt in front of the house,” she wrote. Moreover, Mrs. Adams took note of their condition—and her observation stands at odds with O’Reilly’s:

Two of our hardy N England men would do as much work in a day as the whole 12, but it is true Republicanism that drive the Slaves half fed, and destitute of cloathing, ... to labour, whilst the owner waches about Idle, tho his one Slave is all the property he can boast.

Adams’s rebuke to O’Reilly is not the first time that a benign recollection of slavery has broken apart on the shoals of reality.