Though Americans argue over whether income taxes should be higher or lower, there is consensus that they are a part of life. Inheritance and estate taxes, though, enjoy no such acceptance. Americans simply don’t like the concept of taxing inheritances, but the estate tax actually meshes well with the cherished American ideal of fairness—in fact, its vilification is partially the result of a calculated campaign on the part of those whom it benefits most.

An estimated $1 trillion is predicted to be passed down from estates annually in the coming decades, but the tax aimed at that money applies extremely infrequently: The left-of-center Center on Budget and Policy Priorities estimates that only two out of every 1,000 estates pay any taxes on what’s handed down to the next generation. And that number represents a significant decline from what laws required a decade ago. In 2014, the IRS reported there were only 11,931 estate tax returns filed for the entire year, down 74 percent from 2005.

Moreover, in 2001, up to $650,000 of any American’s estate was exempt from being taxed, and the top rate on wealth beyond that was 55 percent. Since then, the bar for exemptions has risen every year to the point where last year, up to $5,430,000 of each estate was exempt, with the top rate on wealth beyond that falling to 40 percent.

With an estimated 99.8 percent of all estates exempt from taxes, the Congressional Budget Office reports that in 2014, estate tax revenues only amounted to $19 billion. Considering that the IRS collected $1.4 trillion in income taxes that same year, it is not hard to imagine a reversed, but equally sustainable, system: What if a $5.43 million exemption applied to accumulated income, and no such exemption existed for estates? Making a change like this revenue-neutral would require some working out, but it would send a message that the country promotes equal opportunity, and distributes rewards based on contribution and work.

Today, estates represent an immense potential tax base, yet there is little political will to tap them. This is not some American tradition but in fact a departure from it: As The Economist has noted, America’s early state governments threw out laws that encouraged the accumulation of wealth over generations, following the example Thomas Jefferson set with the Virginia legislature in 1777. They got rid of the English legal precedents of primogeniture and entail—under which titles and property were inherited in their entirety by the oldest male heir—forcing families to divvy up their wealth among more children. Jefferson cited Adam Smith, who called the idea that people should control their estates well beyond their death “manifestly absurd.” Jefferson insisted that “the earth belongs in usufruct to the living.”

Similarly, Alexis de Tocqueville identified the breaking-up of estates as one of the cornerstones of the young country’s success. “What is most important for democracy is not that great fortunes should not exist,” he wrote, “but that great fortunes should not remain in the same hands. In that way there are rich men, but they do not form a class.”

The estate tax that exists today was enacted in 1916. In the years since, the tax’s critics—who, unsurprisingly, tend to be wealthier than its proponents—have cleverly labeled it a “death tax.” But when did it come to carry a negative connotation? Bill Gates Sr., a philanthropist and Bill Jr.’s father, co-authored a book arguing for a strong estate tax called Wealth and Our Commonwealth: Why America Should Tax Accumulated Fortunes. Gates attributes some of the resistance to the estate tax to a well-funded public-relations campaign and lobbying efforts prior to 2001.

That was the year when a major tax-reduction bill was passed and signed by President George W. Bush. This legislation, the Economic Growth and Tax Relief Reconciliation Act of 2001, decreased income-tax rates, lowered capital-gains taxes, and allowed for a phasing-out of the estate tax with yearly increases in the exemption, along with a yearly lowering of the top estate-tax rates. Gates describes how a myriad of wealthy families have funded efforts to fight the estate tax in advance of this legislation, including members of the families behind Mars Chocolate, Gallo wines, L.L. Bean, and Campbell Soup. Meanwhile, Gates writes, Frank Blethen, a fourth-generation publisher of The Seattle Times who vehemently opposed the estate tax, encouraged other independent newspapers to run editorials advocating repeal.

The estate tax has also gotten a bad rap because a number of politicians have suggested that paying it forces some people to sell off their small family-owned businesses—specifically farms. But when George W. Bush said in 2001 that he wanted to get rid of the estate tax in order “to keep farms in the family,” the journalist David Cay Johnston reported in The New York Times that when he asked the White House and the American Farm Bureau to point to examples of farms in jeopardy, they couldn’t provide one. “Professor Neil Harl, an Iowa State University economist whose tax advice has made him well known among Midwest farmers, said for 35 years he searched without finding one family farm lost to the estate tax,” Johnston wrote in 2001. “It’s a myth.”

This still appears to be the case today: An analysis by the Center on Budget and Policy Priorities found that in 2013, only 20 U.S. estates that owned farms or small businesses paid any taxes—and those 20 paid an average of 4.9 percent of their value.

These publicity campaigns have been shrewdly run, but the reason they have stuck is likely that Americans believe in their right to do as they please with their property, even after they’ve died. There seems to be a belief that property ownership is an inalienable right without an expiration date, but the concept of property can only exist when there’s a society—outside the purview of an organized government, a truck owner is just Mad Max. There are a number of limitations that society imposes on property owners, and for good reason. Zoning restrictions are an example: I’m glad my neighbor can’t raise pigs on the lot next door to me in the city. The idea that one should have limitless control over one’s assets after death, through this lens, seems a bit shaky: Buying a beachfront house does give someone the right to enjoy the use of that property, but it hardly qualifies her to control that piece of earth for the next thousand generations.

The idea of bolstering the estate tax still may sound objectionable to many, but a good deal of that resistance comes from how the tax has been framed. Calling it a “death tax” cleverly shifts the focus from the receiver to the giver—but from the perspective of the receiver, it’s simply an income tax. As the economist Irwin Stelzer wrote in the British magazine The Spectator, even a tremendously powerful official “could not get a corpse to sign a cheque. It is a tax paid by the recipient of this income, the inheritor, the lucky winner in the sperm lottery.”

Some raise a related concern: that an estate tax represents taxing someone’s earnings a second time. But, as Stelzer also noted, the inheritor is the one paying the tax on the windfall, not the deceased. And in any case, the CBPP estimates that 55 percent of the value of estates worth more than $100 million is made up of unrealized capital gains—which haven’t yet been taxed and never will be taxed under current estate-tax rules.

Given how clearly the estate tax lines up with American notions of fairness, it should enjoy wider support. The beauty of a free-market system is the absence of a special elite that judges who gets what—consumers vote with their dollars for the goods and services that best fit their needs (at least in theory). Inherited wealth goes against this model: As Warren Buffet has said, “The idea that you get a lifetime of privately funded food stamps based on coming out of the right womb strikes at my idea of fairness.” Indeed, it’s surprising that many of the same people who oppose welfare on the grounds that its benefits are not tied to work can so stridently denounce estate taxes, thus endorsing a system that allows people to receive vast amounts of money without putting in any work.

Critics of the estate tax frequently argue that taking away even a fraction of someone’s wealth upon dying also takes away his or her motivation to work while alive. That’s something to worry about if every asset was forfeited to the state, but a fair estate tax doesn’t necessitate a total redistribution of wealth. The threat of taking a larger portion of the dead’s assets may make people marginally less motivated to work, but they’ll still be very interested in enjoying the fruits of their labor while they’re alive, which isn’t exactly inhibited by the existence of a strong estate tax.

It may sound far-fetched, but there’s even an argument that strengthening the estate tax would be doing the rich a favor. For the last dozen years, I have advised wealthy people with taxable estates, and I’ve come to notice an important difference between inherited wealth and earned wealth. I concede that my observations are purely anecdotal, but I did notice that clients who were entrepreneurs displayed a strikingly different attitude towards their wealth than clients who were trust-fund beneficiaries. The former were proud of their companies, eager to talk about their journey, and seldom fearful that they didn’t have enough; the latter often displayed insecurities about their windfall, worried about whether they would run out of money, and were more like to experience family strife, addiction, and depression. Andrew Carnegie wrote, “Great sums bequeathed oftener work more for the injury than for the good of the recipients.”

It is probably not an accident, then, that most of the large wealth-management firms have specific programs targeted at addressing the challenges of what they call “second-generation wealth.” Like most wealth-management programs, these predominately focus on making sure clients’ portfolios are properly diversified, but they often have additional components, such as providing information about family psychology and offering advice about the governance of family businesses and the management of philanthropic foundations.

Limiting the inheritance of wealth represents more than just an attractive means of raising revenue. It would communicate a commitment to allocating society’s resources fairly and promoting equal opportunity. Working to earn money can be admirable, and indeed it should be celebrated when someone offers a product or service in the marketplace that generates great returns for them and increases everyone’s standard of living. But that can’t be fully, or fairly, realized in a system where some people are forced to work two minimum-wage jobs while others skate by on what they’ve inherited.