Last week’s ABC mini-series chronicled the most famous financial fraud in recent American history: Bernard Madoff’s $50 billion Ponzi scheme, which devastated elite institutions and families of the American Jewish community. The scale of Madoff’s crimes was breathtaking. There’s much to be said about his crimes—not least about the incompetence of the regulatory apparatus that failed to stop him despite repeated warnings and what researchers Greg Gregoriou and Francois Lhabitant quite appropriately called “a riot of red flags” over many years.

The tragedy itself was also its own sort of warning. Madoff’s victims were not a random assortment of the well-off; he decimated a segment of the wealthy Jewish community and several Jewish charitable organizations. Knowing the general magnitude of Madoff’s crime, I’m still taken aback by the particulars, which betray Madoff’s lack of conscience and any sense of limits. He wiped out Elie Wiesel’s life savings, and stole $15 million from Wiesel’s foundation. Madoff defrauded Hadassah. Who defrauds Hadassah? That’s like mugging your grandmother.

Such crimes would not have been possible without the cultural ease and social entre Madoff enjoyed in the Jewish community. To put a name on things, this was one of the worst affinity frauds in Americans history, whereby unscrupulous people exploit their cultural connections and people’s communal identities to rip them off. In that sense, Madoff’s crimes were a warning to everyone about how in-group feelings of trust leave people vulnerable.

Affinity fraud is depressingly common. My own incidental exposure would be comical if the stakes weren’t so serious. I happen to live in a majority African American neighborhood in south Chicagoland. Although you wouldn’t know it to look at me, marketing algorithms based on my street address and some of my purchases apparently reveal that I am a middle-aged, middle-class African American man.

Based on that identification, I get an impressive volume of sales pitches. If the letters, Facebook ads and robo-calls are to be believed, President Obama is obsessed with refinancing my mortgage. Our 44th president particularly wants me to take out a 15-year loan, one much better than what can be found at the bank. He also wants to sell me health insurance and some other things, which would be good for me, and good for the community. Many such deals offer a low interest rate, and are targeted to people in fear of losing their homes. But then they charge hidden or opaque fees for "refinance consultations." Some ask for people’s home titles or pose as intermediaries, falsely promising to make payments to the actual lenders. The details vary, but purveyors of such sales pitches hope that their too-good-to-be-true offering will go down easier when combined with appeals for political or ethnic solidarity.

I sometimes chuckle when I hang up after fielding such a sales call. But there’s nothing funny about efforts to defraud people or to deceptively market products by exploiting people’s personal and communal ties. In light of the empty houses and fading for-sale signs not far from my front door, such pitches are especially terrible, as they purport to offer a way out to many people in deep financial difficulty.

Other communities face similar challenges. In the lead up to the foreclosure crisis, unscrupulous lenders deployed Spanish-speaking saleswomen to target recent immigrants. Once they won consumers’ trust with a persuasive sales pitch, lenders presented consumers with incomprehensible English-language financial documents that often steered people into risky, overpriced, or fraudulent loans. In other cases, con artists offer to help immigrants of the same nationality, and use the opportunity to obtain people’s Social Security numbers and other financial information, which is then used for mortgage fraud.

Pretty much any powerful connection humans make with others provides some correspondingly powerful opportunity for affinity fraud. Here in Chicago, an ex-Marine exploited his service ties to defraud fellow veterans. Affinity fraud is an issue within the LGBTQ community, as well. A terrific Economist story, aptly titled “Fleecing the Flock,” quotes an Alabama regulator’s estimate that half the affinity frauds in that region arise in religious communities. It’s easy to see why. Such communities’ intimate ties of faith and mutual trust create particular vulnerabilities. Traditions such as tithing normalize provision of substantial resources to co-religionists whose financial acuity and probity may be difficult to fully discern. The Church of Latter-day Saints has experienced such a spate of cases that the Economist noted the “hook of Mormon” as a distinctive concern.

As those Obama mortgage pitches suggest, political tribalism provides another potent opportunity for connection—and thus fraud—in a polarized nation. Right-wing talk radio, for example, features an array of advertising for dubious financial products, whose sales pitch conveniently matches the programs’ scaremongering regarding economic policy.

Glenn Beck’s connection with Goldline was one notorious example of such products. Beck’s program relentlessly depicted an American economy on the verge of hyperinflation and collapse—an apocalypse naturally hastened by Barack Obama and other liberal political leaders. During commercial breaks, Beck’s audience was then conveniently exposed to Goldline, a purveyor of dicey gold investments.

Perpetrators of affinity fraud seeking to win our trust do the most lasting damage when they recruit community leaders as explicit or implicit endorsers. Some seek to provide a sense of status or peer pressure through nominally exclusive opportunities to join them. Madoff wouldn’t let just anyone invest in his closed funds. Many practitioners of affinity fraud imply that they have secret and timely information, which can turn into serious money if one acts quickly and discreetly.

Practitioners of affinity fraud subtly exploit a community’s distrust of outsiders to discredit alternative sources of information. It becomes a mark of collective identity to spurn conflicting information and advice. Of course the lame-stream liberal media looks down on us for putting so much of our retirement savings into gold. Of course the same Washington experts trying to cut Social Security and Medicare look down on this variable annuity which might plug the holes in your retirement plan. It’s ironic: The most cynical and distrustful among us are often the easiest marks.

The weird information economics of herding also matter. You entrust your money with someone. Many people you know are doing the same thing. Each of you hopes and believes that someone has done the due diligence regarding these investments. If no one actually has, how would you know? After a while, it’s embarrassing to even ask.

Whatever public policymakers do to address these problems, individuals should be especially skeptical of any financial product embraced by influential people in their religious, cultural, or political communities. Given such realities, it’s a mistake to allow personal familiarity, community affiliation, or time pressure to become a substitute for proper written contracts and the same due diligence one would apply to any stranger selling investment products or advice.

In the end, any person's best protection against affinity fraud is to avoid complex or speculative investment products offered by anyone, since even the honestly-offered fancy investment products very rarely provide much benefit. Following a simple investment plan is pretty boring. It remains the best bet.

Once Americans internalize the reality that vanilla stock-and-bond index funds basically outperform everything else, and that vanilla  fixed-rate mortgages are the safest way to get a loan, they’ll save themselves a lot of time. They just might save themselves a lot of heartache, too.