I did nothing wrong. That’s the gist of how many individuals connected with the “Panama Papers”—the millions of leaked documents exposing how a Panamanian law firm, Mossack Fonseca, helped the rich and powerful park their wealth in tax havens and offshore bank accounts or shell companies—are responding to the revelations this week. And in many cases, they may be telling the truth. As The New York Times points out, “Holding money in an offshore company is generally not illegal, although such financial arrangements can be used in illegal ways—for example, to facilitate tax evasion or money laundering.”

Wear a ski mask around town on a warm day, perhaps for a perfectly good reason (you’re a celebrity trying to avoid paparazzi, you ran out of sunscreen), and you’ll probably invite suspicion. Because it looks like you might rob a bank.

The release of the Panama Papers is a bit like the end of The Truman Show, the 1998 film about a man, Truman Burbank, who gradually realizes he’s spent 30 years on the set of a reality TV show in which he’s the star. In the last scene, Truman is sailing a boat when its bow pierces the artificial sky—hinting at a world beyond the show’s set. The leaks, from just one law firm in one country, have similarly pierced the screen that normally conceals a vast network of financial secrecy. But on the other side, there’s a much bigger world that, for most people, remains unexplored.

And this elite world has an impact on the world your Average Truman inhabits. In a report earlier this year, the British charity Oxfam argued that the use of tax havens by individuals and corporations is contributing to growing economic inequality around the world. It cited a finding by Gabriel Zucman, an economist and the author of The Hidden Wealth Of Nations: The Scourge Of Tax Havens, that $7.6 trillion, or 8 percent of individual financial wealth in 2014, is held in offshore tax havens, resulting in $190 billion in lost annual tax revenue for governments from Africa to Latin America. (Zucman estimates that roughly 80 percent of offshore funds aren’t reported to tax authorities and that the amount of money in tax havens has been increasing in recent years; other experts claim that the amount of private offshore wealth may be two to four times as high as Zucman’s figure of $7.6 trillion. Needless to say, measuring the size of an industry whose purpose, in part, is to obscure its size isn’t easy or precise.)

“This global system of tax avoidance is sucking the life out of welfare states in the rich world,” Oxfam wrote. “It also denies poor countries the resources they need to tackle poverty, put children in school and prevent their citizens dying from easily curable diseases.” This at a time when the recent global financial crisis has helped make the gap between the rich and the poor a major public-policy issue in many countries; in advanced and emerging economies alike, there is widespread public concern about inequality.

The Panama Papers offer a somewhat skewed, if unusually vivid, view of the global system of tax avoidance. Consider, for instance, the 10 most popular tax havens that appear in Mossack Fonseca’s files. One out of every two companies that the law firm established for its clients was incorporated in the British Virgin Islands.

But this geographic distribution doesn’t align with the results of the 2015 Financial Secrecy Index, a ranking produced by the Tax Justice Network, an advocacy group opposed to tax havens. The index relies on “a qualitative measure (a secrecy score, based on 15 secrecy indicators),” drawing on a given jurisdiction’s laws, regulations, ratified treaties, and so on, combined with “a quantitative measure (the global weighting to give a sense of how large the offshore financial centre is).” These form an aggregate “financial secrecy” value for more than 90 countries and territories. The goal is to plot on a spectrum, from more to less secretive, “secrecy jurisdictions,” a term the organization uses interchangeably with tax havens.

Critics—including jurisdictions listed in the ranking—argue that the Tax Justice Network’s approach is subjective and insufficiently rigorous, and some have questioned specific aspects of the FSI’s methodology. Christian Humborg, formerly of Transparency International, has noted that the weighting the Tax Justice Network employs tends to shoot large financial centers like the U.S. and Germany to the top of the list, even when their “secrecy scores” aren’t particularly high. “If there are reliable estimates of the size of illicit financial flows into a country, this might be an alternative number to consider,” Humborg suggests.

But if nothing else, the index is a valuable exercise in complicating binary thinking about financial secrecy—the notion that a state is either an otherworldly no- or low-tax haven, or it’s not. Instead, “a global industry has developed involving the world’s biggest banks, law practices, accounting firms and specialist providers who design and market secretive offshore structures for their tax- and law-dodging clients,” the Tax Justice Network writes. “‘Competition’ between jurisdictions to provide secrecy facilities has, particularly since the era of financial globalisation really took off in the 1980s, become a central feature of global financial markets.”

Think of the jurisdictions with higher values in the map below as the places that, in the Tax Justice Network’s view, have contributed most to the global financial flows we tend not to see. (The dark red indicates high values, the dark blue low values. Zoom in to explore particular regions; some jurisdictions are not visible.)

Since several of the highest-ranked jurisdictions—Hong King, Singapore, the Cayman Islands—are small territories that are hard to see on a map, here’s a breakdown of the top 15 “secrecy jurisdictions.”

“[T]raditional stereotypes of tax havens are misconceived,” the Tax Justice Network asserts. “The world’s most important providers of financial secrecy harbouring looted assets are mostly not small, palm-fringed islands as many suppose, but some of the world’s biggest and wealthiest countries. Rich [Organization for Economic Cooperation and Development] member countries and their satellites are the main recipients of or conduits for these illicit flows.” The organization notes that if British overseas territories or crown dependencies like the Cayman Islands and Jersey were combined, the United Kingdom’s network would top the list. (Indeed, the Panama Papers reveal that Mossack Fonseca frequently worked with intermediaries in countries such as Hong Kong, Switzerland, and the United Kingdom.)

But there are limits to what geography can teach about financial secrecy. In a 2012 paper for the Tax Justice Network, James Henry, the former chief economist at the consulting firm McKinsey, noted that the reality of offshore tax havens is more convoluted than it appears on a map:

[P]rivate banking has long since become virtual. So the term “offshore” refers not so much to the actual physical location of private assets or liabilities, but to nominal, hyper-portable, multi-jurisdictional, often quite temporary locations of networks of legal and quasi-legal entities and arrangements that manage and control private wealth—always in the interests of those who manage it, supposedly in the interests of its beneficial owners, and often in indifference or outright defiance of the interests and laws of multiple nation states. A  painting or a bank account may be located inside Switzerland’s borders, but the all-important legal structure that owns it—typically that asset would be owned by an anonymous offshore company in one jurisdiction, which is in turn owned by a trust in another jurisdiction, whose trustees are in yet another jurisdiction (and that is one of the simplest offshore structures)—is likely to be fragmented in many pieces around the globe.

The geography of financial secrecy—which manages to be fragmented, far-flung, and interwoven all at once—helps explain why reforming the practices on display in the Panama Papers is so difficult. Consider the United States, for example. The U.S. government has made real progress in combatting such behavior: Congress recently passed legislation requiring foreign banks to report information about their American clients, and the Obama administration is now considering issuing a new rule under which financial institutions would have to determine the identities of people who have established shell companies.

At the same time, the United States has not joined nearly 100 countries in agreeing to participate in an embryonic, multinational financial-data exchange organized by the Organization for Economic Cooperation and Development. And U.S. states like Delaware and Nevada have recently become hubs for low taxes and corporate confidentiality. Americans “discovered that they really don’t need to go to Panama,” Henry, of the Tax Justice Network, told Fusion earlier this week. “Basically, we have an onshore haven industry in the U.S. that is as secretive as anywhere.”

Similarly, British Prime Minister David Cameron has railed against tax secrecy and is even planning to host an international anti-corruption summit in May and launch a public registry for the “beneficial owners” of British companies in June. But he himself once owned a $40,000 stake in his father’s offshore trust, and “the British Empire has shrunk into a number of tax-avoidance jurisdictions … for companies to bury their wealth,” as Geoffrey Robertson, a human-rights lawyer, told the BBC this week. Offshore financial activity is critical to the United Kingdom’s economic growth.

The Panama Papers show “the weakness at this stage of the international community,” Robertson argued. “That community has finally a few years ago set up an international court to deal with atrocities. We have quite a good organization in Vienna that inspects potential nuclear developments. We have taken some action against trafficking in humans and drugs. But we haven’t really tackled international tax avoidance. We haven’t tackled the secret movement of the money of the wealthy for various often nefarious, sometimes legitimate purposes. And that’s the real problem.”

Political will is lacking in part because, as we’ve seen this past week, many politicians take advantage of financial secrecy to manage their own wealth. But another reason is the formidable challenge of “getting international agreement for some form of convention,” Robertson said. “The first thing is to ensure that offshore trusts, offshore companies are registered with their beneficial owners. It’s transparency. … And then you have to have an international enforcement body which can have the power, and that has to be granted by an international convention, to go into countries, to go into lawyers’ offices and banks and find evidence of sanctions-busting.”

Such obstacles stand in the way of a host of potentially transformational reforms, ranging from an international court for financial crime to a global financial registry to help tax authorities monitor assets. Until those obstacles are overcome, leaks like the Panama Papers may offer illuminating glimpses into the geography of financial secrecy. But they’re unlikely to scramble or shrink it.