Robert Galbraith / Reuters

Time and again, when facing criticism about Facebook’s role in elections, the company’s executives have promised to provide maximal transparency about the advertising that runs on their services.

The digital tools will ensure this transparency will be unprecedented for not just Facebook and the digital advertising industry, Facebook CEO Mark Zuckerberg has repeatedly contended, but for any medium. “Even without legislation, we’re already moving forward on our own to bring advertising on Facebook to an even higher standard of transparency than ads on TV or other media,” Zuckerberg told investors last year. He said similar things in a blog post announcing the effort, as well as before Congress.

This response has, in part, been designed to stave off Congressional action with teeth, even if Facebook publicly committed to supporting the Honest Ads Act, the only real piece of online-ad legislation to come out of the last 18 months.

Given the weight that Zuckerberg and Facebook have placed on this measure before the public and legislators, the company’s actual execution of this idea is of paramount importance—and the bar should be high. Is what Facebook is building a useful tool, or is it merely transparency-washing, to coin a horrible phrase?

According to a new report from Upturn, a progressive policy group focused on the tech industry, the early versions of some of the tools that Facebook has built don't provide the depth of information or access necessary to hold advertisers accountable. “Unfortunately, we find that Facebook’s concrete plans fall short of its laudable commitments,” the report’s authors, Miranda Bogen and Aaron Rieke, conclude.

Facebook’s efforts branch into two categories. One tool, given the role the platform played in the 2016 presidential election, focuses on political and “issue” advertisements. For these areas, Facebook has committed to showing not just an ad’s content, but also roughly how many people saw that ad, how much money was spent on the ad, and broad demographic information about the ad’s audience. No one has seen how this works yet, including Upturn, but it’s been promised ahead of the U.S. midterm elections.

The other tool, called View Ads, allows any Facebook user to see a slate of the ads that any page is running at that moment. Facebook has launched that tool in Canada and Ireland, and a global launch is imminent.

Upturn’s core critique of both tools concerns the depth of access that Facebook provides to the ads. For example, when Upturn completed its report, Facebook had not announced plans to produce results in a machine-readable format in either scenario. That’s important because campaigns, for instance, might run thousands of ads. You need to use code to make sense of all of them.

In response to questions about the new report, Steve Satterfield, Facebook’s director of public policy and privacy, revealed that the company will build in the kind of computerized access that will allow researchers to delve deeply into the thousands of ads from political and issue campaigners. “We will build an API that will allow groups to learn the kinds of things that they want about the ads in the political-ad archive,” Satterfield told me.

Because the political- and issue-ad archive tool is not available yet, most of Upturn’s detailed critiques are about the broader tool, View Ads. The researchers were able to log in to Facebook with Canadian IP addresses to see how the system works.

If you go to a Facebook page in Canada that’s targeting Canada specifically, you can click on a link that takes you to a page of all the ads that are, as Zuckerberg wrote, “currently running.” That’s where the first problem Upturn identifies shows up. Facebook advertising campaigns vary in length. “Currently running” is not like a TV commercial that might run for weeks or months. For example, a flurry of tests to see what content performs the best might run for 4 hours, 8 hours, 20 hours, only enough time to gather data. If you weren’t looking at that moment, you wouldn’t see that flurry.

The next problem is that—unlike what Satterfield promised for political and issue ads—Facebook has not committed to creating a machine-readable format for all ads, so researchers (or journalists or whoever) can’t suck in that data and try to learn from it. It’s an entirely manual process.

Another problem: There is no way even to search across ads—except as promised for political and issue ads. So, if a researcher were looking for discriminatory housing advertisements in a given city, they’d have to go through all the different real-estate pages looking at the ads that they are currently running to spot any issues.

Yet another problem is that Facebook is not showing the metadata associated with ads that it does not deem political or issue-based. That means researchers don’t know how many people other ads reached or what demographic groups those people fall into. All they have is an ad’s content. Keep in mind that some companies will run hundreds or thousands of ads. How can someone know which ones might have had a significant impact and which ones didn’t? Under the current View Ads system, that’s impossible to determine.

For just about all the uses that civil-rights groups might contemplate, Upturn found that the Facebook transparency system is not usable for uncovering the kind of malfeasance or discriminatory behavior that transparency is supposed to expose. “Facebook’s ad-transparency tools don’t include an effective way for the public to make sense of the millions of ads running on its platform at any given time,” the report found. “If public actors cannot search, sort, and computationally analyze ad data, they have little hope of holding advertisers accountable.”

For example, when I asked the report’s authors if the new View Ads transparency tool would have helped journalists and researchers discover the Russian Internet Research Agency’s meddling on Facebook, they had a simple answer: “No.”

Facebook’s Satterfield defended what the transparency effort will achieve when it launches globally. “Every currently running ad on Facebook is now viewable on the page on which it is running,” he said. “Yes, we have dug a level deeper for political advertising, but we’re proud of View Ads.”

At the same time, he acknowledged that it was just a first step, and that criticism like Upturn’s was causing them to improve the tools and consider how to allocate their resources. And he hinted that the approach that Facebook is taking with political and issue advertisements would spread to other types of ads.

“What we’re building is generally consistent with the recommendations that they’ve made, we’ve just chosen to start with the area of ads where we’ve heard from third-party groups and consumers that they want additional transparency,” Satterfield said.

It could be that Facebook inserts more categories of ads—say, housing and employment—into the political- and issue-ad archive, which allows for deeper access. Or it could be that View Ads evolves into a product that contains more data and accessibility.

Over the next few months, as Facebook’s transparency tools roll out across the world, journalists, researchers, and civic groups will test whether the level of transparency Facebook is offering translates into meaningful accountability for advertisers. Upturn, at least, doesn’t think the first publicly accessible tool will do so.

Satterfield emphasized that Facebook wants to make these tools better, and that the criticism they face helps accelerate that process, each level of transparency prompting calls for more. “One of the reasons that we build transparency into our products is that we want to hear feedback,” he said. “The occasion for Upturn’s report is that we announced these tools.”

The big question is whether Facebook should be allowed to self-regulate, deciding what it considers meaningful transparency, or whether some other legislative or regulatory mechanisms should be put into place.

“Facebook has basically been printing money based on the complexity and obscurity of its ad system,” Bogen said. “Now that we’ve seen the problems it can wreak, we’ve come to a decision point in society about what kind of oversight we need to make sure that doesn’t happen again.”

We want to hear what you think about this article. Submit a letter to the editor or write to letters@theatlantic.com.