What Transparency Reports Don't Tell Us

There was a time in the not so distant past when hardly any Internet company wanted to release a transparency report—a report that summarized the number of law enforcement and intelligence requests that they received and responded to. What started with just Google and Twitter in 2010 and 2012, respectively, has become a steady stream of companies joining the bandwagon in the wake of Edward Snowden’s revelations. Companies that had no interest in reporting one year ago now hold out their reports in an attempt to earn back eroded customer trust. The problem is that transparency reports actually tell us very little about whether we should trust these companies.

According to Google’s latest transparency report, in the first six months of 2013, they received 25,879 requests for user data, and complied with 65 percent of them. Sounds like big numbers. And they are. As Google points out in their report, the number of requests has doubled since 2010. But what does that tell us about Google? Less than you might think.

The number of requests for user data that companies receive speaks only to the aggressiveness of law enforcement and intelligence agencies. The number of requests companies comply with is only slightly more within their control; although companies have some discretion in determining what data falls within a request and what requests are overbroad, they can no more reject valid legal requests than they can ignore environmental laws or wage laws in countries in which they operate. When we see a company comply with fewer requests than their peers, we’d like to think that’s because they were fighting on our behalf, but it could be simply because they received a larger number of requests that were too vague to answer. From a numbers standpoint, the two scenarios look identical in a transparency report.

Recently, Kashmir Hill of Forbes, placed side-by-side data from all the major Internet companies’ transparency reports. In her article, Yahoo’s 40,000 requests for user data towered over Twitter’s 1,100. Does this mean that Twitter is a safer or more trustworthy company? The huge disparity has more to do with the fact that Twitter operates a largely open platform—in other words, law enforcement doesn’t need a warrant to see my Tweets, only a web browser. By contrast, Yahoo runs one of the largest and oldest web e-mail platforms, making it a constant target of law enforcement and intelligence agencies.

But it gets worse—the numbers in transparency reports can actually mislead us about company trustworthiness. When implementing PRISM, the NSA approached several Internet companies in order to obtain their compliance in the program. One Internet company took the fight to the secretive Foreign Intelligence Surveillance Court, arguing that the government demands were unconstitutional. That company was the one whose total number of requests tower over the rest: Yahoo. The numbers in the reports tell us nothing about the one thing we should we need to know: corporate responsibility.

That’s not to say that Yahoo is better than Twitter or Google, only that transparency reports, as currently implemented, don’t help us answer a question of trust. Trust isn’t about the number of requests a company receives or responds to, it’s about the steps they take in responding to any given request. And that’s a lesson Yahoo learned the hard way in 2002. A Chinese dissident named Shi Tao used Yahoo Mail to allegedly send state secrets to an anonymous website. China demanded that Yahoo identify the sender, and because Yahoo was operating in China at the time, Yahoo quickly complied. Thanks to Yahoo’s decision, Shi Tao only completed his eight year sentence a few months ago. A single request for user data can have devastating consequences.

In response to the story about Shi Tao, and the resulting congressional investigation, Yahoo made systemic changes to how it approached digital human rights issues. It pulled out of China, created a Human Rights and Business Practices division, and helped found a human rights practices assessment organization, called the Global Network Initiative. It addressed the problems not through providing numbers of requests, but by changing its process for how it handled those requests.

What company transparency reports do provide is a sense of the size and scope of our surveillance state. Among the recommendations made yesterday by the president's NSA review panel was that the government begin disclosing data about the orders it has issued, but until they do so, company transparency reports are our only metric. The panel also recommended allowing companies to say more about national security requests, which is a needed improvement. However, it would be wrong to mistake transparency reports as any indication of corporate trustworthiness (or lack thereof). If companies want us to trust them, it is through transparency of effort and process through which they should earn it.