What Can We Learn from the Numbers That Slipped by NSA Censors?
As part of its ongoing push to increase transparency, the Director of National Intelligence on Wednesday evening released the latest compliance report it submitted to Congress. For the most part, the actual data is obscured behind black bars — except on one pie chart, left unredacted, that appears to give more information on the program than the NSA has ever released publicly.
.As part of its ongoing push to increase transparency, the Director of National Intelligence on Wednesday evening released the latest compliance report it submitted to Congress. For the most part, the actual data is obscured behind black bars — except on one pie chart, left unredacted, that appears to give more information on the program than the NSA has ever released publicly.
Particularly following a similar report leaked to The Washington Post, the question of how often the government exceeds the rules on data collection have become a subject of enormous debate. In a report released earlier this month, the agency downplayed the frequency of those errors, implying that they occurred only a small fraction of the time. Considering that such violations could mean violations of the constitutional rights of Americans, however, civil liberties groups like the ACLU worry that the thousands of known violations — even if a small percentage of the total uses of the data — are already excessive.
The report released on Wednesday was part of the DNI's effort to assuage concerns about those violations. You can see the entire document at the bottom of this article. The only number that the DNI clearly left visible was this one — which, as you might expect, offers only the percentage of incidents. The actual number is hidden.
The visible figure shows that only 0.49 percent of the times in which the agency reviewed material was there a compliance incident — in other words, a review that exceeded the agency's authority. That figure, we'll note, is higher than the 0.197 percent of violations that triggered a 2011 critique from the surveillance system's governing court.
Some of those incidents that added up to 0.49 percent included failure to report the violation on time. Take those out, and the trend looks like this by reporting period.
Outside of the context of the number of total searches, as the ACLU notes, this loses meaning. If you were offered one percent of someone's money, you'd be far happier about getting one percent of a billion dollars than one percent of a buck.
However, a separate chart appears to slip, revealing actual figures pertaining to the incidents.
If the numbers shown on the slices are the raw numbers — the labels match the terms described in a previous section "Categories of Compliance Incidents" — then in the six-month period covered by the report, there were 338 violations. That appears to be fewer than the violations reported for any quarter in the Washington Post's data last week. More than half of the violations in the most recent report were delays (yellow) as indicated in the first chart above. Most of the rest were "tasking" (inappropriate identification of an American as an appropriate target) or "detasking" (failure to stop tracking someone identified as American) errors.
But that total number also gives us a sense of scale of the NSA's surveillance — or at least the scale covered by this report: there were about 69,000 total queries during those six months (338 divided by 0.49 percent).
We've reached out to the DNI for more information about these figures. If the number of non-delay-related violations is indeed 126 over this period, that is a lower rate than for any quarter detailed in the Post's article. But it's still hard to tell what is or should be acceptable. There is very little context to the figures — what those 69,000 queries were; what happened the 48 times an American was improperly targeted. Without that sort of detail — which it's not clear is given to the members of Congress who receive this report — it's hard to know how to consider the numbers.
There's one last bit of data in the report: compliance trends by agency. The number of violations identified by the CIA has gone down. The number of selectors — targets of data collection — identified by the FBI has continuously increased. And the DNI expects it to increase more, as the "FBI has made its nomination process more widely available to its field office personnel." (The FBI also maintains its own database of data collected under the Foreign Intelligence Surveillance Act.)
What the DNI hoped people would take away from this document is that the oversight of surveillance programs is robust and effective. Instead, we're left with more questions than before — including some about the skill of the government's redactors.
Update, Thursday: We reached out to the office of the Director of National Intelligence for a response to this story. That office suggested we speak with the National Security Agency; the NSA indicated that the best agency to answer any questions was the Department of Justice. No one, in the end, had any comment.
However, since we originally published, we came to realize our estimate of incidents based on the chart was likely high. On the page preceding the unredacted chart there is this sentence:
The "notification delays," for example, generally apply to slowness on the part of the NSA in reporting other violations (as in the bar chart above), so incidents in that category may be counted in the other categories as well. While the total of the incidents may be high, the total is still not known.
And then we came to realize that we would not receive any additional clarity on the subject.
Photo: Director of National Intelligence James Clapper. (AP)