Death by Police, Cont'd

Editor’s Note: This article previously appeared in a different format as part of The Atlantic’s Notes section, retired in 2021.

My colleague Adrienne wrote a post last week about the efforts of various online groups and publications—namely The Washington Post and The Guardian—to create a running tally of the number of Americans killed by police officers. Her post begins:

It wasn’t until recently that it became easy to find a number to go with the gruesome reality that black people—and black men in particular—live with every day: the ever-present threat of police violence. Police officers fatally shot nearly 1,000 people last year, according to The Washington Post’s ongoing count. Halfway through 2016, police have shot and killed 506 more. “Unarmed black men are seven times more likely than whites to die by police gunfire,” the Post wrote last year.

Below is a lengthy reader dissent from Nick Selby, a Texas police detective and lead author of the new book In Context: Understanding Police Killings of Unarmed Civilians. Adrienne gave the go-ahead to post Selby’s response:

With respect to Adrienne LaFrance’s “Death by Police,” there are some serious structural problems with the data on which she bases much of her analysis. I don’t for one second believe she is intentionally manipulative, but I do want to point out, as with most things in law enforcement (the one place where Occam’s Razor does not apply), these numbers are not as straightforward as they might appear.

Some of the contextual issues from the piece that should be clarified are around The Washington Post’s numbers, because in the way they are quoted by Ms. LaFrance, they only tell half the story. It is undoubtedly true that newspaper’s Fatal Force study showed that, by the end of 2015, U.S. police had fatally shot 986 people (it was later raised to a total of 990). That number sounds truly terrible given Ms. LaFrance’s apparent inference that the 990 people were innocently going about their day when the police killed them. That is, of course, not the case. Yet Ms. LaFrance implies that it is. “In one in five fatal shootings,” she writes, “the names of the police officer responsible is never disclosed. Even when they are, many officers face no consequences.”

As a police detective who has intensely studied this issue, the underlying datasets, and media efforts to count them, I must point out that the implication of that passage is based, at best, on a lack of understanding of the numbers. As my co-authors and I wrote in our book, that passage fails “to differentiate killings by police who are behaving badly from those in which cops were doing what we want them to do—that is, standing between civilians in danger of losing their lives and those who would take their lives from them.” I think that to tell readers of The Atlantic, who want to better understand the truly important questions about how Americans—especially non-white Americans—are being policed, it’s not helpful to imply (even unintentionally) that all police shootings are unjustified. We as citizens should not be as interested in police using appropriate force as we are in their inappropriate use of deadly force.

In an interview on this topic with David Krajicek of The Crime Report, I said:

When Muhammad Youssef Abdulazeez opened fire on military personnel at a Chattanooga recruiting station, you’d have been hard pressed to find an average citizen who didn’t agree that police acting to stop that deadly threat were justified. I’m not interested in looking at that case. What I am (and I think all of us are) interested in is a case like that of Walter Scott, where you’d be hard-pressed to find a reasonable person who thought that officer was justified. I wanted to know how many Walter Scotts are there vs. Muhammad Abdulazeezes.

How do I know The Washington Post numbers did not refer to unjustified killings? Well, because they told us. It is a very important and often missing fact in discussions about this matter, but the fact remains—as Peter Moskos and I wrote in The Washington Post—that most police shootings are justified.

Don’t get me wrong: I don’t blame Ms. LaFrance for missing this, because The Washington Post didn’t say this loudly. In fact, their analysis of this was fairly well hidden in the tenth paragraph of an article. In fact, it was so well hidden that my own Post editor couldn’t find the reference until I gave him the specific URL—but there it is:

[74 percent of those fatally shot by the police in 2015] had already fired shots, brandished a gun or attacked a person with a weapon or their bare hands… These 595 cases include fatal shootings that followed a wide range of violent crimes, including shootouts, stabbings, hostage situations, carjackings and assaults… Another 16 percent of the shootings came after incidents that did not involve firearms or active attacks but featured other potentially dangerous threats. These shootings were most commonly of individuals who brandished knives and refused to drop them.

So with their own analysis, The Washington Post has taken us from an image of police run amok and shooting black men to one in which, on its face, looks as if five percent of police shootings from last year look “bad.” Nine in ten people killed by police, according to the newspaper, were in fact justified killings of criminals in the midst of fatal or potentially fatal attacks. Our research independently confirms this.

What is more, The Washington Post leaves out what we estimate to be 46 percent of unarmed people who died after an encounter with police—whom the police DID NOT SHOOT. Focusing on shootings limits the true scope of the concern.


Are Black People More Likely To Be Shot Dead by Police?

Another statement by The Washington Post quoted in “Death By Police” is the statement that unarmed black men are “seven times more likely to die” at the hands of police than are white men. This statement has been repeated by many well-intentioned journalists and it is, sadly, not just intensely misleading but demonstrably incorrect. I believe The Washington Post didn’t intend to be misleading; I believe they believe the number to be accurate. It’s just that the Post reporters have leaped to some rather sophomoric statistical conclusions.

The figure was arrived at through an attempt to control for the population disparity in the United States between black and white people[1]: While the raw number of white people shot dead by the police is higher than the raw number of black people, remember that black people only comprise about 13 percent, and black males only 6 percent, of Americans. The reporters (after the release of a report by ProPublica, written by reporters I happen to know were warned by scholars before publication that their numbers were inaccurate and misleading) sought to adjust the numbers to represent a count. For some reason they settled on the unorthodox “X-per-million” designation as opposed to most rate calculations which are given as “X-per-100,000.”

The mistake here, of course, is that by leaving out the context of what the decedent was doing at the time of his death, the calculation becomes truly meaningless. In the words of Joseph Cesario, director of the Social Cognition Laboratory at the Michigan State University, “To adjust the raw shooting numbers on population proportions assumes that ... an officer buying a cup of coffee is as likely to shoot the cashier selling him the coffee as he is to shoot a citizen with an outstanding warrant who has just been pulled over for speeding. Not only does common sense suggest this is wrong, the data do not support this assumption.”

Cesario’s research showed whites as 1.27 times more likely than blacks to be killed during a violent crime arrest. I would caution against relying completely on that number because, again, this stuff is super complex.


So how can we answer the question of “Are black people more likely to be shot dead by police?” The problem in answering comes down to the data itself. As someone who led a project in which 15 people pored over these states for almost a year, I can say that the problems are around capture of the data, coding of it, and the fact that ultimately there are issues of veracity, since as we know, cops sometimes lie. Not a lot, not as much as you might think, but there’s just no dataset that does not have falsehoods and errors.

Additionally, non-police scientists attempting to code police narratives have fallen victim repeatedly to what I call Starsky & Hutch Effect: Having grown up on TV shows and movies, we Americans believe truly that we understand policing, when what we really understand is policing on cop shows.

So now comes Roland G. Fryer, Jr., a frickin’ genius at Harvard, who just released (as you noted earlier this week) a study titled “An Empirical Analysis of Racial Differences in Police Use of Force.” The beauty of this study (it’s a working paper, covered gushingly in The New York Times) is actually not the conclusions, but the methodology. Even if you’ve never read a methodology section before, I urge you to read Fryer’s. His method of capture of narrative into coded data was simply wonderful—in that, for the first time, someone took the trouble to treat police data like most other data. The result was, as Fryer himself stated in the Times, “the most surprising result of my career.” After studying 1000 shootings at ten major agencies, Fryer found (wait for it) no racial bias in fatal shootings by police.

It is highly important to point out that his study did find racial bias in lesser (non-deadly) uses of force, particularly that officers of any race were more likely to go “hands-on” or use non-deadly tools like OC spray, clubs, or TASER on blacks than whites. This is a different problem.


Finally, in terms of counting, Ms. LaFrance is absolutely correct that, shamefully, the U.S. government has not provided us with a comprehensive count of civilians killed by police. This is changing, but only slowly.

However, her post mis-states an important difference between the extant projects (which, for the record, began with D. Brian Burghart’s Fatal Encounters, on which all other data-sets including mine are based). “Of course,” writes Ms. LaFrance, of The Washington Post’s dataset (506), “this is just one count. The Guardian’s tally is 561 deaths, including 526 shootings. And that discrepancy suggests that as important as these efforts have been, in the absence of a comprehensive federal effort to track such shootings, the full scope of the problem remains unknown. Still, attempts to track police shootings are meaningful.”

Respectfully, that is simply incorrect. The discrepancy is both understandable and predictable and speaks directly to the totally different, but published and readily available, methodologies employed by the two projects. The Washington Post counts line-of-duty fatal shootings. The Guardian tracks all DEATHS—shootings and other deaths that include car wrecks, overdoses, strangulation, personal conflicts, and other non-line-of-duty or off-duty incidents, prison deaths (prison guards are not police officers), jail guards (ditto), and others.


As you can see, this data collection is really complicated and difficult to determine. But as a journalist, I think Ms. LaFrance wasn’t skeptical enough, and that caused its own issue, because she based much of her theses and statements (some of which I happen to agree with) on data from a source that wasn’t as complete as it appeared to her to be. In support of that, I quote Professor Fryer, in what has become my favorite footnote of the year: “Data constructed by the Washington Post has civilian demographic identifiers, weapons carried by civilian, signs of mental illness and an indicator for threat level but no other contextual information.”

Update with another part of Selby’s correspondence I forgot to include:

None of all the above (including the findings of Mr. Fryer) is to suggest that there is a “correct” number, and I think that is my overall TL;DR.

But I was remiss not to mention the excellent work of Cody Ross in PLOS [excerpted by an Atlantic reader in a previous note], which looks to the U.S. Police-Shooting Database (USPSD) for answers to these questions and found, "...significant bias in the killing of unarmed black Americans relative to unarmed white Americans, in that the probability of being {black, unarmed, and shot by police} is about 3.49 times the probability of being {white, unarmed, and shot by police} on average.” I have a panoply of bones to pick with the USPSD, but not with Mr. Ross' superb analysis—though W. David Ball has done an extraordinary analysis of that, plus a description of Bayesian analysis in general and a super-cool critique of the Fryer paper in particular.

And then, this morning came Radley Balko, who wrote in his superb blog, The Watch, that it is quite simply impossible to calculate the percentage of police shootings that are legitimate. [2]

The reason the Fryer paper and the Ross paper and Bell’s critique and Fatal Encounters and StreetCred: Police Killings in Context and my book are so important, I think, is that they are serious attempts to take almost comically imperfect data about complex, insanely multi-variate situations and try to provide meaningful insight. The fact is, we can geek the hell out over these numbers and grant points and get great takeaways, but in the end, this is worth repeating:

Law enforcement data is the one place where Occam’s Razor does not apply. Journalists who state facts based on studies do so at their peril.

[1] Since 1997, the federal Office of Management and Budget has required all federal agencies to use a minimum of five race categories: White, Black or African American, American Indian or Alaska Native, Asian, and Native Hawaiian or Other Pacific Islander. Note the lack in the federal OMB requirements of the term, “Hispanic or Latino.” Since the 2010 Census, the U.S. Census Bureau has treated “Hispanic or Latino” as an ethnicity separate from race (White, Black, American Indian, etc.). The Census Bureau’s document, “Overview of Race and Hispanic Origin: 2010”, attempts to clarify: “‘Hispanic or Latino’ refers to a person of Cuban, Mexican, Puerto Rican, South or Central American, or other Spanish culture or origin regardless of race.” While this is probably ultimately the correct decision, it has created heaps of confusion in the statistics and demographics community, and resulted in a large range of descriptions from non-statisticians and non- demographers that conflate “race” with “ethnicity.”

[2] Radley and I spoke by direct message after the printing of that column (I like the hell out of Radley, who I think is a great guy, but in my opinion, he got the 74% analysis only partly right in his column. I aver he mistakenly believes that the "16%" of additional attacks were part of the 74% of killings that interrupted a fatal attack. But if you read the original you will see that the article flatly stated that "74 percent" had fired shots or been in the midst of a deadly attack, and "another 16%" involved people who didn't have firearms but involved "other potentially dangerous threats", "most commonly", knives.