There are problems with this too. Andrew Mack, director of the Human Security Report Project and a former U.N. official, described how surveys performed at refugee camps during the Darfur crisis had the effect of distorting the reported death count. "If you take your surveys shortly after people arrived, they would report the mortality rates that were prevailing before they reached the refugee camps. You would expect them to be a reasonable indicator of death rates from some parts of Darfur," Mack said. "If you went back to those same camps a year and a half later you would find the mortality rates had dropped dramatically." Almost by definition, most refugees will have had a first-hand experience of the war, inflating the reported mortality rate at the beginning of a refugee survey. Then, after a few months, refugee camp services have a tendency to reduce the mortality rate to pre-conflict levels.
"In something like 4-6 months, mortality rates in a reasonably well-provided for refugee camp will have come down to the rate prevailing during the prewar period, or actually lower than that," Mack said. NGOs frequently perform refugee surveys in order to determine the needs of the populations they're serving. But they're less useful for counting a conflict's dead.
As Lacina explains, the "fog" that pervades the entire enterprise of establishing a death toll has no quantitative or numerical value - there's no easy way to translate an observed number into an estimate, never mind a definitive, actual number. "The other thing researchers might want is some sort of sense of how much these numbers tend to change between the sort of fog of war and the revision that comes later when people are found and what happened becomes clearer. And that's a subject of intense disagreement."
Similarly, there is no set multiplier for observed deaths and actual death. "There's not even good data on the relationship between direct deaths on the one hand and indirect deaths from disease and malnutrition on the other," says Mack. "The multiplier goes between anything from 2 to 70 or 80," depending on the conflict. Mack pointed out that even if there were a standard multiplier, it wouldn't be terribly useful. "Even if it was a true average figure how would you know whether the conflict that you're investigating is average or not?."
Luckily, there's "multiple systems estimation," which, in a rather macabre irony, is related to methods used to track wildlife population sizes. As Lacina explains, researchers tag animals one year, recapture them over subsequent years, and use the observed probability of recovering an animal in a given year to a population size. Researchers essentially calculate discrepancies within their own methodology in order to reach a more accurate sense of the population they are dealing with. In the case of conflict death calculations, researchers can look at the frequency with which names appear on individual human rights monitors' lists to determine the probability of appearing on no list, with the aim of calculating a "population size" of the dead. Intuitively, a high frequency of names appearing on only one or two lists -- rather than four or five -- suggests a high probability of not being counted. In contrast, if all names appeared on all lists, it would suggest near-flawless documentation.
It isn't quite as simple as that, as Patrick Ball explained. What works in a nature preserve doesn't necessarily work in a conflict zone. Take the lists, for instance: calculating a simple probability ignores the way in which the different data sets -- in this case, lists of people killed in the conflict, provided by various human rights monitors -- relate to one another. "The logic is we want to get the best prediction of how this interaction process works between these systems," says Ball.
Ball's task is determine a population size based on detailed but nevertheless incomplete information. "What a statistician wants to do is say ok, let's divide the world into little slices of time and space, so we can make estimates over time and space," says Ball. Researchers then test various mathematical assumptions within these individual "strata" -- a "strata" being, for example, Aleppo in March of 2012. This requires math so complicated that Ball resorted to metaphor in order to explain it.
"Imagine that you have two dark rooms, and you want to know how big the rooms are," he said. "You can't go into the rooms and measure. They're dark, and you can't see inside them, but what you do have is a bunch of little rubber balls." Throw the balls in one room, and you hear frequent hollow pinging sounds as they bump into each other. Throw them into the next room, and the sound is less frequent. Even though you can't see inside of either room, you can intuit that the latter room is the larger of the two.
The dark rooms are Aleppo in March of 2012. The rubber balls are the various data sets. The act of throwing the balls into the dark rooms is akin to the complex mathematical analysis that allows Ball to "see how frequently the data set encounters itself," and the aggregate results from every "strata" will be an estimate of the number of people killed in Syria's civil war. He says his team will have this estimate in another two to three months.
For civilians in Syria, it's unclear how any of this matters. It's unclear how it even matters to global decision makers with the potential ability to hasten the end of Syria's civil war.