Lies, damn lies and statistics

The success of the military surge in Iraq aside, little about the war has been more controversial than the subject of "excess" civilian casualties. Understandably, collecting reliable data in the middle of an insurgency while the population teeters on the edge of civil war presents a less than ideal scenario for data collection. Nevertheless, more than a few methodologies have been used to attempt to get to a reasonably accurate number of Iraqi deaths.

The anti-war group Iraq Body Count (IBC) collects data on civilian deaths "from cross-checked media reports, hospital, morgue, NGO and official figures to produce a credible record of known deaths and incidents." IBC's latest tally of deaths attributable to the war ranges between 80,621 and 88,044 as of December 14, 2007.

A second method allowed for extrapolation of civilian deaths by way of population surveys conducted in specific areas of Iraq. The estimate of casualties derived by this technique - 655,000 deaths between January 2002 and July 2006 - was presented in a study published in The Lancet. Predictably, there was considerable back and forth between those who supported the conclusions of the study and those who rejected them out of hand.

A result of this magnitude would seem dubious on its face, as it is higher than figures resulting from comparable analyses by a factor of twelve (see graph). The timing of the publication was also curious, as it was released three weeks prior to the 2006 mid-term election. But the figure sunk into the collective consciousness of those who opposed both the war and the Bush administration more generally. (It was recently cited in a Washington Post op-ed by George McGovern.)

Academics and journalists have expended considerable effort to debunk the results of the Lancet study since its publication. A National Journal article raised significant questions about both the study's methodology and the objectivity of the authors.

Skeptical commentators have highlighted questionable assumptions, implausible data, and ideological leanings among the authors, Gilbert Burnham, Riyadh Lafta, and Les Roberts.

[National Journal] has identified potential problems with the research that fall under three broad headings: 1) possible flaws in the design and execution of the study; 2) a lack of transparency in the data, which has raised suspicions of fraud; and 3) political preferences held by the authors and the funders, which include George Soros's Open Society Institute.

National Journal reports that some of the same authors did a preliminary study - also published in The Lancet - utilizing a similar methodology. (Perhaps not coincidentally, it was published just before the 2004 presidential election.) That article concluded that there were a minimum of 98,000 "excess" Iraqi deaths. But National Journal goes on to note that "[p]erhaps because that estimate contrasted sharply with the observations of embedded reporters, human-rights activists, and others on the ground in Iraq, the media gave it limited coverage." The problems with the original research were immediately evident to the unbiased observer.

In Falluja, Lafta recorded 52 deaths in 29 households, which amounted to 71 percent of the violent deaths recorded by the first Lancet survey. If representative, Lafta's sample translated into 50,000 to 70,000 dead in Falluja by September 2004 -- two months before the start of the second major American military operation to restore order. Falluja's prewar population was estimated to be 250,000, although U.S. officials said that the vast majority of residents had fled before the battles began. Lafta's Falluja death estimate was so far off the chart that his colleagues dropped it from the study, the authors said. (Emphasis added.)

By the time the second study was presented, circumstances on the ground in Iraq were such that public skepticism was universally reserved for proponents of continued military action.

Even though the second study was even further out of line with other sources' estimates than the first, it got tremendous attention -- probably because its findings fit an emerging narrative: Iraq was a horrific mess. The February 2006 bombing of Samarra's Golden Mosque, in particular, had sent the country spiraling toward sectarian warfare. (Emphasis added.)

Media sensationalism notwithstanding, it does appear that the skeptics will carry the day, as the study authors "have declined to provide the surveyors' reports and forms that might bolster confidence in their findings." The Lancet results are also at variance with data recently presented in the New England Journal of Medicine that estimate the number of violent deaths in Iraq between March 2003 and June 2006 to be 151,000.

To be sure, the timing of both articles' publication can also be called into question.

Soros is not the only person associated with the Lancet studies who had one eye on the data and the other on the U.S. political calendar. In 2004, Roberts conceded that he opposed the Iraq invasion from the outset, and - in a much more troubling admission - said that he had e-mailed the first study to The Lancet on September 30, 2004, "under the condition that it come out before the election." Burnham admitted that he set the same condition for Lancet II. "We wanted to get the survey out before the election, if at all possible," he said.

A well-worn aphorism posits that truth is the first casualty of war. It seems more than manifestly clear that anti-war bias animated this research from the very start. The fact that supposedly objective investigators would allow prejudices to color their research is troubling enough, but it does not entirely surprise. As discussed elsewhere, liberal bias has contaminated much of what passes for research in the area of climate change, all of it enabled and encouraged by an establishment that will brook no dissent from its orthodoxy. The insurmountable irony is that all of this comes from the very people who claim that conservatives are the ones who are "anti-science."