PZ Myers seems to have pissed off quite a few people, this time for taking Steven Pinker to task. His take is worth reading in full, but I’d like to add another angle. In the original interview, there’s a very telling passage:

Belluz: But as you mentioned, there’s been an uptick in war deaths driven by the staggeringly violent ongoing conflict in Syria. Does that not affect your thesis?

Pinker: No, it doesn’t affect the thesis because the rate of death in war is about 1.4 per 100,000 per year. That’s higher than it was at the low point in 2010. But it’s still a fraction of what it was in earlier years.

See the problem here? Pinker’s hypothesis is that over the span of centuries, violence will decrease. The recent spike in deaths may be the start of a reversal that proves Pinker wrong. But because his hypothesis covers such a wide timespan, we’re going to need fifty or more years worth of data to challenge it.

Let’s look at the homicide rates of the USA, Canada, and the UK (England and Wales) over the last 60 years.

The net trend for the USA is an increase in homicides, with only 2.9% of the trend-cloud suggesting a decrease. That could be due to a violent period between the early 1970’s and 1990’s, however; the current homicide rate is about the same as it was sixty years ago, so that violent period could have skewed the trend, or the USA could be on the low part of a cyclic trend. Both possibilities are speculation, alas.

England and Wales show a stronger upwards trend since 1967. The current levels aren’t much higher than they were back in 1967, though that might be a temporary spike downwards much like the spike upwards of the early 2000’s.

Canada’s almost the mirror image of the UK. Only 6.7% of the trend-cloud is positive, suggesting we’re trending down, but the current rates are higher than they were back in 1960. Unlike the UK, we can’t point to an obvious spike to wave that away, so we’re in a bit of a Monkey’s Paw scenario here.

So in three countries, we find two show an upward trend in homicides over time and one that shows a decrease but to levels that are still higher than they’ve been in the past. This would seem to be evidence against Pinker’s thesis, except Pinker himself considers those graphs evidence in favor of it!

In most countries the homicide rate has continued the Great Crime Decline that began in the 1990s, itself part of a much larger decline of homicide in the West since the Middle Ages, when the rate was 50 homicides/100,000/year.

By adjusting his timescales, he also adjusts his trendline; if you ignore all but the last decade or two, you can justify saying the homicide trend is downward, but then you’d be deliberately ignoring data that contradicts your hypothesis. Climate deniers are infamous for doing this; by placing an outlier near one or both ends of the trend, they can skew it however they wish.

Confront Pinker with an uptick in violence, like during that interview, and he can wave it away as a statistical blip that’ll disappear over longer terms. Confront him with a downtick in violence, even if it’s only in the last decade or two, and he counts it as evidence in favor of his century-spanning thesis. It’s win-win for Pinker, no matter what the evidence actually says. We have a term for that.

There’s another problem here, too.

During the last two decades, many police departments substantially undercounted reported rapes creating “paper” reductions in crime. Media investigations in Baltimore, New Orleans, Philadelphia, and St. Louis found that police eliminated rape complaints from official counts because of cultural hostility to rape complaints and to create the illusion of success in fighting violent crime. The undercounting cities used three difficult-to-detect methods to remove rape complaints from official records: designating a complaint as “unfounded” with little or no investigation; classifying an incident as a lesser offense; and, failing to create a written report that a victim made a rape complaint.

This study addresses how widespread the practice of undercounting rape is in police departments across the country. Because identifying fraudulent and incorrect data is essentially the task of distinguishing highly unusual data patterns, I apply a statistical outlier detection technique to determine which jurisdictions have substantial anomalies in their data. Using this novel method to determine if other municipalities likely failed to report the true number of rape complaints made, I find significant undercounting of rape incidents by police departments across the country. The results indicate that approximately 22% of the 210 studied police departments responsible for populations of at least 100,000 persons have substantial statistical irregularities in their rape data indicating considerable undercounting from 1995 to 2012. Notably, the number of undercounting jurisdictions has increased by over 61% during the eighteen years studied.

Yung, Corey Rayburn. “How to Lie with Rape Statistics: America’s Hidden Rape Crisis.” Iowa L. Rev. 99 (2014): 1197–1431.

The FBI and other law enforcement agencies have a motive for fudging the numbers, as it makes them look more effective than they actually are. Few people have the resources to audit their numbers, so the odds of getting caught are low. If you duly reported the FBI statistics, though, you’d be led to believe sexual assault is on the decline instead of on the increase.

How a statistic is collected matters at least as much as the statistic itself. If you’re vaguely feminist on the internet, you’ve had the National Crime Victimization Survey of 2014 tossed in your face as a refutation of 2007’s Campus Sexual Assault survey. The latter found “19% of undergraduate women reported experiencing attempted or completed sexual assault since entering college,” while the former found “4.3 victimizations per 1,000” for “rape and sexual assault.” I’ve already spent a fair number of words on the difference between the two surveys, so I’ll refer back there for the details, but suffice to say that different methodologies capture different views of similar data.

If methodology issues between experienced researchers can lead to dramatically different outcomes in the last decade, try imagining what sort of methodology issues pop up when comparing datasets spanning centuries.

Statistical datasets are a lot like wizards; they say exactly what they say, but they may not say what we want them to or what we think they do. There’s good reason to think the 19% statistic is more representative of sexual assault on campus than the 0.43% one, but that doesn’t make the latter statistic useless. It’s telling us something about our methodology, in the same way the FBI sexual assault statistics were in Yung’s paper. There’s no such thing as a “bogus statistic,” just statistics that you don’t understand or make you uncomfortable.

According to another media-driven moral panic, the United States is in the middle of a rape epidemic, particularly on college campuses. But this is based on bogus statistics; the latest FBI data show that rates of rape in the US have continued to decline.

Oh Pinker, you fool of a took. Setting aside your ignorance of the evidence for undercounting in the FBI’s data, you can look at your own graph and see a decline?

When Sargon of Arkkad tried and failed to discuss sexual assault statistics, he at least had the excuse of never having gotten a higher education, never studying up on the social sciences.

Steven Pinker is an experimental psychologist and one of the world’s foremost writers on language, mind, and human nature. Currently Johnstone Family Professor of Psychology at Harvard University, Pinker has also taught at Stanford and MIT. His research on vision, language, and social relations has won prizes from the National Academy of Sciences, the Royal Institution of Great Britain, the Cognitive Neuroscience Society, and the American Psychological Association. He has also received eight honorary doctorates, several teaching awards at MIT and Harvard, and numerous prizes for his books The Language Instinct, How the Mind Works, The Blank Slate, and The Better Angels of Our Nature.

I wonder what Steven Pinker’s excuse is.