In response to Tom Sheldon’s dire warnings of the dangers of preprints, “Preprints could promote confusion and distortion,” I’ve suggested that what really promotes confusion and distortion is credulous reporters failing to apply basic journalistic standards:

Peer review isn’t a magic wand that guarantees that only solid work gets published, and it isn’t a substitute for skepticism. Reporters have a responsibility to evaluate the evidence in a paper whether it is peer reviewed or not.

A couple of recent examples are relevant. First, the claim by mathematician Michael Atiyah to have proven the Riemann Hypothesis, an immensely important number theory problem related to the distribution of prime numbers. Remember, along with promoting “confusion and distortion,” Sheldon had warned that preprints could rob journalists of “time and breathing space,” pressuring them to rush to sensationalize bad science. Reporting on Atiyah’s claim shows what utter nonsense this is.

Here’s Gilead Amit​, writing for New Scientist [emphasis added]:

One of the most important unsolved problems in mathematics may have been solved, retired mathematician Michael Atiyah is set to claim on Monday.

If a solution to the Riemann hypothesis is confirmed, it would be big news.

Everything is conditional: all Amit is saying is that Atiyah is making the claim, not that it is true. Furthermore, Amit is transparent in his effort, and ultimate failure, to corroborate Atiyah’s claim:

New Scientist contacted a number of mathematicians to comment on the claimed proof, but all of them declined. Atiyah has produced a number of papers in recent years making remarkable claims which have so far failed to convince his peers.

Contrary to Sheldon’s concerns, Amit was perfectly capable of responsibly reporting on non peer-reviewed science (well, math). His makes no spectacular claims, repeatedly cautions that the work has not been vetted, and keeps a skeptical tone throughout (even more so in a followup article three days later).

Contrast that with Asa Svensson’s reporting on the Katla volcano in Iceland in The Sunday Times [paywalled, but you can read enough]:

Scientists have warned that Katla, a “highly hazardous” giant volcano in Iceland, is showing signs that it could erupt on a scale that dwarfs the explosion that disrupted worldwide air travel eight years ago.

Thing is, scientists have said no such thing. Doyle Rice, writing for USA Today, interviewed the lead author of the study Svensson was reporting on:

“I said explicitly that we are in no position to say whether or not Katla volcano is ready to erupt; and that air traffic disruption in case of an eruption is unlikely to be as serious as in 2010,” said study lead author Evgenia Ilyinskaya, a volcanologist at the University of Leeds in the United Kingdom.

Dr. Ilyinskaya also complained about The Sunday Times‘ reporting on Twitter:

This article is untrue and should be retracted. You misrepresented the scientific facts I gave you in order to create a piece of sensationalist clickbait https://t.co/1fE5gIiBBz

— Evgenia Ilyinskaya (@EIlyinskaya) September 23, 2018

The study, in Geophysical Research Letters, is about the amount of carbon dioxide the volcano produces, not about eruptions at all:

We discovered that Katla volcano in Iceland is a globally important source of atmospheric carbon dioxide (CO2) in spite of being previously assumed to be a minor gas emitter…Through high‐precision airborne measurements and atmospheric dispersion modelling, we show that Katla, a highly hazardous subglacial volcano which last erupted 100 years ago, is one of the largest volcanic sources of CO2 on Earth, releasing up to 5% of total global volcanic emissions.

So we have one example of perfectly responsible reporting of non peer-reviewed research and one of stunningly irresponsible reporting on peer-reviewed research. Tellingly, this is actually in line with Sheldon’s own article, in which he admits that “we do not yet have examples of harm” from journalists reporting on preprints, and instead gives examples of bad science in peer-reviewed articles.

Peer review does not magically guarantee good science, and it certainly doesn’t have the power to prevent distortion and exaggeration by journalists. It’s a useful filter but not a perfect one, and it doesn’t relieve journalists of their professional responsibility to exercise appropriate skepticism.