PLoS ONE is a strange journal. Until recently, it was the largest journal in the world, publishing as many as 30,000 papers a year. It’s definitely the low end among the Public Library of Science (PLoS) stable of journals, but opinions of its quality vary over a wide spectrum. PLoS ONE‘s mission is explicitly different from most journals, as they say impact (i.e. significance) is not a consideration:

Often a journal’s decision not to publish a paper reflects an editor’s opinion about what is likely to have substantial impact in a given field. These subjective judgments can delay the publication of work that later proves to be of major significance. PLOS ONE will rigorously peer-review your submissions and publish all papers that are judged to be technically sound. Judgments about the importance of any particular paper are then made after publication by the readership, who are the most qualified to determine what is of interest to them.

My opinion of PLoS ONE is higher than that of some other scientists I’ve talked to. In my very limited experience, the peer review process at PLoS ONE is rigorous; in fact (and I am not proud to say this), I’ve had a manuscript rejected from PLoS ONE.

I also think there’s a niche in the community of academic publishing for an outlet that judges only the rigor of a study, not its importance. Science is a cumulative enterprise, with earlier results laying the groundwork for later results. As my Ph.D. advisor once told me, not every paper has to be a paradigm-changing discovery. Some are bricks that contribute to building a larger wall (recent events have made that imagery seem unfortunate, but never mind). In my opinion, science is better off if those bricks end up in the wall than in the file drawer, so that others can build on top of them.

I can attest that some papers published in PLoS ONE, which probably aren’t important to many scientists, have been important for my work. Jang & Ehrenreich 2012, for example, has only been cited ten times in the five years since it was published, but it has been a key resource for my lab’s microbial evolution experiments.

The danger for a journal publishing tens of thousands of papers per year is that it could easily slip into the realm of pay-to-play, predatory spam journals (see “Say it ain’t so! Beall’s list shuts down“). The only thing that separates a high-volume, open access journal like PLoS ONE from the dark underbelly of scholarly publishing is a rigorous peer review process.

That’s why the paper I wrote about on Tuesday is so concerning. That paper has several flaws, both in the methods and in the interpretation of results, that should never have gotten through peer review. That’s also why I’ll be so interested to see how the editorial staff responds. I know they’re aware of the problem, because they’ve responded on Twitter:

.@xprinceps Thank you for raising this to our attention. we are currently looking into the concerns raised.

— PLOS ONE (@PLOSONE) September 28, 2017

I don’t know what remedies they might consider. I suppose the most extreme would be to retract the paper. We’ll see what happens.

By the way, I’m trying not to be mean about this paper. It is bad, in its current form, but it could have been good. Just as when I’m reviewing a paper, I try to remember that this is most likely someone’s master’s thesis or a chapter of their PhD dissertation, and the lead author might have a couple of years of their life invested in it. I feel bad heaping criticism on someone’s hard work, but the peer review process exists for a reason.

I see this as a failing of the reviewers, the editorial staff, or both. I don’t have any inside information about this article’s peer review, but it reads very much like a submitted manuscript that hasn’t yet been revised in response to the reviewers’ comments. It may even turn out that that’s what it is; maybe this is as simple as the journal publishing the wrong version of the manuscript. I doubt it, but I hope so.

I’ll keep you posted.