In early February, a number of bloggers brought to your attention a peculiar paper on mitochondrial proteomics, a paper which was obviously odd on even casual inspection, containing grandiose claims of a theoretical revolution that were entirely unsupported and ludicrous assertions of evidence for God in the genome. Deeper examination revealed that much of the paper had also been plagiarized from various sources. To the credit of the journal, the paper was quickly retracted one month ago today; however, the retraction was entirely based on the plagiarism, and none of the other failings of the paper were addressed, nor were any of the patent errors in the review process at the journal Proteomics discussed. This is strange, especially in light of the fact that the Warda/Han paper was the most accessed article in the journal. This is not an issue that should be swept under the rug!
Today, several of us — Steven Salzberg, Lars Juhl Jensen, and Attila Csordas — are repeating our call for an explanation of the events that led to the leakage of such an egregiously ridiculous paper into print. Bad papers are a dime-a-dozen, and we aren’t so much concerned with the detailed discussion of the flaws in this one paper as we are with seeing the integrity of the peer-review process maintained, or better, improved. The Warda/Han paper had obvious red flags that marked it as potentially problematic in the title, the abstract, and scattered throughout the body, and it’s hard to imagine how any reviewer or editor could have let them simply slip by without comment, yet that is exactly what seems to have happened.
We want to know how this paper slipped through the cracks, because we want to know how large the cracks in the peer review process at Proteomics are. It’s a journal with a good reputation, and we are not presuming that there was any wrong-doing or systematic failure of peer review there, but we do think that a lack of transparency is of concern: there is no assumption of a crime, but the ongoing cover-up is grounds for suspicion. Let’s see some self-criticism from the journal editor, and an open discussion of steps being taken to prevent such errors from occurring again.
Alternatively, if the journal wants to outsource its quality control to a mob of bloggers, that works, too … but we tend to be less formal and much more brutally and publicly critical than an in-house process might be, and we’re also going to be less well-informed than the actual principals in the review process. Better explanations are in order. Let’s see representatives of the journal provide them.
Right on.
As a (too) active peer reviewer for journals (editorial board on 4) and grant applications, I don’t see how a blog posse has much to offer to secure the integrity of the peer review process.
Well, what do you expect — it was linked from Pharyngula, for Pete’s sake. :-)
But if we don’t let them plagiarize other people’s works how will IDers and Creation Scientists (I throw up a little when ever I use that phrase) be able to publish any papers?
Surely you can’t expect them to do real research like real scientists. That would be impossible.
Creation Science (swallow the bile, breath in, out, in, out, OK) has two premises that it can seek to experiment on. One, they can prove god/intelligent creator/whatever exists. This is impossible for obvious reasons. They more commonly try to prove their points by highlighting seeking evidence that evolution is impossible, or doesn’t work. In this case they are trying to prove a negative. It’s like trying to prove that the Flying Spaghetti Monster or the Invisible Pink Unicorn doesn’t exist. It’s a null arguement.
So we’re back to square one. Creationists have no experimental data because it’s impossible to create an experiment that proves or disproves any of their points. Experimental data is a hallmark of real science. Since Creation Science (ugh, gonna have to brush my teeth after that one) can’t produce experimental data, or even a proper testable hypothesis it can’t be real science.
Mike Dunn has got some explaining to do. I predicted we wouldn’t get an explanation.
Still wouldn’t hurt to lean on the publisher, the editor, and anyone else relevant.
My best guess is Dunn is a creo, picked fellow creos as reviewers and tried to slip creo nonsense into the journal. A theory to be sure but it fits the available facts better than, oooppppps, we did a boo boo.
Doesn’t matter that much. In my mind Proteomics is a suspect journal and a joke until they come clean and fire a few people. Mike Dunn should be the first.
This whole incident was an attack on a major foundation of science, integrity. We rely on the assumption that the science is honest and use peer review to confirm it. That is one glaring difference between science and creos. They lie to support their original lies.
I think that’s the point: a blog posse by itself can’t improve the process, except indirectly, by nagging the journal to clean house. This is a clear-cut case where something in the process was broken, and I’d like to see the journal explain what went wrong and what they’re going to do to improve it.
You’re entirely right. Pharyngula can’t fix it. Only the journal can. And their lack of action so far does not convince me that they are going to try to fix it.
I wouldn’t make such an assumption about Dunn. Proteomics is not Rivista, and the editor has some clear self-interest in maintaining the reputation of the journal — there had to have been some slip-up in the process somewhere, because even if he were a creationist sympathizer, he wouldn’t have wanted a paper this bad to represent the journal.
I think someone’s ashamed to admit that they used the stair-toss method to pick what got published one month.
What would maintain the reputation of the journal and Dunn is explaining a few things.
Hasn’t happened yet despite the wolf pack howling outside the door.
Dunn is acting guilty and should expect people to assume the worst. He could prove me and everyone else wrong in a heartbeat by a novel procedure. Telling the truth. Ball’s been in his court for a month and he hasn’t hit it back.
(.)(.)
First, journal editors and the reviewers who they deploy are volunteer scientists providing a service to their research community. There is no way the editor of a busy journal can carefully read all of its articles. A reviewer screwed up. That much is obvious.
Second, any public accounting of the affair is complicated by the fact that the careers of the submitting authors are probably on the line and legalities are or can be involved. They have a higher standard of proof in making public judgments than bloggers. If I was the reviewer or editor, I’d also keep my mouth shut and let due process take its course.
Third, the value of a journal is intimately entwined with its reputation. Publishing in the more respected journals matters to scientists. If Proteomics really has a systemic peer review problem that anybody really needs to be concerned about, that will become evident over time, as measured by the quality and impact of its articles.
Rage on, but it looks to me more like self-righteous spittle directed at an isolated incident.
I agree with as much pressure as possible. The peer review process is critical for science. I hope that proteomics realizes their responsibility and gives a full and transparent accounting of events and proposed reforms that we can see and all agree is adequate.
Denisc
Well, the question is: who were the reviewers?
I’m pretty sure that’s kept confidential, of course. But it’d be interesting to see the comments provided by them (which should be anonymous). Did they raise criticisms but have them over-ridden? Or did they not note the problems which others have?
If I remember (I’ve been out of the sciences for 3 years) some journals let you (the author) give a list of reviewers from which the journal might choose (I don’t know if they really follow your suggestions, though) while others choose the reviewers according to their own process. Is that a factor here?
Ah, but if it’s a reviewer that screwed up, it’s easy. The editor says “A reviewer screwed up, didn’t read the article critically, and rubber-stamped it. We will not be using that reviewer any more.” End of story. We’d be content. Nobody is asking for a step-by-step series of details with names and addresses and phone numbers, but just a basic explanation of what went awry.
This isn’t that hard.
An editor who sits silently and pretends nothing happens, and uses the plagiarism as a good pretext to avoid the issue, though…that’s a problem that fuels poor perception of the journal.
Don’t get me wrong, I now have a poor perception of that journal because of this incident. I’ll think twice about what I see in that journal because of it-not that its on my ‘must read list’. Applied natural selection, if you will allow.
Still, careers are involved and it takes time for professional misconduct in academia to be thoroughly investigated. I simply wouldn’t expect any explanation from the journal so early in the process, less it expose itself to liability.
First, journal editors and the reviewers who they deploy are volunteer scientists providing a service to their research community. There is no way the editor of a busy journal can carefully read all of its articles. A reviewer screwed up. That much is obvious.
It is also obvious that the editor screwed up as even a quick read of the paper should have raised red flags.
Second, any public accounting of the affair is complicated by the fact that the careers of the submitting authors are probably on the line and legalities are or can be involved. They have a higher standard of proof in making public judgments than bloggers. If I was the reviewer or editor, I’d also keep my mouth shut and let due process take its course.
There are legalities involved, but that doesn’t mean the editor just has to sit there and say nothing. Why can’t the editor make a statement identifying the problem and indicating what steps have been taken to make sure the process works better the next time? All that could be done without naming the reviewers.
If I had been one of the reviewers, as soon as I saw it was accepted I would have written the editor indicating I wouldn’t be reviewing for Proteomics anymore. The paper was that bad.
Third, the value of a journal is intimately entwined with its reputation. Publishing in the more respected journals matters to scientists. If Proteomics really has a systemic peer review problem that anybody really needs to be concerned about, that will become evident over time, as measured by the quality and impact of its articles.
Rage on, but it looks to me more like self-righteous spittle directed at an isolated incident.
Think of it as a public letter to the editor if it makes you feel better. A mistake this large indicates that there are some fundamental things wrong at Proteomics.
Again, that’s fine. I can appreciate that they want to negotiate the legal minefield carefully. But there should at least be a statement that they are examining their procedures carefully (and maybe that eminent proteomics researchers X, Y, and Z have been asked to consult), with improved processes to be announced later. As it is, it looks like they heaved a sigh of relief that the authors were plagiarists, so they can pin the blame simply and cleanly on Warda and Han without bothering to do any introspection.
If we assume that the problem is that two or more peer reviewers overlooked the glaring shortcomings of the manuscript by Warda and Han, then I cannot see what the legal problems would be. Warda and Han have already been accused of plagiarism – I cannot see how admitting errors by the peer reviewers would put them in a worse position than they are already in. We are not asking for names of the anonymous reviewers, so even if the blame was to be pinned on the reviewers, it would have no implications for anyone.
Considering that Michael Dunn has chosen to stonewall, I cannot help but suspect that the situation is not quite that simple. Most likely, the changes were introduced during revision, and the revised version was never send to the reviewers. In this case, the editor of Proteomics would be to blame, which would explain the deafening silence from Michael Dunn.
Just for the record, I have sent an email to the Editor in Chief informing him of the joint blog posts. Just to make sure that he cannot claim that he didn’t notice our blog posts.
But we can reasonably expect someone to at least read the titles!
That is the position I took a month ago. However, IMO it has been far too long for us not to hear *anything* substantial, and for my money, they have expended all their benefit of the doubt.
It could hardly be more evident than it was made in the article under discussion. What the editor(s) and publisher owe the scientific community is some explanation of how such a glaring faux pas happened, and what they are doing to make sure it doesn’t happen again.
“Isolated incidents” like this simply are not tolerable.
Yes, they can be fault-free mistakes – e.g., maybe the nonsense was added to the final version of the paper and the journal’s procedures do not call for a sanity check at that point. But if that was the case, the journal owes the community an explanation of what did in fact happen, and of what procedures they are putting in place to prevent a repeat performance. And there has been *plenty* of time to detect and correct that sort of fault.
Notice that a month ago I was the one arguing “give them a little time”. But for my money, the timer has expired.
Oh, I understand the problem you’re raising is that they simply are not addressing the issue at all. I was mentioning the reviews because of one of three possibilities.
(1) The reviewers all (usually 2-3, correct?) felt it was reasonably good for publication. In this case, yes, the answer seems simple: “we trusted the reviewers… they (all?) felt it was good. We will review our processes for more oversight.” Or something like that. As you say, plenty of so-so stuff makes it through the process all the time, it’s nothing new.
(2) The reviews were mixed, either some being cool with it and one or more panning it OR one or more of them being somewhat hesitant to give it a hearty “OK”. Presumably the authors would have attempted to address these concerns with a re-write, or possibly not. In either case, the journal prints it anyway. This raises some questions re: the journal and it’s ultimate editorial decisions.
(3) The reviewers all panned it. It got published anyway. This raises major questions. Why publish a panned paper?
By the way, I had understood that case #3 DOES happen on occasion in some journals – for reasons that are definitely not science-worthy or even very respectable (a favor to a friend, an editorial disagreement with the reviewers) but aren’t necessarily nefarious.
But in the cast of this paper, as I understand it, additional concerns are raised.
I’m not sure who could organize it and how much effort it would take, but one sure way to bring Dunn and Proteomics to heel would be to have a group of scientists in the field announce that they will publish online, post-publication “peer reviews” of every paper the journal publishes until there is a satisfactory response.
One month is simply no time at all on an academic scale.
So shortly after the paper was exposed, it doesn’t make any sense to doubt the general idea that those in charge of Proteomics, those in charge of any journal, aren’t already weighing alternatives to tighten up procedures. If you doubt that, you don’t have a lot of insight or know much about academic publishing.
And my take on the pattern of events, and I may be wrong, is the Korean guy has sort of hung out the Egyptian guy to dry, blaming him on the hole plagiarism thing. If the Egyptian doesn’t take that sitting, you can pretty much bet your college fund you won’t have heard the last on who deserves blame for some time to come.
And for some pleasure reading on how witch hunts of academic impropriety can both spin out of control and drag on forever, try googling Imanishi-Kari + scandal.
#20
Since it was a review article–whether invited or solicited–and given how bad it was (I read it, btw) my guess is only one reviewer was assigned to it…if that, because review articles in some respects are really just editorial/opinion pieces.
If true, my prediction is that the journal will revise procedures to get two reviewers minimally for review articles.
I have to disagree with the person who claims that one month is no time on the academic scale. I work on topics related to those covered by Proteomics, and as a peer reviewer the journals give me between 10 and 14 days to review a manuscript.
#23
Funny, I was literally just back-reading about the whole thing from the earlier posts (I just started reading here) when I realized that it was a review article. I stand corrected.
This raises different issues. I had understood that usually you’re invited to write a review article. Somehow, this put you on a different track than if you were submitting a research manuscript.
I was co-author on a few reviews and the one thing I remember wondering how it didn’t go through any apparent peer-review process in the way research articles do. Perhaps I missed it, but the only bits we had to address were grammatical or consistency errors. We were never really called on substance – as would be common with research manuscripts.
How DOES the process work for review articles? I had assumed someone at least read it to make sure it’s not bizzaro or profanity-laden or something…
At the minimum, even if this is simply a matter of review articles just being given a pass, this should raise awareness that there is a potential loophole where pseudo-science can seep into the literature.
I mean review articles are often a first stop for a person to get introduced to a field of science. How many people actually take the time to go and read the referenced papers in the reviews? I mean everyone should, but…
Of course, the “single common fingerprint initiated by a mighty creator” quote should have made anyone stop in their tracks.
#24
My screen name is me so you’re supposed to say, “I disagree with me” ;>
They can make editorial decisions in no time at all-in less than a day if necessary-but I’m referring to digging down to the bottom of what went wrong with the Han paper, from start to finish, on the part of the authors and on the part of the editorial side. This would likely become an investigation involving a few institutions. Stuff like that takes a lot longer than getting a paper accepted.
I can see a situation where the publisher would be liable for any public statements made by the editor that might put the authors or anyone else in a negative light. I’d bet all parties are lawyered up.
That’s mostly my whole point why the blogosphere shouldn’t expect, and pointlessly demand, an explanation. Especially so early in the affair. Those volunteering at the journal expose themselves and their deep pocketed publisher to far greater loss than what they’d hope to gain by providing explanations to an enraged posse.
I’ve had review articles handled almost perfunctorily by editors, and have had them go to a couple of reviewers and received real feedback.
It all depends on the journal.
Copy editors are like the final last check. They usually ask authors in the proofs to revise that stuff unclear to them.
I’ve got to believe “single common fingerprint…” is one of those on-line korean to english or arabic to english translations. Those things are great at spitting out nonsensical stuff.
My favorite fun evolution experiment is to write something in english, have it translated online into another language, and then back into english.
I just did an english to korean to english google translation. These data are consistent with my proposition at the end of comment 27.
This is the last time I have to say on this issue because I have enough time to waste it, and if you do not wish to waste any more persuasive, of course, since I do not have anyone. I say it enough how the protein of interest in this article bumbled, but troopers could get more competent peer review system, we do not understand them. I do not appear in a conspiracy behind this, and if this one, because I am convinced that the process of revealing the conspiracy.
We are lucky that we live in an age of online pre-prints — had this been a few years ago, the first we would have heard of this article was when it appear on paper in journal copies delivered to libraries and offices. That would have been a much bigger mess to clean up. In many ways, the Internet has radically changed the way academics talk to each other.
I’d just like to jump on this chain of (probably innocent) cover-up to draw attention to the Aetogate scandal (see http://www.miketaylor.org.uk/dino/nm/index.html) currently disturbing the fair world of Palaeontology. I have to admit that the continual cover-up and obfuscation in this case seems to suggest that there is something in it, and I’d call for all science-folk to weigh in with thier opinions, particularly if it leads to more transparency on the use (and abuse!) of in-house journals. Although this has nothing directly to do with the usual content of this Sciblog, it’s an issue that I believe affects the whole of the scientific sphere….
Indeed, the Spencer Lucas situation is probably of more immediate concern for the scientific community, at least fr the paleontological community.
As for the Warda and Han paper, suffice it to say that peer review sometimes fails, and sometimes that failure is significant enough that a paper needs to be attacked critically in public. At the same time, I disagree that greater transparency of the review process will help that. Referee anonymity is critical to the peer-review process, as is the privacy of reviewer correspondence. I know that the NIH is currently discussing means of improving peer-review, although their current report seems like a mess of buzzwords and weasel-speak. On the other hand, we could have a situation like that at PLoS, which publishes reviewer comments online, something I find very distasteful and damaging to the peer-review process.
At least, this is my take on the matter.
If that’s the case then I’ve missed a chunk of the story (always possible) – what is that based on?
At least we know that Warda (the Egyptian) has owned up to, even boasted about, the “mighty creator” bit. And given Han’s initial clumsy attempt to paper over the cracks, I reckon they’re in it together. Until proven otherwise, anyway.
I have heard speculation that it was possible that the title change and wierdness only appeared in the final draft, in which case there were two more-or-less unrelated problems
1) The uncaught plagiarism which suggests the reviewers of the review article didn’t know the field well enough to spot the lack of citation to well-known papers where the material came from.
2) The lack of a final review (or readthrough) to catch the crazy (and unsupported) claims in a review article.
If the claims were introduced in the final version, then mechanical tools like DIFF would have flagged them for review, and perhaps we are nitpicking on the quality of the review process. But if the claims were in the original, we have a complete failure of peer review at every step.
Regardless of if it was Han or Warda or some crazy typesetter/secretary/bicycle messenger, new and unsupported conclusions don’t belong in a review article, and the newness and craziness were strongly suggested in the title, and could have been found by a high school student. An automation tool could have caught the copying. But if the editors let their invitation to write this review article cause them to bypass the review process, then the error is at once more understandable and less forgivable.
We don’t need to know WHO failed, but we do need to know HOW peer review failed. At a minimum, the editors need to tell us the “Never Again” story with a specific fix to the process. Embarrassed silence is not the way you deal with a PR problem that diminished confidence. But as it has only been one month, perhaps this will be covered in an editorial.
One can only hope.
On a related note, I notified the EIC of the Glycoconjugate Journal on February 24 about another case of plagiarism in a Warda and Han (two sections lifted from two separate sources, discovered by Sarah W. on this blog) and I’ve heard nothing.
The ‘mob of bloggers’ line got me thinking.
It would be very helpful to be able to submit papers (and parts of papers / ideas) to an informal ‘mob of bloggers’ peer-review process online. However, the material needs to be protected from being scooped or the ideas being plagerized before the paper is actually published. There are also probably copyright / ‘already published’ issues which might make a paper difficult to actually get published after such a process.
Any ideas how we could build a system/community that could accomplish the goal? Coturnix (A blog around the clock) may be interested and have some good ideas. And I imagine Seed (or a variety of other places) would be more than happy to get the ad revenue from hosting such a thing.
Re: travc
I’ve been thinking exactly the same, and I should have the web capacity to host such a system. I’ve been arguing for it for ages.
There’s researchblogging.com for blogging about peer-reviewed research, but it doesn’t achieve a way of rating papers or quickly presenting their pros and cons.
I’d like to have a site where anyone can submit a paper with a link. Then registered users can add comments under four headings: pros | cons | what it says | what it doesn’t say. The pros are totted up against the cons. Registration is by invitation only, and software would detect outliers – people who’s comments are consistently far from the general consensus.
If anyone is interested, I’d love to hear from them… my e-mail is at the top of my blog, http://www.layscience.net .
@ #20 – why publish a panned paper? You’d be surprised!
I reviewed a paper for the European Biochem’ journal in ’05 and both myself and the other reviewer recommended major revision. 2 weeks later the paper came back with virtually no changes and a bolshy response letter. I rejected it. The other reviewer was listed as “terminated” in the on-line system. Now this is where it gets interesting: The editor sends me an e-mail, informing me of the final decision – Reject. 3 months later I see the paper in press in the journal. Listed acceptance data is 5 days after the editor’s e-mail.
I tried to follow this up (i.e. what happened during those 5 days, and why did you accept a paper when the sole reviewer rejected it?) and got stonewalled. Eventually the paper got retracted/corrigendum because of some other issues surrounding the background science being bad. So, I was right all along. Did I get an apology from the journal? Did I hell!
Another example… right now I’m involved in trying to get a letter to the editor published in a different journal, regarding serious problems with a paper they published (major major flaws that are embarrassing to the authors and whoever reviewed this). The editors rejected my letter first time round, without even sending it out for review. It comes to something when a journal won’t even allow its published work to be discussed in open forum on its pages! Bottom line – once its out there, the deal is done, and you’re the one who has to waste your time proving otherwise.
Journal editors are an odd bunch who think they’re above the law. Dunn is no different in that regard. Kudos to PZ and everyone for keeping up the pressure on Proteomics and Dunn. There is a desperate need to wrestle control over OUR science from these mindless idiots. Who the **** do they think they are?
I interupt this blog with a special announcement: YOUR MOTHER IS NOT AN ANIMAL: The earth shows evidence of being very young just as described in the Bible. For example, some have promoted the Grand Canyon as obviously very old with its distinctive ribbon of a river one-mile below the flat upper surface. ‘Surely it must have taken millions of years for that narrow running water to cut such a chasm’, goes the reasoning. Concluding an age of millions of years however is the end of a series of broad assumptions. The walls of the Grand Canyon and surrounding area indicate something quite different. First, the hundreds of sedimentary layers making up the canyon walls appear to come from a single, huge water borne deposition event occurring over a relatively short period. This is known by the lack of surface erosion in between layers. In other words, the thousands of feet of layered canyon walls when dug into, do not show independent weather erosion effects as would be expected if those layers were laid down slowly over millions of years. Moreover, with the old age scenario, there should be root systems and other organic residue from plant growth remaining between the strata layers. Yet the canyon wall diggings have nowhere shown these independent growth layers. Instead the plant and animal remains dug up often transit one or more layers, and when uncovered, show all signs they were deposited in ways a flood buries things. Only the current topsoil layers indicate erosion and exposure long enough for rain and sunlight to produce plant growth. The fact remains that flood born sedimentary layering with these same characteristics covers all major continents, often going uninterrupted for hundreds and even thousands of miles. Here again the only reasonable conclusion for the sedimentary layering making up the canyon walls and surrounding area is a single brief massive flood. So how could the canyon have been cut so deep and wide? Probably the best idea is the ‘dam-break’ theory. There is a large multi-state topographically low area of the United States north of the canyon. The broad width-to-depth proportions of the Grand Canyon itself argue against the canyon chasms having been carved by a thin river- but better for a one-time massive water runoff event, very likely while post flood sedimentary layers were still soft. Indications are present that a large ‘lagoon’ had formed in that northern low area after the floodwater receded. At some point soon after, the immense dammed lake area broke open washing huge volumes of land into the ocean. The final result is the current rain runoff-river winding through picturesque canyons much as it is today. Flying over the arid area, one immediately notices that 95% of the vast water-carved topography surrounding the canyon river has no water within or nearby. A post-flood ‘lagoon breach’ better explains the parks peculiar wide dry washed out canyons as well.
Michael Woelfel — your lies are out of date, please return them to 1801.
1) My mother is an animal — she is made of meat and she does all of her thinking with two kilos of meat, and she communicates by waving, flapping and manipulating gas with meat. Where is the evidence you are thinking? Where is the evidence that you are not bearing false witness?
2) The water features in caves at the top of the Grand Canyon are millions of years older than those at the bottom. This is inconsistant with a timeframe of less than 6000 years.
3) The hundreds of distinct layers most obviously came from hundreds of distinct sedimentation events. They are not ordered by particle size or density as the single-event model implies.
4) Your claim of a lack of features is not supported by studies. Some surface features which are obvious are lava flows. Other surface features like the Surprise Canyon Formation indicate weathing occured in the middle of the sedimentation process, which is completely inconsistent with the magical flood model.
5) The gross geography of the Grand Canyon is totally inconsistant with the dam burst model, which produces a wide, shallow, straight channel, not the actual sinuous, deep channel of the Grand Canyon which has the types of meanders which only slow waters can achieve.
6) You miss the point that the Grand Canyon sediments are billions of years old.
7) You are off topic.
http://www.talkorigins.org/indexcc/CH/CH581.html
http://www.jwoolfden.com/gc_intro.html