More editorial blindness please

Apart from phylogenetic systematics, another topic that I seem to be writing a lot about is scientific publishing, and there especially about peer review and the trade-offs between the system we have now and the potential problems of open access.

Just today I had a short conversation at work that made me think about what I would change if I were made the dictator of science publishing. Really there are two answers, one dealing with the commercial publishing versus open access issue and one dealing with the gate-keeping and peer review issue.

The first one is simple. If I could reorganize scientific publishing, I would run it all through non-profit public utilities. There would be no publication fees for authors and no access fees for the reader. Everything would be open access and financed directly from taxes. Note that scientific publishing is, with the exception of a few advertisements in the journals, also entirely financed from taxes now, only it happens indirectly, through tax-funded libraries and tax-funded scientists paying publication costs. So the only difference would be that we would stop wasting taxpayer money to provide profits for commercial publishers. (Sad, I know.) As an added bonus from the open access perspective, with no author fees there would also be no incentive to accept crappy papers.

That would still leave the second issue, which is essentially one of quality control. If everybody could get published for free, how do you know what research is sound? As mentioned before, I do not think much of post-publication review because it is not clear how an author would be motivated to correct a manuscript that has already been published. It would also make it harder for newcomers to get their work noticed. Under the current system, they can get a good manuscript into an influential journal and people will read it. In a system relying on post-publication review, anybody without a famous name is likely to get lost in the flood of useless papers using the same key words.

So I would, if I were asked to reorganize things, probably try to keep our current peer review practices pretty much in place: Have a few colleagues assess the manuscript and make an editorial decision whether it (a) is sound science and (b) has been submitted to the right journal or archive. It isn't perfect but it allows me to put more trust into a publication in a high ranking journal than, say, into what somebody wrote on their blog.

However, one thing that bugs me mightily about the procedures in my area of science is that while the authors of a manuscript generally do not know who the peer reviewers are, the peer reviewers and editors can see who the authors are. Some people even call for the abolition of anonymous peer review but that would make it harder for young researchers to critically evaluate the manuscripts of influential older colleagues, especially those having a say in funding decisions.

I would go exactly the other way: peer review should be double-blind. If the peer reviewers and the editors don't know who the authors are, many possible sources of bias disappear. They will not as easily be able to wave through a sub-par manuscript from a big name in the field. They will not be able to be negatively predisposed towards articles coming from specific countries or specific universities. They will not be able to see whether an author has got a female or foreign sounding name. And so on.

But is that not technically difficult? It should not be, anymore. Most journals these days have online submission systems for manuscripts. Of course the manuscript management software needs to know the contact details of the authors to communicate with them, but there is no reason why even the chief editor of a journal should know it before they have decided whether to accept the manuscript. The following set of rules would appear workable:

• The manuscript management software hides the names and affiliations of the authors from the editor(s) in charge of deciding whether the manuscript ultimately gets accepted. It organizes all communication between editor(s), reviewers and authors anonymously.
• To ensure that the manuscript is not sent out for review to the authors themselves or to their close collaborators, a different editor or editorial assistant, in other words somebody who does not decide on ultimate acceptance, chooses the peer reviewers.
• Authors may not disclose their name on the manuscript. If they do so, it is rejected outright. They can be invited to resubmit the manuscript without their names on it, but then a different editor is assigned to it who has not yet seen their names.

None of that is all that hard to achieve; it would merely be a matter of changing journal policies.