Do as I say, don't do as I do

Lately awareness seems to be rising that impact factor mania has reached ridiculous proportions, and interestingly it appears as if one of the highest impact scientific journals is trying to hop onto the train. The American Association for the Advancement of Science, the publisher of the journal Science, is one of the signatories of the San Francisco Declaration on Research Assessment that quite sensibly urges researchers and their institutions to stop judging fellow researchers by the impact factor of the journals they publish in.

Perhaps to promote the declaration, the editor-in-chief of Science followed up with an editorial making the same argument, explaining how destructive the obsession with impact factors is and that they were never meant to evaluate people. Now another editorial has been published in Science bemoaning the emphasis on impact on a more general level, such as the increased push for applied as opposed to fundamental research, and the short-term-ism and risk avoidance that come with such a focus.

Basically I wholeheartedly agree with all of this. But here is the thing: I take issue with being lectured about impact mania from the pages of a journal that embodies all that those complaints are about. It's about the messenger, not the message. This is a journal that is well known for accepting publications not primarily on quality but on expected impact, and for rejecting everything out of hand that is unlikely to be a big thing over the next few years.

Don't get me wrong - I am not saying they don't have a generally good peer review, or that they publish more bad papers than would be expected from rare and unavoidable failures of the review process as they can happen anywhere. No, what I mean are those many, many papers, probably the majority of the submissions, that are never even reviewed because some managing editor turned them down on sight. Does anybody really assume they do that because they can immediately see that the paper is poor quality, or that the analyses are wrong? No, they do that because the topic is not sparkly enough, because if they published high quality papers that aren't sparkly enough then their rank as one of the two or three highest impact journals might suffer. Just you try submitting the most excellent taxonomic revision or seed germination experiment you have ever seen to Science, and see how it fares against cancer research, the discovery of a new exoplanet or even just the discovery of a new hominid bone fragment. Go on, I dare you. That is the name of the game.

And so it is rather galling to read these editorials in, of all places, Science, because to me they sound like "do as I say, don't do as I do". And honestly, gems like this, where the editorial argues against impact factor mania with the observation:

And it wastes the time of scientists by overloading highly cited journals such as Science with inappropriate submissions from researchers who are desperate to gain points from their evaluations.

Cry me a river. A journal that advertises itself with a tag line as nauseating as "The World's Leading Journal of Original Scientific Research, Global News, and Commentary" on its website does not really get to complain if people consider it a feather in their cap to get an article in there.