We need to change our current publishing model
I have written about publishing a few times (here and here), but recent conversations with colleagues have prompted me to write again. In essence, we need to change the way we publish science to pull back from the overt greed of publishers that is consuming funds for research. The public funds scientific endeavour through their taxes, the scientists produce the content, edit and conduct peer review. Then somehow, publishers have convinced us that they should charge for this content and even own it. This morally reprehensible situation must change, but how?
We need to explore all options, and in this blog I consider what science would be like if we went entirely preprint.
Do we really need peer review?
As far back as 2002, William Arms suggested that openly soliciting comments on the web might be an alternative for peer review of scholarly articles. Sixteen years later, this has come to pass in the form of preprints.
There is a growing trend for publishing preprints. Preprints are simply manuscripts that are submitted to an online server and available for all to view. Physicists started first with this phenomenon, but biologists have been hot on their heels and there are now a number of prominent pre-print archives including BioRxiv and PeerJ Preprints.
Each of these sites maintains the open access manuscripts, and allows other users to post comments (partial or complete peer-reviews) of these online manuscripts.
There are problems in the world of publishing. Mostly related to the greed of publishers who demand large sums of money for content that they do not pay to produce, but charge for access. We must look for alternatives to the current model, so could we replace publishing with an open platform like BioRxiv?
Could these comments pages really replace peer review?
Peer review is held as a gold standard in scientific publishing, and there’s certainly a lot to that. It ensures that published material has been read and its contents assessed independently. But peer review is fallible, because scientists are humans.
- Not all reviewers can assess all parts of a paper, especially papers that cover several disciplines
- Not all editors will choose reviewers that are independent and objective. Depending on the framework set up for the journal, friendly reviewers can be chosen or critical reviews removed. Perhaps the inverse is more common, although you are less likely to see these manuscripts published.
- Poor peer review is a growing problem.
- Lastly, and not least, there is an increasing difficulty in finding peers who are prepared to review manuscripts. (See the Perry et al editorial: “The Peer in Peer Review” published in 2012, which was a plea to the herpetological community to accept reviewing as a necessary duty).
In 2003, Stefano Mizzaro proposed changing peer review to the format that we now see in preprint journals. Let every reader become a reviewer.
In the preprint model, the first three problems (above) might all be overcome as no-one chooses the reviewers. Instead they choose themselves, are motivated to do the work. Their competence to cover all aspects of the manuscript is not assured, but one assumes that independently motivated reviewers will only comment on parts that they are able to assess.
All of this is very good, but will people actually read and comment?
A quick look at the sites will tell you a lot about the level of reviewing that is currently going on in biosciences pre-prints. A quick look through the top 10 articles on BioRxiv zoology section confirmed my suspicions. Plenty of tweets about the articles, but none of them had any comments. Indeed, a further trawl through PeerJ Preprints also found no comments.
Further, I’d suggest a greater move to this culture might produce comments for well known labs, a certain amount of trolling for labs with ongoing disputes or rivalries, making this kind of comment review a sort of trial by popularity. But I don’t see a situation where potential reviewers will take time-out once a week (for example) and hunt for manuscripts that have received no comments. It seems far more likely that the authors will have reciprocal agreements with other groups to review each other’s manuscripts. This nepotistic tendency then puts us back into the area of peer review that we’ve been working hard to overcome now for sometime (double-blind reviewing, editors codes of ethics, etc.)
Are preprints published?
As they each have a DOI (Digital Object Identifier), they are in their own way already published.
Another point is that these articles are picking up citations. And there is a new concern that these articles are being cited, even when they are subsequently available through a published journal. This is one of my personal concerns with using a pre-print service. I’m happy to put the paper out there for public comment, but the idea that it’ll remain there and that readers won’t necessarily be re-directed to the peer-reviewed version does concern me.
Another question is what happens to manuscripts that are placed on pre-print servers, are then sent out for review but not published because they are fundamentally flawed? It’s not as if the reviews are not made, but there is no automatic link to the reviews by the journal that conducted them.
Whether or not there is a paper inflation (see blog here), there is certainly an ever increasing number of papers. The rejection rate is not insignificant, and while many of the papers are not rejected because they are flawed (they may well go on to be published in another journal), there are certainly a lot of manuscripts out there with fundamental flaws. These are often sent for peer review, but those reviews pointing out the errors won’t necessarily make it back to the comments page on the pre-print server. I think that this is a serious problem. The reviewers have spent time and effort and the very reason they do this is so that manuscripts with fundamental flaws don’t find their way into the literature. However, pre-print servers have, perhaps unwittingly, found a loophole that allows manuscripts that are not scientifically robust a backdoor to citations.
But if they are fundamentally flawed, shouldn’t everyone be able to spot it?
No. Reviewers are chosen with great care because the area is in their particular domain. They have insights that not everyone will be aware of and these are an important aspect of the purpose of peer review.
I edit for the journal PeerJ. Although there can be various reasons to be rejected from PeerJ, normally it means that your paper is not scientifically sound. As PeerJ has no selection for impact, rejection does not normally mean that it can be simply submitted to another journal. I have noticed that manuscripts that I have rejected from PeerJ are available still as PeerJ-preprints without any comment on their failure to go through peer review. In my opinion, this is not good as it essentially ignores the input given by both reviewers and editors. The article appears as if it has had no comments or attention, when this is not the case. In a system where we move to relying more on pre-prints, why would we want to ignore chosen peer reviewers for whom this article was within their specialist area?
Moreover, I note that the pre-print in question is also receiving citations (according to Google Scholar), again raising concerns that rejection by peer review is not a hurdle to entering the scientific literature.
In my opinion, comments pages won’t replace peer review.
If we end up abandoning our current way of publishing in favour of a comments page, I think that we’ll all be worse off.
Having said this, I acknowledge that the current system is broken and that we need to find a new solution.
My feeling is that the system we have is not faulty prior to the involvement of the publishers. We simply need to overcome the vanity of having our manuscripts set out in a pretty way. Once we’re happy to accept unformatted manuscripts as the way science is presented, I think we’ll be able to move on without the involvement and the exorbitant costs that the publishers extort from us.
