- Bishop Hill blog

With commendable speed, Professor Hardaker, the CEO of the Royal Meteorological Society has responded to my email asking for a statement on the Society's position on one of its journals standing in the way of an attempt to replicate a study published there.

Thanks for your note. I've had a couple of emails relating to this discussion and the position currently is as Prof McGregor mentioned. As I have mentioned to others who have emailed in, I'm very happy to consider the requirement for a clear policy statement and as suchI have put this on the agenda for the next meeting of the Society's Scientific Publishing Committee, at which all the Editors of the Society's journals are members.

While I had hoped for at least some sort of recognition of the need for replication, it may be that Prof Hardaker feels he can't commit the Society to a new policy one way or the other without discussing the matter with his colleagues, and this is not an unreasonable position.

I would hope that the Society would start out from a position that every published study should be replicable and that both the data and code to do so should be in the public realm at the time of publication. The oft-cited policies of econometrics journals would be as good a place to start as any.

My concern would be that the committee ducks the issue by putting in place a bland policy stating that authors should make data available on request or some such, which then merely opens the question of what happens should authors refuse to do so - would the journal withdraw the paper? Would they sue the authors for breach of contract?. Assuredly not. It is surely in the interests of the Society to avoid having to deal with any of this kind of unpleasantness by ensuring that the data and code are handed over up front.

It will be interesting to see what they come up with.

Update on Jan 5, 2009 by

Bishop Hill

In response to a follow-up email, Prof Hardaker has provided some more details:

Many thanks for the pointer to the blog and I would be very happy to feed your thoughts into the discussion. I'd also be very happy to share with you the decision of the Committee, and of course any new policy would be publicised through our respective Journal pages. I would wholly expect that if the Committee decided to establish a policy it would do so for all the journals in the Society's portfolio, but that's just my assumption. The Scientific Publishing Committee next meets in early May. I recognise this is a little way away but it will give us the time to take a considered look at the comments we have received and best practice from other respected journals in time for when we send the committee paperwork out in April. I will also make sure the Chair of our Committee sees all the correspondence we have had associated with this.