Bishop Hill More peer review gatekeeping
Mar 24, 2010 Climate: Models Journals Icecap has an interesting new article by three sceptic scientists - John McLean, Chris de Freitas and Bob Carter - describing the successful attempts to deny them a right of reply in the peer-reviewed literature.
The practice of editorial rejection of the authors’ response to criticism is unprecedented in our experience. It is surprising because it amounts to the editorial usurping of the right of authors to defend their paper and deprives readers from hearing all sides of a scientific discussion before they make up their own minds on an issue. It is declaring that the journal editor - or the reviewers to whom he defers - will decide if authors can defend papers that have already been positively reviewed and been published by that same journal. Such an attitude is the antithesis of productive scientific discussion.
Read the whole thing (PDF).
Reader Comments (19)
Dear God. I have never heard of a Response being refused. I can see that an Editor might want to vet it for, say, length He might even want to insist on civility. And he's duty-bound to require clarity. But since it passes all three of these tests, I think that the guys have a legitmate complaint. Shameful.
So, let us create a list of publishers with unscientific behaviour or doubtful scientific behaviour. Or a table with publishers and grading their scientific behaviour and thinking (not to mix up with the quality of the science they publish): 1-6:
Scientific above all means
Very small scientific flaws
Not always scientifically correct
......
Then a second list to be created by exceptional staticians:
statistically beyond doubt
minor flaws
statistics dead wrong
And another
data cherry picked - or not
And a list of publishers who or whose peers accept sentences like: We believe in ......., especially in the abstracts. Belief is not science.
Then a list of peers judging in circles: eg. Jones-Mann-Trenberth-Jones-....
Be the first one to rate publishers and they will get into thre defense position. My 2 cents. Keep on, you'r doing fine.
To add: The goal of such lists is not to get all publishers or partisan publishers in a defense position, but to mark those who act unscientifically, like those who refuse responses as is described above.
Sigh! How utterly sad.
This is really utterly depressing. The state of climate "science" is dire....
Besides who's scientifically right or wrong, the process is rotten until the very core.
No surprise to see Tamino misbehaving here.
This is disturbing. It makes the issue of the make up of various investigatory bodies even more of an issue. Jones' involvement in this incident certainly needs to be brought to the attention of Russell et al.
Appalling.
There is no "right of reply". This is a scholarly journal not a university debating club.
The reason they couldn't get a reply past peer review is the obvious: their attempts at replies were crap, as was their paper.
James Annan has responded to their 'science by press release'.
James Annan has a different take to be sure, but methinks he doth protest too much.
This bears certain similarities to the issue of the Mckitrick and Michaels examination of the surface record. Gavin Schmidt published a critique of it in the IJOC which contained two erros. The first was that he confused autocorrelation in the dependent variable of a regression, not necessarily a problem, with autocorrelation in the residuals, which does matter. And M&M's equation did not have autocorrelated residuals. The second errors was that Schmidt did not realise that his own estimated equation, alleged to refute M&M, actually had coefficients which lay outside the range which he himself said would constitute refutation. Mckitrick drafted a response pointing out these mistakes, in my opinion devastating and unanswerable, which IJOC has recently rejected on the flimsiest of grounds. YOu can find all the relevant material here
http://sites.google.com/site/rossmckitrick/
in the new items section.
Note that Phil Jones gave a very cursory review of the Schnmidt paper - see climategate emails, which failed to pick up either of the mistakes and also seemed driven by the idea of getting a refutation of M&M into the peer-reviewed literature before the AR5.
Check Appendix A Page 12 of the Critique
Last paragraph
(b) inappropiate splicing of different data products
Frank O'Dwyer - you said
QUOTE
There is no "right of reply". This is a scholarly journal not a university debating club.
The reason they couldn't get a reply past peer review is the obvious: their attempts at replies were crap, as was their paper.
UNQUOTE
The appropriate method would have been as follows:
Scientist A has a paper published after favourable peer review.
Scientist B disagrees with the conclusions and has a paper published after peer review, which criticises the first paper.
Scientist A replies, rebutting these criticisms to the best of his ability. This is also peer reviewed.
THEN the scientific community evaluates these papers and individual scientists come to their own conclusions.
Refusing to publish the rebuttal distorts the scientific process and reflects badly on the integrity of the journal.
Frank O'Dwyer:
There is no "right of reply".
Formally, there might be no right to reply but there was an invitation to a reply by an editor.
November 20, 2009. Editor-2 informs McLean et al. that:
“Comments on your JGR-Atmospheres manuscript "Influence of the Southern
Oscillation on tropospheric temperature" have been submitted by Foster et al.
and have passed peer review. I invite you to submit a reply to the comments,
which are attached. ...... Your reply will be reviewed and, if acceptable, will be
published at the same time as the comments.”
I have read the comment by Foster et al. and the unpublished reply to the comment.
The comment appears to me well in line with all the (sometimes weak) rebuttals by the Hockey Team (B.H.'s HSI book contains some other pretty examples).
The usual tactics is: Claim that an author has said something, which he actually has not, and rebut that. Then this author needs to spend a lot of time (and journal space) to make clear what he has originally stated and to express, politely, that the commenters have obviously not read the paper clearly. And the original paper is, in this case, not to blame as it is indeed quite clearly written.
Frustrating, if people do not read your papers as carefully as you have written them before declaring them wrong. Politicians behave like that, some scientiest obviously as well. Listening is a virtue of debating, well, unless you don't have a point but don't like what your opponent is saying.
Sad...
Frank O'Dwyer
It is beyond belief that you can even consider defending behaviour such as that outlined in the SPPI paper. You may, in your wisdom, think the paper is crap (doubtless based on the opinion of team members), but at least 3 qualified reviewers disagreed and recommended publishing.
That is what peer reviewed means. If this was a paper supporting your prejudices, you would declare that it was Peer Reviewed, and therefore true beyond all question.
JGR has clearly breached both their own publishing guild-lines, and what any fair minded observer would regard as proper scientific conduct. Your defence of such behaviour condemns you as a scientific corruption denialist, who will condone any degree of scientific malpractice as long as it supports your pre conceived opinion.
AusieDan,
And that is exactly what occurred. The best of their abilities wasn't good enough. It didn't pass peer review.
What on earth do you think the function of peer review is, if it is never to reject anything?
They are not being censored. Their reply is published, just not in the peer reviewed literature (where only things that pass peer review get published, the clue is in the name).
Indeed there is probably not a scientist in the field, and many a non-scientist, who isn't aware of their reply. Just now they are aware of one other salient fact: they could not, or would not, muster a reply that would pass peer review.
Frank O'Dwyer
It is hard to believe you are seriously suggesting that the reply in question was so deficient as to be unworthy of publication, and that this unworthiness outweighed the obvious bias betrayed by succumbing to pressure from Jones et al to leave a heavily biased criticism (which struggled to pass peer review itself), unanswered.
Of course the problem with this idea is that we have all read the paper and the reply, and this is clearly not the case. You mention no specific reason why the reply was so unworthy, other than Phil Jones' say so. At the very least the scientific community should have been afforded the opportunity to assess the validity or otherwise of the arguments. To not do so strongly implies a desire to avoid scrutiny, which is often the mark of shoddy science.
Peter Wilson,
I have not mentioned Phil Jones at all, you appear to be hallucinating. The reply was clearly unresponsive to the criticisms in the comment. I also note that McClean et all quote approvingly from one of the comment's reviewers:
While neglecting to quote the rest of the reviewers comments, i.e.:
Curious, eh?
Frank O'Dwyer
The problem with your defence of all this is that the climategate emails clearly show the intrigue behind stacking the review panel with Team members. The comment you quote reads eerily like some of the comments in the emails, and it would be unsurprising if they had the same author.
The reply was not at all unresponsive to the criticisms, pointing out that they were misdirected because of a basic misunderstanding of the purpose of the statistical techniques used . Once again you comments indicate that you believe you are addressing people who have not had the opportunity to examine the documents in question. Your comment is simply untrue.