Wednesday
Feb092011
by 
Bishop Hill
Bishop Hill Steig response coming
Feb 9, 2011 Climate: Surface Eric Steig has indicated that he will be posting a response to O'Donnell later today.
I can hardly wait.
Feb 9, 2011 Climate: Surface
Reader Comments (110)
@DM OF WA
>Was it approriate for Steig to be a reviewer of a critical paper?
It is disgraceful not to have immediately said 'sorry - I can't review this - I would have a conflict of interest as I am the primary author of the work that this paper discusses'.
Would it be ok for the Bishop to review all reviews of the Hockey Stick Illusion, prior to their publication? Well - it might be ok for the Bishop(!), but it might tend to skew the types of review of the HSI which see the light of day. (Presumably anyone, warmer or sceptic, can see this point).
It is unusual that the O'Donnell paper should have been sent to Steig by the journal for review and immensely peculiar that Steig did not immediately decline to be its reviewer.
One would think that climategate would have indicated to the team the danger of evidence and peer review tampering - but old habits die hard.
Let us keep in mind that despite Steig being a reviewer the editor of the journal finally tired of him and asked a fourth reviewer who gave the paper the nod.
There isn't a defence for Steig, if he was reviewer A, and he was, he shouldn't have been, he should have declined the offer on the grounds that he had a vested interest. Steig's "review" was totally in line with what we've seen in the climategate emails, he was there to stop publication, I am assuming that he was helped in this by the rest of the team.
What is surprising is that none of this seems to matter to the Singhs, Bradshaw, Nurses etc. who are constantly wittering on about the purity of science. (Although I must say wheeling out a bloke who has survived HIV for 13 years to prove that non-belief in HIV causing AIDS was some sort of quackery wasn't the best course of action).
I wish people would stop harping on Eric being a reviewer. There is nothing wrong with that provided the conflict is disclosed to the author. In fact, readers here should remember that the exact opposite situation existed when SteveMc was asked to review the Annan paper. Perhaps Bishop could remind us whether Annan knew this was happening.
@TimG.
>"provided the conflict is disclosed to the author"
Perhaps you haven't been tracking all the details....in this case the conflict was pointedly hidden by the reviewer (in pretending that he was not aware of the paper in a public forum, for example). The conflict was deduced by the author of the paper, after multiple rounds of bullying and nit-picking that could only have come from one source.
"Steig's 'review' was totally in line with what we've seen in the climategate emails, he was there to stop publication, I am assuming that he was helped in this by the rest of the team." --geronimo
With 88 pages of comments from Reviewer A, one might suspect a team effort of some kind--a desperate effort? I just wonder: How long did it take the editor to tire of the game and line up the fourth reviewer? Or did he have the fourth reviewer in his pocket all along? And who won the pool at the journal with 88?
IMHO, the Editor should simply have given Steig the chance to respond in the Journal. However, putting him or any of his co-authors as "primary" reviewers seems very odd - it almost encourages the game of gotcha. Reviewers B, C and D all made comments that ostensibly improved the paper so it is not as if there was a shortage of folks with sufficient expertise.
For the record...after all the selfless, ceaseless, unstinting, untiring, po-faced, scientific efforts of Eric Steig, the idea of exercising some restraint when preventing papers reaching the open literature might be dawning.
Either that - or Eric Steig is going for the Phil Jones defence: "It was the wicked sceptics that drove me to hide-the-decline, delete the email, scupper the paper, etc." Right, Eric?
Additionally, there may be a hint of climatological prediction in this comment (or is it merely a threat to future generations?). Anyway, I thought I would record it here before Gavin deletes it.
"I hope Eric isn’t serious about refusing to review a paper submitted by this crew (#42). I think one of their goals with their sort of public behavior is to scare off legitimate reviewers, which isn’t good for the peer review process.
[Response: True, but of course if it gets around that no one legitimate will review their work, then it won't get taken seriously at all. But that's not my goal. I simply don't want anything to do with these people. In fact, I may not bother with a rebuttal to Journal of Climate, because in a couple years temperatures in West Antarctica will probably have reached such an extreme that none of our 'reconstructions' will matter. In fact that may have already happened -- see e.g. Record warming in the South Pacific and western Antarctica associated with the strong central‐Pacific El Niño in 2009–10. --eric]
Comment by MarkB — 10 Feb 2011 @ 12:29 AM"
Well, the response from Steig is here: http://www.realclimate.org/index.php/archives/2011/02/odonnellgate/
Beware of cornered animals.
No i'm not saying Stieg's an animal as this would be rude, he's more of a muppet.
As a scientist who frequently writes and referees papers (in a different area), I too would like to say that it seems perfectly legitimate to ask authors of a paper criticized by a manuscript to be referees of that manuscript. I've experienced this situation myself. In fact, it is good practice as those people will be among the most well informed on the topic under consideration. However, the editor must keep the situation under tight control. (a) Other, disinterested people must be used as referees. Finding disinterested referees in climate science does just about seem feasible - the other three referees have done a mostly fair job on the O'Donnell paper, glancing at their reviews. (b) Negative comments from the interested referees must be discounted if they are arguments from authority rather than statements of fact. This is really difficult for the editor to do, particularly in an environment where the there is a much higher perceived legitimacy in the field for the referees compared to the authors. My experience in such cases is that, much as happened here, things sort of work out as they should, but only after much haggling. I would say that the editor has been just about fair on this one.
The lasting outcome of all this is that a paper that implied near-catastrophic evolution of temperatures in the Antarctic got published in Nature, put on the front cover, and bounced around newspapers and TV, whereas the paper showing that this previous result is almost entirely wrong did get published, but only in a much less prominent journal and without all the PR in the media.
thanks j. good input. but you've read the reviews (in part at least) and i assume you've read the publiic criticism of o'donells methodology (by steig). was steig being duplicitous?
I think the issue of whether Steig suggested changing the method and then subsequently criticises the method which, as reviewer, he preferred may be beside the point. Steig may have forgotten/overlooked etc in his excitement to refute O'Donnell et al. (I am trying to be polite...)
Although somewhat overlooked, RyanO makes a very important point very clearly at the start of his rebuttal to Steig. I put that point like this: Steig in S09 claims to be making a "reconstruction" of Antarctic temperatures and of course this makes the cover of Time and everyone shouts "OMG - Its worse than we thought". O'Donnell et al does not make this claim and RyanO is very clear about it: they are not proposing their result is a "reconstruction" of temperatures they are testing the validity of the method used in S09. Steig makes the mistake on RC of trying to assert that his reconstruction is better by comparing to some measurements.
Note that philosophically there are very strong parallels here between Mann et al (MBH98), McKintyre/McKitrick (MM) and the Hockey Stick. Mann claimed a "skillful reconstruction" of past temperatures. MM showed the method was statistically flawed. MM made no attempt at reconstruction, they simply criticised the method/algorithm, showed how the result changed if their method was followed and that little or no confidence could be placed on the MBH98 “Hockey Stick” result.
In both cases Steig and Mann miss the point of the papers (probably willfully). Steig and Mann claim their "reconstructions" are “skillful”. What MM and now O'Donnell et al are asking is whether given this data, method and uncertainties is any reconstruction likely to be reliable?
The more critical point from RyanO, and very well illustrated by the maps showing the response to sensitivity tests, are that the S09 model is sensitive to data changes (tests if you like) and the consequence of these sensitivity tests is to produce absurd results. Eg introduce slight simulated cooling on the top left of the plot on the peninsula actually results in the "reconstruction" showing substantial cooling at the South Pole. The model of O'Donnell et al is more robust to these simple tests and yet even then they do not claim to be able to "reconstruct" temperature trend maps for Antarctica.
To me the message that I take away from all this is that Steig S09 is like the Hockey Stick: it is a piece of propaganda, unsupported by the data, made to look like science by dressing it up in a highly complex statistical model. It has made the cover of Time magazine, so its purpose is served. No matter how useless the reconstruction is, like the hockey stick it will become the image that is remembered. Even now, on Horizon, years after it has been dropped by the IPCC, the Hockey Stick still gets shown.
All of these are Poster Children. Although I have neither the statistical skill or knowledge of O'Donnell, McKintyre etc (or possibly even Mann or Steig), I do have enough related scientific training and experience to see very clearly when experts like the former dissect the problem for me, that really all the Team is doing is cherry picking the parameters that make a "reconstruction" look as dramatic as possible. And by making it very complex and hiding the data, algorithms etc it takes so long to unpick it all that the propaganda damage is already done. Ignoring stuff in blogs and then shouting it must be peer reviewed in Journals is another delaying tactic. And then of course voluminuous review comments also delays publication. By all these means the dramatic image is left unchallenged for longer. That cover for Time magazine was probably primed and ready to go months before S09 was published.
And then when the result is challenged by the likes of O’Donnell et al and MM, enquiring minds, whether with or without science training, can see a result like MBH98 and now S09 are not valid as reconstructions. But the Team quickly moves on and the MSM blithely ignore it all. After all, I can't see the front page of The Independent tomorrow begin "False Alarm - West Antarctica Temperature Cannot Currently Be Reconstructed".
I too have been involved in peer review throughout my career but I fundamentally disagree with "J". I would never expect the authors of a paper being criticised to be on the peer review panel. The peer review needs to be as objective as possible and this simply cannot happen if one of the reviewers is biased by being the author of the other paper.
When this situation has arisen, the editors I have worked for have undertaken the review process and then asked the authors of the other paper for their comments and have alerted the authors of the paper under review that that is being done.
As far as I am concerned what happned with the O'Donnell paper was completely unethical, the editor should not have asked Steig to be a reviewer and Steig should not have accepted.
In reply to Mark, I haven't read all the reviews (88 pages!) but I've got a good flavour of them. Steig was being defensive and obstructive. Maybe he believed he was being objective, but it doesn't appear that way to me. Was he disingenuous in trying not to confirm he was a referee? Well, you're not encouraged to out yourself as a referee. So not doing so is hardly a major case of duplicity. The issue of encouraging the authors to use one method, and then subsequently criticising that method proceeds in my view more from a general worldview whereby he thinks he is right and they are wrong than from outright duplicity. It is hard to remain consistent in life - he probably had forgotten that he'd recommended that.
Arthur, it may well be that practices for appointing referees are different from one field to another. I said what happened in my field (a mainstream science). I've published > 100 papers and refereed many more - and in my field, this sort of thing would happen very often. The approach you describe is not, in terms of the outcome, fundamentally different from the one I describe, providing as I said that the editor is scrupulous in discounting what the interested referee says if he/she feels it is biased. The evidence in this case seems to be that the editor sailed pretty close to the wind in doing that discounting - but I would say that in this case, as the paper got out, without being destructively mangled, we should just about give credit to the system for having worked. That needn't stop us thinking that the process was unfair and the outcome in terms of Time cover pages etc. being inapproriate.
TS - great summary - I too was thinking that S09 has got parallels with the hockey stick and how it is then portrayed by the keen-to-lap-it-up media. What it amounts to is that they are playing an essentially political game, in which they use their statistical modelling and reconstructions as a means to produce the right result (warming) for the press releases, safe in the knowledge that very few journalists capable of anything other than superficial scrutiny of the actual science (even if they were inclined to ask questions about the quality of the data and integrity of the methods). The so called science is almost incidental.
I also get the feeling that Mann had a significant input into the 88 pages submitted by Reviewer A/Steig. The chances are that this was a team effort, and I bet the email trail would be very interesting. Is there still no comment from Revkin, Harrabin, or Pearce on the flawed methodology in Steig's paper? What is it about the Bishop's summary at http://www.bishop-hill.net/blog/2011/2/8/steigs-method-massacred.html that they cannot understand? The silence is deafening.
"The chances are that this was a team effort, and I bet the email trail would be very interesting"
Agreed. But good luck with that. The team will have switched to non-traceable, non-disclosable Google mail or other private accounts to discuss team business.
Their private correspondence is untouchable now.
Steig admits, "I am not a statistician" but reviewed a paper on statistics.
Does that not reveal the truth about how flawed the peer review process has become, non experts casting judgment on those with expertise.
Doesn't it also indicate that non-experts who utilise unfamiliar methodlogies they don't understand undermine science.
O'Donnell's accusation stands that it was more due to luck that scientists like Steig can ever come up with the correct answer.
j: thanks. so maybe eric is guilty of the lesser charge of inadvertant self contradiction. but he does seem to be willfully misinterpreting a methodological critique as an attempted reconstruction. something we are all too familiar with re mbh. is that a fair comment of his post publication (of ryanO et al.) blogging?
It's worth noting that from that RealClimate thread it appears that Steig disputes the facts of the matter - he claims that his suggestion for a change in methodology was *not* acted on, and that subsequent criticism are based on that failure to change. Wouldn't it be delicious if he forgot which side he was pushing: nice proof that he'll disagree with his own words if you put them in the mouths of skeptics...
RE: ZT
==============
@DM OF WA
>Was it approriate for Steig to be a reviewer of a critical paper?
It is disgraceful not to have immediately said 'sorry - I can't review this - I would have a conflict of interest as I am the primary author of the work that this paper discusses'.
==============
Clearly the journal editor knew that Steig was the author of Steig et al. As a reviewer and reviewee myself, my experience is that it is very unusual to have as one of the primary reviewers of a paper the lead author of the paper critiqued. The usual practice is to wait for final draft and acceptance and then send a copy to the other authors to allow a right of reply but not a right to change the paper unless a clear mistake (not opinion difference) is identified.
Given the contentious nature of the likely argument I am very surprised the journal editor invited Steig to be a reviewer. A good question to ask is why did the journal editor ask Steig to review the paper?
I am very surprised Steig agreed to be a reviewer - it was a huge error of judgement on his part because it now looks so bad. Further to this, Steig on RC has stated he is not a statistician so why is he reviewing what is essentially a statistics argument? If I were the editor I would have looked for a heavyweight statistician to be at least one of the reviewers, preferably one well removed from the content. That reviewer would then almost certainly have remained anonymous.
Regarding Reviewer anonymity etc, for journals where I act as reviewer or have been reviewed, the reviewers get a box to tick asking if they want their identity made known to the authors. I usually tick yes, but many tick no. The anonymity issue and the review comments being kept confidential is unlikely to be binding as I am fairly sure in the peer review system that a contract has not been entered into and no sanctions can be applied.
I think the lesson from all of this is that the concerns about a distorted peer review process that were highlighted by ClimateGate have been clearly shown to be true in RealClimateGate (or is it Steigate?)
Seems to always be the way with climate in the MSM
Doesn't Steig's role as a non-expert reviewer of a statistical paper put him in the same category as Simon Singh, being an interpreter of interpretations.
The journal should never have asked that Steig be a reviewer, that is a strange request. Steig in response should have said no to being a reviewer.
In summary, it transpires Steig was playing Lotto with climate science. It is by mere chance that he could have arrived at a correct answer, any correct answer. That is damning.
An important point, one that Steve McIntyre continually highlights.
Climate scientists should not use non-validated statistical methodologies in their studies. It is one of the issues that has done severe damage to the personal credibility of climate scientists like Mann, Hansen, Schmidt, Jones, etc and now Steig.
Coming up with a the latest statistical wheeze to prove to everyone of Global Warming simply won't wash anymore.
Steig's rejoinder names some names at the heart of his corner on the science side (has Hansen now been airbrushed out of the picture?). But who is at the heart of the PR agency which founded RC and which, for all I know, is still the funder of it. We had a fine display of PR skills with the hockey stick - great penetration of the IPCC, and the media. Now discarded, spent. The South Pole Sizzle was also a success, albeit not on the same scale - though front covers of Time and Nature are not to be scoffed at. The spin now seems to be ad hom against O'Donnell, with the wounded righteous playing the innocent and well-intentioned card. That is for the moment. Something more substantial will have to be found, or the South Pole will follow The Stick into the memory hole, along with tropospheric hotspots, more intense hurricanes, disappearing snow in the UK, bio-fuels as a good thing, carbon-trading as a good thing, blowing up children as a good thing, and many more, too numerous to mention here. What was that bittersweet joke from the old USSR, 'only the future is certain, the past keeps changing all the time'? That's the future of doom and gloom, and anything in the past that might intrude a bit of doubt, or even cheerfulness, about it, has it to go. I imagine that even as we opine on Steig vs O'Donnell, PR experts are running things up flagpoles, flipcharts and walls, in whatever is the modern equivalent of the smoke-filled room. Interesting times indeed, but very wasteful ones.
This is exactly the same as happened to His Graces book on the Hockey Stick Illusion.
The hockey stick was used by the great and the good to promote the CAGW theme.
The MSM hailed it and lauded Al Gore, and all concerned.
It was subsequently proved to be invalid and scientifically incorrect.
The MSM didn't even notice, nether did the green lobbyists, or politicians as they stand to make money from scareing people.
This is exactly what will happen this time with Steig et al.
JS
The moderation at RealClimate highlights that it is dealing in dogma - only the righteous have right to comment or respond.
The big problem with the Steig paper is that it came up with a headline and not any evidence of Antarctic warming.
O'Donnell's paper showed that this headline was an artifact of the methodlogy employed, an artifact that Steig tried to defend even though he was forced to concede he didn't understand the methodology or that it was even a valid methodlogy.
To summarise;
Has Antarctica been cooling over the last 30 years? Yes, the actual data is in?
Has Antarctica been warming over the last 50 years? We don't know, and will probably never know!
Finally the conclusion: People like Eric Steig, Mike Mann, Ben Santer, Phil Jones, Stephen Schneider, Andrew Weaver, Kevin Trenberth, Keith Briffa, Gavin Schmidt, Darrell Kaufmann are simply headline writers.
Thank you, ZT (Feb 10, 2011 at 7:29 AM), for posting eric's response to one of the RC posts - saved me from going over there again ...
So let's look at this in depth:
[Response: True, but of course if it gets around that no one legitimate will review their work, then it won't get taken seriously at all. But that's not my goal. Preventive protesting by Eric here, to cloak with human kindness the true goal of The Team: preventing the publication of of sceptic papers, especially if Steve Mc, RyanO et al have anything to do with them.
Now that's what I call 'Team 'science'' - which has nothing to do any longer with scientific debate.
Well done, Eric!!!
I simply don't want anything to do with these people. In fact, I may not bother with a rebuttal to Journal of Climate, because in a couple years temperatures in West Antarctica will probably have reached such an extreme that none of our 'reconstructions' will matter. In fact that may have already happened -- see e.g. Record warming in the South Pacific and western Antarctica associated with the strong central‐Pacific El Niño in 2009–10. --eric
Comment by MarkB — 10 Feb 2011 @ 12:29 AM" (My emphasis)
Eric and The Team are obviously addicted to everything warming. Overlooking the strong La Nina in 2010-2011, leading to cooling, is simply astonishing. It is quite clear that The Team is not capable of scientific debate, even on blogs, because they are utterly, probably wilfully, blind to anything happening on the planet which factually contradicts their warming meme.
I think further engagement with The Team is simply a waste of time. There are far more interesting scientific papers and findings coming out all the time, which merit our attention.
Having read Mac's comment I thought about measurements and real data as opposed to "reconstructions" like Steig 09. Over the last 20 years or so does anybody know if Jones, Briffa, Trenberth, Hansen etc (not Mann because he was still studying) have ever made formal proposals for a better surface temperature measuring system (not satellites), or have they ever proposed a new experiment which involves data collection?
I'll give you an example. Around 1990/91 Jones published papers suggesting that the UHI effect was unimportant. He used already available data that was, to be polite, far from ideal for the purpose and involved lots of assumptions about rural vs urban stations, station moves, data collection in different countries etc. Yet it seems to me that UHI is something that can be studied by constructing a careful, properly designed experiment. Deploy many sensors in a specific design and carefully collect data for 5 - 10 (or even 20 years) under controlled conditions. Then carefully analyse the experiment and publish the findings.
But they never seem to do this. Considering how "far sighted" they are in advocating the AGW hypothesis they seem pretty "short sighted" in terms of designing experiements and collecting new, real data intended specifically for addressing some of the issues.
@ ThinkingScientist, Feb 10, 2011 at 9:42 AM
Excellent post, wish I could have said this as clearly and as well as you did.
Dear Bishop, in view of ThinkingScientist's expose - and since you obviously haven't got anything else to do (Achtung: irony!), wouldn't it be brilliant if you could use this whole saga for another book?
After all, even RyanO said on CA that your explanation of what they did was far better than his attempt ...
Thinking-
j makes excellent point, but in the end, I would have to agree with your analysis. Allowing someone to be reviewer on a paper, whose own product is critiqued by the paper is ok, as long as her/her opinion is sought initially, but to carry on with three rounds of reviewing, with significant changes made to the paper due to that very reviewer's suggestions? Something is broken here.
Why did Steig get involved to this extent?
And now, he even offers that the authors should have gone to a different journal if they felt they were being "bullied".
They would have, if they had known that Reviewer A was Eric Steig!!
But Steig could not resist the temptation of fiddling with, and micromanaging a paper criticizing him in his area of work, from people who had openly offered the paper's ideas on blogs.
Blind because the grant money will slow to a trickle in flow to scientists crowing about fractional changes in temperature.
Thinking scientist,
You have noted a small part of a much bigger problem with climate science -- very little is done with any intelligence or thought. Not only are the thermometers not installed properly, but no one ever thought to bother to check. The quality control for the databases is non-existent, the instances of embarassing screwups are many, yet nothing seems to be done to fix the problems. The software is a disaster, but no one asks for competent help. The statistics are routinely screwed up, but suggestions that they get expert help are pointedly refused. Science is supposed to be "self-correcting", but no one ever checks, audits or replicates anyone else's work and the lack of transparency makes it impossible in any event.
The really interesting question about Mann's disaster, Steig's mess, and Rahmstorf's statistical nightmare is not the stink that was raised by outsiders long after they were published. The interesting question is how these disasters were published, embraced, and broadcast to the world in the first place. When obvious garbage is paraded as quality science on a repeated basis, it speaks volumes about the quality of the practitioners of the garbage science.
There are so many issues surrounding the controversy over Steig's paper and the critique of it by O'Donnell.
Cherry picking data, adjusting data, deleting data, splicing data, mining for data and inventing data should raise the hackles of the general science community.
The shameful acceptence of dogma within climate science when other scientists over the ages have given blood to eradicate it should be causing alarm bells to go off within the general science community.
The double thinking employed by advocate scientists in the climate change debate should be a real worry to the general science community.
The silencing and marginalisation of critics of current climate science to the point that intellectual freedom is now badly compromised should be of concern to the scientific bodies.
The blatant corruption of the peer review process within climate science should be urgently addressed by the journals.
The one word that sums up the current state of climate science is "dodgy".
We have dodgy data, dodgy methodologies, dodgy conclusions, dodgy peer review, dodgy papers that all lead to dodgy science. The reason for that is obvious - dodgy scientists.
Mark, thanks. I'm not trying to argue away the fact that what has happened here is dysfunctional at some level. But I do think that charges of corruption and outright malpractice in the way the manuscript was handled by the referee and the editor are over the top. Whether people think this should happen or not, in many fields, editors do consult referees who are interested parties. I note that ThinkingScientist, like Arthur, suggests this wouldn't happen in his or her field. TS writes that comments by the interested scientist might be sought at some stage, but that these comments would not lead to changes in the paper "unless a clear mistake (not opinion difference) is identified". That's essentially what happens also if the editor asks that person to referee the paper but applies good judgment in sifting through what they write in their referee report. As I indicated previously, I think that this is just about what happened here, though the whole process gave more weight to referee A than might have been appropriate.
TS also says that "If I were the editor I would have looked for a heavyweight statistician to be at least one of the reviewers, [...]". Looking at the reviews, referee C at least looks to be someone who knows a good amount about statistics. And the referees collectively do need to know about the datasets under consideration, not just about statistics. So someone who is statistically not heavyweight but knows about the dataset is a reasonable person to be a referee. The editor has not completely messed up in choosing referees.
In reply to your question about the point Ryan O makes at the start of his post about Steig, namely that his J Clim paper is about statistical analysis rather than about proposing a newer better reconstruction: I agree, shades of MBH in the way Steig has responded to that. I'll leave others to discuss the finer points of that, though.
Arthur Dent
I completely agree with Arthur not only in the above, but the rest of his posting about what should have happened. Steig should have been given a copy of the paper during the review openly by the editor and be permitted to refute it. Only a very serious error should have had an impact beyond passing the refutation on to the reviewers for their review. That is both papers should be reviewed and then published together.
I really find J's position unacceptable. There was clearly conflict of interest in what was done.
About whether it is normal, or acceptable, for one of the panel of reviewers to be the author of the paper that is being rebutted:
In a not to a comment at RC, Eric (presumably Steig himself) says:
"Response: It would be entirely normal for an editor to send a paper criticizing someone's work to that person, for their opinion. You just wouldn't want that opinion to be the deciding factor, which is why normally you'd get several other reviewers; this is presumably the case here. ....
Call me cynical, but there is a difference between "sending a paper criticizing someone's work to that person, for their opinion", and appointing them as a confidential reviewer ? If so, why is Eric not defending his appointment as confidential reviewer ? Could it be that it is indefensible ?
Don P, we're disagreeing on niceties here. I'm not saying nothing untoward happened - I'm saying that most people working in my field with experience of publishing would not find it crazy to ask Steig to referee. I went on to imply that the editor should have discounted A much quicker. The fact that he did so in the end - the third review from Referee A ended up being essentially ignored by the editor, I would say - means that the editor has not either violated any well established rules about what editors should do to avoid conflicts of interest. That doesn't mean he wasn't unfair. I'll shut up now - I've made my point enough.
Having not worked in, or alongside, a profession where peer review operates I find this discussion fascinating - so thanks to all who are, and have contributed their views and experiences.
But i, too, find that reviewing a rebuttal of one's own work a little strange, to say the least.
At 12:31 PM above, ThinkingScientist asks, then restates the problem”
“...it seems to me that UHI is something that can be studied by constructing a careful, properly designed experiment. Deploy many sensors in a specific design and carefully collect data for 5 - 10 (or even 20 years) under controlled conditions. Then carefully analyse the experiment and publish the findings.”
But the research has already been conducted. UHI/LUC are first order temperature forcings, no less important for scientists to consider than CO2. The point has been made by Roger A. Peilke, Sr., many times and places.
One place was in an NAS report, 2005:
"Regional variations in radiative forcing may have important regional and global climatic implications that are not resolved by the concept of global mean radiative forcing. Tropospheric aerosols and landscape changes have particularly heterogeneous forcings......Regional diabatic heating can also cause atmospheric teleconnections that influence regional climate thousands of kilometers away from the point of forcing. "
He’s made the point on his blog. It is a point to which some, like Anthony Watts, have responded to with more research (see his surfacestatons.org site)
He, together with William Cotton, even wrote a textbook on the subject, “Human Impact on weather and Climate” (2007, 2/e), where in they come to the considered estimation of AGW data overwhelmed by natural variations.
The likes of Steven Schneider are simply not up to date with the scientific literature on the matter, he explains in the interview linked above. Thus, Pielke politely excoriates him: "These findings should not be cavalierly dismissed by Dr. Schneider as 'nonsense'."
fran: i think what j is saying is it doesnt really matter if seig was reviewer or had some pre-publication right of reply. it's the editor that decides on publication and he did the right thing EVENTUALLY. as a layman, ryanO's statement that he doesn't want the editor criticized makes pre-publication discussion alone a bit moot. although steig post-publication comments seem disengenuous to me. part of his recent rebutal is he hasn't understood the scientific/statistical arguments of ryanO yet. really? how many pages?
Orson,
Thanks for your thoughts. I know that others have done some work on this, but why have I never heard Jones, Briffa et al in the context of new data, experiment design etc? They only ever work with second hand data plagued with problems. Where is the advocacy from them as scientists to encourage Universities and government organisations (eg Met Office) to put in place programs to acquire better data? When the divergence problem was observed between tree ring and temperatures why did they not propose a new program of data acquisition to explore the problem? Instead they ignored it.
All the work they seem to do is on other people's (sometimes old) secondhand data.
j
Thanks for your comment above.
I would agree with you that there is no malfeasance or ill-will on the part of Steig here.
But the charges of duplicitiousness are hard to avoid, and arise purely from two facets of this incident
1) the position of conflict Steig placed himself in
2) the outcome of this conflicted postion, vis a vis, his Realclimate post supporting his own paper's conclusions
The conflict from the above has to be grasped, and can perhaps be too subtle in this current hyperpolarized atmosphere, for those who support Steig. The visceral reaction to the percieved injustice caused the author Ryan to come out strongly, and his strong reaction further clouds issues.
It is immaterial to even consider whether or not, Steig purposefully suggested/preferred a new statistical method, to understand the conflict of interest inherent in the situation.
A point about Steig as reviewer -- as to the authors, he was presented as anonymous. That is BS procedure. If a journal wants to solicit the first author's opinions when they are reviewing an article that critical of something previously published, that seems reasonable provided that the authors of the critical paper are told whose criticism they are responding to. That allows them to point out to the editor of the journal those criticisms that are merely unwarranted self-defense. That's not what happened here.
Steig was allowed to serve as an anonymous reviewer, was allowed to throw up all kinds of garbage as if he were a disinterested reviewer, and was allowed to force the authors to jump through a lot of hoops in oreder to get published. Granted, his reviews were so over the top that the authors came to suspect what was going on, but it was still speculation until Steig confirmed it later. The skeptics had to deal with all his BS as if it were legitimate criticism from a disinterested reviewer only interested in bettering the science.
stan: good point. i'd forgotten about anonimity issues. but again, its up to the editor and i heard ryanO said he doesn't want him critisized. still seems dodgy, dunnit?
mark
its up to the editor and i heard ryanO said he doesn't want him critisized. still seems dodgy, dunnit?
Ryan is a very wise, honorable and diplomatic man. He works in a small society.
I had another thought about this question of peer review and the way Steig's picture of "reconstructed" Antarctic warming trends gets on the front cover of Time.
I think a clear conclusion from Climategate and now this incident with Steig and RyanO is that the peer review process has been distorted, as I said in a post above. However, I think the Team has done something else as well. They have convinced governments and the MSM that the moment something is published in a peer reviewed journal it becomes accepted science. So S09 is published, the critical picture makes the front cover of Time and influences public policy. This is absurd. In the case of Steig et al we have 6 authors: Steig, Schneider, Rutherford, Mann, Comiso & Shindell. And perhaps the paper was reviewed by 3 referees and the editor. That makes 10 people who say this is ok to publish. That is not accepted science, it simply means the paper is of a sufficent standard to be published in a journal. What O'Donnell et all shows is that there are many ways to "reconstruct" temperatures and to assume Steig et al (a) the correct way and (b) even reliable is crazy. Any old idiot can make a map of data and write up a justification of it. I see it every day in my work.
For it to become science, after publication it needs other independent authors to confirm the findings. Reagrding independence Wegman made a great point about this when asked by Congress to review the MBH98 and MM affair (The Hockey Stick), pointing out all the key papers were authored or co-authored by a very small group of people. Out of interest, 3 of the above co-authors are on the Wegman relationship diagram.
What the Team has done is two-fold. Firstly they have distorted the peer review process, by being obstructive to papers they don't agree with and reviewing each others to get them through more easily. I have an anecdote on this at the end of this post.
The second thing they have done is convince the ordinary, mainstream people that a peer reviewed paper is equivalent to accepted science. Peer review is completely inadequate for this - Steig et al was probably looked at by 10 people, including co-authors. You cannot base public policy on that.
Regarding my anecdote on the distortion of the peer review process, I have first hand evidence of this - a former employee of mine joined my company after working in an academic post. With a co-author they had been trying to get a paper published on which s/he had worked in their academic post before joining my company. The paper questioned whether the reconstruction of a climate ahcnge over an was in fact an artefact of the data collection pattern changing over time. They were having a repeated row with the reviewers and editor about publication (I have copies of the reviews and letters). I reviewed the paper privately for him/her in the light of the reviewer's comments (the paper was on geostatistics, so I in my field of expertise). The paper was, in my opinion, unexciting but worthy of publication. As far as I know it was never accepted.
Imagine my surprise some years later to find that same paper and an earlier one by the same author mentioned in the Climategate emails. I found the reference quite by accident the other day - several of the Team making disparaging remarks about this paper and the authors.
I've added Ryan's material to my Arctic page.
It just struck me, and has been touched upon several times before:
It seems quite conceivable to me that designated reviewer A (Eric Steig) indeed wrote and sent his review comments by himself, but that he communicated with other Team-members while doing so, possibly asking for their input and even help.
Both langage and delay-tactics and the multiple and inconsistent requests to address issues and moving goal posts at least suggest that as a possibility.
What I mean to say is that it is fully possible that Steig as reviewer both 'recommenden to ues iridge instead' and at RC cirticized the very same practice. But unaware of the contradiction (duplicity?). It has been suggested that this might have been a mistake, a memoryslip etc, and not intentional.
It might also have been the result of him relaying other persons opinions through his review comments, points that he did not completely understand himselft.
Again, this is his own responsibility, and he can blame noone else. But it also makes more sense.
Reading though the defence speech at RC, Steig does not strike me as to versed in statistics or at regarding methodology. In fact, my impression is that he still doesnät comprehend what O'Donnell et al is really about. He keeps talking about different points, about the warming he believes is to come, about O'Donnell not reafuting what he thinks was his 'bigger picture message' etc. And of course complaining about people not being nice enough to him.
I have read though his (their?) review comments and it is very hard not to get the impression that the main effort was to thwart the paper, and if that failed delay it as long as possible, and finally to minimze the harm it did to 'the bigger picture'. Possibly even prepare it for an 'easy rebuttal', at least in the public eye.
It was obvious from the outset that this was a very hostile review, and obviously Steig is still more interested in toeing the party- (Team-) line, than conceding that O'Donnel et al have som valid points.
On the other hand, he also strikes me as quite human, with many responses more out of petulance and pique, as from somebody a bit out of balance, than adressing the criticism and defending his version of 'facts'. Obviously he doesn't have Gavin's routine, and I think it shows to more than thos who already knew what is RCs main purpose ...
And that is good!
As to Eric Steig being chosen as a Reviewer:
I would actually expect that to happen, either him or somebody in his vicinity (eg former co-author). I don't think this is a bad practice. Rather that it would be a bit naïve if the submitting authors expected to go free from scutiny from that side/camp.
But I would also expect the editor to be aware of this and treat it fairly. As Broccoli did here, finally. The criticism there is mainly that it seemed to take so long for him to get a fourth reviewer, and that he should have realiszed what's going on much earlier. Well, maybe he did!? Actually I would be very surprised if he were completely unaware both of the Team, its reputation, and earlier controversies. And also would I expect him to have been pressured (or at least atempted to influence him) through other channels than the formal review process.
And I agree with those who said that he got it right in the end, and should be commended for it.
(And Steig's pouting comment at RC, that it was a 'lousy paper' and more of the same, might in the end have the opposite effect, as it well should among decent and competent scientists)
Based on my dealings with him, I don't think Kaufmann should be included as part of the team. He is on Dr Steig's list because of the Arctic warming paper that McIntyre criticized. However, both the ClimateGate e-mails and his subsequent behavior does not place him as a member of the team. He issued the correction for upside-down Tiljander. In the dealings with ClimateAudit, it was Steve McIntyre who refused to call Dr Kaufmann until the issue was resolved of whether ClimateAudit engaged in vicious commentary as Dr Kaufmann had written to Steve in a personal e-mail. I thought Steve behaved poorly there.