Buy

Books
Click images for more details

Support

 

Twitter
Recent comments
Recent posts
Currently discussing
Links

A few sites I've stumbled across recently....

Powered by Squarespace
« From the archives... | Main | The timing of Acton's eleven »
Thursday
Oct282010

Pamela Nash and peer review

Steve M has already wondered how the Russell panel came up with the three instances of (alleged) subversion of the peer review process that they decided to investigate, a question that was put to the panel by Pamela Nash.

Muir Russell spoke of a footnote in my report that says it was unclear, in one instance, what the particular allegation was. The footnote concerned relates to the McIntyre & McKitrick paper in GRL. Anyone who has read the Hockey Stick Illusion will know the story, but briefly, climate scientists are seen discussing getting rid of the editor in charge of McIntyre's paper. The only slight confusion is over what happened to Saiers - it has been claimed by some that Jones et al were innocent of the charge because Saiers remained at the journal for some time after these events. As I pointed out in the GWPF report, the allegation is that these threats seem to have led to Saiers being removed as editor responsible for the M&M paper, not that he was removed completely from the journal. The central claim is the same either way - namely that the journal was threatened.

In other words the confusion was slight, and should not have led the Russell panel to avoid what was an allegation central to the whole Climategate story rather than the peripheral ones they actually "investigated". (Muir Russell's claim that these were the instances everyone was getting worked up about is not correct.)

The immediate  investigation of this aspect of Climategate should have been simple and straightforward: it was simply a matter of finding out what the CRU scientists and their colleagues had said to the journals concerned. It is astonishing that at the end of four investigations in the UK alone, we still have no idea if Jones et al actually followed up on the ideas revealed in the Climategate emails, and issued threats to journals and their editors.

PrintView Printer Friendly Version

Reader Comments (26)

Didn't Acton say something along the lines of that the only thing he was interested in was preserving the good reputation (sic) of UEA. Thus the "independent" inquiries were set up to produce the required outcome - exonerating the CRU of misdeads. The inquiries were set up not to investigate in depth any of those issues that could show that there had been dirty deeds afoot. They were whitewashes with plenty of supporting flannel and misdirections, obfuscations and carefully planted lies. Sir Humphrey would have been proud of how well the monkeys performed.

So far.

Oct 28, 2010 at 8:02 AM | Unregistered CommenterPhillip Bratby

Any chance that someone could find Saiers, and ask?

Oct 28, 2010 at 8:20 AM | Unregistered Commentergolf charley

I've tried. He didn't reply to my email.

Oct 28, 2010 at 9:15 AM | Registered CommenterBishop Hill

A reminder of the quote from Tom Wigley in email 1106322460.txt:


Proving bad behavior here is very difficult. If you think that Saiers is in the greenhouse skeptics camp, then, if we can find documentary evidence of this, we could go through official AGU channels to get him ousted.

Elsewhere in the email conversation, Mann says "We can't afford to lose GRL" and "This guy Saiers has a prior connection w/ the University of Virginia Dept. of Environmental Sciences that causes me some unease". It is clear that Mann telephoned the editor in chief of GRL (Mackwell) to complain about GRL publishing M&M's paper.

Exactly what happened beyond what we can see in that email, I don't know, but the Russell Review should have looked into it. It is remarkable that they didn't, given that they have a section about 'Peer review and influencing editorial policy of scientific journals'.

Oct 28, 2010 at 9:28 AM | Unregistered CommenterPaulM

PB Exactly

These reviews were not independent. Neither did the review panels actually investigate the controversial science or the specific behaviour of scientists. These panels were coralled by UEA and CRU into choosing to investigate uncontroversial aspects. They were simply whitewashes.

The impact of these flawed reviews are clear to see. It has irrepably damaged CRU. It has seriously damaged UEA. It has damaged the reputations of Lord Oxburgh, Sir Muir Russell, the other panel members, the Royal Society and S&TC.

Oct 28, 2010 at 9:40 AM | Unregistered CommenterMac

Odd, isn't it: "In other words the confusion was slight" - and the highly competent scientists populating the inquiries were unable to clear this up? Like, ahem, by asking questions, perhaps even via e-mail?

Perhaps they didn't ask because they didn't want to sully their lofty minds by talking to an 'enemy', that is you, Bish!

Mind - I think this is another indication for the fact that all inquiries, from the start were only to do with getting UEA off the hook, whatever it took.

Oct 28, 2010 at 9:43 AM | Unregistered CommenterViv Evans

Mac
Oct 28, 2010 at 9:40 AM

"The impact of these flawed reviews are clear to see. It has irrepably damaged CRU. It has seriously damaged UEA. It has damaged the reputations of Lord Oxburgh, Sir Muir Russell, the other panel members, the Royal Society and S&TC."

In the huge group of international CAGW weasels, these dodgy institutions and ratbags are regarded as rather good chaps on the whole, totally on message, maybe a dash of economy with the truth now and then (they are human beings after all) but really, they do not give a monkey's, because they do not have to because there is no possibility of retribution. They "investigate" each other at the request of a fellow "independent" weasel and always find that they did nothing wrong.

Anyone outside weaselistan does not count, therefore why should the weasels care?

Oct 28, 2010 at 10:30 AM | Unregistered CommenterBrownedoff

The one thing that intrigues me is the questioning of Jones with regard the deletion of emails.

Sir Muir Russell has stated that he never asked Phil Jones whether he had deleted emails at the news conference when he presented The Independent Climate Change Email Review on 7th July. He repeated that at the S&TC meeting, this time quoting legal complications as the reason for not doing so.

However in the offical UEA response to the report Prof Edward Acton stated on the 7th July, "Sir Muir Russell's team concludes about the staff of CRU that "their rigour and honesty as scientists are not in doubt. Furthermore, they “did not find any evidence of behaviour that might undermine the conclusions of the IPCC assessments" and, the report states, there was “no evidence to substantiate” allegations of perversion of the peer review or editorial process. In summary, the report dismisses allegations that our scientists destroyed or distorted data, tried to pervert peer review and attempted to misuse the IPCC process."

Acton quoted Russell directly on the IPCC and peer-review processes, but didn't quote Russell directly on the destruction of data nor indicate that he as Vice-Chancellor had directly questioned Jones on the deletion of emails. It seems a strange omission in hindsight.

So when did Acton actually question Jones, and did Acton convey that information and the answer given by Jones to Russell, before, during or after the investigation?

Oct 28, 2010 at 11:43 AM | Unregistered CommenterMac

Mac 9:40am
I agree but Acton does not. At the end of the video 10:36:10 Acton reads a pre prepared statement saying the UK Govt endorses the work of CRU.

I wonder who wrote it.

Oct 28, 2010 at 12:59 PM | Unregistered Commentergolf charley

One thing that I was getting from the video was that everyone, including Muir Russell, was coming to the realisation that peer review is pointless but couldn’t actually bring themselves to come out and say it.

1) It’s not a rigorous assessment of the science (Phil ‘nobody asked me for the data or code’ Jones) and it's no measure of right or wrong.
2) It’s subjected to the biases of reviewers, even if not deliberate.
3) There’s no form of version management i.e. when a new paper supersedes or disproves and older paper, the older paper is rarely struck from the records and a note placed in it’s place to indicate it’s no longer valid. There certainly isn’t a big news frenzy like when it was introduced.
4) Similarly papers that refer to invalidated papers aren’t modified and/or withdrawn to reflect the new situation.
5) Papers seem to slip into obscurity when a new paper is published (often by the same author). At no point does a panel sit and say ‘yes, this paper better fits the situation’, discount the old.
6) To get a paper invalidated normally involves writing a new paper which is subjected to all the delays and biases the consensus team can apply. There’s still no guarantee that the new paper will be taken seriously. It is years between a loony idea and a correction (look how long it took the Lancet to withdraw the discredited MMR paper, they waited until the key doctor had been struck off).
7) There appears to be no form of paper indexing other than Google style searches. For every element of climate science there should be the equivalent of the ‘MWP of the Week’ found at CO2 Science. That way the IPCC shouldn’t have to dig about grey literature, they’d know exactly where to find papers on the Himalayan glaciers and if there were conflicting papers. An additional benefit, it would act as a guide for researchers looking for vacant data opportunities.
8) Once an overview of what research is needed, suitable grants can be offered and that would include research into sceptic option.
9) And finally the most obvious issue. All papers relevant to climate change, and paid for by public money must be made freely available. Any paper used for an IPCC report must also be free to view.

The committee picked up on several of those points. Peer review is a dinosaur from the fifteenth century, it really is time we moved into the twenty first.

Oct 28, 2010 at 1:35 PM | Unregistered CommenterTinyCO2

Roger, Re my stirring experience of jousting with Mann, Ehrlich, and Rahmstorf: What a scurrilous bunch. My sympathy to you and anyone else who has to deal with them. They're gravediggers of science. Nature will soon publish my riposte and, I think, a disclaimer of any ties to me by the Marshall Institute. Below, my further exchanges with the low-life trio. Best regards, Dan


http://rogerpielkejr.blogspot.com/

Oct 28, 2010 at 1:37 PM | Unregistered CommenterLDLAS

I haven’t looked for, and don’t intend to look for, my name in the CRU emails, but one of my colleagues did alert me to an email written by Wigley in which he suggested that, if I were a climate skeptic, then steps should be taken to get me “ousted.” Wigley’s suggestion stems, I believe, from the publication of a GRL paper (by McIntyre and McKitrick) that criticized certain elements of Michael Mann’s Hockey Stick paper. This paper caused a bit of a stir and because I oversaw the peer review of this paper, I assume that Wigley inferred (incorrectly) that I was a climate-change skeptic. I stepped down as GRL editor at the end of my three-year term, long after the excitement over the McIntyre and McKitrick paper had passed. My departure had nothing to do with attempts by Wigley or anyone else to have me sacked.
-James Saiers, responding to an email by Roger Pielke Jr

http://rogerpielkejr.blogspot.com/2009/11/grl-and-james-saiers.html

Simply put, he did not answer what RPJr what getting at.

Oct 28, 2010 at 2:12 PM | Unregistered CommenterShub

Tiny CO2? Tiny-amount-of-thought-before-posting, more like! You have latched on to a couple of points that fall into the arguable/tendentious category, but otherwise it's ill informed, ill considered, ill judged, ... Please do some proper research into how the scientific literature works - and think!

Oct 28, 2010 at 2:20 PM | Unregistered CommenterSquirrel Nutkin

A propos, this article goes some way to explaining what is going on the heads of warmists:
http://online.wsj.com/article/SB10001424052702304410504575560323807741154.html?

Oct 28, 2010 at 2:31 PM | Unregistered CommenterJustice4Rinka

TinyCO2

It would appear that science can be superceded but that an outdated or flawed paper can still be cited.

Phil Jones's 1990 paper is till being cited in research.

Oct 28, 2010 at 3:00 PM | Unregistered CommenterMac

"Sir Muir Russell's team concludes about the staff of CRU that "their rigour and honesty as scientists are not in doubt. Furthermore, they “did not find any evidence of behaviour that might undermine the conclusions of the IPCC assessments" and, the report states, there was “no evidence to substantiate” allegations of perversion of the peer review or editorial process. In summary, the report dismisses allegations that our scientists destroyed or distorted data, tried to pervert peer review and attempted to misuse the IPCC process."

As Muir Russell said 'its all a play on words' , they never asked so they never heard or saw evidence.

The 3 monkeys exactly.

Oct 28, 2010 at 3:04 PM | Unregistered CommenterJohnH

Steve McIntrye widens the debate over the deletion of emails.

http://climateaudit.org/2010/10/28/watch-the-pea-2/

Jones' response does not make it clear that it was Acton who actually posed the direct question over the deletion of emails (rather than offering the information), when he was questioned (Jones' statement is dated 26th July, three weeks after Russell review reported) and who else was questioned (no mention of Briffa, etc)

It would appear Jones in offering up this statement was only directly questioned about the deletion of emails after both so-called independent reviews had reported.

Oct 28, 2010 at 3:17 PM | Unregistered CommenterMac

Squirrel Nutkin: are you going to address TinyCO2's points or are you just going to leave us with the impression that you have no worthwhile points of your own, just gratuitous abuse?

The peer review of journals, in climate science at least, has shown itself to be worse than useless.
To be merely useless we simply need the underlying data for a published paper of any description to remain unexamined and unavailable, to be worse than useless, poor papers with one point of view are published, good papers with the opposite conclusion are rejected.
It would take some tortured logic to try to suggest that this hasn't happened, regardless of whatever anyone thinks is actually happening with the climate.

Oct 28, 2010 at 4:18 PM | Unregistered Commenterartwest

Mac,

The IPCC is the Intergovernmental Panel on Climate Change. The rules for its assessments have been decided by governments and are published. Jones, Overpeck, Solomon, and Manning - with the agreement of all the coordinating lead authors - decided, retrospectively, to change the deadline, by when papers had to be published or in press with preprints available to Expert Reviewers. They changed it from before the start of the last expert review of the drafts to two months after the end of the review. Manning claimed "many" of the 600 Expert Reviewers had asked for the change! I have trawled through all 11,000 comments and it is not true.

Wahl and Ammann 2007 failed the original and the revised deadline for other reasons, but was still cited to prevent the "hockey stick" from being dropped from AR4. The Reviewer for the US Government in his comment on the draft pointed out that Wahl and Ammann broke the rules in force during the review, and that the same rules stated unambiguously that all references to the paper had to be removed. Ross McKitrick pointed out that Wahl and Ammann broke a more fundamental rule - and the back channel emails with Eugene Wahl which led to the "delete all AR4 emails" saga were, in part, trying to address that.

Review Editors were added to what governments decreed to be an open and transparent process to ensure fair play. Review Editor, Sir Brian Hoskins, blocked my FoIA requests on this matter by the unsubtle method of saying he had nothing and forbidding his Information Officer to check. Review Editor, Myles Allen, is still refusing claiming that he was on paternal leave at the time!

Both the Met Office, for Review Editor John Mitchell, and UEA/CRU for Jones and Briffa used the Ministerial veto (FoIA section 36) to block my requests for the various emails on this matter. The Under Secretary of State for Defence said disclosure was against the public interest. On the first of this month the ICO persuaded the MOD to release one email which without knowing the context appears to say very little.

The deadline change was emailed to Briffa, all other Lead Authors and Expert Reviewers on 3 July 2006, by Manning. It invited Expert Reviewers to send in suggestions for papers which would improve the balance of the IPCC Report. However, just three days later, Overpeck emailed his Chapter 6 "hockey team" Lead Authors telling them in effect not to add any new citations unless they could take others out, because the chapter was 5 pages too long. This just adds to the evidence that IPCC AR4 WGI Chapter 6 is at least as flawed as the Glaciergate chapter in WGII.

After claiming that "many" Expert Reviewers had asked for the extra opportunity to comment on Chapter 6, just 5 did, including Steve McIntyre - though his suggestion of Wegman et al 2006 and of NRC 2006 was not passed to Briffa. Just one extra citation was added to Chapter 6, made up of half of one Reviewer's suggestion and half of another's - but hey this is climate science! Schneider Von Demling et al. and Schneider et al. share the citation - one is in the text and the other is in the references. But hey, this is climate science.

Funnily enough no Expert Reviewer suggested adding Wahl and Ammann 2007, but as it was all a charade who cared and since Solomon and Manning had no intention of letting the comments and responses get onto the Internet who would ever know. In one of the leaked emails, Jones confirmed that Wahl and Ammann is in AR4 only as a result of the deadline change.

So how did the ICCER deal with this serious accusation?

First they refused to publish the complaint. But then allowed Briffa and themselves to state what it was. Remember, the ICCER thought that natural justice required that the accused be heard but not the accusers. Are there any verbatim transcripts of interviews?

So how did they decide what the rules were? Did they cite the rules and procedures at the IPCC website, which our government had helped write had agreed to? Did they seek guidance from the IPCC or Department of Energy and Climate Change, who send the representatives to the IPCC sessions to vote for the UK? No - they asked the editor of the Lancet. They also asked the accused Review Editors who had also blocked my FoIA requests - but only by telephone!

Oct 28, 2010 at 4:30 PM | Unregistered CommenterDavid Holland

"It would take some tortured logic to try to suggest that this hasn't happened, regardless of whatever anyone thinks is actually happening with the climate."--artwest

Tortured logic? In AGW CloudCuckooland, that's our specialty!

Oct 28, 2010 at 6:41 PM | Unregistered Commenterjorgekafkazar

Re TinyCO2

Well said! There was recently another UN event held by the ITU. They have a far slicker and effective (albeit expensive) method of managing telecomms standards, draft and final and often with many dependencies between standards. Science journals should get online, adopt similar methods or go the way of the rest of the dead tree press.

Oct 28, 2010 at 7:16 PM | Unregistered CommenterAtomic Hairdryer

I’m sorry squirrel Nutkin but I’ve been reading about peer review for years now and mostly it’s stuff on what peer review doesn’t do. Perhaps you can enlighten us on its sterling qualities? I’d suggest the Wikipedia entry isn’t a bad summary of peer review but I fear that before anyone got to read it, the CO2 wiki police will have rewritten it in glowing terms. The comment by Richard Horton, editor of the British medical journal The Lancet is especially apt.

As Atomic Hairdryer suggests, there are much better ways of managing information. Climate scientists need to stop whining and get on with it.

Of course it’s uncomfortable being made to change your habits and spend lots of time and money doing stuff you think is pointless. Well join the club buddy, I feel that way about CO2 emissions.

Oct 28, 2010 at 8:01 PM | Unregistered CommenterTinyCO2

LDLAS

Sorry if OT, but thanks for the heads up. That response by Daniel Greenberg to the Michael Mann, Paul Ehrlich and Stefan Rahmstorf attack is something to see. I gather Greenberg isn't a sceptic, but he has written a pretty forthright reply.

http://rogerpielkejr.blogspot.com/2010/10/daniel-greenberg-meets-climate.html

Oct 28, 2010 at 8:46 PM | Unregistered CommenterSteve2

Steve2 --
[prefaced with usual apologies as this is O/T for this thread...YG, is there a better place for this and related comments?]
The Mann et al. letter smearing Greenberg is notable not so much as an ad hominem, we've seen lots of those, but because it's such a poor ad hominem. Greenberg's connection to the Marshall Institute is extremely tenuous, having participated once in a roundtable discussion which the Institute sponsored, and then published.

It seems apparent that the true intent is to serve notice that anyone who does not toe the party line, to any degree, is going to be smeared in some way, and not just by the usual attack blogs. And thereby attempts to intimidate those who are of insufficient stature or fortitude to withstand such attacks. [The Anderegg et al. "blacklist" paper is a similar effort.] Greenberg's review is insufficiently critical of Pielke Jnr; ergo he must be attacked. Pour encourager les autres, as Voltaire put it.

Oct 28, 2010 at 11:12 PM | Unregistered CommenterHaroldW

David Holland:

Steven Mosher raised several questions about Wahl and Ammann which Gavin Schmidt proceeded to provide contradictory and dismissive answers for.


I would gladly donate an excerpt from my book as a blog post and maybe even add some details.

I suggest the treatment of Amman’s paper by Briffa in Ch 06 of Ar4.

[Response: I'll make some points in response below, but this example is typical of the non-issues that are being brought up. The point of the an assessment is to get the science right and the Amman and Wahl paper was completely apropos in demonstrating that the impact of McIntyre's points amounted to nothing very much. All this brouhaha about how the specific paper was handled is completely irrelevant to that fact. - gavin]

Here is what the record establishes:

1. The paper in question was given special treatment by briffa, overpeck, and the journal editor in charge of it

[Response: Lots of papers get 'special treatment' because issues sometimes arise. Ho hum. - gavin]

2. The paper relied on another paper that was not published until after the IPCC report.

[Response: No. It cited another paper that, for various reasons, was held up. If you think that the IPCC report would have been substantially different if there hadn't been this cite, you are very much mistaken. - gavin]

3. Briffa consulted with Wahl outside of and in direct violation of IPCC processes that he was aware of.

[Response: The IPCC report was a two year plus process and there was no injunction about discussing it with other colleagues. This imagining of mysterious inviolable rules in a process which reinvents itself every time is just post-hoc whining. Sorry. - gavin]

4. Briffa utilized writings of Wahl without proper citation, with full knowledge that he was violating the procedures and was consciousness of the improper nature of the act.

[Response: Oh please. The comment was a correct statement. Are you suggesting that you'd prefer an incorrect response instead? The point here is to be correct. - gavin]

5. Jones suggested that the paper be backdated to avoid discovery.

[Response: Nonsense. You are completely wrong on this one. Jones' comment was a joke related to the fact that there is a typo on the Amman and Wahl paper which says the submission was "Received: 22 August 2000" which is clearly a misprint. - gavin]

6. Hollands FOIA request for Briffa’s, ammand,s and osborne’s correspondence about this paper and the communications that violated the open IPPC process, caused Jones to ask that people delete their emails.

[Response: It is very unclear that personal emails are FOIA-able (something I'm having a lot of fun in the US context for instance ;) ), but regardless Jones' request (as I've stated many times) was ill-advised. There is a huge difference though between FOI requests for data and for requests for personal communications between close friends, colleagues and co-authors that were written with absolutely no expectation that they could be made public. - gavin]

7. The ICO has reason to believe that Holland’s request for information was improperly handled.

Those are the facts. Now, I don’t think these facts have anything to do with the reliability of climate science. They go to the credibility of the people involved.

[Response: Sure... And it's nothing to do with casting aspersions over a report simply because you don't like what they concluded. - gavin]

Not the science, but the behavior of a small group of scientists who argueably perceived that they needed to bend administrative rules to preserve a certain un-settled scientific position. Their behavior doesnt make their position wrong, but neither is their behavior something that you can condone and expect to be trusted by a doubting public.

[Response: Nothing needed bending to preserve 'a position'. There simply isn't some huge mass of evidence that somehow contradicts the statements in the IPCC report related to multi-proxy reconstructions. Whether the Amman and Wahl paper was or was not cited is pretty much immaterial. It was relevant and therefore was cited, but the implication that this was somehow crucially important or that it's absence would have lead to some hugely different outcome is simply fantasy. - gavin]

If it was all so trivial as Gavin tried to make it out to be, why go to such enormous lengths to cover everything up?

Oct 29, 2010 at 12:18 AM | Unregistered CommenterShub

DH and Shub

........ but still email attachments were deleted at CRU concerning this very matter, this was confirmed by UEA earlier this year despite Acton's claims now that no emails were deleted.

It all brings us back to the nature of these so called indpendent reviews. If effect the review members acted like a defence council at a trial. They were not prepared to dig deeper into the allegations in case it confirmed the contents of the emails or that they might discover other unsavoury aspects of CRU behaviour, but chose instead to investigate matters that highlighted that CRU scientists were people beyond approach. It is like arguing that Dr Harold Shipman could not have been a serial murderer, because he was deemed to be a highly respected member of the community.

That failure to investigate properly in a forensic manner coupled with the known mis-behaviour of CRU scientists by the nature of the leaked emails meant that we are dealing with an establishment whitewash.

UEA were never interested in establishing the truth, they were concerned only in hiding it away in a dark corner, away from the light.

Oct 29, 2010 at 10:06 AM | Unregistered CommenterMac

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Post:
 
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>