Bishop Hill Opening the can of worms
Nov 8, 2012 Climate: sensitivity Nic Lewis is best known as the co-author of the O'Donnell paper on Antarctic temperatures, which underwent a famously concerted attempt to stifle it at birth from those on the consensus side of the global warming debate.
Nic's attentions have more recently turned to climate sensitivity, his most notable success being the uncovering of the IPCC's rewriting of the Forster and Gregory results to make them look more alarming - in my view one of the great scientific scandals of our time.
In an new article at Climate Audit he reports on another key IPCC climate sensitivity study - Forest 2006. Although the author refused to provide data and code, Nic has been able to get hold of the data by other means and eventually to persuade Forest to release the code.
Which contains several strange errors, one of which may well turn out to be significant. If so, the knock-on effects are likely to be widespread:
In addition to the Forest 2002 and 2006 papers, I believe these errors also affected the Forest et al. 2008 Tellus A and the Libardoni and Forest 2011 GRL papers, and probably also 2009 and 2010 papers lead authored by Forest’s regular co-author Sokolov. It is to be expected that there will be multiple citations of results from these various studies in the AR5 WG1 report. I put it to Myles Allen – who seems, along with Gabi Hegerl, to be the lead author of Chapter 10 primarily responsible for the sections relating to climate sensitivity – that in view of these serious statistical errors, results from the affected papers should not be cited in the IPCC report. However, whilst accepting that the errors were real, he expressed the view that the existence of these statistical errors didn’t really matter to the results of the papers concerned. His reasoning was that only error (b) had a potentially substantial effect, and that didn’t much matter since there was anyway considerable uncertainty in the ocean data that the studies used. I’m not sure that I agree with this approach.
I would be surprised if the basic statistical errors in the IDL code do not significantly affect the results of some or all of the papers involved. I would like to test this in regard to the Forest 2006 paper, by running the IDL code with the errors corrected, in time to put on record in my “expert reviewer” comments on Chapter 10 of the Second Order Draft of IPCC AR5 WG1 report what the differences in Forest 2006′s results resulting from correcting these errors are, if significant. At least Myles Allen and Gabi Hegerl will then be aware of the size of the differences when deciding whether to ignore them.
Reader Comments (20)
Let me guess...by pure chance, the "error" made the results more alarming...
I can't comment: the first program I wrote some 40 years ago ... didn't work either ... and sadly the government wanted their computer back before I could get it to work.
Mike: I like the humility. Any programmer who's any good is a humble programmer because errors plague all of us. Test-driven development formalises that humility. That's why it's become such a strong emphasis within software engineering in the last ten years.
Nic Lewis is doing something of great import here. He's not fluent in IDL, nor are many of us. But I don't doubt that the errors he's found are real and need to be considered seriously by AR5. The joys of having access to the code.
Perhaps Richard Betts could tweet a few more people about this.
Good work indeed. The producers of AR5 will not like it. It is a foretaste of the rapid and penetrating analyses that can be expected to follow its release.
An aside:
Glad to see a reminder of Lewis' earlier work on the IPCC's devious manipulation of Forster & Gregory's work. Did he ever get a reply to his open letter to Hegerl? (http://judithcurry.com/2011/07/07/climate-sensitivity-follow-up/). A more recent blog post (http://www.worldclimatereport.com/index.php/2012/02/27/bigfoot-the-loch-ness-monster-and-high-climate-sensitivity/) in February this year makes no mention of any retraction or apology, but it notes "If the IPCC continues to hold on to its use of an uninformed prior when assessing the climate sensitivity in its Fifth Assessment Report (which is currently under construction), we will no longer have to wonder whether they are perpetuating a myth or just badly misinterpreting realty. The preference for the former will be clear."
You wonder why they don't what people to see the raw data and code , well its because of issue like this .
In normal science there should be a self checking or peer checking process in place , but in 'special climate science' it seems has long has you get the results you need you don't bother to ask to hard how you got them .
It never "matters", does it?
What has happened is a race to the bottom - how bad the science you can get away with. Until 1995 just a few were involved, then Santer allegedly rewrote a summary of AR3 without consulting those who wrote it. It got worse in 1997 when it was shown CO2 followed T hence we had the Mann/CRU 'frauds'. In 2004, NASA claimed fake aerosol physics to justify 'global dimming'. AR5 is allegedly being written purposefully to scare.
These people have nowhere to go now. The great pity is that many otherwise good younger people are struggling in an area so tainted much of the work has to be repeated to be sure it's not fake.
Post-modern science demonstrates yet again, its enduring flexibility and tolerance of questionable results undetected through the climate pal review process. As these publications become the basis for informing policy decisions and the inevitable political stance of argument from authority and argument through majority, all who willingly participate in anything less than the most rigorous science demonstrate agenda driven motivation well removed from the aspiration of rigorous research.
The merging of post-modern science, marketing and policy, all in a Green thrall dangling on the end of the corporate puppeteers is truly a toxic brew.
John Shade wrote
"Glad to see a reminder of Lewis' earlier work on the IPCC's devious manipulation of Forster & Gregory's work. Did he ever get a reply to his open letter to Hegerl?"
Yes, I did. A recap for those unfamiliar with the story. My complaint about the alteration of the Forster & Gregory 2006 results was rejected on the grounds that it was done to put all the climate sensitivity probability density graphs on the same, uniform prior in sensitivity, basis. Justifying changing a result from a correct to an incorrect basis on the grounds that all the other results were given on that basis seems very dubious to me. But I knew that at least one of the other studies, Gregory 2002, actually had its results shown on the same basis as the original Forster & Gregory 2006 results, being a uniform prior in the climate feedback parameter - that is, a prior inversely proportional to the square of sensitivity. So my letter to Gabi Hegerl complained that the statement that the Gregory 2002 results were stated on a uniform prior in sensitivity basis was incorrect.
Gabi Hegerl, quite properly, brought my letter to the attention of the IPCC WG1 Co-Chairs, and it was dealt with under the new formal "IPCC protocol for addressing possible errors". The result was the issue of a formal Erratum by the IPCC, stating that Gregory et al "implicitly use a uniform prior on transient climate response". I knew that this was also wrong, but Gabi insisted that the WG1 authors were sure it was right. In fact, Gabi was relying on Myles Allen, who I think was primarily responsible for the use of a uniform prior in sensitivity basis in Chapter 9 of AR4 WG1.
It took me several months, with the help of another climate scientist, to find out why Myles Allen thought that Gregory et al "implicitly use a uniform prior on transient climate response" and then ultimately to persuade him, and thus Gabi Hegerl, that this statement also was incorrect - and that I had been right all along in saying that in fact this study implicitly used a nearly uniform prior in the climate feedback parameter.
Gabi and Myles were not keen to get the IPCC to issue a further Erratum, which would obviously be embarrassing, so I agreed not to pursue the matter further.
"...obviously be embarrassing,..."
One more for that fat little file then.
To me, Nic Lewis has again shown that the pronouncements from the IPCC frequently have the flavour of something from a carnival side-show barker and are so terribly far from the pristine product produced be the usual concept of 'scientist' that Joe Public has in his or her mind.
Kudos for the brilliant and tenacious work from Nic.
I too am interested that the same names crop up in the continuing saga.
When you clutch at straws, you shouldn't be too surprised if they keep breaking.
"When you clutch at straws, you shouldn't be too surprised if they keep breaking."
Nov 9, 2012 at 5:37 AM | Alex Heyworth
Sounds like a soliloquy, Alex!
Nic,
Why did you just drop it even though you were right?
Mailman
Re 'Nov 8, 2012 at 7:56 PM Nic Lewis'
Many thanks for answering my query here. I'm glad you got some response from them, albeit a vexatious one. A large effort on your part produced little movement from them, but you have clearly revealed not only a flawed analysis but also a flawed response to your constructive and informative criticism.
Mailman,
"Why did you just drop it even though you were right?"
I had bigger fish to fry. The IPCC's mistake was not very important in this case, other than in that it showed that the scientists involved didn't understand Bayesian prior distributions as well as they thoiught they did. The first Erratum had already established that.
Surely it is better to try to get climate scientists to see where there are errors in their approach or understanding, in this case about Bayesian inference for scientific studies, than to score points off them.
Nic
While it may be pointless speculating on motives, having seen Myles Allen speak he appears to be a pretty intelligent man, unlike some of the more central figures in Climategate. Do you have any idea why he would be so dogged in defending an error that one would have expected him to be able to find himself?
It does seem that some otherwise competent scientists and mathematicians have allowed a belief system to over-ride their analytical capabilities, in the process aligning themselves with the likes of Mann and Jones who are not in any way competent.
David S,
Myles is certainly intelligent, and he has authored some very useful papers. I think one problem here was that he took the Erratum statement about Gregory 02 from a paper (Frame 05) led by an Oxford colleague of his, of which he was a co-author so he assumed that it must be correct.
Shub at 8.54 p.m. on 08 Nov...One more for that fat little file then.
Suggest replace little with large!