Click images for more details
The definitive history of Climategate.
A few sites I've stumbled across recently....
Guess what? Nobody did anything wrong!
View Printer Friendly Version
Since I'm only concerned with the code the following paragraph is relevant:
CRU accepts with hindsight that they should have devoted more attention in the past to archiving data and algorithms and recording exactly what they did. At the time the work was done, they had no idea that these data would assume the importance they have today and that the Unit would have to answer detailed inquiries on earlier work. CRU and, we are told, the tree ring community generally, are now adopting a much more rigorous approach to the archiving of chronologies and computer code. The difficulty in releasing program code is that to be understood by anyone else it needs time-consuming work on documentation, and this has not been a top priority.
Note the ifs, buts and qualifications.
When it set up the enquiry panel the RS said: "The names being put forward by the society will be acting as individuals, not representatives of the Society and the Society will have no oversight of this independent review.”
Now Oxburgh and co say: "The Panel was not concerned with the question of whether the conclusions of the published research were correct."
In short it is a fig leaf rather than a whitewash. No doubt it provides the politicians with cover to say, yet again, that "the science is settled".
"...Lord Oxburgh insists the panel did not have a pre-conceived view."
Is that the panel where some of the members had openly supported the AGW hypothesis and the Chairman stands to make money from carbon trading?
Talk about damning with faint praise though. I don't think reading between the lines they would want to put their life savings as a bet on the reliability of dendrochronology, and they take a passing shot at the hiding of decline by the IPCC.
Pretty weak altogether.
I'd echo JGC's comments here.
The reason you spend time on quality comments and documentation is twofold:
1) So that you can satisfy yourself that you understand that what's going on in your own code is happening by design and not good fortune.
2) So that another programmer who is totally unfamiliar with the codebase can check and correct if necessary. Crucially, "that" programmer might be "you".
Given the obvious warmist bias of the author, this comment is actually quite damning of the "regrettable" alarmism practiced of the IPCC.
Recent public discussion of climate change and summaries and popularizations of the work of CRU and others often contain oversimplifications that omit serious discussion of uncertainties emphasized by the original authors. For example, CRU publications repeatedly emphasize thediscrepancy between instrumental and tree-based proxy reconstructions of temperature during the late 20th century, but presentations of this work by the IPCC and others have sometimes neglected to highlight this issue. While we find this regrettable, we could find no such fault with the peer-reviewed papers we examined
The BBC is reporting it as "absolutely no evidence of anything wrong".
To John & Sean's points:
Yes, I agree. Unfortunately good documentation within IT environments is hard to come by or gets out-dates rather quickly. Still I distinguish public funded science as a different case. How can credible scientists or science organization get away with not documenting method and data because they had no idea that these data would assume the importance they have today and that the Unit would have to answer detailed inquiries on earlier work. WTF?
That was not science. That was fraud.
There should be outrage over this! This report is meaningless....toilet paper.
So, who's going to do the science we so badly need?
We read: "For example, CRU publications repeatedly emphasize thediscrepancy between instrumental and tree-based proxy reconstructions oftemperature during the late 20th century, but presentations of this work by theIPCC and others have sometimes neglected to highlight this issue." Was the decline ever pointed out? If so why try to hide it?
There seems to a recurrent theme developing in paleoclimatology.
The first is comment is from the Oxburgh Report 2010, the second is from the Wegnman Report 2006:
2. We cannot help remarking that it is very surprising that research in an area thatdepends so heavily on statistical methods has not been carried out in closecollaboration with professional statisticians. Indeed there would be mutualbenefit if there were closer collaboration and interaction between CRU and amuch wider scientific group outside the relatively small international circle oftemperature specialists.
Conclusion 3. As statisticians, we were struck by the isolation of communities such asthe paleoclimate community that rely heavily on statistical methods, yet do not seem tobe interacting with the mainstream statistical community. The public policy implicationsof this debate are financially staggering and yet apparently no independent statisticalexpertise was sought or used.
Given that the enquiry was being carried out by those with either a built-in sympathy and support for the AGW theory (conscious or sub-conscious) and in some cases a vested interest in its continuation, I would have thought that the skeptics have got all they could have asked for.In particular conclusion 2 ...
"We cannot help remarking that it is very surprising that research in an area that depends so heavily on statistical methods has not been carried out in close collaboration with professional statisticians. Indeed there would be mutual benefit if there were closer collaboration and interaction between CRU and a much wider scientific group outside the relatively small international circle of temperature specialists."
This is much worse than I even thought. This is hugely embarrassing for CRU. Anyone with a Standard Grade in Science could write a more comprehensive and in-depth report than this. This report is superficial, cursory, and presents no transcripts of the oral evidence.
Compare the House of Commons Science and Technology Committee report with this one by Lord Oxburgh.
This will not bring closure to Climategate. This shows the cack-handed way the University of East Anglia have dealt with this scandal. This must be the worst attempt at a whitewash in recent history - LOL
Would you buy a degree from this university?
Very deep sigh. It's just as bad as we thought..
But, albeit slighlty O/T, speaking of peer-review - and in marked contrast to faults they could not find "with the peer-reviewed papers [they] examined" ...
New research organized by Donna Laframboise indicates that the IPCC overstated its reliance on “peer-reviewed literature” in AR 4 by 30%. Of the 18,531 references in 44 Chapters, 5,587 were not peer-reviewed. 21 of the 44 Chapters, would receive an F (59% or lower peer-reviewed reference citations):
This is like having a trial and only letting the defence counsel speak. "Your Honor, our clients did have the opportunity but didn't use it. They did have what some mean spirited people might argue is a motive but we simply say it was no more of a motive than your Honor's own motive in deciding this case. And they, like many others in the field, did have the means through the abuse of the statistical methods but they really didn't know enough to use them effectively. The obvious facts of the matter clearly indicate that they didn't do it and even if they did, they didn't know what they were doing. Besides, others, including the most prestiguous bodies in the world, were doing far worse. They are guilty of nothing more than good intentions of saving the world. The prosecutions silence on the matter is the final proof that the CRU scientists have done nothing wrong."
I sense a FOIA request is winging its way to UEA requesting the release of the transcripted oral evidence.
The whitewash has been thinned too much for it to be called a cover up. The most important peer-reviewed paper that CRU has been involved in recently was was the IPCC Fourth Assessment Report and a large body of the complaints to Sir Muir Russell, including mine, laid out clear evidence of wrongdoing supported by the disclosed emails. So far no enquiry has ventured into this minefield.
Two days ago I received an email from 'William' at the ICCE Review saying,
The Climate Change Emails Review intends to publish all submissions unless there are potential legal constraints such as defamation or copyright, or they are abusive. The Review has uploaded a first tranche of submissions to the website and we will continue to upload others as soon as is practicable. Your submission has not yet been published as the Review's legal advice is that it could be open to a claim of defamation under English law if it publishes the current version of your submission, as it makes references to, and comments upon, a large number of individuals. The Review, unlike the UK Parliament’s Science and Technology Committee, does not have Parliamentary privilege.I am therefore writing to ask for your permission for the Review to publish contact details for you on the Review website so that interested parties may contact you and seek access to your submission directly.
I am therefore writing to ask for your permission for the Review to publish contact details for you on the Review website so that interested parties may contact you and seek access to your submission directly.
None of us want to be sued so I have replied asking to be told which particular passages their legal advisor considers might give rise to a risk of legal action, and offered to rewrite them. Should anyone want to see this complaint, which Sir Muir Russell dare not publish, in the strictest confidence and on a non disclosure basis feel to email me using crusub at tesco dot net.
On 21 March 2010 I had emailed Prof. Overpeck, Dr Solomon and the IPCC Secretary Dr Christ asking if they would let me have the spreadsheet mentioned in the leaked email of 28 July 2006 from Prof. Overpeck to Prof. Briffa. I received no reply. On 26 March out of the blue UEA sent the spreadsheet to me explaining that they did hold it after all. Some coincidence. The spreadsheet, which did not include Steve McIntyre's important suggestion that NRC, 2006 and Wegman et al. be cited in the section discussing M&M, has now been put up at CA.
So on 2 April 2010 I sent a detailed letter to Drs Solomon, Pachauri and Christ outlining the accusations that are in my submission and inviting them to answer the charges. Dr Solomon replied on 6 April saying that I would shortly get a reply from the WGI TSU. On 8 April the new TSU in Bern replied in an unsigned email which answered none of the allegations.
Then on 12 April the ICCE Review emails to say it won't publish my complaint. More coincidence no doubt.
So intellectual property rights are used to refuse the release of important methodology and no enquiry save Parliament, dare publish the accusations.
And then they do know where I live and they are many and I am just me.
"The potential for misleading results arising from selection bias is very great in this area. It is regrettable that so few professional statisticians have been involved in this work because it is fundamentally statistical...We cannot help remarking that it is very surprising that research in an area that depends so heavily on statistical methods has not been carried out in closecollaboration with professional statisticians."
Same problem with Michael Mann's work.
"The Unit has demonstrated that at a global and hemispheric scale temperatureresults are surprisingly insensitive to adjustments made to the data and thenumber of series included."
Same defence used by Michael Mann, and a complete lie, of course.
So, the potential for misleading results is "very great", but no matter, we know the result we want so we can torture the data to give it us. Just like Michael Mann did.
This might convince ignoramuses, but it doesn't cut any ice with me. We are simply seeing the lies, distortions and whitewash that Mann and his cronies cooked up when the Hockey Stick was shown to be a fraud.
Each day a little bit of the integrity of science dies.
Facts not consensus fiction please.
There is plenty of implied criticism, between the lines. It is apparent that David Hand the statistician on the panel has reservations. The biggest problem is that the overall presentation of the report gives the media a chance to declare it as a declaration of innocence when it really is not. The question I'd like to see answered is 'Is this the quality of work on which to base policy decisions amounting to trillions of dollars?'
Agree with above comment "a fig leaf not a whitewash".No data , no codes, no statisticians but it's quite good enough to justify billions of wasted dollars/pounds and jeopardise U.K. energy supply for years to come.You couldn't make it up.You must feel very lonely Mr Holland, but there are many, many supporters in several countries but the only voice we have is sites like the Bish. and WUWT.As an aside, a Tory canvasser called yesterday. I tried to engage him, an obviously well educated gent., on the question of AGW and energy supply. He had no idea what I was talking about.
Agree, a whitewash, but the more you read it, the less you see by way of ringing endorsement in my view. Certainly, I'd not feel very comfortable seeing that, had I been the object under investigation.
Look at the brevity of the report - FIVE PAGES and even then it contains repeats. In depth report FIVE PAGES. Is that all?
The childish logic and rhetoric applied in the review is frightening, e.g.:
"Although inappropriate statistical tools with the potential for producingmisleading results have been used by some other groups, presumably byaccident rather than design, in the CRU papers that we examined we did notcome across any inappropriate usage although the methods they used may nothave been the best for the purpose. It is not clear, however, that better methodswould have produced significantly different results. The published work alsocontains many cautions about the limitations of the data and theirinterpretation. "
There would appear to be enough in this one excerpt for several pages of real analysis. Written by commmittee clearly.
A small group with limited statistical ability not prepared to seek advice from mainstream statisticians.Seems as though that is the conclusion I derived from Bart's recent escapades with VS, could of saved a lot of time and effort but then again maybe not. There was limited time and effort put into this report, fifteen man sorry person days at the University, most of the time would have been used up in correctly wording the conclusion to give some talking points but not bucking the establishment.The conclusion that I draw from this is that the CRU are too small and inadequate a group to inluence governmental pollicy and the fault for that lies at the heart of the IPCC.
CRU did a public service of great value by carrying out muchtime-consuming meticulous work on temperature records
On the matter of the ownership of the researcher's data.
I really don't think this argument should be allowed to go unchallenged where the data form the basis of decisions that affect millions of people's lives and cost billions of pounds (or dollars)
So a guy who is making a fortune off of promoting AGW finds that the science is sound.And in other news, a criminal caught stealing jewels denies he did anything wrong.True believer syndrome is how victims of con-artists rationalize their being fooled. Con-artists cover up their scams by pretending nothing is going on.
We agree with the CRU view that the authority for releasing unpublished raw data to third parties should stay with those who collected it.
How about releasing it according to the requirements of Freedom of Information law? It isn't for the review panel to make up their own law.
The Bishop should be on the World Service between 2.25 and 3pm this afternoon (Wednesday, UK time)
"For example, CRU publications repeatedly emphasize the discrepancy between instrumental and tree-based proxy reconstructions of temperature during the late 20th century, but presentations of this work by the IPCC and others have sometimes neglected to highlight this issue."
Remind me, who wrote the relevant chapters of the latest IPCC report? Surely not Keith Briffa (lead author, ch 6, WGI-AR4) and Phil Jones (coordinating lead author, chapter 3)? Did the investigation read (or understand) nothing of David Holland's questions on this subject?
Sweet.... Hopefully there will be a web-link.
Been and gone on BBC WS, Bish, I just heard your last sentence at 14:25...
Steve McKintyre is not impressed, quite a broadside fired from Climate Audit.
Apologies if someone else has said it (I am sure they have but have no time to read all comments) - Conclusion 2 - I find pretty damning!
Given that it is the statistics from which all the hyped up alarmist spin is woven.
Hopefully with proper scientific statistics being applied in future we just might get "science" and not hype.
Here's the 'deconstructed' version of the main points:
“the effects of long term temperature variations are masked by other more dominant short term influences and have to be extracted by statistical techniques. The Unit approaches this task with an independent mindset…”
Read: Temperature variations are masked by other influences and only the most robust of statistical analyses stand any chance of being able to detect them with statistical significance. Like Michael Mann, CRU has little expertise in statistical analysis, so is practically incompetent in its use.
“Although inappropriate statistical tools with the potential for producing misleading results have been used by some other groups…in the CRU papers that we examined we did not come across any inappropriate usage although the methods they used may not have been the best for the purpose.”
Read: CRU employed lousy methods of applying well-known statistical tools. Thus the tools were ‘appropriate’, but their method of use was incorrect.
“It is not clear, however, that better methods would have produced significantly different results.”
Read: We didn’t bother to check that.
“With very noisy data sets a great deal of judgement has to be used. Decisions have to be made on whether to omit pieces of data that appear to be aberrant…The potential for misleading results arising from selection bias is very great in this area. It is regrettable that so few professional statisticians have been involved in this work because it is fundamentally statistical.”
Read: CRU don’t have the necessary statistical competence to apply an appropriate ‘cherry-picking’ of data and thus are prone to producing garbage from noisy data.
“The Unit has devoted a great deal of effort to understanding how instrumental observations are best combined to derive the surface temperature on a variety of time and space scales. It has become apparent from a number of studies that there is elevation of the surface temperature in and around large cities and work is continuing to understand this fully.”
Read: The Unit expended a lot of effort, but we have no idea whether that was efficient or not, or whether the results were sound. They are just beginning to realize that they have not properly accounted for Urban Heat Island effects.
“The Unit has demonstrated that at a global and hemispheric scale temperature results are surprisingly insensitive to adjustments made to the data and the number of series included.”
Read: The Unit claims to have demonstrated it, but we were reluctant to check the same as that would have been devastating since no-one in their right mind could believe such a thing. We will nevertheless include this sentence to deceive the unwary reader.
“For example, CRU publications repeatedly emphasize the discrepancy between instrumental and tree-based proxy reconstructions of temperature during the late 20th century, but presentations of this work by the IPCC and others have sometimes neglected to highlight this issue. While we find this regrettable, we could find no such fault with the peer-reviewed papers we examined.”
Read: This only related to the papers we actually examined. We didn’t examine papers that weren’t exclusively written by CRU personnel, for example collaborations of Jones, Briffa, Bradley and Hughes with Michael Mann, because then we would not be able to use this sentence. Anyway, we can always blame Michael Mann (being careful not to name him) as that part of the IPCC report was under his control.
“We saw no evidence of any deliberate scientific malpractice in any of the work of the Climatic Research Unit and had it been there we believe that it is likely that we would have detected it. Rather we found a small group of dedicated if slightly disorganised researchers who were ill-prepared for being the focus of public attention. As with many small research groups their internal procedures were rather informal.”
Read: There was certainly scientific malpractice, but we charitably conjecture that it wasn’t ‘deliberate’: it was more a case of incompetence, especially in the area of data handling, which regrettably is the whole raison d’etre of CRU.
“We cannot help remarking that it is very surprising that research in an area that depends so heavily on statistical methods has not been carried out in close collaboration with professional statisticians.”
Read: Frankly, these men were a bunch of dedicated (statistical) incompetents who spent a great deal of taxpayers’ money to produce reports of doubtful scientific value. However, since we all have our noses in that trough, the less said the better.
Scientistfortruth, very good, but you missed one:
"We have not exhaustively reviewed the external criticism of the dendroclimatological work"
Read: We could not be bothered to read the extensive, detailed and specific criticisms provided by Steve McIntyre or even the clear summary provided by Andrew Montford. Furthermore we do not wish to give these people any publicity by mentioning their names, for fear that people might read these csources and learn the truth. So we just make vague reference to unspecified 'external criticism'.
Here is a quote with which the report on this figleaf ends in the Telegraoh:
"Bob Ward, policy and communications director at the Grantham Research Institute on Climate Change and the Environment at London School of Economics and Political Science, called for an apology from the sceptics."I think those so-called sceptics who have attempted to undermine the credibility of climate change science on the basis of the hacked emails now need to apologise for misleading the public about their significance.”"Link: http://www.telegraph.co.uk/earth/environment/climatechange/7589715/Climategate-scientists-criticised-for-not-using-best-statistical-tools.html
I'm speechless ...
"The university accepted the criticisms of the unit’s statistical techniques and data storage and said these issues would be rectified."
It was interesting to read the comments that followed. Fairly (pun) hostile.
"I'm speechless ..."
Don't be. Bob Ward and the Grantham Research Institute (headed by Lord Nicholas Stern, no less!) are bankrolled to the tune of tens of millions of pounds by Jeremy and Hannelore Grantham, the same people who bankroll Greenpeace, WWF, The Union of Concerned Scientists, the Woods Hole Research Center (not to be confused with the oceanographic body) and a plethora of other eco-fanatics and advocates. Just remember that every time Bob Ward opens his mouth - he is the mouthpiece of his paymasters.
I am left with an overriding impression of amateurism, not just from the report's authors, but from the CRU. Lord Oxborough et al seem to have expended minimal effort on the investigation and have produced a report which hardly stands critical examination, as much for what it does say as for what it doesn't say. In their attempts to justify the CRU, they have offered as an excuse CRU's lack of statistical expertise yet as statistics is the primary tool for distilling information out of noisy data, they are unintentionally condemning the organisation which they are trying to defend. It’s deeply depressing to know that government policy is being made and schoolchildren are being “educated” on the basis of conclusions drawn from such unimaginably sloppy science.
The Lord Oxburgh's 5 page report is receiving a pounding online. It is being described as an utterly feeble response to the growing concerns expressed on climate science. UEA have scored an own goal.
Viv Evans,The whole point of the whitewash is to allow AGW promoters and profiteers to sustain the true believer syndorome of their victims.
Attendance arithmetic - help needed
I've been trying to understand the attendance data presented in the Oxburgh report. Can someone help me find where I'm going wrong?
The report says: - The panel had seven members (according to the list in Appendix A). - They made two visits to UEA. - Two members of the panel were present on both occasions. - A total of about fifteen person days were spent at the University.
So far as I can see, there are several possibilities:
1. Visit A had two members present and visit B had seven members present (including the two who were present at visit A). That would make a total of nine person days for the two visits.
2. Visit A had three members present and visit B had six members present (including two who were also present at visit A). That would also make a total of nine person days for the two visits.
And so on, down to the possibility where all seven were present at visit A and just two of the seven turned up at visit B. Each case, if I have counted right, gives a total of nine person days for the two visits.
To me "about fifteen" means somewhere between 14.5 and 15.5 - my total of nine is very wide of the mark.
Please point out the fallacy in my working - I am embarrassed to say I cannot find it for myself.
One or both visits must have been for longer than a day?
Another possibility: attendance by the chairman at UEA on his own does not count as a 'panel visit' but is included in the person day total.
Pharos One or both visits must have been for longer than a day?
Of course. That's it. I don't know why I assumed a visit had to be a one-day visit.
The report seems eager to play down the size of the CRU. It says "The Unit is a very small academic entity (...). It has three full time and one part time academic staff members and about a dozen research associates, PhD students and support staff".
Yet the CRU's website says it has:- One director (Liss)- One deputy director (Briffa)- Three academic staff (Jones, Osborn, Goulen).
I always regarded professors as academic staff, too. That seems to make a total of five academic staff.
Plus it has, according to its website: - One manager/research associate - Six "research staff" - Eleven "associate fellows" - Four support staff (admin assistant, research admin, librarian/map administrator, IT manager) - Eleven postgraduate students
My arithmetic says this adds up to five academic staff and thirty-three research associates, PhD students, support staff and whatever.
Which is it? Did Oxburgh understate the CRU's size, or is the CRU overstating its size?
The report is actually quite critical - more so than I expected, given the credentials of the panel. In precis, it says their working methods were very likely to produce unreliable results; the lack of statistical expertise on the team was a serious mistake, further undermining confidence in their findings, and that the IPCC reports lack any of the cautions and caveats that the Unit's published papers contain.
This may be couched in platitudes about 'hard-working, best intentions' etc. But a rational and dispassionate reading of the report would conclude that CRU's overall findings on temperature are unreliable and that better methodologies are needed. Indeed, that is accepted and such improved methodologies are being implemented.
How the media, or anyone, gets from this 'CRU exonerated, the science is settled, the skeptics were wrong' is beyond me.
CRU has a new project; The Dendroclimatic Divergence Phenomenon: reassessment of causes and implications for climate reconstruction.
And who is Principal Investigator? Yes, you guessed it, Briffa.
Quote: "It suggests that the degree of warmth in certain periods in the past, particularly in medieval times, may be underestimated or at least subject to greater uncertainty than is currently accepted. The lack of a clear overview of this phenomenon and the lack of a generally accepted cause had led some to challenge the current scientific consensus, represented in the 2007 report of the IPCC on the likely unprecedented nature of late 20th century average hemispheric warmth when viewed in the context of proxy evidence (mostly from trees) for the last 1300 years. This project will seek to systematically reassess and quantify the evidence for divergence in many tree-ring data sets around the Northern Hemisphere."
"Based on recent published and unpublished work by the proposers, it has become apparent that foremost amongst the possible explanations is the need to account for systematic bias potentially inherent in the methods used to build many tree-ring chronologies..."
The project start and end dates are 12/09 – 05/12. Look at the start date. Could this be coincidence?
Who is funding this work?
"Who is funding this work?"
C'mon, who do you think? The Tooth Fairy?
No, we are. The tax payers. Who'll also provide the little dears with their index-linked pensions and pay for their jollies to Cancun etc etc.
Notify me of follow-up comments via email.