Saturday
Jun192010
by 
Bishop Hill
Bishop Hill A challenge
Jun 19, 2010 Climate: Curry Climate: HSI In the comments on the Collide-a-scape thread, Judy Curry has issued a challenge to mainstream climate science:
I am laying down the gauntlet, [The Hockey Stick Illusion] really needs to discussed and rebutted by the paleo researchers and the IPCC defenders.
Most of the responses are fallacious so far - along the lines of "a bad person liked this book". Let's see if anything more substantial appears.
Reader Comments (52)
Who runs the gantlet
To claim the Lady's gauntlet?
Would be such a man.
=================
I have read that Collide-a-scape blog and its comment thread with fascination, and great admiration for Judith Curry's patience and stamina in her thankless crusade trying to coax fellow AGW proponents to produce something remotely more intellectual than a monotone of bile.
That, buried deeply somewhere in that thread, a scientist of her stature confesses not only to requiring the HSI to fully understand the significance of all the CA McIntyre effort, but to fully apppreciate its monumental significance for her community, is praise indeed.
An excellent idea of Judy, It should be easy to rebut the Hockey Stick Illusion for those who have the access to the remaining 1 million emails of Phil Jones' mailbox that were not published in the ClimateGate.
I'm apparently not the only one who's been glued to the computer this weekend. I began immersing myself in the climate change blogosphere when Climategate broke. After the cursory inquires amounting to basically a tap on the wrist and testing my patience reading warmist blogs, I was becoming jaded.
Then I happened to visit this blog!
My mother is coming to visit from out-of-town and I should be cleaning like a crazy woman. But the thread at collidescape has completely sucked me in! Time to break away for a bit and improve the climate in my home! LOL
I have deep respect for the dignity, patience, and wisdom of the lukewarmers. Looks like I need to read your book! Not one shameless plug ;) Judith Curry's endorsement is powerful. A few of the more fair-minded proponents are no doubt intrigued enough to read it, I bet.
Little by little, the tide turns. First one, then two. Then hundreds.
Most of the critics don't seem to have read THI.
Indeed, geronimo. I find that quite a source of amusement. So many in the establishment - those most eager to label sceptics as "denialists" - so much more obviously exist in an insular cloud of almost palpable denial.
I have to stop posting there because I think my derision towards their entrenchment is threatening to seep into my comments. I really must say that my admiration of Dr Judith Curry grows daily. Her fortitude in the face of her colleagues' stubbornness and bloody-mindedness is greatly to be commended. It seems increasingly clear, though, that she's fighting a losing battle trying to coax the climatology establishment out of ad-hominem and into the realm of respect, reason, and most importantly out of blatant political advocacy.
Climatology seems determined to self-destruct and I don't think anything Dr Curry does will defuse its bomb. To Dr Curry I would ask, are you trying to scrub clean a dirty cat or are these really indelible spots on a leopard?
I don't know who dhogaza is, except that he's manifestly classic RealClimate in tone and stance. He's a powerful reminder for those of us from the "other side of the debate" that there is much more than just science which divides the two. If I had no opinion at all on climatology, or if I were new to the debate, I'm certain that I'd automatically gravitate to this "side" because of dhogaza and his ilk's style and type.
I only meet NICE people over here. Have I told you guys I love you recently? :o)
Pharos,
Did a bit of a double take with your ref to HSI. 'Not too sharp yet this morning to be perfectly blunt.
HSI meant to me Horizontal Situation Indicator which began to appear on the planes I flew in the '70s.
It concentrated all the navigation indications in a single confusing display so that I could lose my way at much greater expense.
With head in the sand
Beachboy missed the turn of tide.
Skeptic storm alert.
===========
Dr. Curry is one of the precious few who seems to have both integrity and the boldness to speak in the climate science industry.
Lambert's bizarre excuse for rationalizing the dismissal of skeptics as financially or politically corrupt flies in the face of facts about skeptics and is laughable when one considers just how lucrative the CAGW promotion industry has been for over at least 10 years.
He acts like a Catholic Bishop discussing Luther's theses, dismissing Luther as heretic while taking the great wealth he lives in for granted.
Yep, hunter, also shub and I picked up on the fact that Michael Tobis seems to be throwing Naomi Oreskes under the bus, saying in effect that she is out of date about funding and skeptics. The real elephant in the room, the fleas of which may soon torment Naomi, is the huge mismatch in funding of alarmist vs skeptical science. This elephant is ridden by the IPCC and its mad mahout.
==============
SimonH
"Her fortitude in the face of her colleagues' stubbornness and bloody-mindedness is greatly to be commended"
I agree. Her remark about her own faculty: "one member wrote the following comment: “her outspoken views on climategate have probably hurt our climate program”" both tells us why people like her are few and far between and confirms sceptical beliefs in the relative unimportance of actual science to 'climate programs'.
Continued funding is clearly far more important!
I thought that this was one of the most telling comments:
(Start quote)
"Michael Tobis made this statement on a recent thread at climateaudit with regards to the hockey stick debate: ”I really don’t have an opinion one way or the other on the key controversies of this blog, which is why I usually don’t follow it. Thus honestly I have no comment on the chapters of greatest interest to you. I genuinely don’t consider those matters key to physical climatology or to climate policy.”
I shared that opinion also, although I was very concerned by the “process violations” I saw discussed at climateaudit. I tried to learn more about the technical issues involved in the hockey stick debate, but found it hopeless to wade through the stuff at climateaudit, and I was further hampered by my eyes glazing over everytime i saw mention of bristlecones.
Montford’s book makes it very easy to understand both the technical issues and the process issues that have been a concern to MM and the broader community of skeptics and citizen scientists."
(End Quote)
In other words, she had no understanding of the Hockey Stick controversy or its significance until the very recent publication of the "The Hockey Stick Illusion".
This wasn't from a newbie amateur this was from from the admirably honest Judith Curry, who "is an American climatologist and chair of the School of Earth and Atmospheric Sciences at the Georgia Institute of Technology." !
The Michael Tobis concerned is "a Research Scientist Associate (in practice, mostly a software engineer) at the University of Texas Institute for Geophysics in the delightful city of Austin. " Number one on his own list of "Things I do" is "Climate and climate change, especially contemporary and holocene time scales." (http://sites.google.com/site/mtobis/)
He also had no such appreciation and even less interest and - presuming he hasn't read the book - still sees no significance in the accuracy or otherwise of the Hockey Stick! (yet still feels qualified to write arrogantly dismissive comments on climate audit)
Congratulations to Dr Curry for making the effort to read a book easily accessible to the average layperson, but for crying out loud, is this the level of knowledge and curiosity generally amongst scientists supposedly interested in climate sciences?
What are the chances of scientists whose work has even less relationship to the climate knowing any more, yet they still have the gall to dismiss scepticism?
artwest, I think your observations are spot on.
Regarding this:
I honestly don't know. I think a lot of scientists working in their own particular areas of research often work very much in isolation. They don't have (or haven't had) an appreciation for the level of integrity that underpins the data upon which they've based their work. Honest scientists have ordered and received gridded products from GISS or from HadCRUT and have - perfectly reasonably - assumed that these data sources have a high degree of integrity. There should be no reason for these honest scientists to suspect differently.
It transpires, however, that whether or not there should be, there IS reason to examine/question their accuracy and by extension the accuracy of these scientists' own work where it uses these products as their base.
It must indeed be disconcerting to discover that something upon which you've come to believe you can depend on - upon which you've built your entire career, and which permeates the body of your research - might not, after all, be the honourable thing of integrity you thought it was. This is where cognitive dissonance comes in to play.
The willing perpetrators of "the crime" - the hockey team - have continually launched ad hominem attacks, have been agressive, dismissive or derisive (or all of those) and wholly obstructive towards the sceptics that uncovered their crimes. That's not cognitive dissonance, that's guilt. But for the scientists whose work has been based on the hockey team's misleading products, cognitive dissonance is a far, far easier place to go than recognising the genuine problem that faces them now. Cognitive dissonance is far more attractive a place to go than actually examining the hockey team's activities and seeking to uncover the harm that has been done to their own life's work by using that potentially faulty product. It's easier and far more comfortable to support the hockey team, whether it's a thing of integrity or not, than it is to accept that everything you've come to believe in about your own work, and the work upon which your own work is built, actually needs to be reviewed and might not withstand scrutiny.
So I'm understanding their cognitive dissonance, their vocal and agitated dismissal of the "denialists" they hear so much about these days. I'm not accepting, but I am understanding. I'm not without some sympathy for those scientists, many of whom I believe desire to be honourable scientists of good standing. But this thing is more important than the proclamations of 2,500 duped and angry scientists. This thing has worldwide impact. Millions of lives and livelihoods may ultimately depend on this science being done right, and on policy being based on good, hard science, rather than falsely inflated probabilities and assertions of certain eventualities.
An interesting read ... right up to the point where the usual suspects turned up. "Judith has moved deep into denialists territory"."Sorry, Judy, but many of us just don’t trust creationists [...]".
Consider the channel well and truly changed.
3x2: It gets worse. The egregious dhogaza issues not very veiled threats to Dr Curry for apparently verging on heresy:
"My interest is your credibility, to be honest. From what I’ve read, a growing number of climate scientists share that interest."
I don't know who dhogaza is, except that he's manifestly classic RealClimate in tone and stance.
I had once hoped that ("tommy gun") "dhogaza" was confined to "climate progress". Initially disappointed, I now realize that he has done more for the sceptic cause than most anyone on the net. He is the net equivalent of those individuals who manage to clear any room at any party in minutes.
A while ago I was following a series of articles on "science of doom" when "tommy gun" popped up in the comments with his usual subtlety. Not been back since. Neither I suspect have the beneficiaries of his "facts".
Shame really as I felt that the site owner had done a good job with the articles and certainly seemed to be dealing professionally with the follow up comments and queries. With friends like these ...
Doctor Curry is the tip of the academic Ice Berg. Many more, who are somewhat like her though less brave, are hiding beneath the water, afraid to rise and face the elements above the waterline. Fear is quite an individual experience. Most of us never learn how to deal with it. We're generally afraid to try.
From the information I have read, many scientists would fear peering above the wall as the "hockey team" et al have assasinated or attempted to assasinate the careers of those who have previously voiced concerns.
It will take a long time before some will agree that their own work has been devalued by data they thought was accurate but has subsequently been identified as weak (at best) and fraudulent (at worst).
Dr Curry deserves every praise for sticking to her guns. I found all her comments direct and lucid, and nothing in that long and very revealing thread has so far undermined her case.
It's worth standing back and asking if climate science will ever become a hard numerical science like astronomy, engineering, electronics where scientists can make accurate numerical predictions of future events. Or is it going to be like botany - labelling and describing things that you have found. A past tense subject.
Rutherford said the all science is either physics or stamp collecting. Climate science looks more like stamp collecting - except they burn the stamps they don't like.
Jack Hughes: "It's worth standing back and asking if climate science will ever become a hard numerical science like astronomy, engineering, electronics where scientists can make accurate numerical predictions of future events"
Give it another century and they'll have some decent data to work with and may produce better results. I've not seen the 'since records began' claim used to support AGW lately, and may because too many people are asking 'when was that satellite launched?'
It's a fine question, Jack. Probably as important is whether climate science will ever recover from pretending to be a hard science and being found out. Though I don't think it's entirely the fault of ALL climate scientists, the public were allowed to believe that the science was "settled". This assertion was made many times by politicians and by climatology's most ardent promoters for many years, but was not popularly countered by climate scientists until some time AFTER Climategate.
It was only countered then - by Gavin Schmidt at RC - when it had become clear that the "settled science" claim was not just completely untenable but also a source of merciless mockery.
Will it recover? I've argued at the Collide-a-scape thread that I don't think it will within this generation of climate scientists. I'm not sure it will with the next, either. The subject itself is highly political (an observation, and much of my gripe) and I suspect it's unlikely that it will ever attract enough people who are interested in the science more than they're interested in seeing through their personal socio-environmental political persuasions. Confirmation bias is an unfortunate by-product of such an emotive political subject like environmental science, but conquering that is what will be required if the subject is to recover in the longer term.
Climate science is really just an amalgum of the more traditional science disciplines and specialties - physics, chemistry, astronomy, geology, biology, palaeontology, volcanicity, glaciology, palaeoecology, meteorology, oceanography: and the more modern ones of economics, statistical mathematics, risk analysis, computer modelling. No-one can properly excel in the whole lot, the creation and funding of climate science is purely the result of political request, there is no other raison d'etre. Initially benign, perhaps, but through accident or design, it became increasingly not so. This is primarily the fault of politicians, and secondly academia, for not only directing funding toward a singular agenda, but actively encouraging environmental activism to infiltrate and stifle balanced ethical scientific inquiry in favour of strident advocacy, without any mechanism of self discipline or audit.
It was bound to reach its Waterloo, and did so last November. Phil Jones realised that. But few others, except Judith Curry. Perhaps she should head the IPCC, if they must produce the final report.
I can understand why your book might need to be "debated". But why does it NEED to be "rebutted"?
Surely, it NEEDS to be debated in order to determine WHETHER IT NEEDS to be rebutted.
It needs to be either rebutted OR the implications that it raises acknowledged. To date both warmists and the establishment have done neither (hoping that the book will be categorised as voodoo science?). They cannot continue to ignore the implications raised without a rebuttal.
Following his presentation: - God, Science and Global Warming - last week
http://www.wargravechurch.org.uk/welcome.htm
I asked Sir John Houghton (former co chair IPCC, head met office)
a direct question afterwards:
"Have you looked at the Climategate emails for yourself, in context, and the code, and looked at what people are saying.."
Sir John Houghton's answer: No
There is a well funded fossil fuel exxon mobil denial campaign using the same people that lobbied against tobbaco.
We had a long discussion.
Mainly around the lines of as a scientist..
phil Jones's comment pre-climategate to Warwick Huges:
"why should I supply data, you only want to find things wrong with it"
Which is appalling to any scientists that is NOT a 'post normal scientist'
What if, that scientist, found something wrong (if you supplied the data) that found that the risk/temperature were UNDERESTIMATED, it is indefensible not to publish data/code used to derive results for scrutiny.
You need to see the code as well, because whilst you say you programmed it a certain way (as I am an IT profesional) you may NOT have actually done it in the code correctly, it must be available for audit.
I have no doubt Sir John Houghton sincerely believes what he says:
As do virtually ALL the European and UK politicians.
The rest of the conversation, will be 'hearsay' to many, so I will not repeat it here.
Let me know, if you want tho hear full audio recording of this public meeting/presentation.
How to get the real scientific debate here and eslewhere, to the politicians and vast majority of the general public is going to be a challenge.
Transitions Towns say this:
"Haven’t they disproved Climate Change?
They have tried, but the science has stood up to the onslaught. The discussion among climate scientists is about how bad it is going to be, how quickly it will happen and how best to minimise possible disastrous consequences."
http://www.henley-in-transition.org.uk/about-transition/index.htm
Have a look at their highly emotive movie, using children extensively.
http://www.transitionnetwork.org/about/publications/transition-movie
Small (cute) child saying:
"CO2 is one of the MAJOR greenhouse gases"
"the weather is becoming more and more unpredictable, JUST because we are realesing more and more CO2 into the air."
Transitions Towns have many things I agree with, sustainability, thinking locally, even alternate energy sources, but I would be labelled a 'deniar' by them.
So whilst, we may discuss the science here, 15 years ago the key 'climate science elite' (ie IPCC lead authors, chairs, etc) scientists became political/social public policy activists. Environmental lobby groups, social change political groups all jumped on board the AGW bandwagon to use it for their own political agenda.
I went to my local transitions town meeting in Henley this month, people refused to talk to me and walked off, rather than look at the BBC’s Roger Harribin interview, that says temps were not unprecedented, contrary to what they were saying on their website.
This is my local community, people I know, my wife knows and are patients in her business (for how much longer, as she is clearly married to a ‘deniar’)
I can't seem to get this into the comments section at collide a scape:
(audio commentary is going to be vaialble to confirm my notes)
-------------------------------
I spoke to Sir John Houghton (former IPCC co-chair) last Thursday (17th June 2010) for about 15-20 minutes following a presentation by him:
- God, Science and Global Warming - (1st Power point slide)
http://www.wargravechurch.org.uk/welcome.htm
John Houghton set up the Hadley centre and was instrumental in the early days of the IPCC and he defended and fought to get Michael Mann's 'hockey Stick' into the IPCC reports and to keep it in..
In last weeks presentation John Houghton said:
"don't believe any rubbish you see on the internet.."
Presumably that includes everything Dr Judith Curry has ever contributed. (let alone my contributions ;) )
"The enquiries have completely exonerated everybody."
"All that was wrong with the IPCC, was one tiny typo 2035 instead of 2350 on glaciers."
"There is a consensus amongst thousands of scientists"
"their is a well funded exxon/mobil denial campaign against global warming, they thought the had something with these stolen emails"
Sir John Houghton's presentation last week of course included a 'hockey stick' graph.
(an audio recording by the church - to verify this will be available)
His voice of authority, and many other similar ones, political as well (Al Gore, etc) and this message, from the IPCC, lobby groups, wwf, greenpeace, etc, remains the sole mesage that the UK/EU political establishment get to hear.
It is the only message the general public hear..
It is the prevailing CURRENT message.
Back in the real world outside the internet amongst friends, parents at school, etc
I know of no one who has even heard about climategate, let alone the discussion. I know of no one else that has come across:
Climate Audit
Bishop Hill
Watts Up
or even Real Climate
Sir John Houghton, started up the Hadley Centre, co chaired the early IPCC reports, his replacement at Roger Napier the MET office previously chaired the WWF- UK (a vocal advocate of AGW)
Roger Harrabin(BBC environment analyst), asked various sceptical websites for a list of uk ' sceptical climate scientists' in post...
How could there be any?!
The key figures in the UK scientific and political establishment is in consensus on AGW for 25 years. No one would have ever received funding for, or studied for a phd if they were remotely scepical, because all the 'climate' professors, grew out of, worked with or were connected with, Hadley, Tyndall, met office, etc..
All the UK has done for 25 years is breed consensus 'climate scientists'.
15 - 20 years ago, it was decided that man made global warming was proven. Since then it has not been about science, but the use of science to support this preconceived agenda.
It was by shear chance that I attended this meeting with Sir John Houghton. His brother happens to live in Wargrave, Berkshire UK. Some people are trying to make Wargrave and Henley Transition Towns and they invited him along to speak in suport of this.
Literally on my doorstep, at the church where I have taken all my children to the church toddler group and family services.
Transitions Towns say this:
"Haven’t they disproved Climate Change?
They have tried, but the science has stood up to the onslaught. The discussion among climate scientists is about how bad it is going to be, how quickly it will happen and how best to minimise possible disastrous consequences."
http://www.henley-in-transition.org.uk/about-transition/index.htm
Have a look at their highly emotive movie, using children extensively.
http://www.transitionnetwork.org/about/publications/transition-movie
Small (cute) child saying:
"CO2 is one of the MAJOR greenhouse gases"
"the weather is becoming more and more unpredictable, JUST because we are realesing more and more CO2 into the air."
Transitions Towns have many things I agree with, sustainability, thinking locally, even alternate energy sources, but I would be labelled a 'deniar' by them.
So whilst, we may discuss the science here, 15 years ago the key 'climate science elite' (ie IPCC lead authors, chairs, etc) scientists became political/social public policy activists. Environmental lobby groups, social change political groups all jumped on board the AGW bandwagon to use it for their own political agenda.
Following his presentation: - God, Science and Global Warming -
I asked Sir John Houghton a direct question afterwards:
Have you looked at the Climategate emails for yourself, in context, and the code, and looked at what people are saying..
Sir John Houghton's answer: No
There is a well funded fossil fuel exxon mobil denial campaign using the same people that lobbied against tobbaco.
We had a long discussion.
Mainly around the lines of as a scientist..
phil Jones's comment pre-climategate to Warwick Huges:
"why should I supply data, you only want to find things wrong with it"
Which is appalling to any scientists that is NOT a 'post normal scientist'
What if, that scientist, found something wrong (if you supplied the data) that found that the risk/temperature were UNDERESTIMATED, it is indefensible not to publish data/code used to derive results for scrutiny.
You need to see the code as well, because whilst you say you programmed it a certain way (as I am an IT profesional) you may NOT have actually done it in the code correctly, it must be available for audit.
I have no doubt Sir John Houghton sincerely believes what he says:
As do virtually ALL the European and UK politicians.
The rest of the conversation, will be 'hearsay' to many, so I will not repeat it here.
Let me know, if you want tho hear full audio recording of this public meeting/presentation.
How to get the real scientific debate here and eslewhere, to the politicians and vast majority of the general public is going to be a challenge.
That's good stuff, Barry. It's not the Arctic that is screaming.
==================
Bingo, Barry; Judy calls the Houghton stuff 'astonishing'.
Why am I not astonished? Well, I'm not astonished he'd believe it, I'm astonished that he'd say it. But, if he believes it, why wouldn't he say it? Where the Hell is that looking glass?
==============
As I mentioned to Barry, his posts with multiple links got shunted to my spam filter. But they are on the site now.
While I have put a handful of rude commenters on moderation (dhogoza being one), nobody's comment gets blocked or deleted unless it violates my comment policy and then I always personally email the commenter to tell them I deleted the comment and why.
I do welcome all Bishop Hill readers, so thanks to those of you who have stopped by. Please keep coming back, as I'm sincerely trying to create a neutral and welcoming space where all sides can talk with one another in civil fashion.
@Barry : "Roger Harrabin(BBC environment analyst), asked various sceptical websites for a list of uk ' sceptical climate scientists' in post...
How could there be any?!"
I emailed Harrabin referencing Dr. Keiller as quoted here:
http://www.eastangliaemails.com/emails.php?eid=1064&filename=1256760240.txt
I'd like to show the back and forth with Roger (very ignorant regards issues dendroclimatological) but the BBC boilerplate wouldn't be happy.
Nevertheless, *some* of the issues raised were put to Jones by Harrabin in a subsequent interview.
Dhogaza has posted a clarion call on Tamino's site (20/6 4.31) re being on moderation and this -
"Wouldn’t hurt for people to gang up there. Curry really needs to be held to account, regardless of “manners”.
http://tamino.wordpress.com/2010/04/12/open-thread-19/#comment-42638
Like I said earlier .. with friends like these....
Hi Keith Kloor.
I only ever thought it was a technical problem. (not like RealClimate)
As an experiment, ask Judith to get a unknown friend of hers to post comments (that Judith my make) at Realclimate.
I do not think she is aware of how 'normal' people (ie unimportant members of the public) get treated there. (and other such places..)
I cocked up with a typo in my name, the other day, which must have put one post into your spam.
Or that you were being cautious, and taking time to verify, the links and verification of my post..
A very interesting blog...
Please keep it up, we probably would agree on 95 % environmental issues.
the CAGW advocates (for want of a better word, warmists, alarmists, warmistas, etc) just do not come across very well do they!
Some of the commentators, just do not realise, how they come across (Michael Tobis being one)
I bet he has not looked at any of the climategate emails, or the code or the emails.
That is OK, neither has John Houghton, or a friend of mine that WROTE some of them.
Such is the consensus thinking
kkloor
Thank you and greatly appreciate that blog, although I failed to contribute. It's serves as a valuable confirmation that the Bishop's book seriously cuts to the quick.
Tamino's site name is 'Open Mind' . Go figure!
3x2 How true and most families/groups have members who others wish they weren't. Going by the 'toxic' comments (to borrow an adjective M Tobis used to describe CA) posted over many years, I guess this character is not viewed as such on some blogs. Selective blog moderation is a great tool!
Keith Kloor,
A very interesting and informative post at your site. There should be more of it.
It's your blog of course, but I'm sorry in a way that the wonderful Dhogaza has been moderated. He is an absolute asset to the lukewarmer/sceptic cause.
Barry Woods
I see you picked up on the same irony re 'Open Mind'. : -)
Don Diego
Was it you who mentioned words? There was one I've not come across before used by John Anderson in his post, #330, para 4:
panjandrum [pænˈdʒændrəm]
n
a pompous self-important official or person of rank
[after a character, the Grand Panjandrum, in a nonsense work (1755) by Samuel Foote, English playwright and actor]
I'm not sure if anyone referred to it, but the previous thread on collide-a-scape is also well worth a look here.
There is a long discussion of the use of proxies in Mann08, with Gavin (Schmidt) complaining about having to repeatedly reiterate to questioners what has already been answered/published. Gavin claimed that leaving out both sets of dodgy proxies only made a minimal difference to the temperature reconstruction. Turns out that when Gavin finally gave a clear answer and a link to the graph that showed what happens, well...read what Lucia had to say:
Lucia's other comments are also worth looking at as examples of clear thinking. Full marks to Gavin for engaging in this thread and answering various questions, but not such good marks for the final result of his engagement (from his point of view).
As Bishop Hill pointed out the Judith Curry challenge to climate scientists
I thought it only fiar, to copy here, the challenge I made to 'Climate scientists' in the same thread.
-----------------------------------------
Back on topic..
To All Cllimate scientists, passing by
Have you looked at the contents of the climategate file(leak/hack) Harry_Read_Me.txt.
‘The Hockey Stick Illusion’ merely touches on this (was due to go to print, just before climategate)
Judith
This is my challenge to you and other climate scientists, that have worked with CRU datasets.
Ask them to Look at Harry_Read_Me.txt file,
for non programmers, skip the code extracts and data extracts look at the comments.
Ask experienced software developers, in other fields to take a look, explain what HADCRUT is to them
Anyone can get a copy of Harry_Read_Me.txt for themselves for FREE, so no excuses there.
Note the continued reference to databases, data, before they even get around processing it.
It is 300 pages long, tip of the iceberg problem, and in context much worse.
Dare your colleagues look?
Let me show you why, I have no faith in CRU datasets, data handling and code and the culture that presumably remains
——–comments from: Harry_Read_Me.txt ——————-
“Am I the first person to attempt to get the CRU databases in working order?!!”
I’ve read the papers, and the miniscule amount of ‘Read Me’ documentation, and it just doesn’t make sense.”
“It’s botch after botch after botch.”
“There is no uniform data integrity, it’s just a catalogue of issues that continues to grow as they’re found.”
“So with a somewhat cynical shrug, I added the nuclear option — to match every WMO possible, and turn the rest into new stations … In other words what CRU usually do. It will allow bad databases to pass unnoticed, and good databases to become bad …”
“but I really don’t think people care enough to fix ‘em, and it’s the main reason the project is nearly a year late.”
“Quite honestly I don’t have time – but it just shows the state our data holdings have drifted into. Who added those two series together? When? Why? Untraceable, except anecdotally.”
“You can’t imagine what this has cost me — to actually allow the operator to assign false WMO (World Meteorological Organization) codes!! But what else is there in such situations? Especially when dealing with a ‘Master’ database of dubious provenance …”
“I am very sorry to report that the rest of the databases seem to be in nearly as poor a state as Australia was.”
“But what are all those monthly files? DON’T KNOW, UNDOCUMENTED. Wherever I look, there are data files, no info about what they are other than their names. And that’s useless …”
”It’s the same story for many other Russian stations, unfortunately – meaning that (probably) there was a full Russian update that did no data integrity checking at all. I just hope it’s restricted to Russia!!”
“Oh, sod it. It’ll do. I don’t think I can justify spending any longer on a dataset, the previous version of which was completely wrong (misnamed) and nobody noticed for five years”
“OH F— THIS. It’s Sunday evening, I’ve worked all weekend, and just when I thought it was done, I’m hitting yet another problem that’s based on the hopeless state of our databases.”
“These are very promising. The vast majority in both cases are within 0.5 degrees of the published data. However, there are still plenty of values more than a degree out.”
“This took longer than hoped.. running out of disk space again. This is why Tim didn’t save more of the intermediate products – which would have made my detective work easier. The ridiculous process he adopted – and which we have dutifully followed – creates hundreds of intermediate files at every stage, none of which are automatically zipped/unzipped. Crazy. I’ve filled a 100gb disk!”
“So, good news – but only in the sense that I’ve found the error. Bad news in that it’s a further confirmation that my abilities are short of what’s required here.”
“This whole project is SUCH A MESS …”
“ This surely is the worst project I’ve ever attempted. Eeeek.”
“So, where there are 7 or 8 missing values in each month (1961-1990), a station may end up contributing only in certain months of the year, throughout its entire run! ”
——————————————–
My Msc is IS Engineering – Cybernetics, plus a career in software in business critical ‘real time’ telco/banking applications. I do not need advice from climate scientists about how computer code is ‘different’ in science research
This documents (help file, notes/log)the notes of an apparently sole developer attempts to knock the code and data into shape for HADCRUT 3.0.
From the previous data/code for HADCRUT 1.2 and 2.0.
Any climate scientist will know of these files..
This file, leapt out at IT professionals as being FAR MORE DAMAGING, than any of the climategate emails.
After I found out the context of the emails, I still think that this is more damaging.
For reason Latimer Alder summarised: (ref#303)
How many climate scientists have work based on HADCRUT datasets.
Or any other CRU datasets, any one of the comments below should give massive pause for thought, by anyone who has used these dataset for research.
As the software development, data handling culture appears ingrained, amongst the long standing leadership at CRU.
There must be a software/data audit by indpendent software professional of all the CRU code and datasets, against government standards for any government IT project. As after all huge government policy descision are being made.
CRU, according to NASA, Giss, has better dataset with less problems, than the other world temp datasets
How many thousands of scientists use them..
Silence is amazing,
but my bet is no conspiracy, they just have NOT looked at this yet..
Barry Woods
Re the Anna Haynes/Watts issue. Watts stated on his blog even Tamino agreed that Anna's behaviour was incorrect and provided a link to the 'Open Mind' post titled “Your right to say it,”. I remember reading it.
That link no longer works because a poster, (not Anna) complained to Wordpress which then asked Tamino to "remove all posts and all comments that either name or allude to" the complainant. In response, Tamino has deleted the entire post and comments.
http://wattsupwiththat.com/2010/06/07/a-note-about-boundaries/#more-20287
http://tamino.wordpress.com/2010/06/21/silence/
Barry..
I've revisited the programming code many times. I spent a lot of time on the briffa_98 fudge factor lines, eventually to drop the pursuit after (IIRC) a comment by Steve McIntyre that it perhaps wasn't actually the smoking gun that I'd initially thought that it was. As far as I know, that's still to be confirmed by someone with more understanding of the larger context of that file than me, or by Ian Harris himself, however. I spent quite a lot of time in back/forth with one of the Telegraph's journal bloggers (I can't remember who), who had posted arguing that the code was commented out and therefore wasn't used.. there were several versions of the file, and I think we established that in the last rendition of the file the code was indeed used.
However, in which plot it was used and for what purpose I haven't been able to state with any certainty and so the all-important conclusive link to demonstrate data fakery and malfeasance with far-reaching implications was broken. My best guess is that, in the absence of a particular dataset - reasons unknown - the fudge array would (adequately?) represent that dataset on a combined plot. Whether that's an acceptable method of representing data on a plot may be a topic to debate in itself. I've not committed either way.
JG-C's criticism of the code is that it's disorganised, poorly documented and does not meet the standards demanded by those funding the work in these respects. I would have to agree with him, but that has coding QC implications which are separate from data QC. Additionally, the coding QC failings don't themselves directly indicate failures in functionality.
What the readme does, however, is betray almost incomprehensible data QC issues in CRU data. As you clearly chart, above, Harris was himself appalled by the quality of the data he was working with. The problem here is that the information in the readme is anecdotal. It charts frustrations with difficulties understanding and integrating the different databases he's working with. However, my bug-bear is that it's not necessarily reasonable to conclude absolutely or with any certainty that, because it is not documented, data reconciliation in these files wasn't ultimately achieved to perfection. Whether or not we should have faith in Harris' ability to achieve this, given the documented data issues, this is a sticking point.
All that said, I agree wholeheartedly that it isn't necessary to understand the minuscule detail of Harris' trials and tribulations at this point. A read of Ian's readme, without doubt, raises questions that require answers. As you rightly say, these questions have yet to be asked. If the CRU data is the best of the bunch, and Harris intimates that it's atrocious in the readme, we really need to know the extent of the problem. Each and every one of Harris' complaints in the file need to be addressed and we (everyone, including the climate scientists who are building their work on this gridded product) need to know what level of integrity and what kind of confidence can be placed in the CRU's adjusted data.
Whatever Harry did to improve it... (this now need to be audited)
And there is a lot of comments that show, thngs he was uncomfortable with.
What about all the work that had gone before, by other scientists, based on the earlier versions of the datsets, that Harry had the problem with..
Until I see some auditing of CRU, to government IT standards and methodolgies, we can only assume that the culture of data handling integrity, etc remains at CRU, and is likely endemic across the field.
The code maybe perfect, it may not,
the output depends on the integrity of the data, processed by the code.
This is in question.
Verification that the code actually,is coded correctly and is fit for purpose is also required..
This is evidence that CRU code, etc, would FAIL any, IT audit to published government IT project standards..
Why do so many people not get this.
Ask any IT professional who has ever had to work on a government funded it project, for the procedure/standads required, surely CRU must be seen to live up to those standards
Agreed, Barry. Regardless of whether or not Harris appropriately achieved his goal of bringing the data together (I mostly sensed that Harris just wanted to get to the END of the project, by any and all means.. but that is simply my subjective reading of the file) there is far and above sufficient reason to demand an audit of the work performed by the IT few at the CRU.
The lady is mischievous. She has read the HSI and knows there is nothing to rebut. So this looks like a provocation of the people who depend on the whole thing just being forgotten.
Barry Woods, SimonH
Like many others, I had gone along with the crowd in assuming that the climate scientists' claims were well-founded.
Indeed - when Climategate broke I tended to play down the emails, any reader of C P Snow or David Lodge recognises that academics can be bitchy. But I found the Harry_read_me file astounding, it indicated a total mess in at least the aspect of the CRU's "results" that Ian Harris had been spending so long to unravel. And if that side of the CRU's work was a total mess - it appeared illogical for CRU defenders to argue "that does not apply to everything the CRU has done".
The extreme reaction by Warmists - for instance that CRU was beyond reproach..... it was some unknown hacker who should be attacked....and the denial of any possibility that it was a leak rather than a hack - made me even more intrigued. And it was only then that I started to realise that the whole IPCC / UN business looked far less clear-cut, that the science was perhaps NOT settled.
I then visited a few sites - following one link to another - and could see the vitriol used against anyone who challenged the "consensus". As a mere layman, it did not take long to conclude :
1 Much primary data appears suspect
2 Many of the published papers on which IPCC have based their programmes had not been reviewed properly in terms of statistical competence - let alone the methods used to include, exclude or adjust the (often suspect) primary data.
3 There have been years of resistance to proper re-appraisal of some of the core papers. And the much-vaunted IPCC process for evaluation of evidence appears deeply corrupt.
4 All this is compounded by provably false alarmist claims - glaciers, polar bears etc.
As Dr Judith Curry has been arguing, the Hockey Stick should be seen as a clear example of refusal by the "consensus" to deal properly with open and clearly-stated challenges.
One of my daughters has a much-cited published paper in cardiology. (460 citations the last time I checked). In writing her paper she had to take a lot of advice from a very senior statistician - who is named as a co-author. Further, all her data (mainly MRI readings) were made available. All this was onerous - but she and her colleagues accepted this was necessary, because what they were propounding was a radical change in how a certain set of patients ashould be treated.
That set of patients is a small fraction of the population. The IPCC stuff affects all of us - therefore the "science" on which it is based should be subject to the same stringent safeguards, the same checking and re-checking.
Barry Woods said: Judith
This is my challenge to you and other climate scientists, that have worked with CRU datasets. Ask them to Look at Harry_Read_Me.txt file, for non programmers, skip the code extracts and data extracts look at the comments. Ask experienced software developers, in other fields to take a look, explain what HADCRUT is to them. Anyone can get a copy of Harry_Read_Me.txt for themselves for FREE, so no excuses there.
Note the continued reference to databases, data, before they even get around processing it.It is 300 pages long, tip of the iceberg problem, and in context much worse. Dare your colleagues look?
Professor Jim Norton is a member of the Russell ” Independent Climate Change Email Review” whose report is due to be published shortly. (I had always understood it was somewhat vainglorious to style yourself "professor" in public life if you merely held a visiting appointment, rather than a university chair.) He is supposed to have been looking at the IT aspects, data integrity and so on.
It will be easy to see within a few seconds whether the Russell Review is just another whitewash. If Norton makes no mention of Harry_Read_me.txt file, then it will be immediately apparent that the Russell Review cannot be taken seriously.
If it turns out to be, as I imagine it will, another whitewash in the stream of post Climategate whitewashes, no doubt there will be much jubilation over at RealClimate. But, in the long run, it will not be to the benefit of climate science to have another false vindication, further delaying the time when the field will eventually be sorted out.
Martin A
But, in the long run, it will not be to the benefit of climate science to have another false vindication, further delaying the time when the field will eventually be sorted out.
Funny how that happens. But it does. And from what I see, it has already started with JC taking a position publicly. She will turn a few, and those a few more. And in time, your prediction will come to pass.
However, that is no excuse to lessen the pressure from the rest of us.
Reading through the comments there has greatly raised my opinion of Dr. Curry. I was suspicious of her initial attempts to bridge the gap between both sides, but the conviction she's shown in sticking to her guns and willingness to call out some of the BS she sees on both sides is admirable.
The thing that strikes me the most is, in an environment where making any statement that is contrary to the "consensus" can jeopardize their funding and possibly their job, how can any climate scientist really expect the public to believe them. Yes, some people will have the time and knowledge to read the details of papers and make their judgements based on what's presented. Most people are simply going to go by what they hear through various sources and use their common sense. Any undecideds that see the comments and actions of the "consensus" are going to sense that somethings seriously wrong.